#2016-05-2512:06sveriIs there something in spec that is comparable to string? but for a boolean value?#2016-05-2512:06sveriexcept #(instance? Boolean %)#2016-05-2512:10richhickey@sveri: not yet. Obviously there are several more useful predicates that could go into core. Considering which ones now#2016-05-2512:10sveri@richhickey: Yea, it would be nice to be consistent, much less work for the brain then š#2016-05-2512:19moxaj(spec/def ::foo (spec/cat :kw keyword? :int integer?))
(spec/def ::bar (spec/coll-of ::foo []))
(spec/explain ::bar [[:a 10] [:b 20] [:c "30"]])
;; => val: [[:a 10] [:b 20] [:c "30"]] fails predicate: (coll-checker :spec/foo)
Why doesn't the error reporting go into detail (like: val "30" fails predicate: integer?
) Is this intentional?#2016-05-2512:44sveriHm, is it ok to use ::spec/name for my own specs? Also I have not found the source for ::spec/string, neither by searching github (which is a bit harder, because it ignores "::", nor by looking at the api.#2016-05-2512:44sveriAlso, is it correct that ::spec/name has string?
as predicate?#2016-05-2512:59kingoftheknoll@alexmiller: About a week ago you hinted on Reddit at improved error messages for Clojure. Donāt think anyoneās said that here but it seems like we just got a peek into how that might be accomplished if all of Clojure.core implements specs.#2016-05-2514:08mjhamrickWhat's the best way to have fdefs be checked during lein test? Instrument all works at the repl, but seems to not during test. #2016-05-2514:10anmonteiro@mjhamrick: what Iāve done is (clojure.spec.test/check-var #āfn-speced-with-fdef)
#2016-05-2514:14mjhamrick@anmonteiro: are you just putting that in each test? #2016-05-2514:15anmonteiroso far, yes#2016-05-2514:15anmonteirothe project where I tried it was pretty small#2016-05-2514:24sveri@mjhamrick: I did put (s/instrument-all)
at the top of each test namespace#2016-05-2514:35Alex Miller (Clojure team)@kingoftheknoll: yes, I have done additional work in this area#2016-05-2514:44Alex Miller (Clojure team)@sveri ::spec/name and ::spec/string don't exist - what is that from?#2016-05-2514:46sveri@alexmiller: Thats why I have not found it in the code. Not sure where exactly it came from, its just that cursive offered it via code completion. Maybe I typed it in a different namespace or whatever. Anyway, good to know.#2016-05-2514:52michaeldrogalis@richhickey @alexmiller: I have another proposed enhancement to explain-data
. Several predicates, such as keyword?
, or integer?
are programmatically easily recognizable via :pred
for non-conforming values. It's a straightforward equality check (e.g. (= 'keyword? (:pred error-map))
). Other predicates are, to my understanding, more difficult to recognize. Take s/keys
and s/tuple
for example. In the former, keys
yields a predicate when a required key is missing. While the predicate is somewhat understandable when you eye ball it, picking it out programmatically has been error-prone. The same goes for s/tuple
when the number of arguments is incorrect.#2016-05-2514:53Alex Miller (Clojure team)an example would help make this specific#2016-05-2514:53michaeldrogalisThose functions have privileged access to which predicates are failing. It might be helpful to provide named predicates in those cases so that consumers of explain-data
could be more intelligent. See - https://gist.github.com/MichaelDrogalis/016ac62cf3f1e899fb89fd79f2de2277#2016-05-2514:54Alex Miller (Clojure team)thx for the example! :)#2016-05-2514:54Alex Miller (Clojure team)oh the contains stuff that's inside keys#2016-05-2514:55Alex Miller (Clojure team)I'll have to defer that to Rich, I think he's not around atm#2016-05-2514:55moxajIt seems to me that inner conformed values are lost when using coll-of
or map-of
. Example:
(spec/conform (spec/coll-of (spec/and integer?
(spec/conformer (constantly nil)))
[])
[1 2 3])
returns [1 2 3]
instead of [nil nil nil]
. Meanwhile, cat
and keys
preserves them. How come?#2016-05-2514:56michaeldrogalis@alexmiller: Sure š Ill be back later.#2016-05-2514:57Alex Miller (Clojure team)@moxaj Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this#2016-05-2514:57moxaj@alexmiller alright, thanks!#2016-05-2515:41sveriI am converting a lib from schema to spec and in schema I was able to inline define the return values / args. Is there a way to do that in spec too? The alternative in this case is a lot more code.#2016-05-2515:44Alex Miller (Clojure team)for map keys, no - that's the whole point of the map / attributes split#2016-05-2515:44Alex Miller (Clojure team)the benefit is that you are creating named reusable semantics for ::entityname, ::ns, etc etc#2016-05-2515:44Alex Miller (Clojure team)so you can use those elsewhere too#2016-05-2515:45sveri@alexmiller: Yea, I understand the benefits, just wanted to make sure I did not miss anything#2016-05-2515:48mpenetIs it possible to hook ones own error message if a predicate fails? Ex we have schemas that takes a string that should be a valid parse of our internal query language, and if fails should return a nicely structured error with line/col num and a human readable message. With prismatic Schema it is possible to do with our own schema type extending a protocol for explain/spec.#2016-05-2515:51mpenetCorrect me if I am wrong, but I think so far I can only get a generic "predicate foo? failed" type of message#2016-05-2515:55Alex Miller (Clojure team)there is not currently a way to provide a custom error for a failing predicate#2016-05-2515:56Alex Miller (Clojure team)you can use explain-data to detect problems and produce a custom error message#2016-05-2515:56Alex Miller (Clojure team)but @richhickey can comment on whether that might be something we could do#2016-05-2515:57mpenetthat's what I suspected. Something like ex-info for specs could be useful#2016-05-2516:05Alex Miller (Clojure team)why not explain-data for that?#2016-05-2516:05Alex Miller (Clojure team)or maybe I misunderstand the suggestion#2016-05-2516:05mpenetI actually missed it, that could work yes#2016-05-2516:06Alex Miller (Clojure team)that's what instrument does on a failing function spec#2016-05-2516:15mpenethmm I am not sure that's really equivalent. I would have to wrap all schemas that contains these values with a special validation fn I think. I would like to have my own datastructure returned by explain-data (or something else) when the predicate fails. I am not sure that's doable with the current approach where predicates return just a potentially truthy value.#2016-05-2516:16mpenetbut I could be wrong. I played for 20min with specs so far.#2016-05-2516:24mpenetthis might sound like a small concern, but in a larger context it's very useful: we have a whole api around the manipulation of data containing such queries, being able to use the same schema type and framework and not wrap the whole thing because of a single type in a potentially nested Schema is a nice feature, in our case it also integrates for free with frameworks that just follow this Schema format (rest server, swagger etc).#2016-05-2517:38eraserhdFirst piece of feedback is that clojure.spec makes me really want āboolean?"#2016-05-2517:49sveri@eraserhd: @richhickey said they consider adding this and some more core types (I felt the need for it too :-))#2016-05-2517:52eraserhd@sveri yay š#2016-05-2517:52eraserhdIām still playing with it, but the second thing Iām working through is the repetition, especially for function arguments.#2016-05-2518:16Alex Miller (Clojure team)@eraserhd: we have boolean? in almost every example namespace we wrote. so, agreed. :) definitely near the top of the wanted-predicates list.#2016-05-2518:16eraserhd@alexmiller: cool š#2016-05-2518:18eraserhdAnother interesting problem I encountered: I used a set for boolean: #{true false}. But sets arenāt treated differently in this context, although I might expect that they are. Namely #(#{true false} false) ;=> false. So having a set in a spec which contains nil or false is awkward.#2016-05-2518:19Alex Miller (Clojure team)yeah, don't do that :)#2016-05-2518:19Alex Miller (Clojure team)it is kind of an interesting wrinkle though that the obvious way to write boolean? is with #(instance? Boolean %)#2016-05-2518:20eraserhdIt might at least deserve a call-out in docs.#2016-05-2518:21eraserhdAlso, a lot of things seem to leak their internal implementations wrt error messages.#2016-05-2518:23Alex Miller (Clojure team)but Clojure uses (only) canonical boolean values#2016-05-2518:23Alex Miller (Clojure team)@eraserhd: prob better for a mention in the ref doc (which isn't there yet)#2016-05-2518:24Alex Miller (Clojure team)@eraserhd: example re errors?#2016-05-2518:24eraserhd@alexmiller: I encountered a couple. Just a minute and Iāll paste.#2016-05-2518:55sveriAnyone else seeing problems with reloading test namespaces in the REPL (cursive)?
I just commented out some tests, reloaded the test ns in the REPL, hit run-tests, but still, all tests were run.
I noticed something similar with test-refresh not recognizing commented fdef
s.
I cannot point at anything right now, but, there seems to be something broken#2016-05-2519:01kovasb@cfleming: are there gonna be slick Cursive toolips for providing associated specs when editing functions arguments?#2016-05-2519:02kovasbI imagine I'm editing the arg to some function that takes a big map, and as I'm filling in the components of the map its telling me what each piece should conform to#2016-05-2519:09arohnerWhat is the spec for āmap? i.e. in core.typed, Iād write something like (t/ann map [ [ X -> Y] [X] -> [Y] ). i.e. it takes a collection of X, and a function of X->Y, and returns a collection of Y. Is there a way to do that in spec?#2016-05-2519:25bbrinckIām missing something related to recursive specs. I want to define a vaguely hiccup-like structure where the first element is the name, followed by some attributes, followed by a vector of children. What am I doing wrong here? https://gist.github.com/bhb/6cfcb3b38757442aec4ba5db46148699#2016-05-2519:38hiredmanif you add another (s/spec ....) around the ::tag I get a success, but I am not sure why#2016-05-2519:42hiredmanI guess you are in the regex context that matches the outer vector, so you need another s/spec to create a new regex context for the list of child vectors, then you need another s/spec to get out of the regex context of the vector of children and specify an individual child#2016-05-2519:48anmonteiroIād also be interested to see the spec for clojure.core/map
#2016-05-2519:48bbrinck@hiredman: Ah, thanks! I had to read that a few times, but now it makes sense š#2016-05-2519:56sveriOk, I dont understand how to use instrument
. My guess was, that if I put (s/instrument-all)
into a namespace all functions are instrumented whenever I load the file into the REPL. But thats not the case, instead I have to execute that function everytime after I loaded a namespace.#2016-05-2519:56sveriAlso, it works when I put it to the end of a ns#2016-05-2520:02dryewoHi all, first I want to thank Rich&Co for this awesome innovation :+1:
And I have a question:
(s/def ::even-big-or-small (s/and (s/or :very-big #(> % 1000)
:very-small #(< % 1))
even?))
(s/valid? ::even-big-or-small 10000)
;; throws CompilerException java.lang.IllegalArgumentException: Argument must be an integer: [:very-big 10000]
what am I doing wrong?#2016-05-2520:07dryewojust noticed, works fine the other way around:
(s/def ::even-big-or-small (s/and even?
(s/or :very-big #(> % 1000)
:very-small #(< % 1))))
#2016-05-2520:12anmonteiro@dryewo: in the first case, itāll conform to :very-big
and output [:very-big 1000]
#2016-05-2520:13anmonteiro(s/def ::even-big-or-small (s/and (s/or :very-big #(> % 1000)
:very-small #(< % 1))
#(even? (second %))))
#2016-05-2520:13anmonteirothis works#2016-05-2520:13anmonteiroeven?
gets [:very-big 1000]
so we are interested in the second element#2016-05-2520:13anmonteirothatās the explanation, but Iām also unsure if itās expected#2016-05-2520:14dryewoyes, I also got this idea of whatās happening, but is it the correct behavior?#2016-05-2520:15dryewoshouldnāt s/and
be order agnostic?#2016-05-2520:16anmonteiroIāve read somewhere that itās not, by design#2016-05-2520:16anmonteirobut I canāt recall the reason#2016-05-2520:18dryewoIām reading the guide: http://clojure.org/guides/spec
maybe I should first read it to the end before asking more questions š#2016-05-2520:25kenny@dryewo: Look into https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/conformer#2016-05-2520:26dryewothanks, there is even a section about it in the end of the guide, Iāll look into it#2016-05-2520:35kennyIs there a pattern for whether you spec functions or use spec for validation (pre/post-conditions)? There doesn't seem to be a reason for pre/post-conditions if you spec a function.#2016-05-2520:39sveriI dont understand that, I have the code in the following snippet, and it fails with:
(remove-autoinc-columns [{:foo "some_t"}])
ExceptionInfo Call to #'de.sveri.clospcrud.s-play/remove-autoinc-columns did not conform to spec:
At: [:args] val: ([{:foo "some_t"}]) fails predicate: (cat :cols :de.sveri.clospcrud.s-play/columns), Extra input
:clojure.spec/args ([{:foo "some_t"}])
clojure.core/ex-info (core.clj:4617)
#2016-05-2521:10anmonteiro@sveri: you need to wrap ::column
in s/spec
#2016-05-2521:10anmonteirolike this: (s/def ::columns (s/cat :col (s/* (s/spec ::column))))
#2016-05-2522:34Alex Miller (Clojure team)@arohner: @anmonteiro here's a spec for map
(assumes a seqable? predicate like the one in core.incubator)#2016-05-2522:34Alex Miller (Clojure team)(s/fdef clojure.core/map
:args (s/cat :f ifn?
:colls (s/* seqable?))
:ret (s/or :seq seqable? :transducer ifn?))
#2016-05-2522:35anmonteirosimpler than I imagined#2016-05-2522:35Alex Miller (Clojure team)I've left off :fn because I'm lazy but you could also verify some additional things#2016-05-2522:36Alex Miller (Clojure team)like :f / :transducer or :colls / :seq are the valid combos and even things like whether the :ret cardinality is minimum of the input colls cardinality#2016-05-2522:37Alex Miller (Clojure team)but that's enough to detect things like (map inc 100)#2016-05-2522:37Alex Miller (Clojure team)which is the kind of thing that normally leads to IllegalArgumentException Don't know how to create ISeq from: java.lang.Long#2016-05-2522:38Alex Miller (Clojure team)which, if it's occurring in a nested context can be pretty confusing#2016-05-2522:40tylerAnyone given the test namespace a try yet? Iām trying to figure out how check-var
works. It appears to be hanging half the time I try it and passing tests the other half.#2016-05-2522:45arohner@alexmiller: is it possible to check that fās input matches the coll?#2016-05-2522:45arohnerright now I donāt see a way to do that#2016-05-2522:46arohnerbecause there doesntā seem to be a way to ācompareā specs#2016-05-2522:46Alex Miller (Clojure team)you would have to use a custom predicate to do so#2016-05-2522:47Alex Miller (Clojure team)the :args spec could be (s/and <current> #(custom predicate on the args))#2016-05-2522:48Alex Miller (Clojure team)I guess you might need to describe :f with an fspec to get the right parts to work with#2016-05-2522:48Alex Miller (Clojure team)I haven't done it!#2016-05-2522:57tylerIāve created a gist with my attempt to try and use check-var
it is returning nil, is that expected behavior? https://gist.github.com/tvanhens/28b7f744d799910750d8ae3942680997#2016-05-2523:11hiredmando you have the test.check library on the classpath?#2016-05-2523:34tylerYup gen/generate works#2016-05-2523:46stuarthalloway@tyler: there is a bug there, I hit it too#2016-05-2523:47stuarthallowayI think it is fixed on master#2016-05-2523:52tylerAh awesome thanks#2016-05-2523:53richhickeyyah#2016-05-2523:57eraserhd@alexmiller An example wrt errors: (spec/explain (spec/tuple number?) []) gives an error about #(clojure.core/= (clojure.core/count %) 2).#2016-05-2523:57eraserhds/2/1/#2016-05-2523:59richhickey@eraserhd: "val: [] fails spec: _ predicate: (clojure.core/= (clojure.core/count %) 1)ā says the collection should have count == 1#2016-05-2600:00richhickeytuple is not a parameterized coll (see coll-of for that), (tuple number?) says vector of exactly one number#2016-05-2600:01richhickeytuple is for short tuples of heterogeneous types each of which will be listed#2016-05-2600:01eraserhd@richhickey Oh hi! My comment is about the message, not that it should passā¦#2016-05-2600:01richhickeye.g. (tuple string? number?)#2016-05-2600:01eraserhdI was saying that it reveals implementation detail of spec/tuple.#2016-05-2600:02richhickeyvs what?#2016-05-2600:04eraserhdI was reading the rationale document as saying that we want to capture the predicates the user supplied, so I was hoping for something like: val: [] fails spec: (tuple number?).#2016-05-2600:04richhickeythat doesnāt tell you much#2016-05-2600:04eraserhdHrmm, this is true.#2016-05-2600:05richhickeyand the interior preds can be arbitrarily complex, will walk down#2016-05-2600:06richhickeymuch better than āyou failed the top"#2016-05-2600:07eraserhdHrmm. So my (clearly not thought through) expectation would be that count would fail at tuple, but sub-types would fail more specifically. Thatās not an idea I can make coherent, though.#2016-05-2600:10eraserhd(Anyhoo, super excited about clojure.specās new ideas, cf prismatic)#2016-05-2600:13richhickeyhave fun!#2016-05-2600:38shaunxcodeis it possible create a spec for a map like {[300 200 54] :status/open} e.g. map keyed by vector of integers#2016-05-2600:42tyler@shaunxcode yes I believe so using map-of
#2016-05-2600:44tyler(s/map-of (s/coll-of integer? []) #{:status/open})
#2016-05-2601:47cfleming@kovasb: Sure, sounds doable#2016-05-2606:41borkdudeShould I include test.check as an explicit dependency when using spec?
I got: FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:456)#2016-05-2606:42borkdudewhen instrumenting the adder
example from the guide https://clojure.org/guides/spec (cc @alexmiller )#2016-05-2607:15sveri@borkdude: Yes, you need it for most cases if you want to use spec in your tests. There was only one function that works without test.check (I think it was spec/gen, but not sure). I have read it somewhere in the official docs in the last days.#2016-05-2607:18sveriAnyone else got an idea why this snippet fails?#2016-05-2607:19sveri@anmonteiro: wrapping it in (s/spec...
doesnt fix it unfortunately#2016-05-2608:50sveriAlso, what works is:
(s/conform ::columns [{:foo "some_t"}])
=> {:col [{:foo "some_t"}]}
which is strange, cause all I do next is to put it into the argument definition of my function#2016-05-2609:07sveriCan someone reproduce this?#2016-05-2609:52dominicmIs there space in spec, for a specification to take parameters via lexical scope? This may be out of scope for spec.
For example, on http request, I take a snapshot of my database, and I want to check if a value is in that database. But I want to have my predicates registered via s/def
so that I can take advantage of :via
and such.
Use case is checking if an email is taken during user registration.#2016-05-2612:10anmonteiroLibrary authors might want to add specs to their existing projects but also keep supporting earlier versions of Clojure#2016-05-2612:10anmonteiroIs this a use case that has been thought about, or isnāt there any interest in supporting such use case?#2016-05-2612:12anmonteiroor are we just supposed to solve it through versioning?#2016-05-2612:32Alex Miller (Clojure team)Specs could be provided in an adjunct library too#2016-05-2612:34Alex Miller (Clojure team)Ultimately, things must move forward to get the benefits of new features as we did with records, protocols, transducers, reader conditionals, etc etc#2016-05-2612:34anmonteiroAgreed wrt. 2nd point#2016-05-2612:35anmonteiroand yea, the optional library add-on could also be a possibility for initial compatibility, hadnāt thought about that#2016-05-2612:35anmonteirothanks#2016-05-2612:35bronsa@anmonteiro: having spec in both clojure.core + an external library could cause namespace conflicts#2016-05-2612:36Alex Miller (Clojure team)They can also be in a separate namespace in your library jar that's just not loaded (unless you're on 1.9)#2016-05-2612:36anmonteiro@bronsa: I understand, but I didnāt mean that.#2016-05-2614:10Alex Miller (Clojure team)1.9.0-alpha3 is now out with several bug fixes and improvements for spec https://groups.google.com/forum/#!topic/clojure/WxT9kPIwlYI#2016-05-2614:11Alex Miller (Clojure team)The guide is now out of date but I will have it updated shortly#2016-05-2614:43dominicm@alexmiller: Is there much of a story for what I mentioned previously? https://clojurians.slack.com/archives/clojure_spec/p1464256334000558
Trying to decide how to approach the problem of generating/validating forms.#2016-05-2615:07Alex Miller (Clojure team)@dominicm: you could register (at runtime) an s/def that partialed/closed-over something known at runtime like a database snapshot. This implies that you are using spec to do runtime data validation. Whether or not this is a good idea, I'm not sure. :)#2016-05-2615:14dominicm@alexmiller: Yeah, I did wonder if dynamic vars might be a solution, and a bad idea.
I'm definitely thinking in the domain of variable user-input, as opposed to contracts during development.
I like the idea of spec as a ubiquitous format that the clojure community uses to describe data. Some data just can't be controlled (user input). Maybe this is way out of scope for spec though, and you guys think that should be a different library altogether. There's definitely a lot of overlap in that space (email still needs to be a string, over 5 chars, match a regex, before it is db checked).
Nothing says a validation library can't register things for you of course. But I try to keep core at the top, not other libraries.#2016-05-2616:01gtrakThe registry itself is pretty simple, I think a 3rd-party lib could sub out the behavior you want but use registered specs along the way.#2016-05-2616:03dominicm@gtrak: Yeah, that's what I was thinking about. But if it is in scope for spec, that would be pretty cool.#2016-05-2616:06gtrakmy understanding so far is that spec itself is mostly concerned with the lib-sharing/transmission/self-describing-edn use-cases, but there are a lot of developer-convenience type cases we'd also want to use it for (overlap with Schema), but it can't integrate all of them.#2016-05-2616:15gtrakFor instance, the first thing I wrote was a multimethod type->spec coercer, which is a special case extension of the built 'conform', I wouldn't expect that to get into clojure.spec, but I might expect it to be available as a lib, since it's so easy to write.#2016-05-2616:16gtrakconformer-helpers..#2016-05-2616:19gtrakhttps://gist.github.com/gtrak/9b6d16d0423b284292dbbdd095bf91e9#2016-05-2616:21gtrakusage: (s/def ::my-key-that-only-matters-to-my-app (coercer integer?))
#2016-05-2616:25gtrakthe one thing I'd request from clojure.spec itself was a way to provide a better error message than what s/conformer creates, which in this case is too generic, a failure in the pred: (fn [from] (coerce from to))
. It's possible I can do so by dropping down a level in abstraction but haven't gotten that far.#2016-05-2616:50dominicm@gtrak: oh I agree! I had wondered if that would be possible! Shame you beat me to it.
But I feel like being unable to lexically scope a spec in some way hinders me a little. Maybe I'm looking at it wrong, or maybe I can come up with a clever roundabout way to do it and lib it up.#2016-05-2616:55gtrakI think I overheard that specs are serializable somewhere, so lexical scope could really mess that up.#2016-05-2617:00dominicm
I probably shouldn't refer to scope. What I want is a solution to the problem of runtime values effecting the running of my predicates, without relying on state.
Am I making sense?#2016-05-2618:19gtrakI don't see how it's possible to do that without state, maybe a dynamic/threadlocal registry, but then the state's in the registry.#2016-05-2618:19gtrakor you create a one-off spec every time#2016-05-2618:20gtrakor every consumer of the registry has to let you tunnel things into specs#2016-05-2620:52sveri@alexmiller: I just found a possible bug in the following snippet:#2016-05-2620:54sveriRunning lein test
for this code will run into a never ending loop when it tries to test leiningen.s-play-test
#2016-05-2620:54sveriI would expect an error message here#2016-05-2620:55sveriThe cause is this line (s/fdef foo :args (s/cat))
with the incomplete (s/cat) expression.#2016-05-2621:19Alex Miller (Clojure team)Feel free to file a jira for it#2016-05-2706:52dominicmhttps://clojurians.slack.com/archives/clojure_spec/p1464286849000602
I wouldn't say that's a terrible idea#2016-05-2708:22dominicmhttps://clojurians.slack.com/archives/clojure_spec/p1464278761000588
I was under this impression also. But I just found this message on google groups: https://groups.google.com/d/msg/clojure/5sZdCPQgZz4/7rpuiaj1AgAJ
> OTOH, you may encounter user- or externally-supplied data at runtime and want to use the facilities of spec to validate/process it. Then you can use valid? or conform explicitly to do so.#2016-05-2719:56Alex Miller (Clojure team)I have added a new section to the spec guide about generators http://clojure.org/guides/spec#_generators#2016-05-2719:57Alex Miller (Clojure team)I still expect there to be a bit more to come on that page about generators and testing#2016-05-2719:57Alex Miller (Clojure team)but seemed better to get something out there sooner#2016-05-2719:58michaeldrogalis@alexmiller Was there any discussion with respect to predicate failures being recognizable from keys
, tuple
, etc? I can dig up the Gist link if you need for reference#2016-05-2719:59Alex Miller (Clojure team)there have been so many things coming in I don't honestly remember if I talked about it with Rich - I have it in my "things to talk about" file#2016-05-2720:02sveri@alexmiller: done so: http://dev.clojure.org/jira/browse/CLJ-1934#2016-05-2720:04sashtonis this a good place to mention spec guide (potential) typos?#2016-05-2720:08michaeldrogalis@alexmiller: No prob, thanks!#2016-05-2720:10Alex Miller (Clojure team)@sashton: sure!#2016-05-2720:10Alex Miller (Clojure team)@sveri thx!#2016-05-2720:12sashton@alexmiller: Under Collections, it has the following:
* Collection - (s/coll-of float?)
I think coll-of
needs a []
#2016-05-2720:12Alex Miller (Clojure team)yep#2016-05-2720:12Alex Miller (Clojure team)thx#2016-05-2720:13Alex Miller (Clojure team)I'll fix this one but for future reference, you can send issues/PRs for the site too http://clojure.org/community/contributing_site#2016-05-2720:41Alex Miller (Clojure team)@sveri I have a patch to fix your bug, should go in the next alpha#2016-05-2721:12sveri@alexmiller: Great, looking forward to testing it#2016-05-2721:13Alex Miller (Clojure team)patch is on the ticket if you want to build your own version of clojure#2016-05-2721:13Alex Miller (Clojure team)I added a test to the tests for it too#2016-05-2721:14sveriYea, I just saw it, I have not setup the clojure build chain, so I just wait for it, no urgency on my side. Thanks for the quick fix#2016-05-2801:27cfleming@alexmiller: You mentioned you have a spec for spec. Any interest in showing that? It would help with adding support in Cursive.#2016-05-2801:30Alex Miller (Clojure team)It's almost certainly pretty buggy so it's not something I really want to publish yet#2016-05-2801:30cflemingOk.#2016-05-2801:31Alex Miller (Clojure team)I have turned on instrumentation with it and had it actually find bugs while I was working though :)#2016-05-2801:32cflemingYeah, Iām planning to merge in the validation code I demoed at the conj, having real-time validation in Cursive for specs would be nice.#2016-05-2801:32Alex Miller (Clojure team)I can send it to you privately if you want#2016-05-2801:33cflemingAnd using specs to help with completion is probably nice too, although I need to work with it more to see whatās possible.#2016-05-2801:33cflemingOk, thanks. Iām not in a big rush, I wonāt get to it for a week or two probably.#2016-05-2801:33cflemingBut Iād be interested to see it.#2016-05-2801:35sekaoanyone have an example of using fdef
with variadic args?#2016-05-2802:04sekaoahh i guess itās something like (s/cat :args (s/* string?))
#2016-05-2811:42Alex Miller (Clojure team)@sekao: yep although you don't even need the s/cat there just '(s/* string?)' would be sufficient#2016-05-2811:53francescohow could I write a spec for, say, clojure.core/conj
that enforces that in (conj coll x)
x
satisfies the same spec as the elements of coll
? Is this kind of reasoning within the intended scope of clojure.spec
?#2016-05-2811:59Alex Miller (Clojure team)That's not a true constraint of conj :)#2016-05-2811:59francescototally agree on that.#2016-05-2812:00Alex Miller (Clojure team)But you could spec something like that with the :fn spec in fdef which relates the args and ret#2016-05-2812:01Alex Miller (Clojure team)It's passed a map with the conformed value of both#2016-05-2812:49sveriA bit more feedback. I constantly run into errors like this:
In: [0] val: ([{:name "age", :type :int, :null false}]) fails at: [:args] predicate: (cat :cols :de.sveri.clospcrud.schema/columns), Extra input
:clojure.spec/args ([{:name "age", :type :int, :null false}])
clojure.core/ex-info (core.clj:4617)
Most of the times the problem was that I was missing a (s/spec ::columns)
expression around the spec definition. Still, everytime I see this error it is so far away from what the problem actually is, that I need time to remember this.
Worse is, that at first I look and search if I did something wrong while transforming my schema to spec.
Maybe a somewhat better error message is possible?#2016-05-2812:49sveri@alexmiller:#2016-05-2814:35Alex Miller (Clojure team)Not sure what a generically better error would be#2016-05-2816:50Alex Miller (Clojure team)Interesting - file a jira if you want so we don't lose it over the weekend#2016-05-2817:26sveri@alexmiller: For me it would be something like: "Maybe you forgot a spec around a regex? Try (s/spec ...)" But, I dont know how generic that error is and if it would fit everytime?#2016-05-2817:54moxaj@alexmiller done, see http://dev.clojure.org/jira/browse/CLJ-1935#2016-05-2818:16sekaois there a way to exclude certain vars from being tested by run-all-tests
? i have some side-effecting functions that will cause some havoc if run in generative tests š#2016-05-2818:39arohnerIām not entirely clear on when generative testing happens. In some cases (side-effects), Iād like the ability to check conform but not do generative testing#2016-05-2818:54sekaofor now i copied the code into my project and added an if
statement that checks for :no-check
in the varās metadata#2016-05-2819:33bbrinck@arohner: My understanding is that generative testing only happens if you invoke run-all-tests
. If you just wanted to test specific functions in your test suite, you could do something like:
(deftest test-my-function
(is (= true (:result (t/check-var #āmy-function)))))
#2016-05-2819:34arohner@bbrinck: Iām pretty sure there are cases aside from that that can trigger it#2016-05-2819:34arohnerIIRC, richās example from the other day will#2016-05-2819:34arohnerone sec#2016-05-2819:35arohnerindeed:#2016-05-2819:35arohnerrepl> (defn foo [fnn] (fnn 42))
#'repl/foo
repl> (s/fdef foo :args (s/cat :f (s/fspec :args (s/cat :i integer?)
:ret integer?)))
repl/foo
repl> (s/instrument 'foo)
#'repl/foo
repl> (foo #(when (even? %) 42))
ExceptionInfo Call to #'repl/foo did not conform to spec:
In: [0] val: nil fails at: [:args :f :ret] predicate: integer?
:clojure.spec/args (#object[repl$eval66335$fn__66336 0x55f2e735 "
#2016-05-2819:37arohner@bbrinck: ^^#2016-05-2819:38arohnerthat ran generative testing on a normal call#2016-05-2819:38arohnerwhich is something Iād like control over#2016-05-2819:38bbrinckI may be misunderstanding, but I don't think it did. I think it just checked that the spec matched the invocation: (foo #(when (even? %) 42))
#2016-05-2819:39arohner42 should pass there. The generative example passed zero#2016-05-2819:40arohnerhrm nvm#2016-05-2819:40bbrinckI think this is just a confusing error message. The value passed is not 0#2016-05-2819:40bbrinckit's #(when (even? %) 42)
, which returns nil, which violates the spec#2016-05-2819:41arohner(foo #(do (println %) (when (even? %) 42)))
-1
0
-1
0
-1
0
-1
#2016-05-2819:42arohner(s/unstrument 'foo)
#'repl/foo
repl> (foo #(do (println %) (when (even? %) 42)))
42
42
#2016-05-2819:45bbrinckHm, yeah, you're right, I was reading that wrong. I still don't think (from playing with other examples) that the generative testing occurs unless you do one of the testing calls. I think this example might just have a bug. I could be misunderstanding though.#2016-05-2819:46bbrinck(I'm trying to try it myself, but on my version of clojure, the fdef
is not working for some reason)#2016-05-2819:47arohnerwhich testing call did I run there?#2016-05-2819:53bbrinckHm, you're absolutely right. I did not expect that.#2016-05-2819:55bbrinck@arohner: Yep, you're correct - there was no testing call, but it's doing some generation for the args for the inner function.#2016-05-2819:59bbrinck@arohner: interestingly, that does not seem to occur in all cases:
(s/fdef double
:args (s/cat :x integer?)
:ret integer?)
(defn double [x]
(if (zero? x)
"zero"
(* 2 x)))
(s/instrument 'double)
(double 1)
#2016-05-2819:59arohneryeah, I think fspec is the culprit#2016-05-2820:01bbrinckah, yes, the docs do mention that now that I read them a little more carefully š . Well, TIL#2016-05-2820:05bbrinckAlso, FWIW, that example won't quite work on 1.9.0-alpha3. fspec
seems to require a ret
now#2016-05-2820:08bbrincknvm, i realized it was a copy/paste fail on my part (didn't grab the full fspec above)#2016-05-2820:18sveriHi, did anyone try to generate a boolean spec? Like this: (s/def ::required #(instance? Boolean %))
. Calling (gen/sample (s/gen ::required))
fails for me with: Exception Unable to construct gen...
.
This becomes problematic if ::required is part of a nested spec, how would I provide a generator for that?#2016-05-2820:28seancorfieldThat specific case is covered in the (updated) documentation @sveri #2016-05-2820:30sveri@seancorfield: Ah, thats nice, I just solved it by defining boolean as true / false: (s/def ::boolean #{true false})
for which the generator works too#2016-05-2820:31seancorfield"There are some common cases that currently donāt have standard predicates, but good generators exist. Itās likely there will be changes in this area in the future but for now you might find these useful:
(defn boolean? [x] (instance? Boolean x))
(s/def ::boolean (s/with-gen boolean? #(gen/boolean)))
(defn uuid? [x] (instance? java.util.UUID x))
(s/def ::uuid (s/with-gen uuid? #(gen/uuid)))"#2016-05-2820:31seancorfield(That didn't paste well -- sorry, on my phone)#2016-05-2820:35sveriYea, no problem, I get it, thank you š#2016-05-2821:00borkdudeJust finished reading the guide. Good stuff.#2016-05-2821:12borkdudeWhy is (doc subs)
printing Spec
on the last line. Bug?
user=> (doc subs)
-------------------------
clojure.core/subs
([s start] [s start end])
Returns the substring of s beginning at start inclusive, and ending
at end (defaults to length of string), exclusive.
Spec
nil
#2016-05-2821:33borkdudeWhat about multi-arity functions like subs
, how to write spec for both cases?#2016-05-2821:37borkdudeLike this? (s/fdef clojure.core/subs :args (s/or :two-args (s/cat :s string? :start integer?) :three-args (s/cat :s string? :start integer? :end integer?))
#2016-05-2821:41borkdudeI'm not sure if I should wrap with s/spec
, both seem to work:
(s/fdef clojure.core/subs :args (s/or :two-args (s/spec (s/cat :s string? :start integer?)) :three-args (s/spec (s/cat :s string? :start integer? :end integer?))))
#2016-05-2901:11Alex Miller (Clojure team)@borkdude: there is new code in doc
to find and print specs, but it shouldn't print that if there are none so that's a bug#2016-05-2901:11Alex Miller (Clojure team)@borkdude: for multi-arity, you can cover all options in a single spec via regex#2016-05-2901:14Alex Miller (Clojure team)you can just do (s/fdef clojure.core/subs :args (s/cat :s string? :start integer? :end (s/? integer?)) :ret string?)
#2016-05-2901:18Alex Miller (Clojure team)you can even add a :fn #(clojure.string/includes? (-> % :args :s) (:ret %))
to verify it returns a string included in the original#2016-05-2901:19Alex Miller (Clojure team)I didn't test any of that but it should be pretty close#2016-05-2901:31sekaocould we get a way to exclude vars from run-all-tests
, like ^:no-check
in core.typed? i jerry-rigged a solution earlier today but it required modifying the function directly. i have lots of side-effecting functions that shouldnt be running generative in tests.#2016-05-2901:35Alex Miller (Clojure team)it's best to file a jira for anything so we don't lose it, esp over the long weekend#2016-05-2901:36sekaook will do#2016-05-2901:36Alex Miller (Clojure team)thx!#2016-05-2901:36Alex Miller (Clojure team)that way we can track that stuff and you can see what's happening with it too#2016-05-2901:46sekaoah i shouldāve looked there first, iāll just comment on CLJ-1936 as it sounds similar#2016-05-2907:33borkdude@alexmiller: thanks š#2016-05-2907:44borkdude@alexmiller: promoted this example to a gist: https://gist.github.com/borkdude/0665078edb40fb0e1551c1d29655c2d6#2016-05-2921:00sveriHey, I have again a problem I dont understand:#2016-05-2921:02sveriWhen I copy the part from args: ({:name "QpJ50qrS1m24V", :type :varchar, :unique true, :required false, :autoinc true, :null true, :pk true, :max-length 14, :fk-name "Tp9tG8hwUXK0"})
and run the spec validation on it it succeeds.
What bothers me, is that the last line in the message wraps the columns in an extra list: :clojure.spec/args (({:name "QpJ50qrS1m24V" ...
I am not sure what that means.#2016-05-3008:29cflemingWhen s/spec
is used to create a new sequential context, I canāt find a way to capture the sequential thing itself.#2016-05-3008:29cflemingFor example:
(s/def ::ingredient (s/cat :quantity number? :unit keyword?))
(s/def ::recipe (s/cat :amount (s/spec ::ingredient) :description string?))
(s/conform ::recipe [[2.0 :teaspoon] "Cinnamon"])
=> {:amount {:quantity 2.0, :unit :teaspoon}, :description "Cinnamon"}
#2016-05-3008:30cflemingHere, there seems to be no way to capture the [2.0 :teaspoon]
vector object if I also want to match its contents.#2016-05-3011:17Alex Miller (Clojure team)@cfleming: you are matching its contents via the cat. If you want to receive the vector as the conformed value you could use coll-of to do a different kind of match or use a conformer to transform the matched result into any arbitrary structure#2016-05-3011:20Alex Miller (Clojure team)Probably the latter is what you want#2016-05-3017:33arohneris there a way to update a map spec? Iād like to say āthis fn takes a foo map, and returns a foo map with an extra key added onā. Itād be nice to say (s/def bar (conj foo ::extra-key)), without specifying the keys of the second map explicitly#2016-05-3018:01arohnerarg. repl development overwrites instrumentation#2016-05-3020:00moxajQuestion: why is the second argument (`retag`) necessary in a multi-spec
expression? In the guide, it is the same as the dispatch function, and I don't see any use case where providing a different function would be beneficial. And if it's the same, naming it again is redundant, as it can be retrieved via the public field dispatchFn
.#2016-05-3020:46seancorfieldAs an experiment, Iāve added an optional spec namespace to clojure.java.jdbc: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj ā feedback welcome (the tests attempt to require that ns and instrument clojure.java.jdbc when running tests, so under 1.9.0 the specs are actually checked for all calls in the tests)#2016-05-3021:12potetmIs there any mechanism that will allow you to include possible exceptions as part of a spec?#2016-05-3022:26kovasbHow can I spec a function that takes an atom(x), where x conforms to a spec?#2016-05-3022:28kovasbthinking something like atom-of instead of coll-of#2016-05-3100:00kgzmHow would one use spec with records, when records use unqualified keywords and spec mandates the use of namespaced keywords?#2016-05-3100:36arohner@kovasb: AFAIK, that doesnāt exist currently, but it should be possible to write from cribbing coll-of#2016-05-3100:46arohner@potetm: ATM, no#2016-05-3101:39cfleming@alexmiller: Ok thanks, I donāt fully understand the second option but Iāll investigate it.#2016-05-3101:44potetm@cfleming: https://clojure.org/guides/spec#_conformers#2016-05-3101:44potetmI guess the obvious followup (maybe for #C06E3HYPR?) is: Are there plans to support exceptions in specs?#2016-05-3101:56seancorfieldGiven that the spec describe data structures, and function inputs/output ā what would it mean to "support exceptions" in specs?#2016-05-3101:57potetmYeah that's fair.#2016-05-3101:58potetmMy first pass idea was just to "list the possible exceptions". But I'm totally open to the idea that that's just a bad idea.#2016-05-3101:59potetmI haven't thought it through or anything. Just noticed that I was hitting problems during generative testing because I couldn't declare that, say, arithmetic overflow, was an okay exception.#2016-05-3102:00cflemingIt seems like something you might want to specify.#2016-05-3102:00cflemingItās a potential output from a function, more or less.#2016-05-3102:00seancorfieldI can see pros and cons. Certainly in (unit) tests it can be valuable to say "given these inputs, I expect the following exception"...#2016-05-3102:01potetmcfleming: Yeah that was my thought as well.#2016-05-3102:01seancorfieldBut is that a failure to conform to its spec? It depends on whether the spec is considered to be "valid for all inputs that are valid" (and therefore "undefined" for invalid inputs)...#2016-05-3102:02seancorfieldI just went through this exercise with java.jdbc
because there are lots of ways to generate exceptions from those functionsā¦ but Iām not sure how to specify that...#2016-05-3102:03seancorfieldIām not even sure if it makes sense to try to specify that? What if the DB is down, or the table you ask for doesnāt exist or thereās a syntax error in your SQLā¦? How would you codify that in clojure.spec
?#2016-05-3102:03seancorfieldHow could you even list all possible exceptions?#2016-05-3102:04potetmYeah it's possible that that would be overly restrictive.#2016-05-3102:04seancorfield(each JDBC driver throws its own types ā do you just say "could throw Exception"ā¦?)#2016-05-3102:05seancorfieldItās a hard problem, either way. clojure.spec
definitely has gaps when you have stateful functions (generative testing on java.jdbc
functions isnāt possible in general ā most garbage input will produce an exception, even if it is in the right "form").#2016-05-3102:05potetmI dunno, perhaps not since it's not exactly stopping you from doing anything at the end of the day.#2016-05-3102:11cfleming@seancorfield: Yeah, spec is mostly not that useful for me either, for similar reasons.#2016-05-3102:11mfikesSo Sean and Nolen both seem to have created a my.lib.main-ns.spec
namespace to hold specs. Wonder if that will become a de facto place to put them for libraries.#2016-05-3102:12seancorfieldI took that approach so that you didnāt have to load the spec ns ā so you could use the code with Clojure < 1.9.0#2016-05-3102:13seancorfieldAlthough the build system shot me in the foot since it compiles all namespaces(!).#2016-05-3102:13seancorfieldConsequently, the build plain olā fails on Clojure < 1.9.0. Not sure how to approach that (make the entire namespace conditional? Ugh!).#2016-05-3102:15mfikesHmmā¦ need a good solution to that problem. Every lib will face it.#2016-05-3102:15potetmSpec actually does this for test.check
.#2016-05-3102:15potetmIt's non-trivial.#2016-05-3102:15cflemingIām just cutting and pasting clojure.spec.* and using Clojure 1.7.#2016-05-3102:16mfikesYeah, the dynaload. Hmm#2016-05-3102:16seancorfield@potetm: But you canāt have an optional namespace that uses spec and get it through the build system.#2016-05-3102:16seancorfieldI already deal with conditional loading in the java.jdbc tests.#2016-05-3102:17seancorfieldLocally, I can run lein test-all
and pass tests on every Clojure version from 1.4 to 1.9 ā and only on 1.9 does it use the spec.#2016-05-3102:18seancorfieldBut the contrib build system tries to compile the namespaces in the projectā¦ So maybe thereās some Maven incantation I can use to exclude the namespace from compilation (or whatever Maven is trying to do with it).#2016-05-3102:19potetmRight so this is all in the build tool. Yeah I got nuthin....#2016-05-3102:19seancorfieldhttp://build.clojure.org/job/java.jdbc-test-matrix/410/ and http://build.clojure.org/job/java.jdbc-test-matrix/410/CLOJURE_VERSION=1.4.0,jdk=OpenJDK%201.6/console#2016-05-3102:21potetmWell except for this problem, that tool seems pretty slick š I've never seen anything like it before.#2016-05-3102:21mfikesMaybe someone can make a hacked-up stub do-nothing compatibility lib in the clojure.spec
and cljs.spec
namespaces that consumers on 1.8 can use, just so things can be loaded. Ugh.#2016-05-3102:21seancorfield@potetm: Have you looked at core.typed or Prismatic Schema at all?#2016-05-3102:22potetmNo I mean the version matrix. All the JDKs all the clojure versions. A really nice idea for open tooling. (I've always had control over JDK and clj versions.)#2016-05-3102:22seancorfieldTomorrow Iāll talk to @alexmiller about the build system and see if we can figure out something ...#2016-05-3102:51seancorfieldSean: 1, Maven: 0 š So it turns out a contrib project can override the parent pom.xml
and suppress the Maven-initiated compile phase: https://github.com/clojure/java.jdbc/commit/b224a3e86df8c00a33a16122e6fcc531c5f71e2e#2016-05-3102:51seancorfieldNow I feel dirty for having to learn that much about Maven š#2016-05-3102:59seancorfieldApparently if I want to get really "clever" I could probably create a conditional profile based on the Clojure version and have it still run compile if weāre using 1.9.0ā¦ Ugh! <profiles>
...#2016-05-3112:22andrewhr@seancorfield: lein's {:aot :all}
doesn't kick in for included dependencies, did it? Maybe that's the reason for maven
compile's strategy#2016-05-3114:40akiel@anmonteiro You had an example of clojure.test
integration were you used spec-is
. I canāt find spec-is
in the sources. Iām also interested in a proper clojure.test
integration of clojure.spec.test/check-var
. clojure.spec.test/run-tests
only works in the REPL for me. I need it in leiningen.#2016-05-3114:41anmonteiro@akiel: spec-is
was something I wrote:
(defmacro spec-is [res]
`(clojure.test/is (true? (:result ~res))))
#2016-05-3114:44akiel@anmonteiro: ok thanks this works.#2016-05-3115:58akiel@anmonteiro: Iāve extended the is
macro in the following gist. This looks even better to my eyes. https://gist.github.com/alexanderkiel/931387c7a86c1879b2267ad064067af7#2016-05-3117:33seancorfield@andrewhr: The pom.xml
file indicates the compile is just a "sanity check" so Iām taking that as "optional"...#2016-05-3119:05ghadiIs there anything in between s/keys and s/map-of for heterogenous maps --- maps with keywords and structured values as keys?#2016-05-3120:34seancorfieldAs I specād java.jdbc
I found myself wanting a way to constrain the values of the optional keys in the optional opts
argument. I suspect the only way is (s/and (s/keys ā¦) #(some custom predicate))
?#2016-05-3120:35seancorfield(Iād have to take a look at what s/keys
conforms values to)#2016-05-3120:40kennyAre you guys tending to write your fdef
s in a separate ns or in the same ns above/below where the function is written?#2016-05-3120:41sekaoiāve been putting them above my functions so far#2016-05-3120:45kennyYeah that is great for readability but it seems like it kinda pollutes your ns.#2016-05-3121:53zaneHow would one spec a map with non-keyword keys and heterogenous values (the specs of which are dependent on the keys)?#2016-05-3121:55zaneThat is, how would you spec ::person
from the guide if ::first-name
::last-name
etc were strings?#2016-05-3122:13zaneOr, to put it another way, is there a version of clojure.spec/keys
that works with non-keyword keys?#2016-05-3122:18seancorfield@zane: I suspect youād need to say (s/map-of string? ::s/any)
and then have some custom predicate on the conformed value.#2016-05-3122:19zaneRight, okay.#2016-05-3122:19zaneAs I'd feared.#2016-05-3122:20seancorfield@kenny: I think overall Iād prefer data structure specs in a separate ns and fdef
alongside the functions themselves but I wonāt have a solid feel for that until weāve used it a bit more heavily.#2016-05-3122:20seancorfieldIn clojure.java.jdbc
, I put all the specs in a separate ns so users on Clojure < 1.9.0 could still use the library.#2016-05-3122:33kennyMakes sense.#2016-05-3122:58zane@seancorfield: If I wanted to try to recover the the features of clojure.spec/keys
but for maps with non-keyword keys would you recommend I implement clojure.spec/Spec
myself?#2016-05-3123:05seancorfieldI suspect that will be a lot of work (but I havenāt looked at it). Since the push is for namespaced keywords ā but unnamespaced keywords are also supported ā I guess I would have to question your desire to define an API based on maps with non-keyword keys?#2016-05-3123:05seancorfieldBy which I mean, what specifically is it that youāre trying to spec out here that isnāt a "regular" Clojure map?#2016-05-3123:06seancorfield(and, perhaps therefore, spec "at large" is not designed for your use case?)#2016-05-3123:10seancorfieldMy sense with clojure.spec
is that itās opinionated deliberately to encourage a particular style of API specificationā¦ "idiomatically Clojurey"ā¦ The comments in particular about namespaced keywords being "tragically underutilized" and that namespace-qualified keywords is "a practice weād like to see grow"...#2016-06-0100:03kennyHow are you guys spec'ing recursive functions where intermediate return values have a different spec than the final return value? Are you just adding an or
to the :ret
or are you writing your function in such a way that that isn't possible (e.g. bundling the recursion in the function by creating an anonymous fn inside your function)?#2016-06-0100:19seancorfield@kenny: Can you give an example of such a scenario?#2016-06-0100:20seancorfield(I generally try to avoid functions that can return multiple types although Iāll often have functions that return something-or-nil)#2016-06-0100:20kenny(defn new-matrix-nd
"Returns a new general matrix of the given shape. Shape can be any sequence of
integer dimension sizes (including 0 dimensions)."
[dims]
(if-let [dims (seq dims)]
(vec (repeat (first dims) (new-matrix-nd (next dims))))
0.0))
#2016-06-0100:20kenny(s/fdef new-matrix-nd
:args (s/cat :dims (s/nilable coll?))
;; ret is not a matrix because new-matrix-nd is a recursive fn
:ret (s/or :m vector? :n number?))
#2016-06-0100:20kennyIdeally the spec should be
(s/fdef new-matrix-nd
:args (s/cat :dims (s/nilable coll?))
:ret matrix?)
#2016-06-0100:20seancorfieldso (new-matrix-nd [])
produces 0.0
?#2016-06-0100:21kennyYes#2016-06-0100:21kennyBut the final return value is always a matrix#2016-06-0100:21seancorfieldUnless you accidentally call it with nil or an empty sequence of dimensions...#2016-06-0100:22seancorfieldSo a zero-dimensional "matrix" is a scalarā¦ hmmā¦ thatās an interesting one...#2016-06-0100:23kennyRight#2016-06-0100:24kennyI can wrap up the recursion inside the function#2016-06-0100:24kennyThat would probably be cleaner to the outside world#2016-06-0100:27seancorfieldIām trying to think how specs are going to be much use to you here thoāā¦ since any call site is going to get "vector or scalar" as a result...#2016-06-0100:28kennyHmm.. Yeah it might not make sense.#2016-06-0100:29seancorfieldeven your :args
spec doesnāt buy you much here: nil
or an arbitrary collection ā you probably want (s/coll-of integer? [])
to give you more checking ā and Iād be tempted to disallow nil
and require an empty sequence of dimensionsā¦ unless you have a really good reason for allowing nil
there?#2016-06-0100:31seancorfieldMaybe the Clojure/core folks need to give a bit more guidance in the Rationale as to how they expect spec to be used in the real world? /cc @alexmiller#2016-06-0100:32seancorfieldIād imagined it as a set of specifications for some pretty high-level parts of your code, around your domain objects, or possibly as the definition of a library APIā¦ but Iām not sure about the latter yet...#2016-06-0100:32kennynil is just the terminating value. It seems wrapping the recursion inside the fn may solve this.#2016-06-0100:33seancorfieldAt work weāre looking at spec to write a specification of our domain model and some of the high-level business logic that operates on that. I donāt know how far "down" the call tree weāll go...#2016-06-0100:36seancorfieldThe Rationale says "the supply chain isnāt burdened with correctness proof. Instead we check at the edges and run tests" ā¦ "Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment."#2016-06-0100:38kennyThis cleans up the spec... (defn new-matrix-nd
"Returns a new general matrix of the given shape. Shape can be any sequence of
integer dimension sizes (including 0 dimensions)."
[dims]
(letfn [(new-matrix-nd' [dims]
(if-let [dims (seq dims)]
(vec (repeat (first dims) (new-matrix-nd' (next dims))))
0.0))]
(new-matrix-nd' dims)))
#2016-06-0100:39kenny(s/fdef new-matrix-nd
:args (s/cat :dims (s/coll-of integer? []))
:ret (s/or :m matrix? :s scalar?))
#2016-06-0100:40seancorfieldBut matrix?
still allows a scalar, right?#2016-06-0100:40seancorfieldOtherwise (new-matrix-nd [])
will pass the :args
check but fail the :ret
spec.#2016-06-0100:41kennyEdited to fix...#2016-06-0100:42seancorfieldYour refactoring hasnāt changed the spec thoā...#2016-06-0100:43kennyNo nil allowed in the fn#2016-06-0100:43seancorfieldWell, the spec disallows it, but that was true of the earlier function as well.#2016-06-0100:44kennyYou could pass nil to the previous fn. That should not be allowed.#2016-06-0100:45kennyThough that spec actually isn't working. (s/valid? (s/coll-of integer? []) nil) => true
#2016-06-0100:45seancorfield(then I was wrong, s/coll-of
allows nil
?)#2016-06-0100:46kennyI guess. That doesn't seem right though#2016-06-0100:47kenny(s/valid? (s/* integer?) nil) => true
#2016-06-0100:48seancorfieldboot.user=> (s/conform (s/* integer?) nil)
[]
boot.user=> (s/conform (s/coll-of integer? []) nil)
nil
#2016-06-0100:49seancorfieldboot.user=> (s/conform (s/* integer?) (list 1 2 3))
[1 2 3]
boot.user=> (s/conform (s/coll-of integer? []) (list 1 2 3))
(1 2 3)
#2016-06-0100:50seancorfieldSo coll-of
is the spec for a (possibly nil) collection ā sequence of values ā and *
is a regex to match zero or more items and returns a vector.#2016-06-0100:51kennyWas coll-of
designed such that a possibly nil collection of values is valid?#2016-06-0100:52kennyNot sure how that can be an artifact as nil is inherently false so it would seem to be by design?#2016-06-0100:52seancorfieldboot.user=> (source s/coll-checker)
(defn coll-checker
"returns a predicate function that checks *coll-check-limit* items in a collection with pred"
[pred]
(let [check? #(valid? pred %)]
(fn [coll]
(c/or (nil? coll)
(c/and
(coll? coll)
(every? check? (take *coll-check-limit* coll)))))))
nil
^ this is what coll-of
calls and it has a very specific check for nil
#2016-06-0100:53kennyHmm. Wonder why#2016-06-0100:53seancorfieldBecause nil-punning is idiomatic and encouraged?#2016-06-0100:54kennyI thought the idea was to be explicit about nil with nilable
#2016-06-0100:54seancorfield(thatās a question for Clojure/core really butā¦)#2016-06-0100:55kennyMaybe we can get some input tomorrow on this.#2016-06-0100:57kennyI suppose it does make sense coming from the Java static typing world because you could pass null
in place of any argument. We don't need to adhere to that though.#2016-06-0101:05seancorfieldBut any function that accepts a (general) collection is almost certainly going to accept nil
Iād sayā¦?#2016-06-0101:06kennyWhy make any assumptions?#2016-06-0101:06seancorfieldAs noted above/elsewhere, clojure.spec
is deliberately opinionated š#2016-06-0101:07seancorfield(not saying I agree, just observing)#2016-06-0101:11kennySeems strange, considering: https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare#2016-06-0101:11seancorfieldVery different idioms. If null isnāt treated as falsey, then you have a very different style of programming.#2016-06-0101:13seancorfieldFor example, in some languages youād have a Maybe monad and you would chain operations using monadic functions. Scala has Option[T] and flatMap, as I recall. Haskell has Maybe t and fmap (yes?). But in Clojure youād chain such operations using some->
or some->>
and theyād just be regular functions.#2016-06-0101:14seancorfieldOr you use if-let
/ when-let
#2016-06-0101:15seancorfieldI was chatting with someone the other day and saying that I almost never hit a NPE in Clojure these days. About the only time I still do is in Java interop code (usually calling into the String class for something!).#2016-06-0101:15kennyTrue (though we can follow that style of programming is Clojure e.g. cats). Anyways, I suppose that is a tangent and doesn't really pertain to this situation.#2016-06-0101:16kennyYeah you're totally right. I can't recall the last NPE I got.#2016-06-0101:17seancorfield(the irony of this conversation today is that someone reported an error against java.jdbc
that a particular erroneous call to one of the functions produces a NPE ā for invalid arguments)#2016-06-0101:18seancorfieldThe latest version of java.jdbc
accepts those calls without error thoā...#2016-06-0101:20kennyAnyways, the primary reason I don't want nil to exist in this particular library is there is no such idea in the math world.#2016-06-0102:52andrewhrMy feelings about clojure.spec
are pretty similar to yours @seancorfield... most for library public APIs (think pedestal, with namespaces keywords and data-based DSLs) or for general domain definition. Following the guidelines presented on Clojure Applied, I imagine using specs at domain edges and possibly Record
s only as a implementation detail of that domain (as suggested in the book).#2016-06-0102:55andrewhrI still need more though (or some good rule of thumbs) on the usage of namespaced keywords per se as a design tool. When to use a fully qualified - and by definition more coupled to implementation - vs just simple namespaces aka :foo/bar
. And when it will be fine to use unqualified ones as envisioned by the Clojure/core
style#2016-06-0102:55seancorfieldRecords require non-namespaced keys thoā so I wonder how that sits with specs? (and I donāt think clojure.spec
actually supports records?)#2016-06-0102:57andrewhrYeah, as far as I got it does not. Considering this style of "Records as an implementation detail", that makes sense no? I just following through the documentation snippet you shared#2016-06-0103:00cflemingIām interested to know how spec handles records, too.#2016-06-0103:01cflemingI guess you treat the fields as non-namespaced keys?#2016-06-0103:12mikethompsonI'm considering spec
from an Information Model point of view.
Let's say I wanted to create a spec for car
. It might have keys like colour
and year
. That part is easy. But what about manufacturer
where I don't want containment, and instead I want a reference - an id
? I could say the spec for manufacturer
was integer?
, but how should I be more specific and say it is a reference to another spec.
My mind is wandering forward and imagining a tool which can read the central registry and produce a nice SVG Information Model. But it can't do that unless it knows that the manufacturer
in car
is more than an integer?
... it is actually a reference to another entity.#2016-06-0103:26seancorfield@mikethompson: Sure you can have ::manufacturer-id
that specifies actual valid manufacturer IDs (or whatever you needed) and for testing you could specify a generator that produced (a small subset of) manufacturer IDs.#2016-06-0103:57zane> I guess I would have to question your desire to define an API based on maps with non-keyword keys?
@seancorfield: Oh, believe me: If we were in control of this data we would use keyword keys. We're trying to enforce / check our assumptions about data we don't control.#2016-06-0104:08mikethompson@seancorfield: Yep, understand that. I'm just wondering about how that SVG Information Model Diagram tool could work. By looking in the central registry, how does it know that ::manufacturer-id
is a reference to ::manufacturer
. I can't see how that can be captured.#2016-06-0104:09mikethompsonAlmost seems like metadata on ::manufacturer-id
#2016-06-0104:09seancorfield@mikethompson: maybe :manufacturer/id
and :manufacturer/spec
?#2016-06-0104:10seancorfieldConventions are needed somewhere, somehow.#2016-06-0104:10mikethompsonAhh. Metadata via naming convention š#2016-06-0104:11seancorfield@zane: If the keys are strings when you get them from "the outside" then maybe keywordize them for use "inside"?#2016-06-0104:11mikethompsonAnyway, all good. I was just wondering if I was missing something. Thanks#2016-06-0104:13seancorfield@mikethompson: No idea what Clojure/core might recommend here. I was trying to follow the Datomic lead there but I haven't looked at how referential specifications might work yet... Definitely something we'll run into at work as we move forward with this...#2016-06-0104:16seancorfield@zane: If you keywordize them "as-is" (maybe putting a standard namespace qualification on them too?), then you can validate with the full force of spec? At work, we're considering using a naming strategy with java.jdbc to create namespaced keys from the DB rather than just column names, so we can move straight to spec. Not sure how that will work out yet but we're planning a spike along those lines next week.#2016-06-0105:16josh.freckletonI'm still a bit new and wanting to bring in some typing. I'm not familiar with the respective libs, is spec
effectively replacing schema
and typed.clojure
?#2016-06-0105:21seancorfieldHi @josh.freckleton -- have you read the Rationale for spec on the http://clojure.org site?#2016-06-0105:21seancorfield(it tries to clarify the spec vs type system position)#2016-06-0105:24josh.freckleton@seancorfield: I have, I'm just trying to clear things up a bit, and I guess i'm wondering where I should invest my time š#2016-06-0105:26seancorfieldspec
is different to both core.typed
and Schema
...#2016-06-0105:29josh.freckleton@seancorfield: oh, can I ask another question while we're talking about this? in schema
or spec
, could I have a case
(switch) according to types
, or defmethods
on different types
? Would there be a prefered method?#2016-06-0105:29seancorfieldcore.typed
is intended to provide a type checking system based on static analysis of the code with annotations. Schema
is a runtime validation system.#2016-06-0105:30seancorfieldNot sure what you're asking... I think I'd have to see an example of what you're trying to do...?#2016-06-0105:32josh.freckleton(ahh, your typed
vs. schema
example makes sense. i still need to digest this)
So say I have a few different "types", and I want to map a fn over them which is specific to their type, I've just started to learn about defmethods
, and of course there are switch statements (`case`) switching on the type
of the object under consideration. If I, say, had a list of things that could be all different types, what would be the best way in to switch on their types?#2016-06-0105:33seancorfieldSorry, I still don't understand...#2016-06-0105:33josh.freckletonmy fault, I'll be more specific...#2016-06-0105:39josh.freckletonI'm working with the free monad + interpreter pattern trying to build some code that takes a data model, and can interpret it in various ways. The important part is that the data model, essentially the Free Monad instance, is a recursive "type" where each "node" of this data model can be different types. Currently, I signify their "type" as a key :type
in a mapping, and I (case (:type obj...
to decide what to do with each "node". For example, one node can be a "User", and another can be "Owned Books", and another can be "Email", "Password", etc. As I interpret this data model, I will want to treat Users differently from Emails, and Passwords, etc.
The 3 options I know of are to 1. defmethod
according to different, real types, 2. to case
over their type if spec
or schema
allows me to switch on types, or 3. what I have now with a map, and a "`:type`" key. Maybe I'm completely misguided in how I'm trying to solve this though, and I'm completely open to you annihilating my idea if it's far out-field!
Does that make more sense?#2016-06-0105:40seancorfieldSo you want polymorphic dispatch on the :type
key in the hash map?#2016-06-0105:41seancorfielddefmulti
/ defmethod
would work for that. Or use a protocol with multiple defrecord
implementations.#2016-06-0105:42seancorfield(the latter would be actual types and would no longer need the :type
key)#2016-06-0105:43seancorfielddefmulti
would use a dynamic dispatch based on aspects of the data structure, in your case the :type
key.#2016-06-0105:44seancorfieldSo, basically, defmulti
is a sort of case statement dispatch š#2016-06-0105:47josh.freckletonOk, I'm glad that way would work, thx. And would their be an idiomatic way to do it without declaring polymorphic methods, like:
(case (spec/type obj) :a "a" :b "b")
, or
case (schema/type obj) :a "a" :b "b")
I haven't seen anything in the docs that fits that ability to check the type of an obj, but I maybe I've missed it?#2016-06-0105:47seancorfieldYeah, I think you're misunderstanding what spec
and Schema
are about.#2016-06-0105:48seancorfieldWhy wouldn't you just do (case (:type obj) :a "a" :b "b")
at that point? The :type
key gives you what you need.#2016-06-0105:50seancorfieldOr (defmulti foo :type) (defmethod foo :a [obj] "a") (defmethod foo :b [obj] "b")
#2016-06-0105:50josh.freckletonThat's probably why I couldn't ask it clearly š
I was just curious if that was an option, it seems similarish to me of Haskell's matching on different types. I think that's the "feel" I was going for, but again I could be wrong since I'm new the functional world.#2016-06-0105:52josh.freckletonfor example:
f Nothing = foo
f (Just x) = bar
You've been a big help sean, helping me prune my search tree of what I need to study, haha. Thank you so much, and thanks for staying up late to help us noobs out!#2016-06-0105:55seancorfieldIn Clojure, Maybe
would more likely just be nil
or not-`nil` and you wouldn't pattern match, just (defn f [x] (if x 'bar 'foo))
#2016-06-0105:56josh.freckletonoh sure, and there's also a maybe
monad, I'm thinking for matching on many (10-100) custom types.#2016-06-0105:57seancorfieldRight, so if you want polymorphic dispatch on a single argument (type), you probably want a protocol and defrecord
.#2016-06-0105:58seancorfieldIf you want more ad hoc polymorphism, defmulti
is probably your tool#2016-06-0105:58seancorfieldClojure is very different from things like Haskell, since there's no extant type system, even tho' Clojure is strongly typed (at runtime).#2016-06-0105:58seancorfieldSo the idioms are very different.#2016-06-0105:59seancorfieldClojure's polymorphism is very different from OOP languages as well.#2016-06-0106:01josh.freckletonK, I had been considering these different options, and with your suggestion I'll zoom in on defrecord
/`defmulti`, that helps me cut out tomorrows work š#2016-06-0106:01josh.freckletonThank you!!!#2016-06-0106:05seancorfieldHave fun!#2016-06-0106:24ikitommi@mikethompson: @seancorfield was thinking also for doing registry visualization to learn things. Did a schema->graphviz tools some time ago, which helps a lot. Are you already working on this?#2016-06-0106:25mikethompson@ikitommi: thought experiment for me. So, no, not working on it.#2016-06-0106:55seancorfieldI'm not a very visual person so it's a "no" on my end too. #2016-06-0114:23nwjsmithI'm having difficulty spec
-ing a map. Would like to re-use spec for the keys and values. In particular, I'm trying to spec this bit of Datomic Pull: map-spec = { ((attr-name | limit-expr) (pattern | recursion-limit))+ }
. I've already got specs for ::attr-name
, ::limit-expr
, ::pattern
and ::recursion-limit
, but I can't figure out how to re-use them in the spec for ::map-spec
#2016-06-0114:23nwjsmithI thought s/map-of
would be the ticket, but it works with predicates not specs.#2016-06-0114:40manutter51@nwjsmith: See the Entity Maps section of the guide http://clojure.org/guides/spec#2016-06-0114:41manutter51you want s/keys
basically#2016-06-0114:44nwjsmithAFAICT s/keys
is too strict, e.g I can't see how to specify that all keys conform to (or :attr-name ::attr-name :limit-expr ::limit-expr)
. It will only let me specify particular keys.#2016-06-0114:45nwjsmithI think what I'm looking for is something half-way between s/keys
and s/map-of
.#2016-06-0115:14zane@seancorfield: Those suggestions are great. Thanks! Full support for maps with keys that aren't keywords would still make sense to me, though. There are plenty of domains where what you're modeling with data really does require keys of other types.#2016-06-0115:51seancorfield@zane: I suspect maps with "other types" of keys are more likely to be homogeneous since you wouldnāt be able to enumerate all the keys (i.e., to list which are required and which are optional).#2016-06-0115:51seancorfieldWe have some maps from strings to hash maps, and some from longs to strings etc. But those donāt have specific required / optional keys ā theyāre "lookup tables".#2016-06-0115:52seancorfield(itās an interesting area of discussion thoā)#2016-06-0115:52zaneThat makes sense, and matches my experience.#2016-06-0115:56nwjsmith@zane reading through your conversation w/ @seancorfield it looks like you and I have the same issue. Have you looked into implementing clojure.spec/Spec
?#2016-06-0115:57zane@nwjsmith: Only superficially.#2016-06-0116:00zane@seancorfield: I'm realizing that one problem with the keywordize-before-running-specs approach is that any resulting error messages will refer to the keyword names rather than the string ones. That limits their utility significantly (makes them less useful to the calling client, for example).#2016-06-0116:02zaneMaybe validating JSON data via spec just isn't an intended use case?#2016-06-0116:04seancorfieldHow are you getting the JSON data? When we get JSON from a web service etc, we have clj-http
keywordize the result, and we do the same with Cheshire
as well. Basically, if we ingest JSON in any manner, we always convert to keywords at the boundary and validate afterward.#2016-06-0116:05zane(Aside: I'm learning a lot from your responses! Thanks!)#2016-06-0116:06zane@seancorfield: Right, right. But our intent was to then return any error messages explain-data
produces from speccing the request to the calling client.#2016-06-0116:07zaneexplain-data
's output is going to refer to the keywordized request, which could be confusing to the client if they're, say, unfamiliar with Clojure.#2016-06-0116:07seancorfieldHmm, Iām not convinced explain-data
is a good format for clients since it refers to the internals of your specs ā¦ and what about localization of messages, and error codes for lookup in a reference manual for your API?#2016-06-0116:15zaneHmm. I suppose I didn't notice anything about specs that made me think they were fundamentally internally-facing. I'm curious what your thinking is there.#2016-06-0116:17zaneI had imagined that error codes could be generated via a transformation of the explain-data
output. (Localization isn't necessary in this particular case since this is API is only used internally.)#2016-06-0116:40seancorfieldIf youāre going to transform the explain-data
output, wouldnāt you then want to convert the keywords (including the spec names) to strings anyway?#2016-06-0116:41seancorfieldTaking this example from the Guide: (s/explain-data ::name-or-id :foo)
;;=> {:clojure.spec/problems
;; {[:name] {:pred string?, :val :foo, :via []},
;; [:id] {:pred integer?, :val :foo, :via []}}}
#2016-06-0116:46seancorfieldYouād want to convert :name
and :id
and whatever was in :via
since those are keywords labeling parts of specs and paths through things ā so why not convert the actual keys as well?#2016-06-0116:47seancorfield(having said that, we havenāt gotten as far as trying this sort of thing to generate responses to clients ā we currently have custom validationā¦ and that validation checks the types of values passed in as well as the structure so Iām not sure how much we could delegate to spec
ā¦ but itās an interesting option)#2016-06-0116:56zaneYes, you could totally convert the keywords back to strings, but you wouldn't have to if spec had better support for non-keyword keys in the first place. That's my whole point.#2016-06-0116:56zaneBut this is totally something that could be added via an extension library, so I guess I should roll up my sleeves. :relaxed:#2016-06-0117:01seancorfield(BTW, Alex Miller just replied on clojure-dev that heās on vacation this week which is why heās not participating here right now ā expect more feedback next week!)#2016-06-0117:39iwankaramazowIs it possible to dynamically generate specs from some sort of schema?
I have a rest api which exposes a schema like this /contacts/schema
(json)
{ "type": "Struct",
"keys": {
"first_name": {"type": "String"},
"last_name": {"type": "String"},
"is_organization": {"type": "Boolean"},
"updated_at": {"type": "DateTime", "options": {"format": "%FT%TZ"}},
"emails": {
"type": "List",
"schema": {
"type": "Struct",
"keys": {
"label": {"type": "String"},
"email": {"type": "String"}}}}}}
From reading the docs, it wasn't clear if I could generate dynamic specs from the above schema.#2016-06-0118:20seancorfield@iwankaramazow: Given that clojure.spec
s API is pretty much all macros, my initial reaction would be "No". I would expect you could generate a Clojure source file from your JSON and use that to define the specs, however.#2016-06-0118:21seancorfieldIām not sure how much of the underlying spec representation is publicly exposed to support dynamic generation of specs...#2016-06-0118:22iwankaramazow@seancorfield: Thanks for the response! Feared this, I'll experiment a bit#2016-06-0118:25arohnerIāll take the contrary stance#2016-06-0118:26arohnerit is possible to build a spec from that, but it requires building some new āprimitives'#2016-06-0118:27arohnerit is possible to build a fn that validates a map containing strings, and it is possible to build a test.check generator for for that schema#2016-06-0118:27arohneritāll take a bit of work, but itās doable#2016-06-0118:29arohner@seancorfield: specās two primitives are predicates and test.check generators. If you can build a predicate for it, you can use it in spec. If you can build a test.check generator, you can use it to generate#2016-06-0118:30arohners/keys is big because it generates a bunch of predicates, one for map? and then one for each key/value. thereās nothing special about it#2016-06-0118:32iwankaramazow@arohner: sounds logical, I'll brew me some algorithm. Thanks for the input#2016-06-0118:43seancorfield@arohner: What about dynamically (programmatically) registering those freshly built specs? Is enough of the machinery exposed for that?#2016-06-0118:44arohneryes#2016-06-0118:44seancorfield(and, yes, I know itās all feasible since even private functions are accessible and spec
is all built in Clojure ā but my "No" was meant to indicate whether it is a realistic goal to attempt and I still maintain the answer is negative there)#2016-06-0118:45arohnerSure, we have the tools to rewrite spec. I didnāt mean to go that far#2016-06-0118:46arohnerMy point is just that 1) any predicate is usable in spec 2) s/keys uses no special machinery 3) you can build a predicate to validate maps of strings 4) you can build a test.check generator to generate maps of strings#2016-06-0118:46arohners/keys is the way it is because Rich is being opinionated, not that other map predicates shouldnāt exist#2016-06-0118:57seancorfieldSome of the machinery youād need is private in clojure.spec
(e.g., res
) and some of the macros that youād need to replicate as functions are a lot of code. So, yeah, "itāll take a bit of work" is certainly true.#2016-06-0119:13arohnerok, youāre right that youād lose the per-field error messages#2016-06-0119:16arohnerbecause currently, a predicate would look like (defn struct? [m] (and (map? m) (contains? m ātypeā) ā¦)#2016-06-0119:18arohneractually#2016-06-0119:18arohnerone sec#2016-06-0119:20arohner(s/conform (s/and map? (fn [m] (contains? m "type"))) {"type" "structā})
#2016-06-0119:20arohners/keys just builds up a set of #(contains? % <key>) predicates. So you just (s/and map? <chain-of-key-checking-preds>)#2016-06-0119:21arohnerhrm, so that gets you key checking, but doesnāt conform values yet#2016-06-0119:24arohneryou could use more and
s to check values, but ideally we could reuse the pass-specs-to-check-values thing#2016-06-0119:31seancorfieldāļø:skin-tone-2: the sign of a Clojure developer with too much time on his handsā¦ š Itās easy to get drawn into the "Hmm, thatās an interesting problem!" rabbit-hole!#2016-06-0121:00kennyWhich is preferred?
(s/conform (s/cat :v (s/spec (s/* number?))) [[1 2 3]])
=> {:v [1 2 3]}
(s/conform (s/cat :v (s/coll-of number? [])) [[1 2 3]])
=> {:v [1 2 3]}
I assume the latter as it feels cleaner.#2016-06-0121:03mario@kenny: As far as I understand, the former will allow any seq
, while the latter will only allow vectors. So depends on what you want.#2016-06-0121:04kenny@mario: Nope:
(s/valid? (s/cat :v (s/coll-of number? [])) '((1 2 3))) => true
#2016-06-0121:05kennyThe vec at the end is just the value s/conform
conforms to.#2016-06-0121:06marioI see. Then I donāt know š#2016-06-0121:17jebberjeb@kenny: why are you using the s/cat
there? Do you really want to spec that it's a collection with only one other nested collection (of numbers)?#2016-06-0121:24kenny@jebberjeb: Yes.#2016-06-0121:33stathissiderisis there any way to say that I want 2 optional elements at the end of a cat
but they have to be both present or not at all?
(s/conform (s/cat :e1 string? :e2 (s/? keyword?) :e3 (s/? keyword?)) ["foo"])
#2016-06-0121:34stathissiderisa bit like that, but is e2 is there, e3 should be there as well#2016-06-0121:34jebberjeb@kenny @mario That second argument to s/coll-of
is only used when working with the generator for that spec, I believe. This succeeds but probably isn't a good idea: (s/conform (s/? (s/coll-of number? '())) [[1 2 3]])
#2016-06-0121:38jebberjeb@stathissideris: think you want (s/cat :s string? :opt (s/? (s/cat :k1 keyword? :k2 keyword?)))
#2016-06-0121:38stathissideris@jebberjeb: that looks like it could work, let me try!#2016-06-0121:39stathissiderisworks, many thanks š#2016-06-0121:41jebberjeb@stathissideris: yw#2016-06-0122:31arohnerstathissideris: use s/& around the cat to pass in a pred#2016-06-0122:32stathissideris@arohner: thanks! that's an option too, but I think I'm happy with @jebberjeb 's solution#2016-06-0122:33arohnerah yes, I missed the āat the endā part, so you can use an optional cat#2016-06-0122:33arohnerin general, & is used to add extra constraints to a seq#2016-06-0123:03kennyWhat is the best way to spec multiple arity functions where the args in one overload aren't necessarily related to the args in other overloads. Example:
(defn triangular-matrix
"Returns a triangular matrix created from `coll`. `coll` is a 1D vector where each element will be used to create the
triangular matrix. `upper?` is set to true to create an upper triangular matrix, false for a lower triangular matrix.
`diagonal` is a 1D vector of elements on the diagonal. `off-diagonal` is a 1D vector of upper or lower matrix elements."
([coll upper?]
;;impl
)
([diagonal off-diagonal upper?]
;;impl
))
#2016-06-0123:07arohnerI think thatās: (s/alt (s/cat :coll (s/coll-of ā¦) :upper? boolean?) (s/cat :diagonal (s/coll-of ā¦) :off-diagonal ā¦.))
#2016-06-0123:08arohner@kenny: ^^#2016-06-0123:09kennyYeah that makes sense to me#2016-06-0123:11kennyWhy is the common pattern to use cat
instead of tuple
for spec'ing args?#2016-06-0123:33stathissiderisspec looks amazing so far, the only thing missing is coercion#2016-06-0123:33stathissideristhis is the only point where I think schema is a bit more useful#2016-06-0123:33stathissiderisbut still, spec is much more expressive#2016-06-0123:50kenny@arohner: The only thing slightly annoying about that method is needed to specify keys for alt. The keys in this case are pretty arbitrary.#2016-06-0123:52stathissiderisI have a map that looks like this, but if the key is say :x
I have a special spec for that value only: (s/map-of (s/or :k keyword? :k string?) ::s/any)
#2016-06-0123:52stathissiderishow would I express that?#2016-06-0123:53stathissideriscustom predicate?#2016-06-0123:59stathissiderisvalidation works with a custom predicate:
(s/def ::simple-attr-map
(s/map-of (s/or :k keyword? :k string?) ::s/any))
(s/def ::attr-map
(fn [{:keys [transform] :as m}]
(and (s/valid? ::simple-attr-map (dissoc m :transform))
(or (not transform)
(s/valid? ::transform transform)))))
#2016-06-0200:00stathissiderisbut as I expected, you lose conform
destructuring and explain
can't explain the problem anymore#2016-06-0200:00stathissideris...so different solutions would be welcome#2016-06-0200:01george.w.singerClojurescript is failing to compile my file if my fdefs
are located above their corresponding defns
. Is this the way things are intended?#2016-06-0200:01george.w.singerI.E., you can't spec your function before you define it?#2016-06-0200:02kenny@george.w.singer: Yes. The var is not defined yet. You could declare
the var if you must write the spec above the function. I have just been writing my specs below my function definitions.#2016-06-0200:03george.w.singerOk, thanks.#2016-06-0200:05george.w.singerMacro sugar needs to be added to spec IMO#2016-06-0200:05george.w.singerSomething like#2016-06-0200:07george.w.singer(fn-spec
fn-name ::arg1, ::arg2 -> ::ret-spec
:such-that ::ret-value-greater-than-max-of-its-inputs)
This would at once declare
our fn-name
and annotate it with corresponding specs using intuitive Haskell-like syntax.#2016-06-0200:13stathissideris@george.w.singer: you could write that macro š#2016-06-0200:14george.w.singeryea š#2016-06-0200:23arohner@george.w.singer: I think thatās a CLJS-impl bug, Iād report it#2016-06-0200:24arohnerin CLJ, you can (s/fdef new-fn) ahead of the var#2016-06-0200:26kenny@george.w.singer: Does it work if you do (s/fdef 'your-sym
:args ...
:ret ...)
#2016-06-0200:35george.w.singerIt does sometimes but then the whole thing becomes buggy. Sometimes it doesn't compile. All tests pass#2016-06-0200:35george.w.singerReally strange behavior is exhibited#2016-06-0200:36george.w.singerThe strange behavior goes away if put them before#2016-06-0200:54kennyI seem to be getting a strange error when creating a spec for a function with a type hinted argument.
Error:
java.lang.ClassCastException: clojure.spec$spec_checking_fn$fn__11414 cannot be cast to clojure.lang.IFn$DO
Example:
(defn bar
[f]
(f 0.0))
(s/fdef bar
:args (s/cat :f fn?)
:ret number?)
(defn foo
[^double value]
(bar (fn [_] value)))
(s/fdef foo
:args (s/cat :v number?)
:ret number?)
(defn foo2
[value]
(bar (fn [_] value)))
(s/fdef foo2
:args (s/cat :v number?)
:ret number?)
(foo 1.0) => error
(foo2 1.0) => 1.0
(s/instrument-all)
#2016-06-0200:57kennyProblem exists in 1.9.0-alpha4.#2016-06-0202:05seancorfieldLooks like primitive hinting encodes the argument and return type to be encoded in the underlying type (if you add ^double
as a return type hint, the cast is to IFn$DD
, for example).#2016-06-0202:51seancorfieldVery surprised to see that calling an instrumented function can actually load and call clojure.test.check
to do generative testing as part of the conformance test, without explicitly calling clojure.spec.test/run-all-tests
etc.#2016-06-0202:51seancorfieldhttp://dev.clojure.org/jira/browse/CLJ-1936#2016-06-0207:59stathissiderisis this a bug?
spec> (gen/sample (s/gen #{1 0}) 10)
(0 0 1 0 0 0 1 0 1 0)
spec> (gen/sample (s/gen #{false true}) 10)
(true true true true true true true true true true)
#2016-06-0208:02hiredmans/gen takes a spec, which is a predicate#2016-06-0208:02hiredman#{false true} will only return true, as a predicate, for true#2016-06-0208:14stathissideris@hiredman: thanks for your explanation, but I still don't get how I would generate true/false randomly#2016-06-0208:17slipsetIāve defined these specs and vars#2016-06-0208:17slipset(s/def ::point (s/cat :x number? :y number?))
(s/def ::body (s/+ ::point))
(s/conform ::point [2 3])
(s/conform ::body [[2 3]])
(s/explain ::body [[2 3]])
#2016-06-0208:18slipset::point
conforms nicely, but ::body
does not conform, and the explanation is:#2016-06-0208:18slipsetIn: [0] val: [2 3] fails spec: :snake-game.utils/point at: [:x] predicate: number?
#2016-06-0208:19slipsetI would have thought that (s/+ ::point)
would mean one or more points?#2016-06-0208:31stathissideris@slipset: (s/def ::point (s/spec (s/cat :x number? :y number?)))
#2016-06-0208:32stathissiderisThe combination of s/+
and s/cat
means that body is defined as a series of :xs and :ys without nesting#2016-06-0208:32slipsetthanks š#2016-06-0208:37stathissiderisfor the record, I ended up with this for booleans:
(defn- boolean? [x] (instance? Boolean x))
(s/def ::boolean (s/with-gen boolean? #(gen/boolean)))
#2016-06-0209:25stathissiderisis there any way to make s/cat
specs generate a vector?#2016-06-0209:57slipsetWhat @stathissideris said :)#2016-06-0220:38surreal.analysisThis is a pretty minor note, but this appears to be one of the only channels with _
separation instead of -
. Any chance an admin could change that?#2016-06-0220:39arohnerand as we all know, clojure names should prefer -
to _
#2016-06-0220:40seancorfieldI see no stinkinā underscores here...#2016-06-0311:16jcfI'm having some trouble spec'ing a fn that walks maps allowing you to hyphenate keys etc. I wonder if someone can point out my mistake as I'm having a hard time decrypting the error messageā¦
(defn walk-map
"Recursively apply a function to all map entries. When map is nil returns an
empty map."
[f m]
(if m
(walk/postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m)
{}))
(s/fdef walk-map
:args (s/cat :f (s/fspec :args (s/tuple ::s/any ::s/any) :ret ::kv)
:m ::maybe-any-map)
:ret ::any-map)
(defn hyphenate-keys
"Recursively transforms all map keys from underscored strings to hyphenated
keywords."
[m]
(walk-map (fn [[k v]] [(hyphenated-keyword k) v]) m))
(s/fdef hyphenate-keys
:args (s/cat :m ::maybe-any-map)
:ret ::any-map)
#2016-06-0311:16jcfThe error:
ERROR in (t-hyphenate-keys) (core.clj:4631)
expected: (= (sut/hyphenate-keys {:a 1, "a" 2}) {:a 2})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ({}) fails at: [:args :f] predicate: (apply fn), nth not supported on this type: PersistentArrayMap
:clojure.spec/args (#object[example.common$hyphenate_keys$fn__20568 0x754a38a0 "
#2016-06-0311:17jcfI've tried a s/cat
to capture ::s/any
but I'm sure I'm missing something.#2016-06-0311:18jcfWith the s/cat
that I'd expect I need:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn), clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args (#object[example.common_test$fn__21912$fn__21926 0x7787a602 "
#2016-06-0311:20jcf(s/fdef walk-map
:args (s/cat :f (s/fspec :args (s/cat :kv ::s/any) :ret ::kv)
:m ::maybe-any-map)
:ret ::any-map)
Hmm.#2016-06-0311:22jcfShouldn't an fspec
with :args (s/cat :kv ::s/any)
match any and all args?#2016-06-0312:27jcfThis is what's confusing me. This spec looks right, but doesn't work in the fspec
s :args
.
(s/explain-data (s/cat :kv (s/tuple ::s/any ::s/any)) [[:a {:b 2}]])
#2016-06-0312:31manutter51what is your hyphenated-keyword
fn doing? Iām not sure Iām following whatās happening there, but it looks like itās blowing up in that function, and thatās causing the spec failure#2016-06-0312:33jcf(defn- hyphenated-keyword
[x]
(if (or (string? x) (keyword? x))
(-> x keyword->string infl/hyphenate keyword)
x))
@manutter51: I've used this code in a number of projects for a few years now. The fns are pretty reliable.#2016-06-0312:36jcfThe relevant specs I'm using:
(s/def ::any-map
(s/map-of (s/nilable ::s/any) (s/nilable ::s/any)))
(s/def ::maybe-any-map
(s/nilable ::any-map))
#2016-06-0312:36jcfNot sure if ::s/any
is already nilableā¦ š#2016-06-0312:41jcfIn the walk-map
test I get a cast exception:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn), clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args (#object[example.common_test$fn__21912$fn__21926 0x5ea46e79 "
In the tests that rely on walk-map
:
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys {"a-b" 1}) {"a_b" 1})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args (#object[example.common$underscore_keys$fn__20550 0x1f36bed9 "
#2016-06-0312:42jcfThese error messages aren't intuitive. I've got a feeling this will be another thing newcomers to Clojure really struggle with.#2016-06-0312:45jcfThis looks like it might help with the vague error message: https://github.com/clojure/clojure/commit/68fe71f0153bb6062754442c0a61c075b58fd9bc#2016-06-0312:45jcfIt recurses into a ::pcat
to expand out the error explanation, and I think I'm looking at a ::pcat
above.#2016-06-0312:48jcfOkay, so only the commented out test fails:
(deftest t-walk-map
(are [f m x] (= (sut/walk-map f m) x)
identity nil {}
identity {} {}
identity {:a 1} {:a 1}
;; (fn [[k ^long v]] [(name k) (if (number? v) (inc v) v)])
;; {:a 1 :b {:c 2 :d {:e 3}}}
;; {"a" 2 "b" {"c" 3 "d" {"e" 4}}}
))
#2016-06-0312:53jcfOkay. Looks like spec was exercising my walk-map
with keys like []
and ()
. I was then calling name
on vectors etc.#2016-06-0312:54jcfFixed that by making my test fn valid. š
(deftest t-walk-map
(are [f m x] (= (sut/walk-map f m) x)
identity nil {}
identity {} {}
identity {:a 1} {:a 1}
(fn [[k v]]
[(if (named? k) (name k) k)
(if (number? v) (inc ^long v) v)])
{:a 1 :b {:c 2 :d {:e 3}}}
{"a" 2 "b" {"c" 3 "d" {"e" 4}}}))
#2016-06-0312:54jcfOne down. One to go!
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys nil) {})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args (#object[example.common$underscore_keys$fn__20550 0x6dd8e64b "
#2016-06-0312:56jcfI wonder if it's the same problem. I need to make sure I have named keys.#2016-06-0314:01jcfIt was. Because I had enabled instrumentation my functions were being called with vectors, sets, etc. and those weren't supported.#2016-06-0314:50benzapso i'm converting from schema, and i'm having some issues with respect to preventing clutter within my namespace. maybe i'm not structuring my data correctly#2016-06-0314:51benzapex.
`(def TextBlock {:type (schema/eq "Text")
:foreground-color Color
:background-color Color
:style {(schema/optional-key :bold) schema/Bool
(schema/optional-key :underline) schema/Bool
(schema/optional-key :italic) schema/Bool}
:text Letter})
(s/def ::foreground-color ::color)
(s/def ::background-color ::color)
(s/def ::style (s/keys :opt-un [::bold ::underline ::italic]))
(s/def ::textblock
(s/keys
:req-un [::type
::foreground-color
::background-color
::style]))`#2016-06-0314:52benzaphow would I apply s/def to ::bold, ::underline and ::italic, without s/def? I don't want those in the namespace#2016-06-0314:52benzapif I did have them in the namespace, i'd prefer to have them called something like ::font-style-underline, but the underlying spec would need to accept :underline as the style key#2016-06-0314:54benzapIs there any way to do this? I'm confused on whether I should be inlining the spec stuff. I'm sticking to what I did with plumatic.schema by placing all of my schemas in one file called schemas.cljs#2016-06-0315:20jcf@benzap: I don't think you can. You need to def the keywords in order to enable reuse.#2016-06-0315:21jcfAnd I'd think you only want really common schema in a global namespace as spec is embracing namespaced keywords.#2016-06-0315:22jcfSo, :font/bold
and :font/underline
instead of :schemas/font-style-underline
etc.#2016-06-0315:24jcfI've used specs like :common/any-map
because I want nilable maps in a lot of places, but there might be a better way I haven't yet found.#2016-06-0315:25jcf(And I've done away completely with my shared schema namespaces for now.)#2016-06-0315:29benzap@jcf: I see, so I guess i'll have to start embracing namespaced keywords as well š#2016-06-0315:30benzapI already have a font.cljs, would it be common to place spec in-line with the code?#2016-06-0315:33jcfThat's what I'm doing, and what I've seen in all the examples.#2016-06-0315:34jcfIt makes sense keeping everything starting with font
in font.cljs
; that's where I'd look at least!#2016-06-0319:08nwjsmithHrm, is there a way to specify a transform to a spec's conformed data?#2016-06-0319:09nwjsmithI'm speccing datomic query, which accepts either a list or a map, and I'd like them to conform the same way#2016-06-0319:32nwjsmithFound it! Looks like clojure.spec/conformer
is what I was after.#2016-06-0321:03hiredmanI haven't seen this discussed elsewhere, but as a warning, protocol functions called in a non-higher order way (where clojure knows the protocol function is being called an creates an optimized call site for it) don't properly check args and returns against specs if instrumented#2016-06-0415:48nwjsmithI'd like to ship a library without requiring test.check
, and have the generators dynamically loaded as they are with clojure.spec
, but I can't figure out how to do that. Do I wrap all calls to clojure.spec.gen
functions in clojure.spec.gen/delay
?#2016-06-0415:50nwjsmith(the library is essentially a bunch of specs, some of which require custom generators)#2016-06-0418:40hiredmanthat is kind of interesting to think about, if you are defining custom specs with generators in your source, then test.check needs to be a regular dependency not a test scoped dependency#2016-06-0418:41hiredmana delay delays runtime code from running, not compile time stuff#2016-06-0418:41hiredmanlike var resolution#2016-06-0418:42hiredmanyou likely need some combination of something like dynaload from clojure.spec.gen and the delay there and maybe clojure.core/delay#2016-06-0418:43bbloomor just put your generators in another namespace?#2016-06-0418:45hiredmanthat would still have issues with any kind of tooling that loads all namespaces (in the absence of test.check)#2016-06-0418:45bbloomwhat kind of tooling does that? seems kinda like a bad idea š#2016-06-0418:47hiredmanI was thinking of stuff like lein check, but that only loads stuff from the current project#2016-06-0418:49bbloomon another trackā¦. 1st order predicate expressions and regular languages are closed under intersection ā would be cool if somebody wrote an āintersectā function for spec, could be useful for static analysis#2016-06-0418:52arohner@bbloom: elaborate?#2016-06-0418:55bbloom@arohner: well you could just write an abstract interpreter that walks the clojure AST and instead of an environment of symbols -> values, have an environment of symbols -> specs#2016-06-0418:55arohneroh, yeah#2016-06-0418:55bbloomeach time you use something, you call intersect on it#2016-06-0418:55bbloomif the result is void, you have a "type error"#2016-06-0418:55arohneryeah, just wasnāt sure what you meant by intersect#2016-06-0418:55arohnerI have about half of that written#2016-06-0418:56arohnercurrently reimplementing the regex part of spec to handle it#2016-06-0418:56bbloomif each spec defines a set, intersect = set intersection#2016-06-0418:57bbloomintersecting two predicates is just and-ing them, but you could do better for some predciates, cat, repeat, alt, etc#2016-06-0418:58arohneryeah, itās a lot more work than that š#2016-06-0418:59bbloomcertainly not trivial, but should be at least straightforward, no?#2016-06-0418:59arohneryeah#2016-06-0418:59arohnerparsing specs and the regex stuff is straightforward, but a ton of code#2016-06-0419:00arohnersame w/ flowing predicates through a fn body#2016-06-0419:01bbloomyeah, i have a clojure interpreter that iāve copy/pasted and modified 4 or 5 times for code walking purposes - someday iāll make it a library#2016-06-0419:01bbloomin the mean time: https://github.com/brandonbloom/metaclj/tree/master/src/metaclj/impl#2016-06-0419:02bbloomthe transform namespace is an example of a code walk on that ast & you could replace that with a type check walk#2016-06-0419:03bbloomhereās an interpreter: https://github.com/brandonbloom/eclj/blob/master/src/eclj/interpret/meta.eclj#2016-06-0419:03arohnerIām already walking the tools.analyzer output, so itād be a good amount of work to swap#2016-06-0419:04bbloomnot suggesting that you switch to my thing š just showing that it can be much less code than tools analyzer, which does all the analysis all at once#2016-06-0419:26bbloomanyway @arohner: excited to see what you come up with š#2016-06-0419:27arohneryeah, me too. Just need to finish the regex matching crap#2016-06-0421:04gfredericksthe first big question about clojure.spec that I'm going to try to dive into is how I can get functionality similar to https://github.com/gfredericks/schema-bijections#2016-06-0421:05gfredericksso I suspect (ha!) I'll be reading a bunch of the clojure.spec source soon#2016-06-0421:20gfredericksis it weird that s/tuple takes preds instead of specs?#2016-06-0422:11benzapI thought preds were specs#2016-06-0422:12benzapI have a tuple '(s/def ::color (s/tuple ::byterange ::byterange ::byterange ::byterange))' that seems to work @gfredericks#2016-06-0422:19benzapas of now, I have successfully converted my project from plumatic.schema to cljs.spec!#2016-06-0422:19benzapstill have a lot to learn about it, but it sure is nice#2016-06-0423:19gfredericksbenzap: yeah it does seem to work, but that means the docs are misleading I think: preds are specs but specs aren't preds#2016-06-0500:08Alex Miller (Clojure team)the terminology drifted a bit during dev#2016-06-0502:55adambrosIs there anything planned in terms of having predicates/specs with custom error messages/data?
Iām speaking from a validation perspective, because the alternative is to wrap explain-data#2016-06-0502:56adambrosit also seems that for related reasons, I would be writing a conform-or-explain!
function that throws with explain-data when the conform fails#2016-06-0503:02adambrosWhile Iām asking, it would be a useful feature to be able to somehow get the path at which a predicate is being run. This one is coming up as Iām trying to secure a datomic transaction wrapper by looking up if a user has access to a certain :db/id
thatās potentially nested within a map.#2016-06-0503:25seancorfield@adambros: which alpha are you using? I thought they'd added a path result in explain in one of the later alpha builds.#2016-06-0503:27adambrosiām referring to accessing the path from within the predicate#2016-06-0503:28adambrosmight be bad idea (TM) but its something that would ameliorate my problem#2016-06-0503:30seancorfieldBut each predicate only knows what it is called on -- it can't know the whole path, since a predicate is a general function.#2016-06-0503:32seancorfieldIf your predicate is string?
for example...#2016-06-0503:32adambrosit could if there was a dynamic var *path*
or something.
I think i could solve this another way so its fine#2016-06-0503:33seancorfieldDynamic vars are evil š#2016-06-0503:33adambros*evil*
!#2016-06-0503:33seancorfieldš#2016-06-0503:33adambrosim more interested in my other questions, so i probably shouldnāt have asked about it#2016-06-0503:37seancorfieldOk, what are the other questions? I think I missed those...#2016-06-0503:38adambroscustom error message for predicates/specs
something like (s/def :title (s/either string? {:message āinvalid title"}))
#2016-06-0503:39adambroswhere either
ās predicate either succeeds or puts the 2nd argument in explain-dataās result#2016-06-0503:40adambrosand the other was about a potential builtin for validation, conform-or-explain!
that throws if it cant conform#2016-06-0503:41seancorfieldI get the impression from the discussions here that explain-data
is indeed the preferred basis for any custom error messages. I think there is a JIRA ticket for something like conform-or-explain
tho'... let me go look...#2016-06-0503:46seancorfieldHmm, no, maybe it was just a discussion on the clojure-dev mailing list.#2016-06-0503:47adambrosseems like itād be a simple wrapper, with an extra arity for a function to call with the data when it fails#2016-06-0503:47adambros(defn conform-or-explain [spec x & [on-fail]] ...)
#2016-06-0503:48adambrosby default itād throw the error in an ex-info
#2016-06-0503:51seancorfieldAh, I remember now, it was a valid or explain wrapper that was discussed -- for use in :pre
assertions.#2016-06-0503:55seancorfieldBut, yeah, I think the wrapper is so easy to write -- and some folks would want to throw explain-data and some would throw explain as a string so I don't think there's a one size fits all here.#2016-06-0503:56adambrosfair enough, sounds like I just need to have some sort of spec namespace that declares these wrappers#2016-06-0503:58seancorfieldOne of the great things about Clojure: it's easy to write an adapter layer to make the supplied libraries match each specific application's needs š#2016-06-0503:58seancorfieldWe have a bunch of wrappers at World Singles. We have several of our own string functions in a namespace that supplements clojure.string
.#2016-06-0504:07seancorfieldWe also have some collection functions, a bunch of date stuff, wrappers around system level stuff. Wrappers around JDBC stuff (and, heck, I maintain clojure.java.jdbc
!).#2016-06-0513:30benzapanyone know of a good way to define a spec for an atom holding a piece of structured data?#2016-06-0513:31benzapseems like I would need to deref in order to perform the spec, so a lot of wrapping even for simple checks#2016-06-0514:01nwjsmith@benzap: I think someone asked this a few days ago and the answer was "There isn't because atoms aren't 'data'". IIRC there was also a suggestion to take a look at coll-of
, which might have some bits you could crib to implement an atom-of
. I might look into this today.#2016-06-0515:09Alex Miller (Clojure team)personally, I think you should work with state by
a) defining your problem in terms of data
b) writing nearly every function as a pure function from data to data
c) isolating the use of stateful entities to as small number of functions as possible
d) focusing on writing great specs for b in terms of a#2016-06-0515:10Alex Miller (Clojure team)not every function has to have a spec#2016-06-0518:50gfrederickshas anybody started a clojure.spec utility library yet because I keep thinking about doing that#2016-06-0518:51gfredericksshould probably call it spook#2016-06-0519:00Alex Miller (Clojure team)you may want to check your urban dictionary before you venture down that path#2016-06-0519:00gfredericksurban dictionary has all the best words#2016-06-0519:01gfredericksspock#2016-06-0519:01Alex Miller (Clojure team)totally on a tangent, the urban dictionary was hiring for a full stack clojure position recently#2016-06-0519:02Alex Miller (Clojure team)spock is much better :)#2016-06-0519:02Alex Miller (Clojure team)although I think there is a scala (or groovy?) testing library called that already#2016-06-0519:02Alex Miller (Clojure team)https://github.com/spockframework/spock#2016-06-0519:03Alex Miller (Clojure team)spackle?#2016-06-0519:06gfredericksshpec#2016-06-0519:07gfrederickssplec, shplec#2016-06-0519:08bronsashpec sounds like something sean connery would say#2016-06-0519:08gfredericksokay so we already have a mascot#2016-06-0519:09gfredericks(require [com.gfredericks.shpec :as s'])
#2016-06-0519:10bronsajust realized 'foo'
is valid clojure#2016-06-0519:10gfredericksI only wish the syntax highlighters were more often aware of that#2016-06-0519:10gfredericks...because I use that all the time#2016-06-0519:34mfikesReplete (standalone ClojureScript iOS app) has an update in the App Store with support for cljs.spec
.#2016-06-0520:16gfredericksÅpĆ©Ä#2016-06-0520:55stathissiderisis there any way to make s/cat
specs generate a vector?#2016-06-0520:59gfredericksstathissideris: both s/coll-of and s/tuple can generate vectors -- are either of those usable for you?#2016-06-0521:00stathissideris@gfredericks: tuple would work for my validation, but ideally I would like to keep the destructuring that s/cat
provides#2016-06-0521:01gfrederickswhat you want is a way to gen/fmap the default generator#2016-06-0521:01Alex Miller (Clojure team)use with-gen
to override the generator with fmap#2016-06-0521:01gfredericksalexmiller: but how do you get the default generator to pass to fmap?#2016-06-0521:02Alex Miller (Clojure team)well you could call s/gen on the original spec but I guess you may end up repeating yourself#2016-06-0521:02gfredericksthat was the hack I was about to suggest#2016-06-0521:03gfrederickswe can put a helper for this into shpec#2016-06-0521:04stathissideristhank you both! I'm not very well-versed with test.check, but I'm saving this part of the log so that I can go do some reading. It will hopefully make more sense after that š#2016-06-0521:05gfredericksstathissideris: we're suggesting (s/def ::foo <definition>) (s/def ::foo (s/with-gen <definition> (let [g (s/gen ::foo)] #(gen/fmap vec g))))#2016-06-0521:05gfredericksand <definition> being there twice is the repetition alex mentioned#2016-06-0521:07stathissiderisgreat thanks, I'll give it a go. In my case <definition> is not too long, so it could be ok#2016-06-0521:08gfrederickscould even try factoring it out to a let
and see if that works#2016-06-0521:08stathissideris...or even a macro is that doesn't work#2016-06-0521:11gfredericksalexmiller: the default generator for s/and is necessarily pretty hacky; should there be some documentation somewhere suggesting an override?#2016-06-0521:12gfrederickssomehow overriding test.check's normal error message for such-that
would be a fancy approach#2016-06-0521:17Alex Miller (Clojure team)I think there is some desire to have a better (programmatic) way to have a conversation about filtering with such-that than just boolean yes/no#2016-06-0521:18Alex Miller (Clojure team)but I'm sure Rich would have more to say about that#2016-06-0521:20gfredericksthat sounds sooper fancy#2016-06-0521:21gfredericksalexmiller: do you know if recursive specs is the only reason for the no-arg-function-returning-a-generator pattern?#2016-06-0521:22Alex Miller (Clojure team)Stu worked on that but I assume the dynamic loading is another aspect#2016-06-0521:22gfredericksoh that sounds plausible#2016-06-0612:08mandersonCurious if there has been any thought given to adding a doc string parameter to clojure.spec/def
like there is in clojure.core/def
? I understand the spec should be self-explanatory, but it's nice to be able to capture additional information/description about the spec. I've found myself adding comments that would be nice to have as doc strings that could then also be parsed programatically.#2016-06-0612:09Alex Miller (Clojure team)yes, there has been thought about it#2016-06-0612:11Alex Miller (Clojure team)not sure if anyone noticed, but you can now call doc with a registered spec keyword to get it's spec definition. that's a place that docstring could show up.#2016-06-0612:11Alex Miller (Clojure team)but it's something Rich is still thinking about afaik#2016-06-0612:14mandersoncool, thanks Alex. I think the built in doc string could be really useful.#2016-06-0612:14mandersonAlso, didn't notice the doc
addition. That's great.#2016-06-0613:53mike_ananevHello all! I learn clojurescript. Today I discovered that clojurescript doesn't throw Exceptions in code above like Clojure. Can clojure.spec help in such situations? I use reagent and want prevent state updating from using inappropriate function.#2016-06-0614:02nwjsmith@mike1452: Hey Mike, welcome to ClojureScript! clojure.spec can definitely help you here:#2016-06-0614:03mike_ananev@nwjsmith: Thank you!!#2016-06-0615:26gfredericksnwjsmith: the spec needs to be (s/cat :arg integer?)' instead of just
integer?`, right?#2016-06-0615:26gfrederickss/integer/number/#2016-06-0615:27nwjsmithThat's what I thought at first, but maybe this is a situation where a pred is implicitly converted to a spec?#2016-06-0615:29gfredericksI guess I'll just go try it :)#2016-06-0615:29nwjsmithwhen passed to valid?
or conform
, predicates will be implicitly converted to a spec, which I think is what happens here.#2016-06-0615:31gfredericksit throws an error when you call it with a number as well#2016-06-0615:31gfredericksit's not about spec vs predicate it's about spec-for-a-single-arg vs spec-for-an-arglist#2016-06-0615:33nwjsmithooooh#2016-06-0615:36nwjsmith@mike1452 looks like the example I gave above is wrong. The call to s/fdef
should be:
(s/fdef inc :args (s/cat :x number?))
thanks @gfredericks#2016-06-0615:41gfredericksnp#2016-06-0616:47mike_ananev@nwjsmith: got it! Thanks!#2016-06-0618:03angusiguessIs there a nice way to spec the following?#2016-06-0618:04angusiguess"I want a map of arbitrary keywords whose values must conform to some predicate"#2016-06-0618:05nwjsmithmap-of
#2016-06-0618:06angusiguess@nwjsmith: Beauty, thank you š#2016-06-0618:06nwjsmithnp#2016-06-0622:42wilkerluciowith clojure.spec I'm finding myself wanting to use some kind long namespaces for my keywords, they are just getting to be a borden to write, for example (s/keys :req [:my-app.some-ns.clock/at, :my-app.some-ns.clock/namespace])
, the namespace my-app.some-ns.clock
doesn't actually exists, it's just helpful to avoid name clashing, I'm writing those in the my-app.some-ns
#2016-06-0622:42wilkerlucioI was thinking that would be useful to be able to alias those namespaces so they can be used in a simpler way, for example:#2016-06-0622:44wilkerlucioit's possible to alias like this with the currently clojure features?#2016-06-0622:45noonianI believe if you require a namespace like (ns my-ns (:require [my-app.some-ns.clock :as clock]))
you can use ::clock/foo
and it will resolve to :my-app.some-ns.clock
.#2016-06-0622:46wilkerlucio@noonian: yes, but that would require me to create a file with that namespace, that's what I would like to avoid#2016-06-0622:47noonianAh got it, sorry I didnāt read your message carefully enough.#2016-06-0622:47jr@wilkerlucio: see this proposed feature http://dev.clojure.org/jira/browse/CLJ-1910#2016-06-0622:48wilkerluciothanks @jr, I had saw that one, and I agree that will help, but only on map cases... when you are setting it as keywords for lists (on the s/keys
for example) you can't use that, so we would still have to repeat the typing, I believe aliasing is a more general solution here that could be very useful with long namespaces#2016-06-0622:48hiredmanyou don't actually need to create a file now#2016-06-0622:49hiredmancreate-ns#2016-06-0622:51hiredmanuser=> (create-ns 'my-app.some-ns.clock)
#object[clojure.lang.Namespace 0x3b7d3a38 "my-app.some-ns.clock"]
user=> (alias 'clock 'my-app.some-ns.clock)
nil
user=> ::clock/foo
:my-app.some-ns.clock/foo
user=>
#2016-06-0623:19wilkerlucio@hiredman: thanks, I'm going to try that soon, do you think that can work on cljs as well?#2016-06-0623:21hiredmancljs is a different kettle of fish, I am not sure#2016-06-0623:31wilkerlucioyeah, I have to fix some compilation issues here and then I'll try it out#2016-06-0702:33Alex Miller (Clojure team)@wilkerlucio: in addition to the new namespaced support in CLJ-1910 and CLJ-1919 we are considering changes related to aliasing of non-existent namespaces too#2016-06-0702:34wilkerlucio@alexmiller: cool, that would be nice, specially for cljs where I can't do the trick mentioned by @hiredman#2016-06-0702:37Alex Miller (Clojure team)I did some recon on it a few weeks ago and we talked about it briefly again today#2016-06-0702:39wilkerlucioI'm happy to hear that this is being considered, I think with clojure.spec the usage of fully namespaced keywords is going to have a great increase in usage#2016-06-0708:08jcfFound some clojure.spec benchmarks over on Reddit: https://muhuk.github.io/validation-benchmark/
N.B. I haven't independently verified any of the work.#2016-06-0708:29slipsetplaying with specs and tests in Clojurescript I get this error:#2016-06-0708:29slipset actual: #error {:message "Call to [object Object] did not conform to spec
#2016-06-0708:29slipsetThe error is probably correct, but the message leaves something to be desired.#2016-06-0708:58slipsetHaving said that, specs are awesome!#2016-06-0711:41mishadoffHi, could you help me understand clojure.spec
Here is the little snippet:
(s/def ::name string?)
(s/def ::specific-name #{"John" "Jack"})
(s/def ::age (s/and integer? #(<= 0 % 100)))
(s/def ::person (s/keys :req-un [::name ::age]))
(s/explain ::person {:name "John" :age 10})
Is there a way to validate that map contains specific key like :name
, but value should be matched against another spec, ::specific-name
?#2016-06-0711:46slipset(s/def ::person (s/and (s/keys :req-un [::name ::age]) #(#{āJohnā āJackā} (:name %))))
#2016-06-0711:47slipsetnot answering your question, but should work.#2016-06-0711:49mishadoffyes, but would be good to use already defined spec ::specific-name
instead of copying its implementation#2016-06-0711:55slipsetDon't know your code, but you could (def specific-names #{...}) and use that var in the specs#2016-06-0711:57mishadoffI can, but that not a spec anymore#2016-06-0712:12Alex Miller (Clojure team)you can do (s/def ::name ::specific-name)
to just alias an existing one#2016-06-0712:12Alex Miller (Clojure team)spec will "chase" registered names like that#2016-06-0712:14Alex Miller (Clojure team)or you could define a predicate that is reused for multiple specs#2016-06-0714:12bronsalooks like spec might need some performance tuning https://muhuk.github.io/validation-benchmark/#2016-06-0714:27Alex Miller (Clojure team)I don't think that test is very good, but that may also be true. will be looking at it today#2016-06-0714:31seancorfieldThe use case for clojure.spec
is different to Schema tho', right?#2016-06-0714:33seancorfieldYou might have conform
in several places for destructuring and some calls to valid?
but you're not going to have everything instrumented in production code. #2016-06-0714:33seancorfieldWhereas folks do have Schema enabled in production don't they? Hence the focus on performance there. #2016-06-0714:34gfredericksschema aims to be used similarly though#2016-06-0714:34seancorfieldAh, so not in production code then?#2016-06-0714:34gfrederickswhen you decorate functions with schemas they don't run by default#2016-06-0714:34gfredericksand similar to spec you can turn them all on#2016-06-0714:34gfredericksfor testing#2016-06-0714:35gfredericksand you can call them explicitly to validate in/out in production#2016-06-0714:35gfredericksso maybe they focus on performance primarily for that last case?#2016-06-0714:44Alex Miller (Clojure team)we still want spec to have good performance, even when not used in production :)#2016-06-0715:09bronsa@seancorfield: we disable schema in prod, performance hit is way too much#2016-06-0715:19wilkerlucio@mishadoff: like alexmiller said, you can create a new one, if you still need that key on the current namespace, make up a new namespace, eg: (s/def :other-ns.anything/name ::specific-name)
and then (s/keys :req-un [:other-ns.anything/name])
#2016-06-0715:21mishadoff@wilkerlucio: thanks, for now Iāve decided to use same names for keys and specs#2016-06-0715:22wilkerlucio@mishadoff: cool, that is the preferred way I believe š#2016-06-0715:23mishadoff@wilkerlucio: yes, but sometimes you have maps with unqualified keys from third-party libs#2016-06-0715:24wilkerluciothat's true, maybe with the clojure.spec people will start using more fully qualified keywords from now one, and possibly with specs already coming from the library itself, will be awesome š#2016-06-0715:53gfredericksis :opt
in s/keys
just for documentation purposes?#2016-06-0715:53gfredericksI can't figure out how else it affects anything#2016-06-0716:09angusiguess@gfredericks: does it work with generators?#2016-06-0716:09gfredericksoh that's probably true#2016-06-0716:10gfredericksyes indeed#2016-06-0716:11gfrederickscool.#2016-06-0716:42Alex Miller (Clojure team)yes, those reasons :)#2016-06-0716:44Alex Miller (Clojure team)bunch of new predicates (most with gen support) just landed in master https://github.com/clojure/clojure/commit/58227c5de080110cb2ce5bc9f987d995a911b13e#2016-06-0716:45Alex Miller (Clojure team)also, the long-awaited seqable?
:)#2016-06-0717:01andrewhrAlso, a nice bonus to see an abstraction for Inst
š#2016-06-0717:05seancorfieldAnd Alpha 5 is comingā¦ when? šø#2016-06-0717:29gfredericksalexmiller: oh boy this inst generator#2016-06-0717:30Alex Miller (Clojure team)@seancorfield: winding its way through the tubes#2016-06-0717:31gfredericksTIL that java.util.Date does weird things with negative years#2016-06-0717:31Alex Miller (Clojure team)ha#2016-06-0717:31Alex Miller (Clojure team)probably better to use something like inst-in
with that :)#2016-06-0717:33gfrederickscoming up with reasonable generators in a lot of cases is difficult :/#2016-06-0717:33gfredericksreasonable defaults I mean#2016-06-0717:34gfredericksstrings are my best example of that#2016-06-0717:34Alex Miller (Clojure team)yeah#2016-06-0717:34Alex Miller (Clojure team)here's a bunch of stuff that in no way resembles reality #2016-06-0717:34gfredericksa uniform distribution over unicode characters would give you 99% unprintable things#2016-06-0717:40Alex Miller (Clojure team)1.9.0-alpha5 https://groups.google.com/d/msg/clojure/D_s9Drua6D4/CTWk12cXDQAJ#2016-06-0717:41Alex Miller (Clojure team)notably for spec is the new unform
which lets you conform ... backwards#2016-06-0717:42Alex Miller (Clojure team)also in addition to the vast quantity of new predicates in core, there are now specs for long, double, and instant ranges in spec#2016-06-0717:42Alex Miller (Clojure team)and everything gens yay#2016-06-0717:45gfredericksyay#2016-06-0717:46seancorfieldThatās an awesome new release! Thank you (to everyone involved)!#2016-06-0717:52angusiguessyessssss#2016-06-0718:10brabsterhey folks, I have a question on map-of
in spec that I'd appreciate some help with#2016-06-0718:10brabsterhopefully this is a good forum, even if it turns out to just be me being dumb!#2016-06-0718:11brabsterif I use s/or
in the value spec in a map-of
, I lose the qualifier I get when I use it on its own#2016-06-0718:12brabstereg. as per docs (s/def ::thing (s/or :name string? :id integer?))
conforms eg. (s/conform ::thing "bob")
to [:name "bob"]
#2016-06-0718:12brabsterbut... (s/def ::map-of-kw-to-thing (s/map-of keyword? ::thing))
doesn't behave the same was under conform#2016-06-0718:13brabster(s/conform ::map-of-kw-to-thing {:foo "bob"})
gives {:foo "bob"}
, not {:foo [:name "bob"]}
as I was expecting#2016-06-0718:14brabsterAm I missing something? I expected ::thing
to behave the same way regardless of where it was used#2016-06-0718:16brabsteroh sorry I see a message in the history that I missed from @alexmiller: Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this - sorry!#2016-06-0718:20brabsteris there somewhere I can go to see whether this is planned or to ask for it?#2016-06-0718:22gfredericksqueue the warnings from all your favorite libraries that clojure.core/boolean? is being replaced by their local copy#2016-06-0718:25pheuteryep.#2016-06-0718:26Alex Miller (Clojure team)hopefully all benign if the work we did in 1.7 was successful :)#2016-06-0718:29gfrederickswhat sort of work?#2016-06-0718:30hiredmanisn't http://dev.clojure.org/jira/browse/CLJ-1591 still open? I didn't accidentally re-open it did I?#2016-06-0718:32Alex Miller (Clojure team)it's open but it's very narrow in scope iirc - only when defining a new fn of the same name in terms of the old function of the same name, right?#2016-06-0718:32Alex Miller (Clojure team)the general case of just overlapping the name was resolved afaik#2016-06-0718:33gfredericksah#2016-06-0718:33Alex Miller (Clojure team)prior issues came up when we added "update"#2016-06-0718:33bronsayeah, I don't actually think that's a bug#2016-06-0718:33Alex Miller (Clojure team)we have since added other things that were in general use without issue (although I can't remember an example now)#2016-06-0718:34bronsathe fact that def
declares the var at analysis time is the only way to write recursive functions with defn
#2016-06-0718:34hiredmanmmm, I guess I am note sure what version of clojure the guy was running when I w as helping him debug this#2016-06-0718:34Alex Miller (Clojure team)@hiredman: I'm not sure if your final example in the comments is actually the same thing or not, not sure?#2016-06-0718:34bronsaFWIW I couldn't reproduce that example#2016-06-0718:35gfredericksshould it be possible to write a function that verifies that none of the keyword-references used in any specs have typos?#2016-06-0718:35gfredericksusing keywords instead of vars makes the whole thing feel a lot more typo-sensitive#2016-06-0718:36hiredmanyeah, I didn't look at the history to see the current state of the bug, just spent a while trying to figure out why this guys defmulti was resulting in an unbound var, and after way too long I remember that bug, and he said changing the name of his function fixed it, but that is kind of loose, who knows what is happening on the other end of irc#2016-06-0718:36gfredericksby "typos" I mean a keyword that is supposed to refer to a spec but no spec has been registered#2016-06-0718:36Alex Miller (Clojure team)that's not a typo#2016-06-0718:36Alex Miller (Clojure team)necessarily#2016-06-0718:37Alex Miller (Clojure team)as long as it's registered by the time you use it, it's fine#2016-06-0718:37gfredericksright, so I mean "never registered"#2016-06-0718:37Alex Miller (Clojure team)well never is a long time#2016-06-0718:37gfredericks:)#2016-06-0718:37gfredericksI could have a test in a codebase that looks at this for example#2016-06-0718:38gfredericksso it would have the responsibility to ensure that all pertinent code is loaded already#2016-06-0916:25gfrederickss/or/for/#2016-06-0916:26gfrederickseven if there's something minor it could do better I feel like there's more leverage to be had using The One True Tool#2016-06-0916:26gfredericksI'll make a clojure.spec utility library if I have to though, for specialized stuff that clojure.spec decides not to support directly#2016-06-0916:27gfrederickse.g., maybe for the super succinct map syntax plumatic/schema has for defining map schemas#2016-06-0916:27rickmoynihanthe only things I think plumatic/schema has over clojure.spec right now are:
1. it's perhaps a little closer to json schema - and possibly less work to bridge into json schema
2. no automated coercions#2016-06-0916:28wilkerlucioI'm wondering here about sort of generic specs
, I'm thinking on the channel
case, I can spec the return of a function to return a channel, maybe would be nice to be able to annotate also the expected value that will come from the channel, do you people have any thoughts on how to deal with stuff like this?#2016-06-0916:45richhickey@gfredericks: having the fn return data which you throw is better - (s/keys :opt-un [:ex-message :ex-data])?#2016-06-0917:34ikitommi@rickmoynihan: the plumatic->json schema is done within the lib (the json-schema ns, based on protocol & supporting multimethod). Swagger (OpenAPI nowadays) only supports only a "pragmaticā subset of JSON Schema as itās original target was the OO-languages like Java - has client code generators for those. Things like oneOf
or anyOf
are not supported - the requests to add those are over 2y old now. I think having a separate pure spec<->json schema would be awesome! Having it work with swagger requires some extra work, happy to do/help with that.#2016-06-0918:30gfredericksrichhickey: yeah that's essentially what I was thinking; can you think of any args that should be passed? if we make it a one-arg function that's passed an empty map that would be better for backwards-compatibly adding more things later :) I'm wondering because if there's no need for any args then it's not clear why it even needs to be a function and not just passing the data directly#2016-06-0918:52gfredericksthis fancy defn is fun; I suppose it's a lot like core.match#2016-06-0918:56gfredericksuh oh I just made it stack overflow#2016-06-0919:03gfredericksokay here it is -- defn+spec
, where each arg can be decorated with a spec and you can overload a function by spec: https://gist.github.com/gfredericks/e4a7eafe5dcf1f4feb21ebbc04b6f302#file-defn-spec-clj-L73#2016-06-0919:05gfredericksthere's a note in there about a stack overflow that I haven't tried to debug, that happens when I add a spec to the defn+spec
macro itself#2016-06-0919:08fxposterhi everyone
I have 2 questions about clojure.spec, which are not that obvious from the beginning:
1. is it okay to use predicates that connect to external resources for validation? ie: I have a tree of "paths" and want to validate that it matches the filesystem or posts on some website? I don't see any techncal problems with that, but maybe there are other solutions for that?
2. for example, I have 2 data structures: new {"application": {"branches": {"release": "1.0", "snapshot": "1.1"}}}
and old {"application": {"branches": {"release": "0.9", "snapshot": "1.0ā}}}
and I want to actually check that the new value is a "valid update" of the old one, based on some constraints (for example: "new value contains at least all branches from the previous one" and "all versions in new value are greater than the same ones in the old one"). is there a way to at least partially express that in clojure.spec and get validation + error reporting?
Thanks#2016-06-0919:12gfredericksI think the predicates are supposed to be pure functions#2016-06-0919:12gfredericksI couldn't say where the first place you would run into trouble would be though#2016-06-0920:28akielCan someone explain what I found in my snippet?#2016-06-0920:36benzapdoes replacing sp/alt with sp/or make it work?#2016-06-0920:37akielYes!#2016-06-0920:37benzaphaha, i've had the same issue#2016-06-0920:38benzapsomeone worked out the differences. It has something to do with how sp/alt is used in regex, and sp/or is used otherwise.#2016-06-0920:38akielSo sp/alt on something which is not a regexp doesnāt work? Ok I see...#2016-06-0920:38benzapSimply put yes. Someone else on here had a really good explanation, but I didn't completely follow#2016-06-0920:39benzapHe was wondering why sp/alt and sp/or were so similar, and produced the same results in some contexts#2016-06-0920:39benzaphad to do with the regex distinction#2016-06-0920:40akielso maybe we need spec for spec - I mean itās all macros - everything can happen#2016-06-0920:41benzaphaha, i'm curious to see how much spec is used š#2016-06-0920:41benzapI wonder if they'll use clojure.spec on all of the clojure.core to try and get away from the java stracktraces#2016-06-0920:42akielcat and alt says: āreturns a regexā, and says: "returns a specā - so maybe a spec is not a regex š#2016-06-0920:42benzapah ok#2016-06-0920:42akieltype systems would help rant#2016-06-0920:43benzapI was talking to someone about clojure.spec on freenode#programming, and he said clojure.spec is like contracts in racket#2016-06-0920:44benzapthe idea being that they're supposed to be more powerful, since you can apply predicates#2016-06-0920:45akielyes spec is more powerful as a type system, because you can inspect actual values at runtime#2016-06-0920:45benzapya, the whole instrumentation is pretty neat#2016-06-0920:45benzapi've used it, but it's rather slow. I need to start using it per namespace#2016-06-0920:46akieljust use it only in dev and test#2016-06-0920:46benzapI have been, but it's even slow then. Maybe 20 times slower#2016-06-0920:46benzapmakes testing rather slow#2016-06-0920:46benzapIt isn't an issue if I run it once in a while, so i'll throw down instrumentation every once in a while#2016-06-0920:47benzapBut I wonder if there's any room for improvement in performance, it would make it very appealing to just leave instrumentation enabled even in production#2016-06-0920:48benzapThere's a reddit post comparing different schema/data validation libraries. No one has commented on it yet, and I wish someone more knowledgable would#2016-06-0920:48benzaphttps://www.reddit.com/r/Clojure/comments/4mxqcy/spec_performance_comparison/?ref=share&ref_source=link#2016-06-0920:48akielThan I think, you apply spec to inner loop things. I would apply it only to my public API.#2016-06-0920:48benzapya, that makes sense#2016-06-0920:49benzaplike, only validate data that comes in from an external source for production?#2016-06-0920:50akielIn production I would check data coming over wire directly without instrumentation, just in normal code.#2016-06-0920:51akielI do this with Schema already.#2016-06-0920:51benzapI've been using the clojurescript version of clojure.spec, cljs.spec#2016-06-0920:51benzapoh nice#2016-06-0920:51benzapyeah, I converted my project i've been working on from schema to spec#2016-06-0920:51benzapdidn't take that long, but it did require quite a bit of refactoring#2016-06-0920:52benzapI'm still not sure where I should put the specs. I kind of placed them at the beginning of the files, and would s/fdef after each function#2016-06-0920:53benzapI suppose that's good enough, I don't think i've tested s/fdef when the function hasn't been declared yet, so I don't think I could place them in another file?#2016-06-0920:53akielIām not completely sure either. But I have seen putting s/fdef before each funtion often.#2016-06-0920:53benzapoh, before?#2016-06-0920:53benzaphmm#2016-06-0920:54akielyes it is possible#2016-06-0920:54benzapthat's good to know#2016-06-0920:54akielits like contract first and than the implementation#2016-06-0920:54benzapthat makes sense#2016-06-0920:54benzapi'll have to consider moving my stuff into a separate folder#2016-06-0920:55benzapit's hard to get used to, really. I considered it an eye-sore at first#2016-06-0920:55benzapdefinitely useful though, I caught a lot of bugs early#2016-06-0920:56akielopposite to schema, the annotations are not inline to the function - so the possibilities are broader - like in core.typed I think - never used it#2016-06-0920:57benzapYeah, that would explain my confusion. At first, I had a hard time figuring out how to convert from schema#2016-06-0920:58benzapmade a lot more sense once I got started#2016-06-0920:58benzapwhen I used schema, I had originally defined the structures in a separate file, sortof like how you would define a jsonschema#2016-06-0920:58arohner@benzap: itās probably slower because itās doing generative testing at runtime. Hopefully that becomes a config setting#2016-06-0921:00benzap@arohner: that's interesting, I thought it was trying to conform each function with respect to it's defined spec. If it's also throwing in generative testing, I can see that causing some performance issues#2016-06-0921:01arohnerwell, fspec does. checking fdef now#2016-06-0921:02benzapa somewhat related topic, there's a library in clojure called specter, which compiles itself to increase performance#2016-06-0921:03benzapI wonder if the same concept could be applied to clojure.spec, where you could pre-compile the validator for your functions#2016-06-0921:19fentonwhat would a spec look like that does the following: if a map has one key, it should have another key#2016-06-0921:19bbrinckAnd in plumatic schema, you can create validators ahead of time for performance.#2016-06-0921:24fentonoops wrong description of problem. if a key in a map has a certain value, then another key should be present. writing specs can be a challeng...like a whole nother language! š#2016-06-0921:31angusiguess@fenton That is likely a multispec#2016-06-0921:31angusiguessWhere you write a multimethod looking for the first key, and that dispatches another spec to check for the second.#2016-06-0921:32fenton@angusiguess: ok, will look into that. thx.#2016-06-0921:32angusiguessš
#2016-06-0921:58arohnercould also be a simple s/or#2016-06-0921:58arohner(s/or map? (s/and (s/keys [::foo ::bar]))#2016-06-0921:58arohnerdepends on how complex it needs to be#2016-06-0922:01danielcompton@rickmoynihan: we use coercions which are extremely handy#2016-06-0922:03danielcomptonIf spec (or surrounding tooling) doesnāt support automated coercions, then weāll need to keep schemas for our boundary interfaces between webapp and RethinkDB#2016-06-0923:09rickmoynihandanielcompton: Yeah - I've used coercions before too - and I don't dispute their utility at boundaries... I guess you could easily build a specialised coercion library on top of spec though... don't know enough yet how you might do that...
Regardless I much prefer what I've seen of spec to schema; and don't think coercions are enough of a feature on their own to either not use spec, or use schema as well as spec... I'd definitely much prefer something that worked with spec#2016-06-1002:23sparkofreasonIs there a straightforward to define a function spec and reuse with multiple fdef's?#2016-06-1007:43mpenet@rickmoynihan @danielcompton : same here, also free type hinting, custom explain messages, performance and the list goes on. So far we have no reason to migrate really, we're waiting to see where clj.spec is going, it's probably a bit too early#2016-06-1013:51ikitommiBesides coercions, my top feature request to spec: helper-fn to create a spec from a vanilla clojure (function) var. It would understand the Clojure destructuring syntax. Something like this:
(require '[clojure.spec :as s])
(s/def ::age integer?)
(s/def ::fullname string?)
(s/def ::role keyword?)
(defn doit [{:keys [::fullname ::age ::role] :or {:boss ::role}}] [fullname age role])
(-> #'doit s/extract-spec s/describe)
; => (keys :req [:user/age :user/fullname] :opt [:user/role])
* the responsibility to extract (and use) the specs would be on the user - I would use these on the web-tier to auto-extract docs & do coercion in the web-api tier with our libs
* would not add new meta-data to vars (arguments are already in :arglists
)
* no need to describe the shape of the data twice (both for the function arguments & for it's spec)
* (optionally the :ret
and :fn
could be read from the Var metadata too)
Thoughts?#2016-06-1014:01angusiguessI think this is rad, my only real question is why might we rely on spec to do it?#2016-06-1014:01angusiguessThere are pretty good hooks to pull validation information out of a defined spec and an alternate defn like this should be a macro.#2016-06-1014:02angusiguessI don't work on spec so this is grain of salt stuff, but it seems like the opinion of spec is that people could conceivably bring their own sugar but under the hood there's a common language for validation.#2016-06-1014:09Alex Miller (Clojure team)@ikitommi: you could build that from what exists now. since it wouldn't generically apply, I don't think we would do that as part of core or anything. One thing that would help is a spec for destructuring, which I have and which will be released at some point in some form (details TBD still)#2016-06-1014:16ikitommi@alexmiller: spec for destucturing sounds cool. Did a dummy version of the extractor, will play more with it. Are there any caveats in playing with :arglists
? #2016-06-1014:18Alex Miller (Clojure team)there are a few cases where people have abused it a bit in what was put in it (data.generators is one that comes to mind) but generally should be fine#2016-06-1014:19Alex Miller (Clojure team)I think I would also consider allowing overrides via lookup in the registry - s/fdef
registers stuff there under the fn symbol and those can be obtained via s/fn-specs
#2016-06-1014:19Alex Miller (Clojure team)so you have an existing registry for overrides of things you couldn't build automatically#2016-06-1014:21Alex Miller (Clojure team)also note that CLJ-1919 will add a new syntax for namespaced key destructuring.#2016-06-1014:21Alex Miller (Clojure team)your example there is not syntactically correct btw - the keys of :or
should always be unqualified symbols (matching the bindings that are created). there are some bugs in this area in current Clojure that will be fixed in CLJ-1919.#2016-06-1014:22Alex Miller (Clojure team)so that is, what you have there probably works now, but by accident not intent, and will change#2016-06-1014:24ikitommiuh, copy-paste error in the code. But thanks! will check out your pointers.#2016-06-1015:10wilkerluciowhat's the correct way to express to an fspec
that a function takes no arguments?#2016-06-1016:13gfredericks(s/cat)
I'd guess#2016-06-1016:13gfredericksmaybe #{()}
would work too#2016-06-1016:16gfredericksalexmiller: there's no reason not to add specs to test.check is there?#2016-06-1016:17gfredericksassuming it accounts for older clojures#2016-06-1016:18gfredericksI guess this question is a superset of seancorfield's question on the ML, but also about test.check in particular since it's used in clojure.spec#2016-06-1016:24seancorfieldI haven't moved forward with that since the first cut. Want to see more discussion on the ML first. #2016-06-1016:24seancorfield(sorry if I don't follow up for a few hours -- doors closing en route for a cat show!)#2016-06-1016:28Alex Miller (Clojure team)None other than that it then requires Clojure 1.9#2016-06-1016:28Alex Miller (Clojure team)Which requires test.check#2016-06-1016:46gfredericksokay, cool; I'll probably do a separate .specs
namespace like sean did#2016-06-1017:11jcfAnyone tried writing specs for stateful objects like database connections, or a Datomic database?#2016-06-1017:14jcfI can't think of a nice way to specify a function takes a db
, and some other args. To generate a db
I need a database connection that isn't available when I define my specs.#2016-06-1017:16jcfImagine a trivial example like this:
(s/fdef load-entity
:args (s/cat :db ::d/db :tx ::entity-tx)
:ret ::entity)
(defn load-entity
[db tx]
(d/entity db [:entity/id (:entity/id tx)]))
#2016-06-1017:22jcfI can't generate Datomic entities either. I need a DB, which needs a connection, which needs a URI.#2016-06-1017:24jcfIs there a way to say a spec can't be generated automatically so I can test other specs in this namespace maybe?#2016-06-1017:27arohnerisnāt your DB predicate just #(instance? datomic.whatever.Db %)
?#2016-06-1017:27pheuter@jcf: i just went through that exercise of creating a Datomic Db#2016-06-1017:27pheuter(s/def ::db
(s/with-gen #(instance? datomic.db.Db %)
(fn []
(gen/fmap (fn [facts] (-> (helpers/empty-db)
(helpers/transact facts)))
entities-generator))))
#2016-06-1017:27jcfI need a generator to go with that spec @arohner.#2016-06-1017:28pheuter(s/def ::db
(s/with-gen #(instance? datomic.db.Db %)
(fn []
(gen/fmap (fn [facts] (-> (helpers/empty-db)
(helpers/transact facts)))
entities-generator))))
#2016-06-1017:28jcfI've got basic predicate fns like these:
(defn db?
[x]
(instance? datomic.Database x))
(defn entity?
[x]
(instance? datomic.Entity x))
#2016-06-1017:28jcf@pheuter: what's facts, and what does (helpers/empty-db)
look like?#2016-06-1017:29pheuterwhere entities
is a vector-distinct-by
of :db/id values#2016-06-1017:30pheuterempty-db just creates an empty datomic db that has been primed with a schema#2016-06-1017:30jcfSo you've got some hardcoded Datomic URI or something?#2016-06-1017:30jcfYou must have global state floating around, right?#2016-06-1017:31pheuterNo, we just generate random URIs, we use in-memory databases for development / testing#2016-06-1017:32jcfThat is global state. My Datomic connection is managed with components, and they're stopped/started around tests.#2016-06-1017:33jcfI could have two Datomic databases with separate connections. That wouldn't be possible with (helpers/empty-db)
.#2016-06-1017:33pheuteryes, we use mount to start and stop our connections as well#2016-06-1017:34pheuterwe dont use mount for testing though, and thatās when we generate db values#2016-06-1017:34pheuteractually, we do use mount for certain state, and rely on dynamic values using test/use-fixtures
#2016-06-1017:35pheuteryour predicates look fine, but you wonāt be able to use them for generating out-of-the-box, will need to use s/with-gen
#2016-06-1017:56dominicm
https://clojurians.slack.com/archives/clojure-spec/p1465578891000846
I've been trying to figure this out also. My solutions have involved macros and sideband data. Not elegant at all. I also ignored generators. #2016-06-1018:05gfredericksseancorfield: were you thinking of having a conditional require
in the .jdbc
namespace? otherwise you'd have the problem of up-to-date users having to opt-in to the specs#2016-06-1018:05gfrederickswhen there are use cases for the specs besides explicit testing, e.g. clojure.repl/doc#2016-06-1019:34jcfIs there a way to merge two s/keys
specs?#2016-06-1019:34jcfI must be missing something. Back to the manual!#2016-06-1019:35donaldballIāve been idly thinking about the problem of db args as well, though in the sql context. The problem seems the same for datomic and jdbc though: a spec saying the db
arg is e.g. a jdbc connection isnāt sufficient. You really want a spec that says this value is a jdbc connection to a database with at least a certain schema and maybe even a certain set of entities.#2016-06-1019:35jcfOh wait, just me being stupid apparently.#2016-06-1019:36jcf@donaldball: I'm using this at the mo:
(defn- with-datomic
[f]
(let [running (-> (config/read-config :test)
(assoc :uri (str "datomic:mem://" (UUID/randomUUID)))
map->Datomic
component/start)]
(try
(f running)
(finally
(component/stop running)))))
(defn entity?
[x]
(instance? datomic.Entity x))
(s/def ::d/entity
(s/with-gen
entity?
(fn [] (with-datomic (fn [{:keys [conn]}]
(gen/fmap
#(d/entity (d/db conn) %)
gen/int))))))
#2016-06-1019:37jcfDon't love it if I'm honest. Creating a new connection for every test is pretty inefficient, but it works.#2016-06-1019:38jcfIf you've not used Component or Datomic that's probably meaningless.#2016-06-1021:02gfredericksoh woah#2016-06-1021:04gfredericksI hadn't thought about this generator setup encouraging people to use stateful resources in their generators#2016-06-1021:12dominicm@gfredericks: I'd say that the encouragement and bias towards namespace level hoisting (with s/def) takes away our ability to lexically scope and generally pass around explicit arguments.
Diving too deeply into that statement takes you to mount vs component.#2016-06-1021:15gfredericksdominicm: the way I've used generators in the past is purely data-driven, so there's not even component-like stuff until the test starts running; but if you have a spec that's explicitly for a stateful thing, then you can't write a generator for it that way#2016-06-1021:16gfredericksmy gut would be to try to keep doing the data-driven thing, and so not use generators for specs that describe stateful things#2016-06-1021:16gfredericksnot sure how easy that is with datomic#2016-06-1021:16gfredericksa datomic entity is a map of attributes, is that right?#2016-06-1021:17gfredericksif that's the case you could make most of your specs just expect maps#2016-06-1021:17gfredericksand could do low-level testing with maps instead of entities#2016-06-1021:24dominicm@gfredericks: It's definitely difficult to manage with generators. The discussion (in my opinion) transcends just testing. Doing explicit s/explain-data
on a runtime database is also a use-case.#2016-06-1021:24gfredericksthat should work with map specs though I would think?#2016-06-1021:29dominicm(s/explain-data {:email "
The unique email checker needs a database to make it's check. If I make it part of the map data (assoc m ::db (d/db conn))
, then when I check, the returned data gives me an incorrect path to the error position.#2016-06-1021:30dominicmI have just got a macro working, which would take the error generated, and readjust the path for you. But it's still somewhat unnatural.#2016-06-1021:32gfredericksso your spec is making database queries?#2016-06-1021:33ghadithat is a bad idea#2016-06-1021:36gfredericksI think specs should be pure functions#2016-06-1021:43ghadiabsolutely#2016-06-1021:44gfrederickssomething that's not a pure function can be a plain ole test :)#2016-06-1022:14eggsyntaxI need to write a script to generate some Datomic seed data, and I'm experimenting with using spec to do so. Two questions:
1) One part of the seed data I need to write is the key/val that'll generate a temporary db/id. If I were hand writing the seed data, it would look like {:db/id #db/id[:db.part/user -1015948] ... }
. Can anyone help me understand how I would go about creating a spec for that?
2) ideally, it'd be nice to just spec that part of the schema, and then somehow use that to generate the seed data, but I'm not quite sure how go about that. Any hints?#2016-06-1022:15eggsyntaxIn other words, how can I spec a tagged literal like that?#2016-06-1022:30eggsyntaxGot it.
(defn db-id? [v] (instance? datomic.db.DbId v))
(s/def ::db-id db-id?)
#2016-06-1022:35eggsyntax(or at least that's the spec for the tagged literal itself. I'm not quite sure how to get from that to a spec that'll produce seed data as above. I guess I'll need to write a generator for it, but it's gonna take some experimentation for sure.#2016-06-1022:37hiredmanusing spec just to generate data seems super weird, why wouldn't you use the generators from test.check or https://github.com/clojure/data.generators directly?#2016-06-1022:39eggsyntaxWell, if I can figure out the relationship between a spec for the schema and a spec for the seed data, I can use the spec to do validation as well as generating seed data, and maybe find other uses for it as well.#2016-06-1022:40eggsyntaxAnd it's also partly a clojure.spec learning exercise for myself š#2016-06-1111:05borkdudeDoes spec offer anything in relation to coercing like Schema does?#2016-06-1111:36Alex Miller (Clojure team)Not really. It does have conformers but their intent is a bit different.#2016-06-1111:38Alex Miller (Clojure team)@rymndhng: this is the intended behavior - map-of samples its values for performance reasons so doesn't conform everything. This been a common question though and something will probably be added for it.#2016-06-1208:02patrkrisHi Everyone. How do you name aggregate stuff with namespaced keywords? Say I have a map that would look like this without namespaced keywords
{:first-name "John"
:last-name "Doe"
:address
{:street "Example Street"
:street-number "413"
:city "Example city"}}
The keys at the first level might be namespaced like :customer/first-name and :customer/last-name, but how would I name the keys inside the address map?#2016-06-1208:31hiredman{:customer/first-name "John"
:customer/last-name "Doe"
:customer/street "Example Street"
:customer/street-number "413"
:customer/city "Example city"
:customer/address-fields #{:customer/street
:customer/street-number
:customer/city}}
#2016-06-1208:40patrkris@hiredman: thanks. when would you say it's appropriate to create a "child namespace", e.g. :customer.address/*
?#2016-06-1214:27sanderWhat is more idiomatic,
(ns work.invoice
(:require [clojure.spec :as s]))
(s/def ::invoice (s/keys :req [::number ::date ::amount]))
or
(ns work.core
(:require [clojure.spec :as s]
[work.invoice :as i))
(s/def ::invoice (s/keys :req [::i/number ::i/date ::i/amount]))
? The first is less typing work, but I don't really like :work.invoice/invoice
as a spec name. Maybe this?
(ns work.invoice
(:require [clojure.spec :as s]))
(s/def :work/invoice (s/keys :req [::number ::date ::amount]))
#2016-06-1217:06danstoneDoes anyone know of a way to compose key sets? e.g something like (s/merge ::foo ::bar)
edit: and
works for validation, but does not yield a working generator...#2016-06-1217:10t3chnoboyHi! Iām trying to validate a javascript class and not sure how to make it work:
(s/def ::transport (s/or :websocket js/WebSocket
:long-poll js/Phoenix.LongPoll))
(s/conform ::transport js/WebSocket)
For objects I use #(instance? Class %)
and it works as expected.#2016-06-1217:34t3chnoboyok, Iāve just made it work:
(s/def ::transport (s/or :websocket #(= js/WebSocket %)
:long-poll #(= js/Phoenix.LongPoll %)))
(s/conform ::transport js/WebSocket)
#2016-06-1300:00t3chnoboyIs there a way to validate core.async
chan
type?#2016-06-1300:02t3chnoboyI want to define a spec for a function which returns the following map:
{:in-chan (chan)
:out-chan (chan)}
#2016-06-1300:05jfntn@t3chnoboy: you should be able to make a predicate for a ManyToManyChannel
instance, or perhaps more generically test that (satisfies? ReadPort in-chan)
and (satisfies? WritePort out-chan)
also both defined somewhere in core.async...#2016-06-1300:11t3chnoboy@jfntn something like:
#(instance? cljs.core.async.impl.channels.ManyToManyChannel %)
?#2016-06-1300:12t3chnoboyDoesnāt look pretty...#2016-06-1300:17jfntnindeed, think you could import ManyToManyChannel
and reference it directly though#2016-06-1300:21jfntnThe protocol check is probably a better way to go however#2016-06-1308:48dominicm@gfredericks: Yep, my specs make db queries. Spec has s/valid?
to accomodate runtime validations (not just function checking in development time). I honestly think having two libraries, one for "types" and one for "db connecting checkers" would be wasted effort.#2016-06-1311:10jimmyhi guys, I want to discuss a bit about spec. Spec is amazing, I can see the workflow that we can try out things in dynamic clojure, then add spec to it later to ready for production. But would spec have an optimization behind to check for example: we define a function with the args which is an integer? then spec will also generate a function with type hint integer beside doing the validation ? Then we can have something that is very flexible at output error and very fast code as well. Just my thought.#2016-06-1312:10shemis there a way to write a spec that says "this map should contain these keywords, and only once each"?#2016-06-1312:17stathissideris@shem: but keys can only appear once each in a map... by definition!#2016-06-1312:21shemergh, right. was eyeing generator output that spat several instances of the map. need more coffee#2016-06-1312:42jimmyhi guys, how do we define a spec that can validate both namespaced key and non namespace key, for example : user/first-name
and first-name
one of those would be valid.#2016-06-1314:51manderson@nxqd:
(s/def ::first-name string?)
=> :user/first-name
(s/def ::map-spec (s/keys :opt [::first-name]
:opt-un [::first-name]))
=> :user/map-spec
(s/valid? ::map-spec {::first-name "joe"})
=> true
(s/valid? ::map-spec {:first-name "joe"})
=> true
#2016-06-1315:27settingheadhas anyone tried using core.async
with clojure.spec
? i ended up having a lot of functions that returns a channel in my code (`(defn [] (go ā¦))`). iād like to spec these functions beyond checking its type is ManyToManyChannel
(i.e. checking what comes out the channel is valid based on a spec). Iām thinking of adding transducers to the returning channels. but is there a better overall approach? should i write my functions this way to begin with?#2016-06-1315:41jimmy@manderson: I meant
(s/valid? ::map-spec {:user/first-name "joe"})
and (s/valid ::map-spec {:first-name "joe"})
#2016-06-1315:43manderson::first-name
is the same as :user/first-name
so either should work with above spec. Is that what you're asking?#2016-06-1315:43manderson^ is the same if you are in the same namespace. :: simply appends the current namespace qualification to the keyword#2016-06-1316:18jimmy@manderson: sorry, I was out. I will check on this, I think I miss understand something here.#2016-06-1316:27jimmyyeah I did mis understand, it works fine. with user/first-name
. thanks#2016-06-1316:32jimmy@manderson: ah I have another question, if I refer the spec in another ns, it shouldn't work with :user/first-name
case as long as I understand ?#2016-06-1316:38mandersonusing the definition of :user/map-spec
as defined above, it will validate fine for :user/first-name
or :first-name
. The namespaced keywords reflect the namespace they were defined in if set with ::
.#2016-06-1316:41jimmyok, I see. in s/keys
is there a way that we can validate something like :first-name
or :user/first-name
, one of those is required.#2016-06-1317:02mandersonhm, good question. this seems to work:
(s/def ::map-spec2
(s/or :qual (s/keys :req [::first-name]
:req-un [])
:simple (s/keys :req []
:req-un [::first-name])))
#2016-06-1317:03mandersonthe keywords in or
are just tags...#2016-06-1319:24danstoneQuestion, when should one specify a property of a function in the :fn
of an fdef
rather than in a proper test.check property?#2016-06-1320:48danielcomptonIām pretty sure the answer will be ānoā, but is there any way to use spec to define newtypeās, i.e. I have a username
and a full-name
which are both strings. Can I spec functions to stop passing a username
where a full-name
is required? For this discussion, assume that both strings have no distinguishing features you could use.#2016-06-1320:50danstoneYou might be able to use conform with a custom conformer for 'full-name'. edit: never mind, it won't work - as you say such a conforming fn would never know the difference between the two values#2016-06-1403:25mfikesItās too bad (s/describe (s/spec #(< % 3)))
has to return something that doesnāt quite look like an anonymous function literal#2016-06-1403:34mfikesIām not sure what to make of the fact that
(s/def ::a (s/and #(> % 2) #(< % 5)))
(s/def ::b #(and (> % 2) (< % 5)))
look the same. Perhaps it is OK because they behave the same way.#2016-06-1405:02jimmyhi guys, how do we use variable inside of (s/def (s/keys))
like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
It's a value in a macro in a macro. And I'm no macro master ...#2016-06-1405:44jimmyI have an error while trying to generate from clojure.spec, I think this would be the try out such-that limitation in clojure.spec implementation.
(defn str-limit [lower upper] (s/and string? #(<= lower (count %) upper)))
(s/def :project/description (us/str-limit 116 1000)) ;; the gen doesn't work if the lower value is around above 100
(s/def ::project-gen (s/keys :req [:project/description]))
(gen/generate (s/gen ::project-gen))
;; -- error
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4703)
#2016-06-1407:09stathissideris@nxqd: This is because string?
generates totally random strings and then checks the second predicate to see if they conform, and gives up after 100 tries. Consider wrapping the spec with s/with-gen
and providing your own generator.#2016-06-1409:07jimmy@stathissideris: thanks !#2016-06-1411:26skapoorhi guys, am playing with clojure spec and running into an un-expected behavior: (s/valid? #{nil "a" "b"} nil) ;; returns false when it should be true
#2016-06-1411:28minimalnil
is not truthy so the set as a predicate is returning a non-truthy value#2016-06-1411:29minimal(s/valid? #(contains? #{nil "a"} %) nil)
true
#2016-06-1411:30skapoor@minimal yeah, I tried that and it works.. but it should return true when a set is used in specs too right?#2016-06-1411:33minimalIf you are relying on the value returned from the set then it needs to be truthy to be valid#2016-06-1411:36skapoorokay, (map #{"a"} ["a" nil])
and (map #{"a" nil} ["a" nil])
both return ("a" nil)
#2016-06-1411:37minimalThe not-found
value of a set is also nil so it is troublesome if you donāt expicitly check using contains?
.#2016-06-1411:38skapoori'm using sets in specs to define an enum of values with the possibility of nil.... so it looked like a convenient way. guess, I'll have to either use s/or nil? ...
or #(contains?)
#2016-06-1411:39minimalor you can use a defualt value that isnāt nil. (#{} 1) => nil
(get #{} 1 :not-found) => :not-found
#2016-06-1411:40minimalYeah itās tricky#2016-06-1411:41skapoorok thanks. i'm wondering if I should file an issue for this on jira..#2016-06-1411:42minimalI donāt think itās an issue#2016-06-1411:42skapooronly because the contains?
behavior is right but when a set is used as a function call it's not.#2016-06-1411:43minimalUsing set as a predicate is a convenience but you are relying on an implicit conversion to boolean based on the value in the set#2016-06-1411:45gfrederickseither choice is surprising#2016-06-1411:46gfredericksto somebody#2016-06-1411:46gfredericksdepending on whether you expect the set to be treated specially or to be treated like a predicate#2016-06-1411:46skapoorthe implicit conversion to boolean is being done by the internal clojure implementation..#2016-06-1411:49skapoor@minimal: but I get it, it won't be considered a defect. so I'll just use a different way. thanks!#2016-06-1415:20Alex Miller (Clojure team)@mfikes from your question way back on describe - s/form is useful for distinguishing these#2016-06-1415:23Alex Miller (Clojure team)@skapoor: you could also wrap s/nilable around the set (s/nilable #{"a"})
#2016-06-1417:39arohner@alexmiller: whatās the rationale behind instrument only checking :args?#2016-06-1417:53Alex Miller (Clojure team)shift in perspective - instrument is about checking invocations are correct, not verifying the function itself is correct. (this is similar to how only the :args are checked in macro fdefs). The check and test functions are for verifying functionality of the code.#2016-06-1418:05bfabryis there still a way to set up your app so every spec is checked on every invocation?#2016-06-1418:41arohner@alexmiller: this seems to severely hamper specāing non-pure functions (i.e. things that are difficult/impossible to write generators for)#2016-06-1418:42arohnerand IMO, validation and generative testing are still too tightly coupled#2016-06-1418:49Alex Miller (Clojure team)@bfabry: no, but that was not really the intention of instrument
, which is about verifying that callers of a function are calling it correctly. If you want to test the the behavior of your functions are in accordance with your specs, you should use the spec.test functions to test your functions.#2016-06-1418:51bfabry@alexmiller: sure, and no doubt were we to use spec we would use the spec.test functions. but like @arohner mentioned not all functions are pure, and if I'm going to write all those specs then I might as well get some extra value from them for free by having them always be checked in lower environments where I do not care about performance#2016-06-1418:51Alex Miller (Clojure team)@arohner: there are still more things coming that will help with verifying aspects of non-pure functions#2016-06-1418:53arohner@alexmiller: one nice feature of schema is that you can choose to use validation at e.g. user-input boundaries, in production. Instrument seems more designed for dev-time atm#2016-06-1418:53Alex Miller (Clojure team)@arohner spec does not remove the need to write tests for your functions. those tests can use invoke specs to validate as appropriate#2016-06-1418:54Alex Miller (Clojure team)@arohner: instrument is only designed for dev time#2016-06-1418:54Alex Miller (Clojure team)you should not instrument in production#2016-06-1418:55Alex Miller (Clojure team)you can choose to explicitly conform with spec at boundaries if you like#2016-06-1418:56Alex Miller (Clojure team)there will be a conform-ex
that validates or throws if not, not quite in yet#2016-06-1419:03bfabryif I explicitly conform, then it will happen in production, when I only want it in staging. it also adds a whole bunch of boilerplate to every single function. I don't really understand the reasoning here, being able to reuse the :ret and :fn specs for extra checking when performance isn't a consideration just seems like an obvious win. and I mean, I definitely can still do that, writing my own macro that wraps all functions or whatever, but it sounds like I won't be the only one#2016-06-1419:10Alex Miller (Clojure team)there is an assertion facility still coming as well#2016-06-1419:11Alex Miller (Clojure team)the point is that checking ret/fn every time should be redundant to what you have (presumably) already confirmed in testing - that your function works.#2016-06-1419:16bfabryI'm maybe a bit skeptical that adding generative testing (while definitely awesome) is going to straight away mean I stop writing functions that produce unexpected outputs when they encounter production data. and I'm a big fan of fail fast with a good error message when that does happen#2016-06-1419:19Alex Miller (Clojure team)you (will be) able to assert that return (if instrumented at dev time) or choose to explicitly validate it at production time if you want#2016-06-1419:20Alex Miller (Clojure team)it's unclear to me if you're talking about dev or prod#2016-06-1419:20bfabryactually talking about master/staging#2016-06-1419:21Alex Miller (Clojure team)fair enough - so you can turn on instrumentation in staging#2016-06-1419:21Alex Miller (Clojure team)that will check args on functions#2016-06-1419:21bfabryon my laptop/travis I run unit and generative tests, in master/staging the application runs "production-like" but with assertions turned on for :args :ret :fn, production the app runs with no assertions <-- this is my ideal scenario#2016-06-1419:22Alex Miller (Clojure team)there will be an assertion facility that you can use to (explictly) check ret/fn for instrumented functions#2016-06-1419:22danstoneAs the channel is a little bit louder this evening, I thought I'd pose a question I asked yesterday again:
What is the intended usage of :fn
in fdef
- In the docs it says something like 'relationships between args and the return'. Is the idea here to restrict it to type-y properties (e.g arity 1 of map returning a transducer rather than a seq)?
The reason I ask is it's possible to define many more general properties of functions as part of the spec. As spec gets richer I imagine it may be possible to auto-generate the code for many properties (idempotency is easy if you have spec'd the args)#2016-06-1419:23Alex Miller (Clojure team)I think what I'm talking about is close to that, but varies in that ret/fn are not automatically checked but require an explicit assert (which is not active in production)#2016-06-1419:29bfabryright. and so I'll probably end up writing a macro that wraps every function to add that explicit assert, and I've got a feeling that a whole lot of people will do that. we use plumatic/schema atm which checks arg/return values when validation is turned on, and I'd say the same number of bugs are caught by the return value checking as the arg value checking. aaaanyway, writing the macro is nbd, and maybe it'll turn out I don't actually need it or I'm the only person who does š#2016-06-1419:41Alex Miller (Clojure team)one question I have is whether you're getting the same level of generative testing from schema that you can get from spec (that is, whether more of those bugs could/should have been caught earlier)#2016-06-1419:41Alex Miller (Clojure team)also keep in mind that the return value often is an input to another function, which can check its args#2016-06-1419:44bfabryno, we're definitely not. but like I said I'm just a bit skeptical that generative testing will suddenly mean these issues disappear, and it costs me like an extra $5 per month to run extra validation in the staging environment so why not? the return value will likely be the input value to another function, but maybe that function doesn't have a spec yet, because we didn't feel it was worth the time to write yet, or maybe it's too broadly defined etc. If I believed I could perfectly specify every function up front so that bugs were impossible I'd be writing haskell š#2016-06-1419:48Alex Miller (Clojure team)maybe you should just write your code without the bugs?#2016-06-1419:48Alex Miller (Clojure team)š#2016-06-1419:53bfabryhaha#2016-06-1420:49wilkerluciohey people, does anyone here found/created a generator for cljs to create strings from regexps? I was hoping to use the string-from-regexp
from test.chuck
but I just realised it's implementation is for CLJ only#2016-06-1420:56Alex Miller (Clojure team)I would ask @gfredericks#2016-06-1421:09wilkerluciothanks Alex, I opened an issue on test.chuck
, he will see it I think š#2016-06-1421:09wilkerlucioone more thing, given I have my fdef
all spec set, I remember seeing a simple command to test that function but I'm not finding it, what's the simplest way to run generative specs on a function?#2016-06-1421:15Alex Miller (Clojure team)clojure.spec.test/check-var#2016-06-1421:21wilkerlucio@alexmiller: thanks, would you consider adding that to the clojure spec guide?#2016-06-1421:21Alex Miller (Clojure team)yeah, that's on the todo list - I was actually working on some alpha6 related updates right now#2016-06-1421:21wilkerlucionice š#2016-06-1421:51ikitommiIf I have understood correctly, the registry doesnāt have any tools for handling duplicate definitions?#2016-06-1421:53ikitommiso, if there are multiple definitions fos person/id
, the last one stands?#2016-06-1421:56bfabry@ikitommi: yeah just replaces https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L261#2016-06-1422:10ikitommiIs this good? should the specs be immutable by default? If there are name clashes, depending on the import order of namespaces, the specs might mean different thing.#2016-06-1422:11ikitommior some hooks for the registry to resolve those clashes.#2016-06-1422:14wilkerlucio@ikitommi: this is the reason they are encouraging namespaced keywords I think, do you have a situation where the same namespace is loaded on multiple files?#2016-06-1422:15bfabryfunctionally the same as multiple (def
's isn't it?#2016-06-1422:20bsimadoes anyone have examples of using fdef
and check-var
? I'm having trouble getting it to work in my test suite#2016-06-1422:23ikitommi@wilkerlucio: true that - the specs must have a namespace, but it doesnāt have to be a clojure namespace. One might have multiple :order/id
s in a large system. Should not, but could.#2016-06-1422:23wilkerlucio@bsima: check this snippet, may help you:#2016-06-1422:25wilkerlucio@ikitommi: that's some sort of thinking shift, moving from those to fully qualified, there are going to be some nice helpers to deal with longer namespaces in 1.9#2016-06-1422:26wilkerlucioyes, the problem exists like you said, but it's the same for def
as mentioned by @bfabry, I guess it's just about people starting moving towards fully qualified namespaces to avoid name clashes, I believe when it's the norm will be very positive for everyone#2016-06-1504:44jimmyhi guys, how do we use variable inside s/keys
like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
#2016-06-1509:54danstone@nxqd: I don't think we can because s/keys
is a macro. Though I'm not sure if this is worth it for keys, as it doesn't capture predicates, I would expect it to be a function. As far as I can tell the only thing macro'ey it does is look at forms like (or ::foo ::bar)
.#2016-06-1517:50Alex Miller (Clojure team)At the moment you can't do this except via eval or something#2016-06-1517:50Alex Miller (Clojure team)But there has been talk of adding a fn entry point that would allow it#2016-06-1518:57seancorfieldWith check-fn
and check-var
, is there a way to get a "nice" explanation of the failures, like explain
produces?#2016-06-1518:57seancorfieldI'm trying to see how the changes in Alpha 6 affect the testing workflow. #2016-06-1519:47eggsyntax@nxqd: (somewhat belatedly) you can replace the var with the function that populates it (it can cache its results, if it's an expensive call), if it's a var that it makes sense to populate at macro eval time.#2016-06-1519:48eggsyntaxUnrelated: test.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one.#2016-06-1519:59eggsyntaxThe only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen)
, but it's pretty ugly to put in a generator and then ignore what it generates.#2016-06-1520:52jannisFolks, is it a good idea to define fdef
function/macro specs next to the actual function/macro definition or would you rather define them in separate my-project.specs
kind of namespace?#2016-06-1520:53jannisI'd like to associate most of my functions with specs for automatted random function testing but at the same time I don't want to clutter my code base with specs too much.#2016-06-1520:53jannisI guess there are no best practices established yet?#2016-06-1520:54eggsyntaxI've been debating that myself, haven't come to any particular conclusion.#2016-06-1520:55eggsyntaxCurrently defining them on their own because it's still fairly experimental for us...but as we fully integrate them, I'll definitely consider moving them to live next to what they're speccing.#2016-06-1521:11seancorfieldI think weāll have data specs in separate namespaces. Not sure yet about function specs. Part of me would like them above function definitions ā that seemed natural for when we used Schema and core.typed.#2016-06-1521:12seancorfieldWe went back and forth between core.typed and Schema several times before we abandoned them. With spec being in core, I think weāre more likely to stick with it.#2016-06-1521:12jannisYeah. It also means you don't have to require the function namespaces and the function spec namespaces when you want to test functions.#2016-06-1521:13seancorfieldAlthough, https://github.com/clojure/java.jdbc/blob/master/src/test/clojure/clojure/java/test_jdbc.clj#L28-L32#2016-06-1521:13seancorfieldhttps://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2016-06-1521:14seancorfieldThere the function specs are in a separate clojure.java.jdbc.spec
namespace, below the data specs.#2016-06-1521:14seancorfieldBut that was mostly to ensure pre-1.9 code can still use the library#2016-06-1521:15jannisHere's my first attempt at writing function specs alongside the actual functions: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L9#2016-06-1521:18jannisIt would feel a little more natural to define the function spec like`:pre` and :post
, e.g.`(defn my-fun [arg1 arg2] {:spec {:args ... :ret ... :fn ...}} <body>)` or something like it, although there are good reasons to keep them separate.#2016-06-1523:22bbrinck@jannis: What do you think about putting the fdefs before the function? IMHO, that's a little clearer. I like having the function specs near the functions since that provides documentation when reading the function. Not sure about data specs, though.#2016-06-1523:29bbrinckor rather, put each fdef before its associated fn#2016-06-1523:45bfabrywhat have I messed up here...?
Clojure 1.9.0-alpha7
Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17
Exit: Control+D or (exit) or (quit)
Commands: (user/help)
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Find by Name: (find-name "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Examples from : [clojuredocs or cdoc]
(user/clojuredocs name-here)
(user/clojuredocs "ns-here" "name-here")
boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (require '[clojure.spec.test :as t])
nil
boot.user=> (defn foo [x] (inc x))
#'boot.user/foo
boot.user=> (s/fdef foo :args (s/cat :x integer?) :ret integer?)
boot.user/foo
boot.user=> (t/check-var foo)
java.lang.IllegalArgumentException: No :args spec for
#2016-06-1523:51bfabryboot.user=> (s/fn-spec foo)
nil
boot.user=> (doc foo)
-------------------------
boot.user/foo
([x])
Spec
args: (cat :x integer?)
ret: integer?
nil
boot.user=> (s/fn-spec 'foo)
#2016-06-1600:35seancorfield(t/check-var #āfoo)
ā it accepts a Var
.#2016-06-1600:36seancorfieldSimilarly (s/fn-spec #āfoo)
ā although the result of that is less useful to print since it is an object.#2016-06-1600:37seancorfieldYou could also do (t/check-fn foo (s/fn-spec #āfoo))
#2016-06-1600:38seancorfieldcheck-fn
is so you can provide an (s/fspec :args ā¦ :ret ā¦ :fn ā¦)
for an arbitrary function without fdef
.#2016-06-1601:23leifpHi, all. I hacked up specs for most fns in clojure.core and 1/3rd of the macros (WIP). I don't know much about clojure.spec (of course), though, so it wasn't the good kind of hacking,. Hold your nose if necessary: https://gist.github.com/leifp/abe50082f6baa0063f8b7840e80657af#2016-06-1607:08bfabryahhhhh thanks @seancorfield#2016-06-1609:58jannisShouldn't ((instrument #'my-fun) :foo)
throw an exception with an explanation if my-fun
has a spec defined for it with fdef
and if the returned value does not conform to the spec of my-fun
?#2016-06-1609:59jannisInstead it just returns :clojure.spec/invalid
.#2016-06-1610:04jannisOh, that's because that's what my-fun
returns in this case. So it doesn't even fail, it just returns its value, even if that doesn't conform to the function spec.#2016-06-1610:41patrkrisI see clojure.spec uses "named arguments" in several places (e.g. (clojure.spec/fdef :args ... :ret ....)
). Is there any convention as to when to use named arguments vs. a map? Maybe named arguments are primarily used for macros?#2016-06-1611:10pithylessHi, all. Has someone come across the need to have an s/keys
that is more strict (does not allow extra keys)?#2016-06-1611:14jrychter@pithyless: I have. I am guessing Rich thought it isn't a good idea, as in the long term it could hurt expandability and composability. But I still think we should be able to do it in some cases.#2016-06-1611:22pithylessI'm sure it's been brought up internally; I wonder how likely it would be to get an :only
option added to s/keys
.#2016-06-1611:56patrkris@pithyless: some semi-related discussion here: https://groups.google.com/forum/#!topic/clojure/UVgXXcIxhJQ#2016-06-1612:00jrychterAs I suspected ā "spec embraces the idea of open maps". But there are cases where you don't want open maps. A good example is the current world of lein-cljsbuild, where I regularly pull my hair out because it isn't clear if a particular setting is in the right place. Extra keys do no harm, so people tend to put config options all over the place, and you end up with a terrible mess. Figwheel recently started doing some great work towards fixing this ā and this kind of config checking should really be "these keys, and these keys only".#2016-06-1612:02pithylessI've got some half-baked solution with clojure.set/difference
and keys
, but I wish it were simpler.#2016-06-1612:05jrychterI am hoping someone will write a validations library similar to bouncer, based on clojure.spec
.#2016-06-1612:10jannisThings are getting a little meta over here... I'm spec'ing out a data format and a parse function that first uses clojure.spec/conform
to validate and normalize the input and then passes it on to a transformation function. For that transformation function the input data format is the output of conform
, so I now have a spec for what conform
returns as well. š#2016-06-1612:46ghadi@pithyless: https://clojurians.slack.com/archives/clojure-spec/p1465423760000580#2016-06-1612:53pithyless@ghadi: Thanks for linking to the archives. I understand why the default is what it is, my use case falls clearly in the "don't want to accidentally be leaking stuff" camp. But it's a minor thing... clojure.spec
is nice š#2016-06-1613:16dominicm@jrychter: As someone working on this: spec isn't suited to the task.#2016-06-1613:17jrychter@dominicm: oh š I was hoping that one could build a validation system based on what explain-data
returnsā¦#2016-06-1613:17dominicmThat's the easy bit.#2016-06-1613:19jrychterI currently use bouncer, but I have to work around its problems, and as I migrated from schema to spec, I was hoping I could get rid of it entirely.#2016-06-1613:19dominicm@jrychter: When you do user registration validation, you need to check email uniqueness. So that requires 2 things:
1. Data to validate, the email
2. Sideband data, the database (connection).
Spec has a funnel for 1. Not one for 2.#2016-06-1613:20dominicm@jrychter: https://github.com/leonardoborges/bouncer/issues/43 š I know bouncer. It would be nice.#2016-06-1613:20ghadispec has nothing to do with integrity checks (#2)#2016-06-1613:21dominicmYour only options for validating in that way, and neither of these are particularly spec-y, are dynamic vars, or the weird macro I started working on, and gave up on because it was a big ball of mud.#2016-06-1613:21jrychterAs for (2) above, I currently use global state encapsulated using mount.#2016-06-1613:22dominicm@jrychter: You may not have as much trouble.#2016-06-1613:22dominicmHowever, I land on the weavejester side of the camp. š#2016-06-1613:22jrychterMeaning, some of my bouncer validations actually use database or even (gasp) network connections.#2016-06-1613:22dominicmYep, it's a necessity.#2016-06-1613:22dominicm@ghadi: What kind of check does spec do?#2016-06-1613:22jrychterEU VAT numbers are an example.#2016-06-1613:28dominicm> weavejester side of the camp
I should clarify this means that I use component.#2016-06-1613:48dominicm@ghadi: I'm mostly interested in the terminology and reading on the subject.#2016-06-1615:32seancorfieldAs I recall, the discussion about closed maps ended with consensus that using select-keys
was appropriate for "not leaking additional keys", but there wasn't much consensus on validation of closed maps. #2016-06-1615:33seancorfieldI got the impression @richhickey is "considering" adding support for it due to repeated requests from the community but he doesn't consider it a good idea. #2016-06-1615:37seancorfieldI think designing systems to simply ignore extra keys is more robust, but I also accept there are cases where that is difficult (the example I gave is clojure.java.jdbc/insert!
where a map is converted to a SQL insert statement without reflection and therefore extra keys become extra columns and will be rejected by the database; the alternative is to introspect the database and ... well, do what? Silently ignore the extra columns? Throw a different exception?)..#2016-06-1615:39seancorfieldAnd there in lies the issue: the behavior for extra keys is not knowable at that level - it would have to be an application-level decision (and the application has the means to do that reflection and call select-keys
if it wants).#2016-06-1615:41seancorfieldWhich is why I side with Rich that it's not clojure.spec
's job to directly support restricting keys, since it shouldn't be the norm, and you can do it via custom predicates already if you really have one of those odd cases where you really do need to check. #2016-06-1615:42donaldballGoes back to my point that if you want that level of validation, just specifying that the db arg to insert! is a Connection or a Connectable or whatever is not sufficient; it would need to specify that the database in question has such and such a schema#2016-06-1615:44seancorfieldRight, and that's a whole different thing. java.jdbc
provides metadata/reflection APIs but doesn't use them internally (for performance mostly),#2016-06-1616:46bhaumanA strict key set constraint can make a lot sense depending on the situation. This depends on the api of course, but I can't imagine a higher level api, that third parties are going to rail against, not having tighter key set constraints.#2016-06-1616:48bhaumanThat being said, you can compose a strict map key set and description with spec fairly handily.#2016-06-1616:51bhaumanSpec is beautiful.#2016-06-1617:48wilkerlucio@pithyless: if you really want to get just the strict keys from a map, I would suggest using select-keys
as people mentioned before, as a plus, I wrote a simple utility that can extract the keys from a keys spec, it's still a naive implementation (it doens't consider req-un
or opt-un
) but can be starting point if you want to go further on the idea:#2016-06-1617:52wilkerlucioso I believe if extra keys are harmful on your case, you can use this kind of trick to remove unwanted ones, so we need to force the restriction of the keys, but just eliminate the ones you can't deal with (and only if you can't silently pass it on)#2016-06-1617:59seancorfield@wilkerlucio: The s/def
for ::name
and ::email
donāt do anything in that exampleā¦?#2016-06-1618:01seancorfieldAre you doing something else to check the values of the ::my-aggregate
map conform to those specs, by convention (based on the key names)?#2016-06-1618:01wilkerlucio@seancorfield: the idea here was just to show an example of how to do a select-keys
while reusing the specs, you are supposed to call s/valid?
yourself before doing the strict-keys
on this case#2016-06-1618:02seancorfieldOK, so youāre relying on convention that the (qualified) key names are the same as the specs that apply to their respective values?#2016-06-1618:03wilkerlucioyes#2016-06-1618:03wilkerluciobut like I said, this is a naive implementation, if you really wanna rely on it, will need more work there#2016-06-1618:04seancorfieldFYI, instead of (->> (s/form spec) (next) (partition 2) (map vec) (into {}))
you could just do (->> (s/form spec) next (apply hash-map))
#2016-06-1618:05wilkerlucio@seancorfield: thanks for that, I'll update the snippet š#2016-06-1618:11bfabryhas anyone come up with a good way of speccing non-keyword maps with known keyval pairs?#2016-06-1618:30zaneDoes spec just not play well with tools.namespace
?#2016-06-1618:30zaneOr with the REPL?#2016-06-1618:32zanedev=> (defn f [x] (inc x))
#'dev/f
dev=> (s/fdef f :ret pos?)
dev/f
dev=> (s/instrument #'f)
#'dev/f
dev=> (f 1)
2
dev=> (f -3)
-2
#2016-06-1618:33bfabryinstrument doesn't check :ret or :fn in the latest alpha#2016-06-1618:33bfabryonly :args#2016-06-1618:33zaneOuch.#2016-06-1618:33zaneIs that going to change?#2016-06-1618:33zaneDid it check :ret
and :fn
in previous alphas?#2016-06-1618:33bfabryyes#2016-06-1618:34zaneā¦ Huh.#2016-06-1618:34bfabrythe reasoning is that :ret and :fn are for checking whether the function is correct, which should happen when you're using the functions in clojure.spec.test. :args are for checking the function was invoked correctly#2016-06-1618:38zaneSo, we have a function that reads environment variables. That function has a :ret
spec on it that validates that required environment variables are set and have valid values.#2016-06-1618:38zaneMy understanding now is that instrument
will not help me here and I should use s/conform
?#2016-06-1618:40bfabryyeah, s/conform or s/valid? or whatever explicitly. there's also an s/conform-ex coming iirc#2016-06-1618:41zaneWhere should I look for info on s/conform-ex
?#2016-06-1618:41bfabryin a subsequent release š#2016-06-1618:43bfabryI'm sure it's coming soon, they've been evolving very rapidly#2016-06-1618:44zaneUnderstood.#2016-06-1618:44zaneDo you know what s/conform-ex
going to do?#2016-06-1618:45bfabryconform or throw an exception on failure I assume#2016-06-1618:45zaneAh, I see.#2016-06-1618:46settingheadlooks like there was a commit 3 hours ago:
https://github.com/clojure/clojure/commit/aa9b5677789821de219006ece80836bd5c6c8b9b#2016-06-1618:54leifpAs mentioned before, I've (roughly) spec'ed a good chunk of clojure.core. I made that an actual repo in case someone wants to test and/or beautify them: https://github.com/leifp/spec-play#2016-06-1618:57zane@bfabry: When is the :ret
argument to fspec
ever used, then?#2016-06-1618:58bfabrywhen clojure.test.* functions run generative tests#2016-06-1618:59zaneGot it. So only for generative testing.#2016-06-1618:59zaneOof.#2016-06-1619:05zaneThat seems like a very weird design choice to me.#2016-06-1619:08bfabryI'm not real sold on it either#2016-06-1619:08bfabrybut I haven't used spec on anything big enough to be confident#2016-06-1619:18robert-stuttafordso, i'm new to test.check in general. anyone know how i might generate a set of keywords, from a known set of possible keywords?#2016-06-1619:19robert-stuttaforde.g. i have #{:a :b :c :d :e}
and i want generated subsets of same#2016-06-1619:20bfabry(s/exercise #{:foo :bar})
=>
([:bar :bar]
[:bar :bar]
[:bar :bar]
[:bar :bar]
[:foo :foo]
[:foo :foo]
[:bar :bar]
[:foo :foo]
[:bar :bar]
[:foo :foo])#2016-06-1619:21bfabrythis is probably better#2016-06-1619:21robert-stuttafordinteresting#2016-06-1619:21bfabry(s/exercise (s/coll-of #{:foo :bar} #{}))
=>
([#{} #{}]
[#{} #{}]
[#{} #{}]
[#{:bar :foo} #{:bar :foo}]
[#{} #{}]
[#{:bar :foo} #{:bar :foo}]
[#{:bar :foo} #{:bar :foo}]
[#{:bar} #{:bar}]
[#{:bar :foo} #{:bar :foo}]
[#{:bar :foo} #{:bar :foo}])#2016-06-1619:22robert-stuttafordi'm modelling Magic the Gathering cards as an exercise#2016-06-1619:22bfabryhaha, nice#2016-06-1619:22robert-stuttaford(s/def ::type #{:land :creature :artifact :enchantment :sorcery :instant :planeswalker})
(s/def ::types (s/with-gen
(s/and set? (s/+ ::type))
#( ? )))
#2016-06-1619:23bfabryanyway yeah s/exercise generates data that suits a spec, a spec of "sets of these keys" is (s/coll-of #{:keys} #{})#2016-06-1619:23robert-stuttaford::types
works, but it can't be generated because of how s/and
generators work: generate for the first and discard anything that doesn't satisfy the rest of the ands#2016-06-1619:23bfabryI think you just want (s/coll-of ::type #{})#2016-06-1619:24robert-stuttafordindeed, thank you#2016-06-1619:25bfabry(s/def ::type #{:land :creature :artifact :enchantment :sorcery :instant :planeswalker})
=> :kafka-google-connector.runner/type
(map first (s/exercise (s/coll-of ::type #{})))
=>
(#{}
#{:planeswalker}
#{:artifact}
#{:land}
#{:creature :planeswalker}
#{:sorcery}
#{:land :planeswalker :sorcery}
#{:instant :enchantment :land :planeswalker}
#{:instant :enchantment :creature :land :planeswalker :sorcery}
#{:land :sorcery})
#2016-06-1619:25bfabryman.. that's pretty neat#2016-06-1619:26robert-stuttafordok. my next question (which is the thing i really want to solve, now that i've softened you up š ) is how might i write a generator when i'm using s/and
on two s/keys
specs?#2016-06-1619:26robert-stuttaford(s/def ::base-card (s/keys :req-un [::name ::types ::metadata]
:opt-un [::sub-type ::legendary? ::world?]))
(s/def ::cost string?) ;; todo
(s/def ::spell (s/and ::base-card (s/keys :req-un [::cost])))
#2016-06-1619:27bfabryI've not actually looked into generators sorry#2016-06-1619:27robert-stuttafordah š worth a try!#2016-06-1619:27robert-stuttafordit's a super-interesting problem to solve#2016-06-1619:27robert-stuttafordto me, anyway#2016-06-1619:47wilkerlucio@robert-stuttaford: just a suggestion, since you are modeling something new, maybe would be better to use the namespaced keys instead of clear ones, with namespaced keys you can for example validate a map keys even if you don't know the aggregate name for it#2016-06-1620:18bhaumanSpit-balling on strict keys just for the fun of it. I would love some feedback.#2016-06-1620:27leifpbhauman: What are the advantages of this vs. just (s/& (s/keys ...) #(only-these-keys % [:k ...]))
? Or a macro that expands into that.#2016-06-1620:29bhauman@leifp: the only interesting thing here is the explain data#2016-06-1620:29bhaumanwhere it points exactly to the key that failed#2016-06-1620:30bhaumanIn: [:there] val: :there fails spec: :howdy/fine at: [:there] predicate: #{:builds :server-port :server-ip :http-server-root}
#2016-06-1620:34bhaumanit will create explain data for all the keys that failed#2016-06-1620:51bfabry@robert-stuttaford: I wrote this which works. I don't know how sane it is. my guess is "not very"
(defmacro extend-keys [spec-name & {:keys [req-un opt-un]}]
(let [spec-m (apply hash-map (rest (s/form spec-name)))]
`(s/keys :req-un ~(into (:req-un spec-m) req-un)
:opt-un ~(into (:opt-un spec-m) opt-un))))
=> #'kafka-google-connector.runner/extend-keys
(s/def ::spell (extend-keys ::base-card :req-un [::cost]))
=> :kafka-google-connector.runner/spell
#2016-06-1621:11leifpbhauman: Hmm... I guess there is no explain
equivalent of conformer
or with-gen
, so I can't really think of another way to do it than reifying Spec. Your impl. looks fine, but it doesn't seem to explain the extra keys if one of the required keys fails its spec.#2016-06-1621:13bhauman@leifp: I haven't looked at that, must be because of the s/and#2016-06-1621:14bhaumanbtw I've iterated on it a bit#2016-06-1621:16bhauman@leifp: what do you mean by extra keys? you mean in the explain data?#2016-06-1621:23leifp@bhauman:
user=> (s/explain (strict-keys :req [::r]) {::r "bad" ::extra 2})
In: [:user/r] val: "bad" fails spec: :user/r at: [:user/r] predicate: number?
In: [:user/extra] val: :user/extra fails at: [:user/extra] predicate: #{:user/r} ;; <<< expected, not present
#2016-06-1621:23leifpThat output line was expected and not present, I mean.#2016-06-1621:24bhaumanoh yeah it short cuts#2016-06-1621:27bhaumans/and short cuts#2016-06-1621:27bhaumanso that makes sense#2016-06-1621:33bhaumanI would need to compose over the keys-spec to get that behavior#2016-06-1707:04Oliver GeorgeHi Specy Specers. What's the most human friendly way of viewing the output from check-var? I see that reporter-fn can be provided, is there a commonly used one? (I'm using CLJS so perhaps that makes a difference).#2016-06-1709:27jrychterI find myself writing lots of (s/and string? seq)
to specify non-empty strings. Also, I'm missing a predicate for strings of length from n to m (analogous to int-in-range?
).#2016-06-1709:34jannisIs it possible that s/with-gen
alters the spec it defines a generator for? I have a simple (s/cat :base ... :children ...)
spec that works fine but as soon as I wrap it in (s/with-gen <spec> #(gen/tuple (s/gen ...) (s/gen ...)))
, data that would previously conform to the spec now becomes invalid.#2016-06-1709:35jannisNote: I', not generating the data using the generator yet. I'm using hand-written data.#2016-06-1709:42jannisHere's a minimal example: https://gist.github.com/Jannis/5dcc91473f20861d154dc8be2fff2bfd#2016-06-1715:53seancorfield@olivergeorge: there's a discussion about that on the main Clojure mailing list. I'm very interested in the answers to that question. #2016-06-1716:37leifp@jannis: It looks like it's introducing a new regex context like spec
does:
user=> (s/explain (s/cat :x ::pair) '[a [b c]])
Success!
user=> (s/explain (s/cat :x ::pair-with-gen) '[[a [b c]]])
Success!
#2016-06-1717:34sparkofreasonI'm working on some code that does simulations over state machines. The transition functions tend to have a lot of detailed conditional logic, and as a result test.check doesn't seem to be a great fit for validating the functions (random inputs tend to be ignored or lead to errors, and writing generators to provide valid input is essentially the same as writing the state machine model). When instrument-all
checked the return value it was very useful, since I caught a lot of errors of omission, misspelled keywords, etc. I'd like to suggest we have an option to check :ret
and :fn
for cases like this.#2016-06-1717:50seancorfieldYeah, whilst I agree in principle with the justification @alexmiller offered as to why :ret
and :fn
checking was removed in Alpha 6, I also agree that there is potentially a lot of value in having the option to be able to instrument functions in a way that does conform
the result at least.#2016-06-1717:50seancorfieldThis is the commit that changed the behavior: https://github.com/clojure/clojure/commit/30dd3d8554ff96f1acda7cbe31470d92df2f565a?diff=split#2016-06-1717:55sparkofreasonThanks, I may use that to hammer out my own version of instrument
for now.#2016-06-1717:56bfabryI think there's gotta be something coming for spec'ing impure functions that will cover this#2016-06-1717:56bfabryotherwise you could never spec them, really, or the spec would be pointless#2016-06-1718:05Alex Miller (Clojure team)@seancorfield: Rich has some ongoing work, I'm not sure what the endpoint will be on this. you might have noticed that explain-out
was made public today in master#2016-06-1718:06Alex Miller (Clojure team)@bfabry: not every function is a great candidate for generative testing via spec (but the spec may still be useful for docs or other purposes)#2016-06-1718:07eggsyntax@alexmiller: where was the explanation @seancorfield mentioned of why :ret
and :fn
checking was removed? Iām curious to read it.#2016-06-1718:08Alex Miller (Clojure team)mailing list#2016-06-1718:08Alex Miller (Clojure team)https://groups.google.com/d/msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ#2016-06-1718:08eggsyntaxThanks š#2016-06-1718:09Alex Miller (Clojure team)@seancorfield: I think Stu is looking at some testing-related mods too btw#2016-06-1718:27seancorfieldre: explain-out
ā I already added a comment on that commit thanking him for that š#2016-06-1718:29seancorfield@alexmiller: I really do appreciate the steady stream of alpha builds so we can all try this stuff out and provide feedback.#2016-06-1718:30Alex Miller (Clojure team)I'm sure there will be more :)#2016-06-1718:34seancorfieldHaving an option on instrument
to use the old version of spec-checking-fn
with :ret
and :fn
conforming would be very nice. I think checking just :args
is the right choice for most cases of instrumentation, but I think being able to "fully instrument" certain functions would be very valuable ā especially for functions that cannot easily be tested the generative way.#2016-06-1718:36seancorfieldFor example, working on java.jdbc
ās specs, they canāt reasonably be tested generatively because many of them are side-effecting (updating the database) and writing generators that conformed to the database schema would be ā¦ a huge amount of work, if itās even feasible (e.g., unique key constraints etc?).#2016-06-1718:36seancorfieldSo losing the ability to conform the :ret
and :fn
specs there is kind of a big deal, IMO.#2016-06-1718:39seancorfieldIāll be interested to see how this all ends up since any given code base is going to have a mix of functions that can reasonably be tested generatively and functions that canāt, so (clojure.spec.test/run-all-tests)
needs a way to distinguish those, right?#2016-06-1718:51seancorfield(mind you, right now I canāt run generative testing on java.jdbc
because the system doesnāt know how to generate a java.sql.Connection
ā¦ which might be an interesting exercise š )#2016-06-1718:52seancorfield(and Iād also need a generator for a java.sql.PreparedStatement
)#2016-06-1718:53seancorfieldWhat is the recommendation for stuff like that? How would you even write a generator for some of these Java objects?#2016-06-1719:34tomcWould anyone be willing to offer some guidance on conventions for attributes shared by entities of different types? For instance, I have "question" and "survey" entities, each of which can have names. Right now I'm using (s/def :entity/name string?)
and entities of either type can have an :entity/name attribute along with their type-specific attributes. The alternative of course is to define both :question/name
and :survey/name
. I haven't seen the :entity/attr pattern elsewhere, and I'm wondering whether there's a reason for that.#2016-06-1720:03wilkerlucio@tomc: I believe you can share the attribute as long as the semantic is the same, for example, a car name may have a different semantic from a person name, but that will depend on the requirements on your system#2016-06-1720:09tomc@wilkerlucio: thanks a lot, that's helpful.#2016-06-1720:12gphilippIām trying to generate dates within a range with spec, and Iām stucked with this piece of code which generates #inst whose year is above 9999, making them unrecognized when i try to def them manually afterwards : (gen/sample (s/gen inst?) 60)
#2016-06-1720:13gphilippex of data generated: #inst"26138-06-03T15:09:43.670-00:00
#2016-06-1720:14gphilipp(def d1 #inst"26138-06-03T15:09:43.670-00:00")
CompilerException java.lang.RuntimeException: Unrecognized date/time syntax: 26138-06-03T15:09:43.670-00:00, compiling:(/Users/gilles/dev/try-spec/src/try_spec/core.clj:23:46)
#2016-06-1720:18wilkerlucio@gphilipp: you can try something like this:#2016-06-1720:18wilkerluciothe first number, 100000
is a cap to limit the increment, and the 1465391285642
is the ms
for a start date, adjust those to met your needs#2016-06-1720:19wilkerlucioah, and this example is for CLJS, please change the Date initialisation if you are using on Clojure#2016-06-1720:25gphilipp@wilkerlucio: thx, I will try this#2016-06-1720:28leifp@gphilipp: There is also an clojure.spec/inst-in
macro.#2016-06-1720:28seancorfieldThe default 100 tests for test.check
can take a really long time on some fairly simple looking specs...#2016-06-1720:29seancorfieldā¦running 50 tests wasnāt too bad but it seems to be taking more than linearly longer to do 5, 10, 25, 50, 100...#2016-06-1720:31leifp@seancorfield: Yeah, as well as being very dependent on ordering. The caveats about "generate-and-test" style vs. constraint satisfaction are in full effect here.#2016-06-1720:32seancorfieldOn the plus side, I figured out how to write generators for stuff like java.sql.Connection
etc š#2016-06-1720:32seancorfieldAnd I also finally figured out how s/keys
and the test.check
integration hang togetherā¦ which answered a question Iād asked someone else here before I understood what was going on...#2016-06-1720:35gphilippThanks @leifp, (gen/sample (s/gen (s/inst-in #inst "2016-01-01" #inst "2016-12-31")) 100)
did the trick#2016-06-1720:39leifp@gphilipp: The end date is exclusive, remember.#2016-06-1720:39gphilippah, correct#2016-06-1720:40gphilippI still wonder why (gen/sample (s/gen inst?) 60)
generates invalid instants.#2016-06-1720:44leifpSee? Generative testing is going to help us avoid the Year 26k Problem.#2016-06-1721:08seancorfieldFYI, this is the spec that takes a crazy long time to gen test: https://github.com/clojure/java.jdbc/blob/spec-gen/src/main/clojure/clojure/java/jdbc/spec.clj#L205#2016-06-1721:10seancorfield(and it took me a while to even get so far as to make run-all-tests
actually start runningā¦ with-gen
taking a 0-arity function that returns a generator is very counter-intuitive and I kept getting that wrong š )#2016-06-1721:57leifp@seancorfield: The (s/* (s/or ...))
combo seems to be the culprit. It would probably be very fast if you could limit it to a max size. I don't know how to do that, though.#2016-06-1721:59seancorfieldI guess I could always add a custom generator around it just to avoid that?#2016-06-1721:59seancorfield(and, thanks for the pointer on that!)#2016-06-1722:18leifp@seancorfield: This works: (s/def ::column-spec (s/with-gen (s/cat :spec (s/* (s/alt :kw keyword? :str string?))) #(g/vector (g/one-of [g/string g/keyword]) 0 6)))
Still not what I'd call lightning-fast, but you can bound the generation time by decreasing the max. And note that g/
refers to clojure.test.check.generators
, the equivalent using clojure.spec.gen
didn't work.#2016-06-1722:20leifpKind of clunky, though, and you'd need to do that everywhere you have (s/* (s/alt ...))
. Maybe if enough people run into performance problems, rep
will be made public, or the regex generators will be optimized.#2016-06-1722:28seancorfieldThanks @leifp#2016-06-1723:58leongrapenthinAssume I parse with conform. Then I have functions that operate on the value returned by conform. I can't get a spec for the value returned by conform (so that I can spec said functions) - It seems like this could be automated though. Imagine (s/conform-spec ::my-spec)
would return the spec of the return value of calling (s/confom ::my-spec 42)
#2016-06-1800:08wilkerlucioI noticed (s/exercice number?)
can generate some NaN
entries on cljs
, is that by design?#2016-06-1800:10leongrapenthinSo conform-spec
as described above can apparently be implemented quite easily via (defn conform-spec [s] (s/and (s/conformer #(s/unform s %)) s))
#2016-06-1800:17leongrapenthinBut it would probably be more valuable if a spec could give a spec of what its conform* returns#2016-06-1803:55wildermuthnIs there a changelog for Spec? Iām looking at some of the recent commits to cljs.spec, and see that the docs have been updated. But would be great to know if there was another place to reference to see the updates.#2016-06-1804:05wildermuthnThe more I learn about Spec, the more I think it is going to be regarded one of the essential Clojure features, on par with immutable data. For instance, I found out about conformer
, and not ten minutes later had the beginning of a solution that Iāve never had for parsing JSON enums in a sane way. I shared this on #C03S1L9DN, but interested in getting feedback if Iām going at this the wrong way:#2016-06-1808:37vikeriA question about spec and generators, is it able to generate data from a spec directly or does it generate data and test it against the spec? As an example I suppose that a generator for string will only generate strings, but what if you have some slightly more complicated predicates like strings with the length 3? Will it understand to only generate strings with the length three? I guess this would need some way of inverting an if statement, which sounds difficult. The main question in hence: How āsmartā can the generator be?#2016-06-1810:21Alex Miller (Clojure team)Well it's not magic :)#2016-06-1810:23Alex Miller (Clojure team)If you have (s/and string? #(= (count %) 3)) the and generator generates based on the first pred, then filters based on the subsequent ones#2016-06-1810:24Alex Miller (Clojure team)So it will generate random strings and keep those that are length 3#2016-06-1810:25Alex Miller (Clojure team)That's probably a restrictive enough filter that you'll need to supply a custom generator#2016-06-1810:28Alex Miller (Clojure team)@wildermuthn: I list incremental changes for each release in the announcement notes in the Clojure mailing list#2016-06-1810:29Alex Miller (Clojure team)@wildermuthn: not clear to me what you're asking about with conformers above #2016-06-1812:20ghadi@alexmiller: s/every has a tiny typo in it causing this:
user=> (s/valid? (s/every #{:foo}) [:foo :foo])
true
user=> (s/valid? (s/every #{:foo}) 42)
true ;; incorrect
#2016-06-1812:21ghadihttps://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L1095#2016-06-1812:21ghadis/:invalid/::invalid
#2016-06-1812:24Alex Miller (Clojure team)Thx#2016-06-1814:31nhaWhat is wrong with the following ?
(defn add []
"a")
(s/fdef add
:ret int?)
(s/instrument #'add)
(add) ;;=> "a"
#2016-06-1814:32nhaI was expecting an error on the return value.#2016-06-1814:33minimalInstrument was changed to only check :args#2016-06-1814:35nhaOoh so how do I check the return value ? (ie. what would be a simple test here ?)#2016-06-1814:37nhaAlso is there a constraint on the order of things ?
Can I s/fdef
before defn
-ing a function ? Can I s/instrument-all
when I want ?#2016-06-1814:41minimalFor now you have to use the functions from the clojure.spec.test
namespace like check-var
#2016-06-1814:43nhaThanks I will dig into it. Those can do "normal" testing as well right ? ie. not generative testing#2016-06-1814:43minimalYou can s/fdef
before. The docs say the instrument fns are idempotent#2016-06-1814:43nhaThanks š#2016-06-1814:44nhaAah I read that but assumed it meant it can be called many times#2016-06-1814:44minimalYeah#2016-06-1814:46nhaAny reason the :ret
has been removed ? Will it come back at some point ?#2016-06-1814:46eggsyntax@nha: see https://groups.google.com/forum/#!msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ#2016-06-1815:03nhaSo it looks like clojure.spec.test
is only for generative testing, I still have to use whatever I was using if I want to explicitly give input arguments to the tested function (ex. for things too hard to generate). Correct ?#2016-06-1815:17gfredericksnha: sounds right#2016-06-1820:56arohnerFYI, Iām reliably reproducing the āno such var encore/bytes?ā thing that @seancorfield reported the other day#2016-06-1820:56arohnerI donāt understand the cause, but Iām experiencing the same behavior#2016-06-1821:08seancorfieldThere's an updated Encore that fixes that. Some interaction with cljx / do / defn in that case - and conflicting with a (new) core function. #2016-06-1821:09seancorfieldBut I also eliminated every single dependency conflict and that solved the other, similar problems I had after that. #2016-06-1821:09arohneryeah š#2016-06-1821:09seancorfieldSo I'm not sure what the root cause is at this point. #2016-06-1821:09arohnerIām hoping the clojure bug gets fixed#2016-06-1821:10arohneris there a way to re-use fdef or fspec on other functions?#2016-06-1821:10arohnerIād like to say āthese two defns have the same fnspec"#2016-06-1821:14arohnerah, looks like I can just (s/def ::name (s/fspec)) (s/def ::foo ::name)#2016-06-1821:15bbloom@arohner: if two functions have the same fspec, wouldnāt they just be the same function? (or your fspec contains no interesting properties?)#2016-06-1821:16arohner@bbloom no, there are lots of things that have the same spec that are interesting#2016-06-1821:16arohneri.e. simple math#2016-06-1821:16arohner+, *, - all take two numbers, return a number#2016-06-1821:16arohnerin this case, parsing#2016-06-1821:16arohnertake some input, return a parsed value and the rest of the stream#2016-06-1821:17bbloomseems like youād want to reuse a spec for args and result, but a different spec for the property relating args to results#2016-06-1821:17bbloomthere might be a missing abstraction here#2016-06-1821:18bbloomweāve got specs for data and specs for functions which contain arg spec, return spec, and relation spec, but we donāt have a spec for args + return (signature?) without relation#2016-06-1821:18arohneryes, Iād like to get more specific in the future#2016-06-1821:18arohnerand I still have an open challenge out to ācorrectlyā spec clojure.core/map#2016-06-1821:19arohnerI assert it canāt reasonably be done right now#2016-06-1821:19bbloomcorrectly = ?#2016-06-1821:19arohnerāmap takes a fn of type X -> Y, and a seq of X, and returns a seq of Y'#2016-06-1821:20bbloomah yeah, spec lacks logic variables#2016-06-1821:20bbloomcanāt do context sensitive parsing#2016-06-1821:21eggsyntax@arohner: how could you verify that itās a fn of type X -> Y in a language that doesnāt do type declaration?#2016-06-1821:22eggsyntaxIs it sufficient to hand it an X, designate whatever comes out as Y, and then go from there?#2016-06-1821:22arohnerwell, I can spec that f is X->Y now, and spec that coll is coll-of X#2016-06-1821:22eggsyntaxAh, gotcha.#2016-06-1821:23eggsyntaxMakes total sense; somehow I was assuming f was unspecced.#2016-06-1821:23arohnerbut that doesnāt narrow down map
ās return spec#2016-06-1821:23arohnerbecause mapās spec is just seq
or whatever#2016-06-1823:42zpinterhello everyone! wondering if anybody knows of a clean way to define the spec for a map and its keys without having to repeat the list of keys again for the map spec#2016-06-1823:43zpintermy first attempt was something like this: (def style-keys
[
(s/def ::padding ::dimen-spec)
(s/def ::padding-left ::dimen-spec)
(s/def ::padding-right ::dimen-spec)
(s/def ::padding-top ::dimen-spec)
(s/def ::padding-bottom ::dimen-spec)])
(s/def ::style (s/keys :opt style-keys))
#2016-06-1823:43zpinterhowever, s/keys doesn't seem to like having :opt passed as a symbol (presumably since it's a macro)#2016-06-1823:44zpinterthis version compiles and works fine (s/def ::padding ::dimen-spec)
(s/def ::padding-left ::dimen-spec)
(s/def ::padding-right ::dimen-spec)
(s/def ::padding-top ::dimen-spec)
(s/def ::padding-bottom ::dimen-spec)
(s/def ::style (s/keys :opt [::padding ::padding-left ::padding-right ::padding-top ::padding-bottom]))
#2016-06-1823:44zpinterhowever, every time I add a new key, I have to update two different places (which I'd ideally like to avoid in this case, since there'd be a lot of keys)#2016-06-1823:45bbloomzpinter: iām far from an expert (i havenāt tried spec properly yet) but i think you can just use s/and, right?#2016-06-1823:45zpinterthe recommendation from #C03S1L9DN was to make my own macro, which seems like a good approach, just wondering there was anything built-in#2016-06-1823:47bbloomoh, nvmd, i think i misunderstood what you were asking for#2016-06-1823:49bbloombut yeah, iād guess making your own macro is the way to go - but depending on what youāre doing, the redundancy might make sense in order to allow decomposition later#2016-06-1823:49bbloomfor example, you may want to limit the styles on some thing to only color related styles or only spacing styles or what not#2016-06-1823:50bbloomso youād do (s/def ::paddingā¦, (s/def ::padding-left ... and then group those with s/and to form ::padded#2016-06-1823:50bbloomthen you can have (s/def style (s/and ::padded ::colored ::etc-etc-etc#2016-06-1823:51zpinterhmm... does s/and work with s/keys?#2016-06-1823:51zpinterI guess you could use s/and to combine multiple (s/keys :opt values....#2016-06-1823:51bbloomiād be surprised if it doesnt, but like i said - i havenāt installed the alpha yet to try it š#2016-06-1823:55zpinterwill keep investigating, thanks @bbloom#2016-06-1823:56eggsyntax@zpinter: I'd be interested in seeing what you come up with š#2016-06-1902:10Alex Miller (Clojure team)You can compose any kind of spec with and#2016-06-1902:11Alex Miller (Clojure team)Note also that s/keys validates all keys in the map regardless of what's in req or opt#2016-06-1902:12Alex Miller (Clojure team)So you don't technically have to list them as opt keys#2016-06-1910:25akielHas someone thought about specs for ex-data?#2016-06-1913:30gfredericks(s/fspec :args ... :ret ... :throws ...)
#2016-06-1913:57cigitiaIn clojure.spec, is there a way to associate custom generators with your own functions, the way that clojure.spec.gen/gen-for-pred
and gen-builtins
associate generators with various clojure.core functions?#2016-06-1915:00gfrederickscigitia: you can make a spec that has a particular generator associated with it using spec/with-gen
#2016-06-1915:01cigitia@gfredericks: Yes, though that requires either registering the spec returned by with-gen
under a keyword, or using with-gen
inline every time you use the function.#2016-06-1915:02cigitiaIām wondering whether itās possible to have generators when using functions directlyā#2016-06-1915:02cigitiaLike how int?
has a generator even when you use int?
directly as a predicate, due to the way gen-for-pred
works.#2016-06-1915:16gfredericksI think you can say (def int? (spec/with-gen #(instance? Integer %) gen/large-integer))
#2016-06-1915:16gfredericksI might be lying#2016-06-1915:59Alex Miller (Clojure team)There is more stuff coming in this area soon#2016-06-1916:16akiel@gfredericks: is :throws
discussed somewhere? I canāt find something about it.#2016-06-1916:57gfredericksakiel: no I just made it up#2016-06-1916:57gfredericksI was imagining an API that related to what you were asking about#2016-06-1917:26akiel@gfredericks: Yes thatās exactly what Iām after. @alexmiller What do you think about exception data specs? As far as I see it, exception data keys are not even documented normally.#2016-06-1917:33Alex Miller (Clojure team)We are not going to spec exceptions#2016-06-1917:34Alex Miller (Clojure team)You could of course create specs for the ex-info map keys and I can see that possibly useful in functions that receive ex-info data after an exception has happened (to generate user errors or error pages etc)#2016-06-1917:37akielDo you think that the shape of the ex-data should be public API of a function or do you are more in line with Joshua Bloch: Effective Java Item 57: Use exceptions only for exceptional conditions were he suggests that exception should not be used for decision making.#2016-06-1918:20Alex Miller (Clojure team)Yes#2016-06-1918:21Alex Miller (Clojure team)Unless you're abusing it for performance :)#2016-06-1919:24akiel@alexmiller: sorry for asking further - yes public or yes not-public?#2016-06-1919:30hiredmannot specing execeptions makes a lot of sense#2016-06-1919:34hiredman1. spec is not a type system 2. spec is primarily about specifying data and exceptions are not data, they are weird control flow operation in a language 3. specs for functions specify what valid argument data looks like, and given that valid argument data, what a valid result looks like, and that is how the generative stuff works, and in that model there is no place for exceptions#2016-06-1919:35hiredmangiven a function F, if F throws E, to ensure correctness you want to look at all callers of F, and ensure that they handle E correctly, which is exactly the opposite of how the generative testing works (if I understand correctly)#2016-06-1919:36akiel@hiredman: in case APIās donāt use exceptions for control flow Iām with you. But than we donāt need ex-info
and ex-data
at all and should stop build APIās which put data into exceptions#2016-06-1919:38akiels/conform
is a good example for an API not using exceptions - it returns ::s/invalid
#2016-06-1919:40bhaumanIs there a way to validate that you have defined all namespaced keywords in the your spec definitions? And get a warning if some are missing?#2016-06-1919:41hiredmanwrite a predicate that checks the spec registry#2016-06-1919:46bhaumanhmmm ... so say compose a macro over spec/keys, capture the key args and and then warn if if the keys are not present in the registry? or were you thinking something else?#2016-06-1919:47bhaumanif you wanted to verify everything with a single call though it seems like you would have to parse the describe of each of the registry members#2016-06-1919:47hiredmanuser=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {:a 1})
true
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {::a 1})
false
user=> (s/def ::a ::s/any)
:user/a
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {::a 1})
true
user=>
#2016-06-1919:49bhaumancool yep, but was thinking of a different use case, where you are inspecting the self referential integrity of the registry itself#2016-06-1919:50hiredmannot sure I follow? that s/map-of spec will fail if given a map with namespaced keys that are not speced#2016-06-1919:51hiredmanoh#2016-06-1919:51hiredmanI think I get it now#2016-06-1919:52hiredmanyeah, spec seems to be slightly later binding than clojure is#2016-06-1919:53bhaumanyeah, it's doable, but I think it would be nice to have some help beyond describe#2016-06-1919:54hiredmanyou'd like ::foo to throw an error earlier then when you run the spec, basically, like clojure does with vars#2016-06-1919:55bhaumanI'd like to do ( check-missing (clojure.spec/registry))#2016-06-1919:56bhaumanto help me in the repl so that I can see if I forgot something#2016-06-1919:58hiredmanthat might not be currently possible, I don't think specs have a way to get all the specs they depend on#2016-06-1919:58hiredmanwhich might be an interesting enhancement#2016-06-1919:59bhaumanexactly, it would be nice to have a spec walker#2016-06-1920:03bhaumanor at least a corollary to describe that returns the actual internal data#2016-06-2002:05Alex Miller (Clojure team)there will likely eventually be a deep-describe, which will require something like that#2016-06-2013:54alqvistI feel that the usefulness of s/inst-in is hampered by cljs incompability. Any words of wisdom?#2016-06-2014:43alqvistOr I could just use the latest clojurescript - Never mind#2016-06-2015:04angeliniis there anyway to reproduce the behaviour of alpha-5ās s/instrument
without using spec.test
? The rational of only checking :args
makes sense, but it was really useful when testing code at the REPL to verify that :ret
and :fn
weāre correct.#2016-06-2015:32adamfreywhen you have a test.check generator that is failing a such-that predicate 100 times, is it possible to print out the failing values, just to make the generator less opaque? Nothing stood out to me in the source code for doing that.#2016-06-2017:39akielAre there plans to support some kind of validation context? I think about a spec for JSON Web Tokens which are signed. Such a validation would need the public key as context. Or do I stretch the intended usage patterns here?#2016-06-2019:41Alex Miller (Clojure team)@adamfrey nothing explicit but you can gen/sample the s/gen of some simpler part to get an idea. It's usually the first pred in an s/and that is creating things that won't pass the later preds in the s/and#2016-06-2019:41Alex Miller (Clojure team)@akiel no plans for that#2016-06-2019:42Alex Miller (Clojure team)Prob a sign you are taking things too far :)#2016-06-2020:10akiel@alexmiller thanks#2016-06-2020:38Alex Miller (Clojure team)@cigitia: this channel is for spec, please take to #off-topic #2016-06-2020:40cigitiaMy apologies; accidentally sent to wrong Slack team; meant to sent to personal friends#2016-06-2108:07borkdudemakes me wonder about the spec for get-in: (get-in {:a "lol"} nil) ;;=> {:a "lol"}
#2016-06-2108:08borkdudeI guess it could be correct if nil is regarded as the empty sequence#2016-06-2111:55akiel@borkdude: nil is the empty sequence š I think it should be possible to write a spec for get-in
.#2016-06-2114:33jannisIs there any chance of opts being added to clojure.spec.test/run-(all-)tests
? I have a couple of functions that exceed the GC limit if I run 100 tests against them. 25 or 50 is fine, so I'd like to set :num-tests
globally instead of having to use check-var
for every single function I want to test.#2016-06-2114:35jannisAn alternative idea would be per-`fdef` opts perhaps?#2016-06-2114:58jannisOr a way to exclude specific functions from run-(all-)tests
?#2016-06-2115:06akiel@jannis why are your functions are so memory hungry? do they have side effects or are the inputs to big? for the latter, I would rather optimize the generators.#2016-06-2115:07jannisGood point, it's probably the generators generating big inputs. I'll double-check.#2016-06-2115:09jannisAlthough... the generator is already generating relatively small data.#2016-06-2115:10jannisThat's the function: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L324 and this is the generator: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L90#2016-06-2115:10angusiguesssuch-thats are a good thing to look out for.#2016-06-2115:12jannisI have overriden almost all generators because my data specs seem to be too complex to use the default generators.#2016-06-2115:12angusiguessOne thing we ran into was the following:#2016-06-2115:12angusiguessIf there's a significant distance between your generator and your validation predicate that generator tends to perform poorly.#2016-06-2115:13jannisOf course, that makes sense. That's why I overrode most of the generators so that they don't have to be tried a lot to have conforming data generated.#2016-06-2115:15angusiguessI haven't tried it, but does increasing the heap size help?#2016-06-2115:16jannisWithout knowing much about it, GC limit exceeded to me sounds like it's generating too much data too quickly for GC to catch up. Let me re-run the tests with a heap size of 2GB.#2016-06-2115:17angusiguessGC Limit exceeded usually means that the ratio of GC pauses to 'useful' computation is quite high.#2016-06-2115:18angusiguessSo more heap space can reduce pauses.#2016-06-2115:18jannisOk#2016-06-2115:23jannisIt's also a test performance issue by the way. With 100 tests, that single function test runs for ~5-10 minutes until it errors out with the GC limit. I'm aiming for a fast test suite, so being able to reduce the number of tests globally - or for this function - would help.#2016-06-2115:56jannisIncreasing the heap size avoids it failing after 5-10 minutes. Instead it has now been running for more than half an hour. š#2016-06-2115:58akiel@jannis I have you source code open. But I never used boot before and Iām using cursive - so its not so easy for me to get it running.#2016-06-2115:58jannis@akiel: Once you have boot installed, all you should need is boot test-once
#2016-06-2116:20akiel@jannis: the generator of ::property-spec
is already very slow. Try (s/exercise ::properties-spec 15)
and than with 20. The thing is that generated samples become bigger and bigger if you generate more. test.check has some internal size thing. You have to improve the generator.#2016-06-2116:21jannis@akiel: Cool, thanks for investigating. I'll see what I can do. š#2016-06-2121:15cigitiaIs there any particular reason that the sequence regex ops do not recognize strings as seqable
s of char
s? For instance, āabā
does not conform to (s/cat :a #(= % \a), :b #(= % \b))
, while [\a \b]
does.#2016-06-2121:19cigitiaIn addition, are there plans for negative-lookahead regex ops, like those from &
? For instance, in PEGs / parsing expression grammars, &
has a negative counterpart called !
Ā .
In particular, an end-of-sequence regex op would be very usefulāit could be called ::end
, and it could be equivalent to (! ::any)
. As far as I can tell, this isnāt currently possible.#2016-06-2122:25bfabrylol, @cigitia your slack-fu is struggling#2016-06-2122:25Alex Miller (Clojure team)@cigitia: you're posting news links again#2016-06-2122:25cigitiaArgh, sorry#2016-06-2122:26Alex Miller (Clojure team)@cigitia There are good tools for string regex. Spec is never going to be great for that and there are no plans to use it for that#2016-06-2122:27Alex Miller (Clojure team)No plans for negative look ahead ops #2016-06-2122:27cigitiaOkay, thanks#2016-06-2122:27Alex Miller (Clojure team)They are generally not needed for the kinds of things we expect you to do with spec regex#2016-06-2122:32cigitiaI had actually been working on a PEG library that could create grammars for generic EDN data, including but not limited to strings, before Spec was announced#2016-06-2122:34cigitiaOnce Spec was announced, it seemed like there wasn't much room for such a library anymore, and so I thought about instead making a library that would utilize specs and convert them into parser transducers#2016-06-2122:35cigitiaMaybe there's still room for the first idea, though, then, hm#2016-06-2202:59Oliver GeorgeI was expecting s/conform to return fully "conformed" data but it doesn't seem to work like that.#2016-06-2203:00Oliver George=> (s/conform (s/coll-of (s/conformer (fn [x] (println "I am conforming" x) (str x))) []) ["1" 2 :3])
I am conforming 1
I am conforming 2
I am conforming :3
["1" 2 :3]
#2016-06-2203:01Oliver Georgeit uses the conformer in order to check the data but returns the un-conformed data. Seems to be "non-recursive"#2016-06-2203:02Oliver GeorgeIs that intended behaviour? Perhaps I'm attempting to use it incorrectly - as a data verification & transformation tool.#2016-06-2203:40Alex Miller (Clojure team)The problem here is that coll-of samples its contents and doesn't flow the entire contents#2016-06-2203:40Alex Miller (Clojure team)There are some things in this area changing in next alpha#2016-06-2203:57Oliver GeorgeThanks Alex, I think I see similar behaviour with map-of.#2016-06-2204:02Alex Miller (Clojure team)Yes#2016-06-2204:02Alex Miller (Clojure team)every and every-kv are new things already in master but not yet released#2016-06-2204:03Alex Miller (Clojure team)One of those sets will conform all args, although I'm not sure which#2016-06-2204:08Oliver GeorgePerfect. Thanks.#2016-06-2211:38akielIs there a way to spec values which will be conveyed over a channel or returned by a promise?#2016-06-2211:53Alex Miller (Clojure team)nothing built in yet#2016-06-2211:54Alex Miller (Clojure team)ultimately I expect there will be some things provided for core.async#2016-06-2211:55Alex Miller (Clojure team)you could supply a map transducer to an async channel that called s/conform or something like that though#2016-06-2212:09akielsome kind of spec wrapper which takes a spec and validates channel values on the go would be nice#2016-06-2212:11akielI only like to be sure that nothing speaks at this conceptionally before spec gets final#2016-06-2214:02eggsyntaxtest.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one.
The only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen)
, but it's pretty ugly to put in a generator and then ignore what it generates.#2016-06-2214:04eggsyntax(I understand that shouldn't be a typical use, but it'd be good to have an escape hatch for cases where you have an existing source of randomness that you don't want to have to duplicate)#2016-06-2214:43akiel@eggsyntax: As you said already, itās not the typical use to implement the root generator yourself. You have to understand that test.check needs to control the generated values in a way to get shrinking work. So I have no answer to your question only the suggestion: Donāt do this! š#2016-06-2214:44eggsyntaxHeh, fair enough.#2016-06-2214:46eggsyntaxAlthough in this case I'm still gonna do it...again, existing (complex) source of domain-specific randomness.#2016-06-2214:46eggsyntaxMaybe I'll have to figure out how to make a Rose tree from it...#2016-06-2214:47eggsyntaxI realize there's a likely cost in terms of shrinking, though.#2016-06-2218:07Alex Miller (Clojure team)@eggsyntax: have you looked at return
?#2016-06-2219:22eggsyntax@alexmiller: I have, but it looks like that'll only generate a constant value, right? I need something that'll call a fn every time it needs a new one.#2016-06-2219:39angusiguesstest.check does have a no-shrink qualifier too I believe.#2016-06-2219:39angusiguessIf the value doesn't benefit from shrinking (like uuids for ex)#2016-06-2223:06bostonaholichas anyone come up with a reasonable clojure.spec/def
for a database argument?#2016-06-2223:06bostonaholicall Iām using at the moment is ::s/any
#2016-06-2223:06bostonaholic(defn get-user [db id] ...)
(s/def ::database ::s/any)
(s/def ::user (s/keys :req [::id ::email]))
(s/fdef get-user
:args (s/cat :db ::database :id ::id)
:ret ::user)
#2016-06-2223:07bostonaholicbut obviously that wonāt work with generators#2016-06-2223:08bostonaholicIām not sure thereās much value here, but interested in hearing what a possible solution might be#2016-06-2223:40eggsyntax@bostonaholic: what should the DB argument look like?#2016-06-2223:41bostonaholicwell, itās a datomic db#2016-06-2223:42eggsyntaxAh, ok, you hadnāt mentioned it was datomic.#2016-06-2223:42bostonaholicyeah, sorry#2016-06-2223:42bostonaholicthatās why Iām thinking this isnāt a good idea#2016-06-2223:44Oliver George@alexmiller picking up on my question about fully conforming yesterday. I tried every-kv this morning (cljs release) and I see the same behaviour:#2016-06-2223:45Oliver George=> (do
(s/def ::tree (s/or
:branch (s/every-kv keyword? ::tree)
:value ::s/any))
(s/conform ::tree {:A {:B 2}}))
[:branch {:A {:B 2}}]
Was hoping for [:branch {:A [:branch {:B [:value 2]}]}]
or similar.#2016-06-2223:45eggsyntaxYou can certainly check the type using a predicate built from instance?
#2016-06-2223:46eggsyntaxThat might be enough for your needs.#2016-06-2223:47bostonaholicyeah, thatās the only other thing I could think of#2016-06-2223:49eggsyntaxThat's enough to show it's a legit database...you could also do something like
(s/and #(instance? datomic.db.Db %)
(s/keys :req-un [::key1 ::key2]))
with further specs for those keys.
(don't have a datomic-using project right in front of me, I'm improvising on the type, and assuming that it's a record
)#2016-06-2223:50eggsyntax(which could be dead wrong š )#2016-06-2223:51bostonaholicyouāre right#2016-06-2223:56Alex Miller (Clojure team)@olivergeorge its not done yet#2016-06-2223:56eggsyntax@bostonaholic: ooh, quite a few keys there, huh? (:id :memidx :indexing :mid-index :index :history :memlog :basisT :nextT :indexBasisT :elements :keys :ids :index-root-id :index-rev :asOfT :sinceT :raw :filt)
#2016-06-2223:57bostonaholicheh, yeaahhhhhhhhh#2016-06-2223:58Alex Miller (Clojure team)@olivergeorge: I believe when it is done map-of will be what you want #2016-06-2300:01Oliver GeorgeThanks again Alex. I suspected that might be the case.#2016-06-2301:08eggsyntax@alexmiller: is there a roadmap online anywhere for what changes are definitely coming? Or even something that discusses some of them?#2016-06-2301:34Alex Miller (Clojure team)no#2016-06-2301:34eggsyntaxOh well š#2016-06-2301:35Alex Miller (Clojure team)there are definitely changes around instrument and the c.s.test namespace coming soon#2016-06-2301:35Alex Miller (Clojure team)and map-of and coll-of work is under way#2016-06-2301:35Alex Miller (Clojure team)there are a bunch of other things on the list but I donāt how that will all pan out#2016-06-2301:36eggsyntaxExcellent! I donāt think Iāve actually said this to yāall before about spec, but thanks š. Iāve been working with it for about a week, and itās just terrific work.#2016-06-2301:36eggsyntaxJust how powerful it is didnāt even fully sink in until Iād dived into it for a while.#2016-06-2301:36eggsyntax^ @alexmiller#2016-06-2301:39seancorfieldIf I have (s/keys :req-un [::foo ::bar])
, am I correct that something in clojure.spec
will expect ::foo
and ::bar
to exist as specs, for some operations? I was trying to add stuff to clojure.java.jdbc.spec
to support generative testing (by writing custom generators for db-spec, java.sql.Connection
, and java.sql.PreparedStatement
etc) and I started getting errors that the "unqualified" keys in some of my specs didnāt resolve...#2016-06-2301:40seancorfieldThat wasnāt obvious to me from the documentation about s/keys
and the -un
keys in maps ā but it makes sense from the p.o.v. of actually trying to generate maps to pass into functions whose arguments are specād as s/keys
...#2016-06-2301:53seancorfield(FWIW, folks can look at the work-in-progress experiment on a gen-testable java.jdbc spec here https://github.com/clojure/java.jdbc/blob/spec-gen/src/main/clojure/clojure/java/jdbc/spec.clj )#2016-06-2302:31Alex Miller (Clojure team)if youāre trying to gen keys in a map, then yes you would need them to have a definition#2016-06-2302:31Alex Miller (Clojure team)otherwise how would you gen them?#2016-06-2302:33Alex Miller (Clojure team)are you getting this while genāing or during something else?#2016-06-2303:01seancorfieldDuring genāing. And, yes, it makes sense. It just wasnāt obvious from the docs...#2016-06-2303:02seancorfieldBecause I missed this (very important) phrase "These variants specify namespaced keys used to find their specification"#2016-06-2303:04seancorfieldI just didnāt associate the ::first-name
etc with the specs given two examples above since they were "unqualified keywords" in my mind.#2016-06-2303:06seancorfieldNot sure what to suggest to make that clearerā¦? Perhaps repeat the specs in the :unq/person
example? (s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::email ::email-type)
(s/def :unq/person
(s/keys :req-un [::first-name ::last-name ::email]
:opt-un [::phone]))
#2016-06-2303:06seancorfield(and now I notice that ::phone
is not provided as a spec anywhere on that page)#2016-06-2303:07seancorfieldObvious in hindsight, of course.#2016-06-2308:34Oliver GeorgeThis is a thought in passing. Often we want confidence that some data is fully realized. It seems like a challenging case. We have realized? as a test and could write a recursive ::realized spec. It is challenging since displaying the data in reporting might "realize" it. So the test modifies the data.#2016-06-2308:35Oliver GeorgePerhaps not a clojure.spec domain issue... data is valid, it just doesn't exist yet!#2016-06-2308:35Oliver GeorgeWe've seen cases where lazy and dynamic aspects cause surprises. Right now we're considering a class of problems where delayed evaluation makes debugging harder.#2016-06-2308:36Oliver GeorgeJust raising in case it's something of interest as spec matures.#2016-06-2308:48thomasdeutschhow can i spec this structure?
{::items-by-id {"id-1" {::id "id-1"
::title "first-item"}
"id-2" {::id "id-2"
::title "second-item"}}}
i would like to spec that every item ::id is the same id as its key on the parent map.#2016-06-2310:50wagjoAre optional docstrings for s/def
(CLJ-1965) something you'd like to have or were they left out intentionally?#2016-06-2312:01Alex Miller (Clojure team)@wagjo: still on the table#2016-06-2312:02Alex Miller (Clojure team)@olivergeorge: I donāt expect spec is going to add anything re lazy/realized data#2016-06-2312:02Alex Miller (Clojure team)@seancorfield in my next guide pass I will see if I can do anything to bring that out#2016-06-2317:10seancorfieldMuch appreciated https://clojurians.slack.com/archives/clojure-spec/p1466683358000870#2016-06-2320:48uwoIs there a suggested way to maintain metadata about which keys a spec is concerned:
(s/def :bill/delivery inst?)
(s/def :bill/pickup inst?)
(s/def :bill/pickup-gt-delivery (fn [{:keys [bill/delivery bill/pickup]}] (> pickup delivery)))
(s/def ::bill (s/and (s/keys :req [:bill/delivery :bill/pickup])
:bill/pickup-gt-delivery))
;some lookup
{:bill/pickup-gt-delivery [:bill/delivery :bill/pickup]}
#2016-06-2321:40Alex Miller (Clojure team)no, I donāt think so#2016-06-2321:40Alex Miller (Clojure team)user=> (s/explain-data ::bill {})
{:clojure.spec/problems {[] {:pred [(contains? % :bill/delivery) (contains? % :bill/pickup)], :val {}, :via [:user/bill], :in []}}}
#2016-06-2321:41Alex Miller (Clojure team)The explain-data will tell you what itās missing in the form of predicates but thatās not what youāre asking#2016-06-2321:41Alex Miller (Clojure team)(btw, as of the next alpha, youāll be able to rewrite pickup-gt-delivery as (fn [{:bill/keys [delivery pickup]}] (> pickup delivery))
)#2016-06-2322:07uwo@alexmiller: cool. thanks for the response!#2016-06-2402:22bsimaHow do you spec protocol functions? Is it the same as spec-ing a regular function? (I haven't actually tried it yet...)#2016-06-2402:23bsimaI tried to spec a multimethod using fdef
a few days ago and it didn't work but I didn't have time to figure out why#2016-06-2402:37seancorfield@bsima: Read this issue for some information about spec and protocols http://dev.clojure.org/jira/browse/CLJ-1941 (regarding instrumentation)#2016-06-2402:42bsimamm thanks seancorfield#2016-06-2415:11jannisHow do I best write a generator for :args
? It seems like even if I use (gen/cat <sequence-generating generators>)
it generates a sequence that is then passed to :args
as the first argument, instead of "splicing" it into the macro/function call.#2016-06-2415:13akiel@jannis Is it not sufficient for you to write individual generators? Do you have many dependencies between your args?#2016-06-2415:16jannis@akiel: This is my example: https://gist.github.com/Jannis/51ebdd54e10074fd9c574a54dd8421c9#2016-06-2415:17jannisThe args are fairly flexible and somehow what gets generated for name
and & forms
is not quite what it should be. Perhaps my :args
spec is wrong (thinking var args perhaps need special treatment)?#2016-06-2415:21jannisEven if I just use (s/cat :name ::command-name)
and drop the & forms
argument in the macro, check-var
complains with "Wrong number of args (1) passed to: command/defcommand"
. Something there is odd.#2016-06-2415:22akielYes (s/cat :name ::command-name)
generates only one argument#2016-06-2415:23akielvaragrs is no special - you have already a s/*
in your args#2016-06-2415:23akielwhat is ::command-name
?#2016-06-2415:23jannisBut one argument should be fine for a macro that takes exactly one argument.#2016-06-2415:24jannisIt's a symbol?
spec#2016-06-2415:24akielah yes you are right - one arg should be possible#2016-06-2415:25jannisI can paste the entire file once my laptop has rebooted.#2016-06-2415:30akielI get the same wrong number of args message with check-var#2016-06-2415:30akielbut if I convert the macro into a function, it works#2016-06-2415:30akielther may be some issues using char-var with macros#2016-06-2415:31akielI never specāed macros before#2016-06-2415:31jannisYep, I never ran into this with specs for functions.#2016-06-2415:32jannisActually, this is a Clojure + ClojureScript macro, which will just pass on the arguments to a function that does the actual work. I could move the spec to that function instead.#2016-06-2415:32jannisThat could work around the problem.#2016-06-2415:34akielbut speccing macros is an interesting feature on its own - but check-var might simply not work with it#2016-06-2415:35jannisCould be?#2016-06-2415:36akielI simply donāt know. Iāll hit this if a spec my first macro. But I have currently none.#2016-06-2415:36jannisOk š#2016-06-2415:37jannisMoving the spec to the function and turning the var-args into a (s/cat :name ... :forms (s/spec (s/cat :description ...)))
subspec structure does the job š#2016-06-2415:37jannisAll the generated inputs are now useful.#2016-06-2415:40akielnice š#2016-06-2416:00kendall.buchananQuestion about the direction of namespaced keywords:#2016-06-2416:01kendall.buchananWith Clojurescript (Om appears to be pushing namespaced keywords) on the front end, specs in the middle, and Datomic on the end, namespaced keywords feel natural.#2016-06-2416:01kendall.buchananBut if youāre working with a Javascript front-end, that doesnāt support them. And a SQL database on the other. Whatās one to do in the middle if you want to leverage specs?#2016-06-2416:01kendall.buchananJust :req-un everything?#2016-06-2416:02kendall.buchananOr create translation layers across the board?#2016-06-2416:02akiel:req-un
works fine - the only thing you loose is automatic validation of all keys even if not present in req or opt#2016-06-2416:04kendall.buchananMakes sense. Iāve hesitated on spec for fear that they canāt live in āthe real worldā (a non-full-Clojure stack environment).#2016-06-2416:05akieleven in clojure itself - nobody can change existing apis - so un-namespaced keywords will be still there#2016-06-2416:06kendall.buchananK, thanks @akiel.#2016-06-2417:49seancorfield@kendall.buchanan: Iām working on making namespaced keyword support in java.jdbc
a lot easier. In the upcoming 0.6.2 release youāll be able to specify a :qualifier "foo"
option on calls to get :foo/col-name
back from all queries. I need to figure out the final story for what happens if you push :foo/col-name
into java.jdbc
.#2016-06-2417:50kendall.buchanan@seancorfield: Thatās very cool. I suspect work will need to be done on all ends of the stack to make namespaced keys more tenable for the average project.#2016-06-2417:56seancorfieldNow I look at it, I probably wonāt need to do anything for pushing namespaced columns into java.jdbc
: user=> (def db-spec {:dbname "mydb" :dbtype "mysql" :user "tester" :password "secret"})
#āuser/db-spec
user=> (j/query db-spec ["select * from status"])
({:id 1, :name "approved"} {:id 2, :name "new"} {:id 3, :name "rejected"})
user=> (j/query db-spec ["select * from status"] {:qualifier "status"})
({:status/id 1, :status/name "approved"} {:status/id 2, :status/name "new"} {:status/id 3, :status/name "rejected"})
user=> (j/insert! db-spec :status {:status/name "test"})
({:generated_key 13})
user=> (j/query db-spec ["select * from status"] {:qualifier "status"})
({:status/id 1, :status/name "approved"} {:status/id 2, :status/name "new"} {:status/id 3, :status/name "rejected"} {:status/id 13, :status/name "test"})
#2016-06-2417:58seancorfieldAnd due to a very recent change, even this works as expected: user=> (j/insert! db-spec :status {:status/name "qualifier"} {:qualifier "result"})
({:result/generated_key 14})
#2016-06-2418:12Alex Miller (Clojure team)@jannis: check-var (very soon to be just ātestā in the next alpha) is not going to work with macros#2016-06-2418:14Alex Miller (Clojure team)@kendall.buchanan: we are considering adding something in s/keys that lets you alias unaliased keys to arbitrary specs, but no guarantees#2016-06-2418:19jannis@alexmiller: Will there be an equivalent for fdef'ed macros?#2016-06-2418:19Alex Miller (Clojure team)you can fdef macros now#2016-06-2418:20Alex Miller (Clojure team)itās just that check-var is not going to work to test them, afaik#2016-06-2418:23akielIs there a way to compose multiple s/key specs into one big one so that a map has to satisfy them all?#2016-06-2418:24bfabry@akiel: I wrote a macro that did that because I couldn't find anything. it was kinda painful#2016-06-2418:26akiel@bfabry: I need it to work with multi-spec because I have a big environment map were I have multiple subsets of varing keysets.#2016-06-2418:28bfabryI don't know enough about multi-spec to know if it would help, but here it is (very ugly) https://clojurians.slack.com/archives/clojure-spec/p1466110303000387#2016-06-2418:30akielAh s/form is nice. Never used it.#2016-06-2418:34akiel@bfabry: Thanks but your macro will only work with static s/keys specs.#2016-06-2418:34bfabryyup, I couldn't find a way around that because s/keys is a macro#2016-06-2418:39Alex Miller (Clojure team)@akiel you can just do s/and of multiple s/keys#2016-06-2418:39bfabry@alexmiller: that won't create good generators though#2016-06-2418:39Alex Miller (Clojure team)true#2016-06-2418:41akiel@alexmiller: Ok I only have to wrap all subsequent s/keys into a one-arg function and ignore the conforming value?#2016-06-2418:48akiel@alexmiller: ok it works#2016-06-2418:48Alex Miller (Clojure team)s/and will create a spec, not sure why you need to make it into a function#2016-06-2418:48akielNo my reading of the doc was just to loose.#2016-06-2418:48Alex Miller (Clojure team)the resulting conformed value should be the map#2016-06-2418:49akielI thought there would be always a predicate which gets the previous value. Bit this counts only for preds and not for specs.#2016-06-2418:51Alex Miller (Clojure team)not sure I understood that#2016-06-2418:54akielThe example is (s/and even? #(< % 42))
and the doc says: āSuccessive conformed values propagate through rest of predicates.ā So I thought you can have one initial spec and than only predicates which are called with the previous conformed value.#2016-06-2418:54bfabry@alexmiller: out of interest is there any move to make some of these macros functions? a dynamic s/keys seems like it could be useful#2016-06-2418:55Alex Miller (Clojure team)@akiel: yes, although I think maybe youāre reading too much into the choice of āspecā and āpredicateā there#2016-06-2418:56Alex Miller (Clojure team)and combines specs which can be either predicates or other things#2016-06-2418:56Alex Miller (Clojure team)the conformed value flows through them#2016-06-2418:57akiel@alexmiller Ah ok. I see. The specs get also the previous conformed value and just use it to be able to return the whole thing at the end.#2016-06-2418:58Alex Miller (Clojure team)@bfabry: yes, thatās possible. things are macros so that we capture forms for reporting purposes. but there may still be function forms as well (which would fill in under the macro forms)#2016-06-2418:59bfabrycool cool#2016-06-2419:19fentonI have a multi-spec. My in data can look like: {:method :search :search-term "blah" :auth "abc123" :gps-lat 3.4 :gps-long 5.3}
or {:method :get-products :product-id 12345 :auth "abc123" :gps-lat 3.4 :gps-long 5.3}
. My multi-spec already keys off the :method
key. My data has some shared keys: :auth
, :gps-lat
, :gps-long
and some keys that are not shared: :search-term
and :product-id
. Ideally I'd like to create a spec that merges two s/keys
specs into a single s/keys
spec, however s/cat
creates a spec that expects two hashes in a list as opposed to a single hash with merged keys.#2016-06-2419:21Alex Miller (Clojure team)I talked through some options for something like this with @luke - maybe he could elaborate on where he ended up#2016-06-2419:24Alex Miller (Clojure team)iirc, he ended up with a spec for the common keys and separate specs for each variant#2016-06-2419:24Alex Miller (Clojure team)and then specs for each variant that anded common and separate#2016-06-2419:25Alex Miller (Clojure team)and then instead of multi-spec, itās an s/or of the concrete variants#2016-06-2419:26Alex Miller (Clojure team)with the downside that itās a closed system#2016-06-2419:26Alex Miller (Clojure team)with multi-spec, youād have to repeat the keys in each variant#2016-06-2419:28fenton@alexmiller: okay, i'll look into what it looks like with s/or
option and compare that to repeating in a multi-spec scenario. Thanks.#2016-06-2419:29fentonthe closed system you mention doesn't use s/keys
?#2016-06-2419:41Alex Miller (Clojure team)no it would#2016-06-2419:43Alex Miller (Clojure team)common = (s/keys ā¦)
A = (s/and common (s/keys Aā¦))
B = (s/and common (s/keys Bā¦))
all = (s/or :a A :b B)#2016-06-2419:44Alex Miller (Clojure team)that doesnāt help you with the conforming question though#2016-06-2419:45fenton@alexmiller: k i'll give that a try.#2016-06-2423:52leongrapenthin@fenton @alexmiller I ran into this too, it seems a common scenario with Datomic "polymorphism" aka {:payment/amount 42, :payment/type :payment.type/stripe, :payment.stripe/payment-id 1234}
- it would be great to have a merge-keys operation that takes two s/keys and merges their implementations - what do you think?#2016-06-2423:55Alex Miller (Clojure team)Talked about it with Rich today#2016-06-2423:55Alex Miller (Clojure team)We'll see#2016-06-2501:28seancorfieldAlpha 8 is looking like a great release, based on the commits over the last few days!#2016-06-2502:14Alex Miller (Clojure team)Gonna be a big one :)#2016-06-2502:32seancorfieldAny likely ETA @alexmiller ?#2016-06-2502:46Alex Miller (Clojure team)Soon#2016-06-2502:47Alex Miller (Clojure team)Didn't quite finish everything today as expected so prob early next week#2016-06-2505:29bbloomrather than complain about having to upgrade fipp for new reader/printer forms, iāll instead boast: open source contribution opportunities! clojure 1.9 functionality PRs welcome š#2016-06-2518:26fenton@alexmiller: small typo in spec guide. the description around function specs uses an int?
predicate instead of integer?
#2016-06-2518:30Alex Miller (Clojure team)That's correct as is#2016-06-2518:30fentonoh, hmm, couldn't find an int? predicate#2016-06-2518:31Alex Miller (Clojure team)int? is a new predicate in 1.9 that matches any fixed precision integer#2016-06-2518:31gfredericksso in particular not bigints?#2016-06-2518:31fentonah, good to know.#2016-06-2518:31Alex Miller (Clojure team)It was briefly called long?#2016-06-2518:31gfredericksbigints are the only difference between int?
and integer?
?#2016-06-2518:32bronsaAFAICT yes#2016-06-2518:33gfredericksinteresting#2016-06-2518:33gfredericksI wonder why that's considered more useful than the other way#2016-06-2518:34bronsawell there are a number of constructs in clojure that don't work with idxs bigger than Long/MAX_VALUE
so I'm guessing int?
was provided to cover those use-cases in specs#2016-06-2518:35bronsaguarding against i.e. (range 0 (inc Long/MAX_VALUE))
#2016-06-2518:35gfredericksinteresting#2016-06-2518:36bronsa(although there are also other constructs that don't work with idx bigger than Integer/MAX_VALUE
like dotimes IIRC?)#2016-06-2518:36bronsanope dotimes
accepts longs#2016-06-2518:37bronsaah, I'm thinking of for
and doseq
, they're constrained by Collection.count
returning int
#2016-06-2519:08Alex Miller (Clojure team)Yes widening from just long to other fixed precision was the reason#2016-06-2519:08Alex Miller (Clojure team)In particular to capture int#2016-06-2519:11Alex Miller (Clojure team)Conceptually we would prefer if users primarily focus not on the Java types but on integer fixed vs integer arbitrary precision as the main "types"#2016-06-2519:40mathias_dwprobably a dumb question, but Iām having to call s/instrument-all every time I make a change to the code or the specs. Is there a way of automating that, or is my workflow simply wrong?#2016-06-2519:42mathias_dw(or is it maybe some nrepl/cider-related thing?)#2016-06-2519:47Alex Miller (Clojure team)You shouldn't have to do that #2016-06-2519:47mathias_dwok, thanks. Iāll try without the cider/nrepl stuff#2016-06-2519:48Alex Miller (Clojure team)That was true of the very first release but was fixed early on#2016-06-2519:48mathias_dwIām using alpha5#2016-06-2519:49Alex Miller (Clojure team)Yeah shouldn't be a problem #2016-06-2519:49Alex Miller (Clojure team)Must be something in the workflow#2016-06-2519:49mathias_dwok, thanks. Will try to pinpoint it and raise it in the relevant place#2016-06-2520:45gfredericksI'm having trouble with a recursive spec on master; not a problem on alpha7: https://www.refheap.com/120803#2016-06-2520:45gfredericksI scanned the commit messages and didn't see anything suspicious#2016-06-2520:46gfredericksif I had to guess I'd say it's something to do with coll-of
suggesting that there's ambiguity#2016-06-2520:48Alex Miller (Clojure team)That code is brand new and on my plate to test more on Monday morning so I will def look at it#2016-06-2520:49gfrederickscool, thanks#2016-06-2607:03seantempestais there a way to test if a spec exists for a namespaced key?#2016-06-2611:05Alex Miller (Clojure team)There is a 'registry' function #2016-06-2611:05Alex Miller (Clojure team)In master for the next alpha there will be 'get-spec'#2016-06-3001:51seancorfieldwhen I run the java.jdbc tests on 1.9, it loads the specs and instruments the ns (not all, just java.jdbc)#2016-06-3001:52seancorfieldSo, for library code that might have pre-1.9 users, separate specs is pretty much the only possibility.#2016-06-3001:52puzzlerCould the main sourcecode namespace do that test on version number, and automatically load the specs for docstring purposes?#2016-06-3001:53seancorfieldFor application code thatās is bound to 1.9, I can imagine having specs intermingled ā maybe.#2016-06-3001:53seancorfieldYeah, I guess java.jdbcās source namespace could auto-load the specs on 1.9. Iām just not sure that would be what all its users would expect.#2016-06-3001:54seancorfieldAfter all, if you (instrument)
your code would you expect all the external libraries to slow down due to specs?#2016-06-3001:55seancorfieldSome specs can have a really high performance overhead so I donāt think the library should dictate that.#2016-06-3001:55puzzlerI'm thinking that my number one use for specs will be to get good error messages when working with other libraries, so yes, I think that's what I personally would want. The biggest source of errors in my code is having a false expectation about how someone else's code should work.#2016-06-3002:06seancorfieldGood to know.#2016-06-3002:07seancorfieldWeāve jumped on the Alpha builds at work (in production) in preparation for using clojure.spec fairly heavily but weāre still in the early exploration phase. Weāre already leveraging the new predicates in production code thoā.#2016-06-3002:09seancorfieldWe havenāt fully decided what weāll spec out yet vs what we wonāt bother with. Iāve been exploring using java.jdbc with namespaced keys as part of this work.#2016-06-3002:10seancorfieldWe have over 30Kloc production code and over 11Kloc tests so whatever we do will need to be a gradual process...#2016-06-3002:11seancorfieldā¦expanding what we can do with generative testing alone might bring us a good "bang for our buck".#2016-06-3002:18Tim@seancorfield: at world singles?#2016-06-3002:19seancorfieldYup. Just pushed a build to QA based on Clojure 1.9.0 Alpha 8 so thatāll go to production in our next build.#2016-06-3002:20seancorfield(we already have Alpha 7 in production)#2016-06-3003:58cap10morganI'm getting Don't know how to create ISeq from: spec_demo.core$full_name
when running (stest/instrument full-name)
in an ns where I have a fn full-name
and have (s/fdef full-name ...)
. It works if I leave the argument off (so it instruments everything), but I thought it was supposed to take symbol args now too in alpha8?#2016-06-3004:22seancorfield(stest/instrument 'full-name)
<-- needs to be a symbol#2016-06-3004:26seancorfieldTo explain the error message, (stest/instrument full-name)
will evaluate full-name
, since it's a function it evaluates to its class name which is a class full_name
inside the package spec_demo.core
because the Clojure is spec-demo.core/full-name
.#2016-06-3004:26seancorfieldHope that helps @cap10morgan ?#2016-06-3004:50puzzlerI'm having trouble ascertaining the purpose of the retag in multi-spec. Can someone clarify when you'd actually see the retag?#2016-06-3011:35Alex Miller (Clojure team)In gen#2016-06-3011:35cap10morgan@seancorfield: ah, yes, thanks. I was trying #'full-name
and getting a var but totally forgot about 'symbol
.#2016-06-3011:37cap10morganI'm giving a presentation tonight on clojure.spec at Den of Clojure (Denver meetup). So putting together as many live demoes as I can.#2016-06-3011:44cap10morganHmm, but it still doesn't work. (stest/instrument 'full-name)
doesn't seem to turn on instrumentation (I get an exception when an invalid arg blows up in the fn body rather than spec telling me it's invalid) and (stest/test 'full-name)
just returns ()
. These all worked with just (stest/instrument)
and (stest/test)
.#2016-06-3011:46Alex Miller (Clojure team)If you use back tick it will resolve the symbol to fully qualified#2016-06-3011:47Alex Miller (Clojure team)The symbol has to be fully qualified#2016-06-3011:47Alex Miller (Clojure team)Symbol, not var#2016-06-3011:49cap10morganOK, that fixed it. Thanks, @alexmiller. I think that's the first time I've used back tick outside of a macro. Makes sense that it would need to be fully-qualified since it's going into a central registry.#2016-06-3011:59cap10morganDid clojure.spec.test.check
go away or change in alpha8? Trying to change how many tests get run in clojure.spec.test/test
and test's docstring says to pass a map with a :clojure.spec.test.check/opts
keyword.#2016-06-3012:11Alex Miller (Clojure team)That's just a keyword namespace, not an actual namespace#2016-06-3012:22cap10morganhuh, OK. not sure how to alias it, then. but I guess I don't strictly need to.#2016-06-3012:32jjcomerDoes anyone have an example of stub or replace? Neither seem to be working for me, the original fns still execute.#2016-06-3012:52puzzler@alexmiller I thought multi-specs couldn't auto-generate data since the possibilities can't be enumerated. And if you have to write your own generator, I don't see how the retag comes into play.#2016-06-3013:25bhaumanI'm sure that this has probably been discussed before, a bunch, but I am late to the game and curious about and
flowing conformed values to successive predicates. Initially, it seems very surprising that a successive predicate is operating on a different__ value. This seems semantically more like and->
. Is this in the Guide? It's also curious that the underlying implementation of merge
is named and-preds
.
I know there is a reason for this and admit that I haven't looked closely enough at it.#2016-06-3014:57fenton@puzzler: you can auto-generate data for multi-specs.#2016-06-3015:47Alex Miller (Clojure team)@puzzler did you read the doc-string for multi-spec
? I think it answers all these questions.#2016-06-3015:47Alex Miller (Clojure team)@bhauman: and
vs and->
is under consideration. for now, and
flows conformed values.#2016-06-3015:48bhaumancool, i didn't know and-> was under consideration#2016-06-3015:49Alex Miller (Clojure team)Rich mentioned it in the back chat#2016-06-3015:51bhaumanoh and a big big + šÆ on every
#2016-06-3015:53flipmokidHi, I'm looking to write a spec for a map in clojurescript which is used to check user input. One of the keys requires that the input be an int and be in a set of allowed values which comes from a server (so it's an async call). What is the best way of achieving this?#2016-06-3015:59bhaumanif its deep in the spec, make the call to the server first, and then make a binding, and have your predicate read the binding#2016-06-3016:02bhaumanbut otherwise I would separate this concern from the spec#2016-06-3016:02bhaumanand do a second pass#2016-06-3016:03Alex Miller (Clojure team)you donāt have to make it a fn spec to use spec - you can explicitly call s/valid? to make that check#2016-06-3016:08flipmokid@bhauman: Thanks, that make sense (having that separate from the spec). BTW thanks for figwheel!
@alexmiller I wanted to originally do something like (s/and integer? (call-service)) where call-service returns a channel which eventually returns a set of ints. I wasn't sure of the best way of doing this as that function isn't a predicate
I'm fairly new to clojure btw so please bear with me!#2016-06-3016:09Alex Miller (Clojure team)have you considered just writing some Clojure code? :)#2016-06-3016:10Alex Miller (Clojure team)Iām not sure spec is buying you anything for what youāre describing#2016-06-3016:12flipmokidI will give it a go. Time to crack open Clojure Applied again š#2016-06-3016:14Alex Miller (Clojure team)well nothing about spec in there :)#2016-06-3016:15Alex Miller (Clojure team)those idiots were too dumb to write about it#2016-06-3016:17flipmokidLol. The book is great, thank you! I meant using the book to write some clojure code. I thought I should do the server check separately from the spec but just wanted to make sure.#2016-06-3016:17flipmokidThanks guys#2016-06-3016:17flipmokidMuch appreciated#2016-06-3016:18Alex Miller (Clojure team)you shouldnāt register a spec thatās calling a server, but you could use s/validate? to verify a value conforms to a spec (which has a predicate, which is based on a set obtained from elsewhere)#2016-06-3016:19Alex Miller (Clojure team)however, that ācheckā is so simple, that you might as well just check if the value is in the set (without using spec)#2016-06-3016:23flipmokidWith the first option, what would be the best way of waiting for the set to come back out of the channel in clojurescript inside the predicate? The best my mind could do was poll the channel until there was something on it.
The current map spec I have does have a lot of keys which are much more involved than this one, but this is the only one that needs to check against a remote resource.#2016-06-3017:10fentonwhere do we report issues found with spec?#2016-06-3017:13fentonbut change :matching-p
to :matching
and it works okay... seems hyphens don't work for matching values.#2016-06-3017:16fentonhmm... hold on that... maybe dirty repl#2016-06-3018:49blanceI just read about clojure.spec and it seems like a great tool for testing with test.check. To utilize it for unit testing, should I specify a spec for every functions using fdef
in my application so that I just call clojure.spec.test/test
to do unit testing?
I've never used schema before so I'm not sure which function/args/data I should specify those and which not to#2016-06-3018:58seancorfield@blance: Have you read the Rationale for clojure.spec
? It talks about when to use spec and what sort of stuff itās designed for.#2016-06-3018:59seancorfieldhttps://clojure.org/about/spec#2016-06-3018:59bhaumanSo here is another thing, the :in
path on key value errors in (s/every-kv ..)
end withs a 0 1 presumably to disambiguate between the key and the value, but this renders the :in
path pretty tough to use to look up a value in a nested structure. I'm thinking that intention was to have the :in
path terminate with the map key and the 0 or 1 should be at the end of :path
only#2016-06-3018:59eggsyntax@blance: & lots more detail in the guide: http://clojure.org/guides/spec#2016-06-3019:00eggsyntaxNotably http://clojure.org/guides/spec#_testing#2016-06-3019:01blanceYea I've read both.. It must be that I read through them too fast. I'll read them again.#2016-06-3019:02blanceI mean I know how to use them, but I'm not sure on where to use it. Does it make sense to say have a fdef
for each single function in my app?#2016-06-3019:03eggsyntaxA reasonable shorthand might be: any function that you would otherwise have hand-written a test for.#2016-06-3019:03eggsyntax(and of course for fns where spec is valuable in other ways, eg runtime validation of inputs)#2016-06-3019:04eggsyntaxThat's just my take on it, though, and I'm a spec beginner too. As I guess are we all š#2016-06-3019:05blanceyou would want your unit test to cover as much functions as possible right? so write spec for as much functions as possible as well?#2016-06-3019:06blanceI feel like some fns would take as long time to write a spec to validate its input&output as to write the actual function itself#2016-06-3019:06eggsyntaxMatter of taste. Personally I don't shoot for complete test coverage, just coverage of fns that a) are tricky, b) are at risk for regression bugs, c) could use a test as usage documentation.#2016-06-3019:07eggsyntaxAnd yeah, it's definitely arguable that hand-written tests are a better tool to reach for in some cases than spec is. It'll be interesting to watch as different ideas about best practices for spec emerge.#2016-06-3019:08eggsyntax(although the new stub/mock functionality makes it a tool I'm likely to reach for a lot more often)#2016-06-3019:10blancethat make sense. and I would assume you only need to use clojure.spec.test/test
for unit testing? Do you still hand write test to cover corner cases?#2016-06-3019:10blancewhat's the stub/mock functionality? is that also in spec?#2016-06-3019:11blancesorry I have lots of questions as I'm new to testing as well:sweat_smile:#2016-06-3019:11eggsyntax:stub replaces a fn with a stub that checks :args, then uses the :ret spec to generate a return value.
https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#2016-06-3019:12eggsyntaxTo me it seems like generative testing is, in general, more likely to catch corner cases than hand-written tests, unless there's a specific corner case you want to test for that isn't likely to be generated.#2016-06-3019:13eggsyntax(oh, :stub
is an optional arg to instrument
btw)#2016-06-3019:13blancethat's cool! didn't see that in the doc#2016-06-3019:13eggsyntaxIt's new as of alpha8. I think the guide isn't quite caught up yet.#2016-06-3019:14blanceIll take a look at it. Thanks!#2016-06-3019:16bfabry@fenton: there's no way and
could combine the generators. there's a new function s/merge specifically for that scenario though#2016-06-3019:16fenton@bfabry: okay, will check that.#2016-06-3019:29seancorfield@blance: If you specify "every" function then your system will be brittle: changing anything will likely break specs and youāll have to constantly rewrite them ā itāll be hard to do refactoring. I think specs make sense on "module" boundaries and on "library APIs".#2016-06-3019:31seancorfieldAs for generative testing, you can test a function against a spec (in the test suite) without needing to fdef
it in the main code. That seems like a reasonable approach to me.#2016-06-3019:31fentonhow to spec for a string that should be an alpha-numeric?#2016-06-3019:31seancorfieldUse a predicate with a string regex?#2016-06-3019:33fenton@seancorfield: sorry not understand, example?#2016-06-3019:34seancorfield#(re-find #"^[a-zA-Z0-9]+$" %)
#2016-06-3019:34seancorfield(thereās sure to be a shortcut for that pattern#2016-06-3019:35seancorfieldSo youād have (def ::alphanumeric (s/and string? #(re-find #"ā¦" %)))
#2016-06-3019:37fenton@seancorfield: okay that makes sense...thx.#2016-06-3019:37seancorfieldboot.user=> (s/def ::alphanumeric (s/and string? (partial re-find #"^\w+$")))
:boot.user/alphanumeric
boot.user=> (s/exercise ::alphanumeric)
(["Q" "Q"] ["7" "7"] ["4" "4"] ["6z6" "6z6"] ["RjN" "RjN"] ["r05" "r05"] ["9" "9"] ["o8m" "o8m"] ["5j7YFcm" "5j7YFcm"] ["RFNe5Xj3" "RFNe5Xj3"])
#2016-06-3019:39fenton@seancorfield: even nicer!#2016-06-3019:39eggsyntaxIt'd be nice to do something like (s/def ::alphanumeric (s/and string? #(Integer/parseInt %)))
, but that'll throw an exception rather than failing...#2016-06-3019:40blance@seancorfield: I'm actually working on clojurescipt. most of my fns are api calls, transition db to new states, and UI stuffs. No need to write spec for any of them I guess?#2016-06-3019:41seancorfield@eggsyntax: you would need both a predicate to match numeric strings and a conformer to convert to Long
I think?#2016-06-3019:41bhaumanI looked into the :in
path problems for maps that I mentioned above, it seems like ::kfn
isn't enough to eliminate the following tuple key.#2016-06-3019:41eggsyntax@seancorfield: makes sense.#2016-06-3019:41eggsyntax(& at that point the regex solution is nicer)#2016-06-3019:41seancorfield@blance: Hard to say in absolute terms. Try a few approaches and see what works best for you?#2016-06-3019:42blancesounds good. Thanks!#2016-06-3019:42seancorfieldIād probably spec the API itself (between client and server) and then maybe some of the bigger components within each side. But I suspect you have a lot of "trivial" functions which wouldnāt be worth specāing?#2016-06-3019:43eggsyntaxNew motto: "Don't spec the small stuff"#2016-06-3019:43eggsyntaxš#2016-06-3019:43blanceapi calls are async, or rather returning a core.async channel#2016-06-3019:43blanceso not sure how to spec that#2016-06-3019:47bhaumanits expressive enough for me to fix it in use though#2016-06-3019:48bfabryI figure fdefs probably live at a similar level to unit tests. if you unit test every function in your application it'll be annoying to change, but unit testing still holds value at some "right" level. same with fdefs#2016-06-3019:48bfabryfdefs have the added advantage that they also give you runtime contract checking and easier to understand documentation than unit tests#2016-06-3019:49puzzler@alexmiller I read the doc-string about multi-spec but (sg/generate (s/gen my-multi-spec))
throws ExceptionInfo Unable to construct gen at: [] for:
so I must be misunderstanding the point about generation.#2016-06-3020:17fenton@puzzler: did u see the example i made for u?#2016-06-3020:29puzzler@fenton no, didn't see that. Where?#2016-06-3020:29puzzler@fenton, ah, I see it now.#2016-06-3020:32puzzler@fenton, hmm, not sure why it isn't working for me. Maybe one of the individual multimethods can't generate but the error message is misleading, making it seem like the overall multispec is failing.#2016-06-3020:32fenton@puzzler: did my example work for you?#2016-06-3020:38puzzler@fenton: no, I copied and pasted your code and tried it at the REPL but I'm getting NullPointerException clojure.test.check.generators/call-gen (generators.cljc:41)
This is on the latest alpha. Not sure what's going on.#2016-06-3020:41puzzler@fenton, oh I called gen and generate in the wrong order on your code. So yours is now working for me. But I had it right on my example which wasn't working. So still trying to figure that out...#2016-06-3020:43puzzler@fenton, ok figured it out. I wasn't calling gen on the keyword associated with the multispec, I was calling it on the actual multispec.#2016-06-3020:43puzzler@fenton confusing error messages#2016-06-3020:54puzzlerIn some cases the docstring info provided automatically by spec isn't terribly useful, especially in the case of multi-specs. Is there any way to manually add info to the spec docstring?#2016-06-3021:19fenton@puzzler glad u got it work....dunno about ur last comment tho. š#2016-06-3023:20blanceWould you put all spec defination at one place? say my.appns.spec
? If so, any trick to refer to a namespaced keyword?#2016-06-3023:21blanceI assume nobody want to type {:my.appname.is.very.long/key1 "foo" :my.appname.is.very.long/key2 "bar"}
`#2016-06-3023:25blanceAlso i'm wondering if clojure.spec can spec arbitrary nested vector? for example, geoJSON coordinates can be [1 1]
or [
[ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ],
[ [100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2] ]
]
or [ [100.0, 0.0], [101.0, 1.0] ]
#2016-06-3023:27Alex Miller (Clojure team)You have optionality and recursion, so without knowing more than those examples, yes#2016-06-3023:28Alex Miller (Clojure team)@blance if you are making a spec namespace, I would propose the plural (specs) rather than spec. We will be doing clojure.core.specs for example.#2016-06-3023:29Alex Miller (Clojure team)For namespaces, you can alias with :as in require then autoresolve with :: keywords#2016-06-3023:31blancelike my.appname.specs :as specs
then ::specs/key1
right?#2016-06-3023:31Alex Miller (Clojure team)Yeah#2016-06-3023:32blancecool thanks!#2016-06-3023:41blanceOn a side note, I feel like clojure team does not want us to put all specs at one place, otherwise we would have the option to just describe un-namespaced key which is more convenient.#2016-07-0103:30seancorfieldIām working with instants and ended up with this code: (defn years-ago
"Given a number of years, return an Instant that long ago."
[y]
(-> (LocalDate/now) (.minusYears y) .atStartOfDay (.atOffset ZoneOffset/UTC) .toInstant))
(defn age-18-120?
"Given an Instant, return true if it represents a date of birth such
that someone would be between 18 and 120 years old."
[i]
(s/inst-in-range? (years-ago 120) (years-ago 18) i))
(s/def ::date-of-birth (s/with-gen age-18-120?
(fn [] (s/gen (s/inst-in (years-ago 120) (years-ago 18))))))
Could this be done more cleanly? (without the duplication of the range of years)#2016-07-0104:24bfabry@seancorfield:
(s/def ::age-18-120? (s/inst-in (years-ago 120) (years-ago 18)))
=> :kafka-google-connector.runner/age-18-120?
(defn age-18-120? [x] (s/valid? ::age-18-120? x))
=> #'kafka-google-connector.runner/age-18-120?
(s/def ::date-of-birth ::age-18-120?)
=> :kafka-google-connector.runner/date-of-birth
(s/exercise ::date-of-birth 1)
=> ([#inst"1970-01-01T00:00:00.000-00:00" #inst"1970-01-01T00:00:00.000-00:00"])
#2016-07-0104:26bfabrythough the age-18-120? fn seems totally redundant at that point, but if you still wanted it#2016-07-0104:28seancorfieldNo, because the years-ago
calls must happen when the predicate is applied, not just once at compile time.#2016-07-0104:28bfabryriiiiight, makes sense#2016-07-0104:34seancorfieldAnd I'm assuming the with-gen
generator-returning fn is called each whenever the spec is exercised / test-gen'd but that's not really as important since tests are short-lived, whereas the spec has to be long-lived and check the correct range of dob each time it's conformed / used for validation.#2016-07-0109:58Oliver George@seancorfield: could "now" be part of the data structure you spec/validate? that makes it a simple input rather than implied context.#2016-07-0111:13Oliver GeorgeWill it make sense to update defn to allow a spec to be provided as metadata? seems like it could overlap with pre/post...
(defn option-match
"Default search for local datasource: case-insensitive substring match"
[simple? option query]
{:args (s/cat :simple? boolean? :option ::option :query string?)}
#2016-07-0111:58Alex Miller (Clojure team)No, we won't be doing that#2016-07-0113:09seantempestaSo, specs can also be used to convert invalid data into valid data? Iām reading thisā¦
conformer
macro
Usage: (conformer f)
(conformer f unf)
takes a predicate function with the semantics of conform i.e. it should return either a
(possibly converted) value or :clojure.spec/invalid, and returns a
spec that uses it as a predicate/conformer. Optionally takes a
second fn that does unform of result of first
So how do you use the feature to get the response (possibly converted)
?#2016-07-0113:11bhauman@seantempesta: you have to provide a function that returns :invalid or a result that is the converted value#2016-07-0113:12bhauman(conformer (fn [a] (if (= a "one") 1 :clojure.spec/invalid)))
#2016-07-0113:20seantempestaholy crap, this is great!#2016-07-0113:20seantempestathanks @bhauman!#2016-07-0114:10ghadiwould love to see this answered @alexmiller https://clojurians.slack.com/files/glv/F1MQPJBQT/Specifying_an____annotated_value_map_.md#2016-07-0115:33wilkerlucio@ghadi: one simple option that I see is to move the cached data to the map meta-data, this way the map validation can stay simple#2016-07-0115:36glvBut Iād like to validate the cached data as well. For purposes of the code, itās not valid if those keys arenāt there.#2016-07-0115:36glv(That was clumsily worded, but you know what I mean ā¦ the code expects those keys.)#2016-07-0115:44wilkerlucio@glv: I had a different idea, what you think on this:#2016-07-0115:44wilkerluciohere, I used a conformer
to remove the namespaced keys before doing the map validation, but checking the keys before doing it#2016-07-0115:44wilkerluciothis way all namespaced keys will be validated, then removed just to check the rest of the map#2016-07-0115:53glvHmm ā¦ thatās better than my current horrific solution. š#2016-07-0116:30seancorfield@olivergeorge: Interesting ideaā¦ The data structure comes from the database and we already decorate it with some computed fields that are transient (short-lived) so adding a current timestamp to the data wouldnāt be a hardship.#2016-07-0116:56Alex Miller (Clojure team)@ghadi: sorry, moving today!#2016-07-0116:56jjcomerIām trying to use spec-test/test in my clojure.test tests (that was a lot of test :)). When I try to realize the result of test I get the following exception
java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
#2016-07-0116:56ghadi@alexmiller: good luck!#2016-07-0116:56jjcomerIf I run the test outside of a deftest it works no problem#2016-07-0116:57jjcomerDoes anyone have a strategy they are using to use spec-test/test within a deftest?#2016-07-0117:18glv@alexmiller no worries ā¦ whenever things calm down is fine.#2016-07-0119:14akielWhat is the idiomatic way to spec an empty :args
seq? I currently use (s/cat)
.#2016-07-0212:03leongrapenthinSeems like the "annotated value map" is also asked for here: https://stackoverflow.com/questions/38151446/how-can-i-spec-a-hybrid-map#2016-07-0314:26Alex Miller (Clojure team)This is explained in the generators with s/and section in http://clojure.org/guides/spec #2016-07-0314:27Alex Miller (Clojure team)You should probably also look at s/int-in for int ranges with gen support#2016-07-0321:50bhaumanI'm curious if it's an eventual goal to make explain's :in
path work with core functions like get-in
? Right now the spec'ing of maps as a sequential collection via (every (tuple k? v?))
is causing some ambiguity in the :in
path.#2016-07-0408:16dhruv@alexmiller: thank you very much! That explanation clears it up for me, plus s/int-in
works a treat for my purposes :+1::skin-tone-3:#2016-07-0413:51eggsyntaxIt'd be useful to have an undef
fn or something like that, to clear a spec from the registry. Or some way to clear the registry entirely. Mostly just for use during development. Or, for that matter, just make the registry-ref atom publicly accessible.#2016-07-0414:03rauh@eggsyntax: Hack: (reset! @#'spec/registry-ref {})
#2016-07-0414:04eggsyntax@rauh: Oooh, slick. Thanks!#2016-07-0414:07eggsyntaxI didn't realize that'd get around the private
declaration. Good to know in general.#2016-07-0414:32Alex Miller (Clojure team)@eggsyntax weāve talked about this and I expect there will be something public like this#2016-07-0414:32Alex Miller (Clojure team)eventually :)#2016-07-0414:32eggsyntaxSounds good. & Andre's hack will do me fine for now š#2016-07-0501:50wildermuthnReading āOut of the Tarpitā (http://shaffner.us/cs/papers/tarpit.pdf), and maybe Iāve just got Spec on my mind, but this looks pretty familiar:#2016-07-0501:50wildermuthnhttps://www.dropbox.com/s/fjh9fi3r0e4ldt2/Screenshot%202016-07-04%2021.50.02.png?dl=0#2016-07-0502:04wildermuthnThe articleās description of "Functional Relational Programmingā includes a section on āEssential Logicā that seems to correspond very well with Spec!#2016-07-0509:58Alex Miller (Clojure team)I think it's no accident that Clojure+Datomic overlap pretty well with Out of the Tarpit#2016-07-0513:06Alex Miller (Clojure team)s/and flows conformed values through the predicates and the s/+ conforms to a vector#2016-07-0513:07Alex Miller (Clojure team)Rich has mentioned a split into s/and and s/and-> to have both choices#2016-07-0514:22vikeri@alexmiller: Thanks! Yes it could probably be useful sometimes to not have it flow through the predicates but evaluate them separately.#2016-07-0515:01Alex Miller (Clojure team)separately, the need to validate this specific case of a regex and vector? is common and one Iāve run into many times in specāing macros and there might be some addition specifically for this#2016-07-0520:27aengelbergHas there been any talk about allowing local registries to use with clojure spec functions rather than only using the global registry? Similar to (make-hierarchy)
and derive
.#2016-07-0520:35Alex Miller (Clojure team)I haven't heard such talk :)#2016-07-0520:51pdlugIām trying to use multi-spec
but it doesnāt seem to work with defmethod
ās :default
dispatch, is this a bug or intentional?#2016-07-0520:55kidpolloI donāt think multi-spec behaves exactly like that. I believe there is no default
to be had.#2016-07-0521:00pdlugThatās unfortunate. Is there a better way to handle the validation where a field in the map determines the type and you want to explicitly constrain some types but provide a default base?#2016-07-0521:01pdlugEx: modeling analytics events, the :type
provides the type of event, for :search
events :terms
are required, for :page-view
events :url
is required but for all other events there are no other constraints on the map#2016-07-0521:02kidpolloIt seems it only dispatches on key defined in s/multi-spec
not on the this
#2016-07-0521:05kidpollohmmm but I just tried (s/def :foo/bad-entity (s/multi-spec entity :does-not-exist))
(gen/sample (s/gen :foo/bad-entity) 1)
and the program goes into an infinite loop#2016-07-0521:06kidpollowait! not infinite loop I just get ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.
#2016-07-0521:07kidpollomakes sense it either matches a spec or it does not. I am back to my original argument š#2016-07-0521:08pdlugI think itās unexpected behavior if multi-spec
works w/ multimethods but then doesnāt support the default behavior of them#2016-07-0521:08kidpollo@pdlug: yah I had the same issue I want to dispatch on type
but it has to be based on a key on your map š#2016-07-0521:08pdlugI still donāt see anything else that supports my example use case#2016-07-0521:11kidpolloyeah, my problem is that I want to dispatch on something else than a namespaced key on the map. I want to dispatch on meta
from the map. Not always you want to expose the ātypeā on the actual entity#2016-07-0521:13pdlugmakes sense#2016-07-0521:14pdlugah, I found this open issue: http://dev.clojure.org/jira/browse/CLJ-1935#2016-07-0521:16kidpolloright!#2016-07-0521:47Alex Miller (Clojure team)@pdlug you might also see if s/merge helps you in combining base+specific s/keys#2016-07-0522:03kidpolloYeah but you would be able to have a generic event
spec for all the other types.#2016-07-0612:23ikitommiIs there a easy way to select a conformer based on external input? E.g. for the web - conform strings to numbers only for query parameters, not for json params.#2016-07-0614:51pdlug@alexmiller: Iām not familiar with the clojure bug process, that issue CLJ-1935 says the status is āTriagedā what does that mean exactly? Is there a way to tell if it will be addressed or slated for a release?#2016-07-0614:57artemyarulinhi, what is the right way to enable spec checks globally for my project? I would like that by default everything got checked. I guess I can put something like that in project file:
:repl-options {:init (do (clojure.spec/instrument))}
and then everything should be checked when Iām in a REPL? Oh whatās the right way?#2016-07-0614:58petterik@pdlug: You can find the entire JIRA workflow here "How a Ticket Becomes a Commit": http://dev.clojure.org/display/community/JIRA+workflow
I found it very useful#2016-07-0614:59pdlug@petterik: ah thatās very helpful. Thanks!#2016-07-0615:53Alex Miller (Clojure team)@pdlug in this case, it means āI think Rich should look at itā but he has not yet#2016-07-0616:23pdlug@alexmiller: ok, thanks#2016-07-0701:50seancorfieldI thought alias
had been updated so you didnāt have to have a real namespace for the second argument?#2016-07-0701:52seancorfieldIt seems not. I was hoping to use alias
to shorten a lot of qualified keywords (without having to actually load a namespace).#2016-07-0702:13Alex Miller (Clojure team)no, weāre just considering that#2016-07-0702:49seancorfieldNow weāre beginning to use namespace-qualified keywords a lot more, I can say that would be a very convenient addition! š#2016-07-0703:18Alex Miller (Clojure team)as a short-term workaround you can do something like this: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L17-L19#2016-07-0703:37seancorfieldThatās an interesting hack! š#2016-07-0706:07mandragoranI was wondering how I would spec an async function (a function returning a core.async channel). Would I need to write an async version of fdef or is there another way?#2016-07-0708:19wagjoI'm a bit struggling to choose between :foo.vocabulary/term-uri
and :foo.vocabulary.term/uri
or just :term/uri
. I know it's probably a bit soon, but any best practices for choosing the name(space)s?#2016-07-0709:14mandragoran@wagjo: I've been taking advantage of the new ::alias/some-keyword feature. So I'd probably call it ::term/uri and map the term alias in all namespaces using it to foo.vocabulary.term.#2016-07-0710:13vikeriHow can I reference the first argument of a function in fdef :arg
when the argument does not have a name since Iām destructuring it immediately with {:keys [a b]}
?#2016-07-0710:48vikerinvm, realized the keywords in :arg (s/cat
are not tied to the variable names#2016-07-0711:07Alex Miller (Clojure team)@wagjo if you're in a lib or other public project, you should add enough context to be distinguishable from anyone else (something like your maven group/artifact id). If in a private app, I would probably do less just for typing sake. I think it will be useful to put specs for the domain in a small number of namespaces (maybe 1). It might be useful to separate generic specs from domain attributes (which alias them) too.#2016-07-0711:10Alex Miller (Clojure team)@mandragoran: we will probably have some async related specs eventually. Until we do I think it's better to do type checks on ReadPort and WritePort than specifically on ManyToManyChannel as those are more generic#2016-07-0712:08gfredericksad-hoc higher-order specs?#2016-07-0712:45lvhHow do I write a spec for a map of unqualified keywords where the keyword itself is something dumb thatās going to collide a lot? In particular, swagger (the api format) has a toplevel key, also called āswaggerā.#2016-07-0713:08Alex Miller (Clojure team)can you give an example? not getting the question.#2016-07-0714:06lvhSure! Letās say Iām parsing swagger#2016-07-0714:06lvh(def sample-swagger
{:swagger "2.0"
:info {:title "my sample api for puredanger"
:version "elventy flive"}})
#2016-07-0714:07lvhLetās say Iām in a ns defining swagger in general (doesnāt matter much, could also be :swagger/whatever
). Iām doing something like:
(spec/def ::swagger
(spec/keys :req-un [:swagger/swagger
::info]))
(spec/def :swagger/swagger
;; The top-level key is called swagger. This feels dumb.
#{"2.0"})
(spec/def ::info
(spec/keys :req-un [::api-title
::api-version]))
(spec/def ::api-title string?)
(spec/def ::api-version string?)
#2016-07-0714:08lvhIt seems like Iām naming :swagger/swagger
what it is because :req-un
expects to find the names of map keys that way, but having a ::swagger and a :swagger/swagger
looks confusing#2016-07-0714:09lvhSimilarly, ::api-title
actually needs to be :
or something, but the tons of nsāes make it annoying to type#2016-07-0714:09lvhI guess Iām asking if thereās a way to distinguish between the key under which something appears in a map, and its spec name#2016-07-0714:10lvhIIUC there are some new reader macros in the current alpha, I donāt know if they help with this#2016-07-0714:10lvhthey might limit the typing to just the spec defs so at least the destructuring looks good, I suppose?#2016-07-0714:12lvhif I had a choice, in this code, I would refer to that specced value (esp. after conform) as ::api-title
ā the only reason I wouldnāt do that is because the JSON Iām getting from swagger (a standard, so I canāt change it) uses keys that donāt collaborate well with that#2016-07-0714:13lvhIf I could provide, say, an alias, or say that under key :title
Iām expecting to see spec ::api-title or something, thatād help too#2016-07-0714:16lvhbailing out and using a fn to just assert that some keys are some particular value doesnāt feel very idiomatic either#2016-07-0714:33lvhI guess you can define specs in terms of other specs? That doesnāt seem to help a lot but Iāll play with it#2016-07-0715:40donaldballHas anyone run across or is writing a lib of commonly useful strong specs, e.g. for urls, hostnames, unix ports, valid sql identifiers, etc.?#2016-07-0715:52Alex Miller (Clojure team)I haven't seen anyone doing so yet but it seems like a reasonable and perhaps inevitable thing to do #2016-07-0716:40Alex Miller (Clojure team)@lvh aliasing is a good and useful thing, so creating api-title and api-version are just fine as base specs (you may want to use some other namespace though like :swagger/api-title
or :swagger.api/title
), then s/def code-localized versions to match the unqualified attributes that alias those base specs, like (s/def ::title :swagger/api-title)
#2016-07-0716:41lvhOK, so I define the ārealā spec under :swagger.ap/title, and then a local alias. Makes sense. Thanks!#2016-07-0716:42Alex Miller (Clojure team)if you create the keyword namespace as an actual namespace, you can alias it, then use the alias in the autoresolved keyword like ::sw/api-title
(after something like (alias āsw āswagger.api)
)#2016-07-0716:44Alex Miller (Clojure team)the key there is that alias currently has to refer to a real (loaded) namespace. That might change still but a workaround is what you see here https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L17-L19#2016-07-0716:44Alex Miller (Clojure team)switch to (and thus create) the namespace, then switch back, and alias#2016-07-0717:32leongrapenthinToday we wanted to create a spec that validates a percentage in five percent steps. It occured to us that int-in doesn't support a step argument like range does. Has this been discussed or is a ticket worth a shot?#2016-07-0717:39sparkofreasonBefore I get too far in reinventing the wheel, is anyone else out there working on converting spec to Datomic/Datascript schema?#2016-07-0717:48Alex Miller (Clojure team)@leongrapenthin: no, it's not going to do that#2016-07-0717:49leongrapenthin@dave.dixon: I have thought about it and realized that generating a spec from Schema is likely more desirable#2016-07-0717:50donaldballIām not sure what Iām doing wrong with this simple spec:#2016-07-0717:50donaldball(s/def ::foo (s/spec #(= :foo %) :gen (gen/return :foo)))
:user/foo
user> (s/gen (s/get-spec ::foo))
ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn clojure.spec/spec-impl/reify--13361 (spec.clj:800)
#2016-07-0717:50leongrapenthin@alexmiller: Ok, thanks#2016-07-0717:50Alex Miller (Clojure team)@leongrapenthin: just do (set (range 0 100 5))
#2016-07-0717:51donaldballOops, I misread the contract for s/spec, carry on#2016-07-0717:51leongrapenthin@donaldball: Wrap (gen/return ...) in a (fn [] )#2016-07-0717:51Alex Miller (Clojure team)yep, or put # at the beginning#2016-07-0717:52Alex Miller (Clojure team)(s/def ::foo (s/spec #(= :foo %) :gen #(gen/return :foo)))
(gen/sample (s/gen ::foo))
#2016-07-0717:55leongrapenthin@alexmiller: Yeah, but that's gonna be quite the error message š - We came up with (s/and (s/int-in 0 101) #(zero? (mod % 5))) if I remember correctly. Generator worked. Projecting this scenario on per permille or larger spaces was what made us wonder why a step is not supported.#2016-07-0717:56Alex Miller (Clojure team)because itās a min/max check, and youāre specifying enumerated values. thatās just not what it is intended to do.#2016-07-0717:56Alex Miller (Clojure team)I think your spec is better anyways#2016-07-0717:57Alex Miller (Clojure team)you could combine int-in with it too#2016-07-0717:57Alex Miller (Clojure team)nvm, youāre doing that#2016-07-0717:58leongrapenthinYeah, the reasoning makes sense#2016-07-0717:59Alex Miller (Clojure team)the hard part of int-in is generating uniformly across a well defined set and youāre leveraging that by building on top of it#2016-07-0718:00leongrapenthinLooking at that from a different perspective it is only enumerated because of the min-max check#2016-07-0718:00leongrapenthinYou could argue that everything in a min-max interval is enumerated then#2016-07-0718:01Alex Miller (Clojure team)well enumerations have obvious usage issues when there are a lot of values :)#2016-07-0718:04leongrapenthinMy case is a very specific enumeration and can't be discarded as just any enumeration. Every step range of allowed values is a subset of an integer range with the same boundaries.#2016-07-0718:05leongrapenthinSo from that perspective I think it still makes sense as a feature of a min/max check.#2016-07-0718:05leongrapenthinThe min/max check is also an enumeration with a step of 1, then.#2016-07-0718:14leongrapenthinBetter said, for our usecase it would be preferable if it was an in-range check than a min/max check with the word range meant in the same way as in clojure.core/range#2016-07-0718:17Alex Miller (Clojure team)well, range does a lot of stuff (reverse ranges, infinite ranges, half-open ranges, support for all number types, etc)#2016-07-0718:18Alex Miller (Clojure team)in-range implies to me a check that would need to enumerate the range to determine conformance and gen would be a lot harder.#2016-07-0718:18Alex Miller (Clojure team)int-in is intentionally much narrower#2016-07-0718:25leongrapenthinAgreed. I think what I want to say is that int-in as a name doesn't necessarily limit the spec to a min/max check#2016-07-0718:27leongrapenthinA fully featured in-range is likely an overkill for the rare amount of usecases#2016-07-0718:44leongrapenthinThanks for discussing. Having thought about it I won't be able to resist to submit a small patch over the weekend for reconsideration, though š#2016-07-0718:46Alex Miller (Clojure team)I think Rich is unlikely to ok that#2016-07-0718:48kidpolloI am running stest/check on an fdef that never finishes. It just keeps going and I don't know If its stuck or what is the deal. I have independently tested the input and output specs for possible problems and am getting nowhere. Any tips on how to debug this?#2016-07-0718:58kidpolloexercise-fn
seems to work fine#2016-07-0719:02leongrapenthin@kidpollo: Isolating the problem by starting with dumbed down specs and adding until the problem occurs?#2016-07-0719:02kidpolloI also tried (s/exercise-fn my-fn 100)
and it takes like 1 min but it works.#2016-07-0719:03leongrapenthin@kidpollo: Probably recursion in s/map-of or every-kv?#2016-07-0719:04leongrapenthinMight be that it just takes so long to print the result because it prints a huge generated value#2016-07-0719:05kidpolloCould be.#2016-07-0719:05kidpolloI am running on the repl#2016-07-0719:05kidpolloon intellij#2016-07-0719:06leongrapenthinYou could eliminate this quickly with setting print-length#2016-07-0719:07Alex Miller (Clojure team)dump stack and see what itās doing#2016-07-0719:07Alex Miller (Clojure team)I think intellij has a button for it#2016-07-0719:08kidpolloohh let me see#2016-07-0719:10Alex Miller (Clojure team)I guess maybe thatās only in the debugger mode#2016-07-0719:14Alex Miller (Clojure team)(map println (.dumpAllThreads (java.lang.management.ManagementFactory/getThreadMXBean) false false))
#2016-07-0719:14Alex Miller (Clojure team)will serve in a pinch#2016-07-0719:16kidpolloyah I was able to dump them in debug mode#2016-07-0719:16kidpollojust see a bunch of locked threads in generators#2016-07-0719:18kidpollodamn 700% cpu usage#2016-07-0719:21kidpolloI guess i need to find what spec is going crazy. Start with simpler specs maybe? The input and output to this fn are multispecs. The maps are not so complicated maps.#2016-07-0719:47kidpollo@alexmiller @leongrapenthin the problem seems to be happening in alpha-9. I went back to alpha 8 and the test finishes. It takes a while but it finishes#2016-07-0719:57Alex Miller (Clojure team)I think check
changed from 100 to 1000 iterations in alpha9#2016-07-0720:20kidpollooh my that would explain it#2016-07-0720:20kidpollowhen it produced errors it did end earlier#2016-07-0720:21kidpolloI guess Ill leave it over night to see how long it actually takes#2016-07-0720:40Alex Miller (Clojure team)That count is an option you can pass to check though iirc#2016-07-0721:12kidpollowhat is iirc?#2016-07-0721:13jr(if I recall correctly)#2016-07-0721:17kidpollonice! snme in spanish š (si no me equivoco)#2016-07-0721:18kidpollolol ^ totally does not exist in spanish š#2016-07-0721:26kidpolloIt seems it would look like:#2016-07-0721:28kidpolloIs it encouraged that all clojure projects start using fully qualified keywords?#2016-07-0721:47Alex Miller (Clojure team)Nothing categorical. Qualified keywords have utility - if they make sense for you, use them#2016-07-0721:48Alex Miller (Clojure team)I think spec and the other syntax changes have increased their utility and decreased their verbosity, maybe changing that equation#2016-07-0721:52danielcomptonAnd their visibility š#2016-07-0721:52Alex Miller (Clojure team)Well I don't think that changes their utility :)#2016-07-0721:53seancorfieldI can see us (World Singles) using qualified keywords very heavily as it will make some of our code much more explicit. Hence the recent changes to java.jdbc
to make it easier to get qualified keywords back in result sets (the new :qualifier
option to everything that produces result sets).#2016-07-0722:43ajssIn spec/fdef, is there any way to make one element of :args depend on another? I've got a function that takes two maps, but wants the keys of each map to be the same.#2016-07-0722:55kidpolloyou can define the relationship between inputs and outputs in :fn#2016-07-0722:55glvRight, but not constrain inputs based on relationships.#2016-07-0722:56glv(For example, I have functions that take a grid and a coordinate within that grid, and obviously the coordinate must be within the bounds of the grid to be valid. But I donāt think thereās a way of expressing that.)#2016-07-0722:58kidpolloSounds similar to the ranger-rand
example in http://clojure.org/guides/spec#_spec_ing_functions#2016-07-0722:59kidpolloinstead of using :ret
just use named args in :args
#2016-07-0723:01ajssso (s/and (s/cat :a map? š map?) #(= (keys (:a %)) (keys (:b %))))?#2016-07-0723:02ajssah - ": b" = š#2016-07-0723:03ajsswill run out of generator iterations won't it?#2016-07-0723:23kidpollowell you would need to replace map?
with a more "narrowed down" spec#2016-07-0723:28ajssah, yes that would work. Thanks š#2016-07-0723:31Alex Miller (Clojure team)@glv you can s/and an arbitrary predicate in the :args spec#2016-07-0723:33Alex Miller (Clojure team)and flows conformed values so it should already be nicely destructured if you add it last#2016-07-0723:35glvAh, gotcha. Iāll try to incorporate that into my specs when I get a chance.#2016-07-0723:38glv(Also, a reminder about https://clojurians.slack.com/files/glv/F1MQPJBQT/Specifying_an____annotated_value_map_.md (and @wilkerlucioās solution https://clojurians.slack.com/archives/clojure-spec/p1467387857001732)#2016-07-0800:49donaldballI started a library with some ostensibly generally useful specs, e.g.: https://github.com/SparkFund/useful-specs/blob/master/src/specs/internet.clj#2016-07-0800:49donaldballWould very much appreciate feedback on design, etc. Itās my second day using spec in anger, as it were.#2016-07-0801:25Oliver Georgeit'd be nice if there was a way to modify how instrument reports. the default spec-checking-fn can be super verbose when turning values to str for the ex-info message.#2016-07-0801:26Oliver GeorgeHere's my quick workaround. Not perfect but gives me something nicer to investigate: https://gist.github.com/olivergeorge/c4a26886b7bbd05f2e9018614a2213ff#2016-07-0801:26Oliver GeorgeI rely on cljs-devtools to make the console.debug output to be expadable#2016-07-0802:07Alex Miller (Clojure team)@glv I have much to report re the āhybrid mapā stuff, but not quite ready yet to do so#2016-07-0802:08Alex Miller (Clojure team)have been working on it much of yesterday and today#2016-07-0802:11Alex Miller (Clojure team)@olivergeorge: you have explain-data ex-data coming back in the exception - why not write a function that customizes output from that, rather than hacking spec-checking-fn?#2016-07-0802:13glvThat tells me that it's a legit problem that y'all are taking seriously and working on, so I'm cool with that. :-)#2016-07-0802:16bbloom@glv: There some considerable advantages to just using another map. Iām curious what @alexmiller is working on, but still: you should just add a :grid key or similar#2016-07-0802:17bbloomwhat if you want to be able to represent a board without a starting coordinate?#2016-07-0802:17bbloomyou can easily take a grid and merge in some extra options, but how easy is it to dissoc them?#2016-07-0802:18bbloomwhat if you want to change the representation? now youāve coupled your param spec with your grid spec#2016-07-0802:18bbloomwhat advantage is there to abusing the fact that keywords and grid coordinates are disjoint? slightly shorter syntax? seems not worth the complexity#2016-07-0802:18glvThis isn't the representation of the grid; it's the result of a particular kind of analysis of the grid, which always, inherently has a starting coord.#2016-07-0802:19bbloomcomposite structures come in two flavors: homogenous and heterogenous#2016-07-0802:19bbloomof course if you have disjoint key sets, you can combine them#2016-07-0802:19bbloombut itās just clearer to keep them separate#2016-07-0802:20bbloomalso when combined, you canāt enumerate keys without filtering extra keys#2016-07-0802:20glvI'm certainly willing to entertain arguments that I should push the coord->distance mapping down a level.#2016-07-0802:20bbloomi guess iām making such arguments š#2016-07-0802:21bbloombut more importantly: what advantage is there to flattening it?#2016-07-0802:21glvSo far, I don't find them convincing. :-)#2016-07-0802:21glvI see three advantages.#2016-07-0802:21Alex Miller (Clojure team)@glv totally legit - I have a couple different solutions with existing tools and something new for just this problem that may help if it comes together#2016-07-0802:22bbloomiām not saying hybrid maps are never useful#2016-07-0802:23bbloomiām saying that itās pretty rare and barring additional info, i donāt think youāve found a justified use case š#2016-07-0802:25glv1: the coord->distance mapping is the primary result of the algorithm. The ::start-coord, ::max-coord, and ::max-distance entries are annotations that could be reconstructed (with some cost) if they weren't present. But the only way to make a non-hybrid map would be to leave the secondary annotations at the top level and push the primary info down into a nested map. That doesn't seem right to me.#2016-07-0802:26bbloomyou can return those keys + a ::grid key and then create a helper function which simply extracts the grid key ā then the caller can decide what is āprimaryā or not#2016-07-0802:26glv2: the code that's involvedāboth the code that builds the map and that which uses itāwould get more complex and clumsy (and slower) that way#2016-07-0802:27bbloomreally? how?#2016-07-0802:29glv3: That's the way it is now, and it has never caused a hint of a problem; the only reason to change would be to accommodate clojure.spec. #2016-07-0802:29Alex Miller (Clojure team)I donāt think itās that rare/weird#2016-07-0802:29Alex Miller (Clojure team)others (including me) have run into same problem#2016-07-0802:30glvHow? Isn't that obvious? The vast majority of the updates and accesses will be to the coord->distance mapping, so pushing that a level deeper means extra lookups in the vast majority of cases. #2016-07-0802:31bbloomalexmiller: any examples you can share? i donāt think iāve ever encountered such a hybrid map that didnāt (surprisingly quickly) need a (->> hybrid keys (filter #(ā¦ that could just be replaced by (-> hybrid :foo keys)#2016-07-0802:31bbloom@glv: just build the grid and then assoc the extra keys in at the end#2016-07-0802:31Alex Miller (Clojure team)sure, I ran into it with map destructuring (so, a syntax example) - the base map is symbol->any but also special options like :keys, :syms, :or, :as etc#2016-07-0802:32bbloomalexmiller: heh, yeeaaaah i was wondering if you were going to mention macros. destructuring is like the ultimate core macro š#2016-07-0802:32Alex Miller (Clojure team)itās fun#2016-07-0802:32bbloom@alexmiller itās funny b/c syntax is the one place where ājust a little bit shorter syntaxā is totally worth it for a common operation#2016-07-0802:32glv@bbloom: even if that were sensible (it's not) it would only help on the creation side of the process, not the use side.#2016-07-0802:34bbloom(:grid (let [start 123, m ā¦build up m hereā¦.] {:start start :m m}))#2016-07-0802:34glvIt's not sensible because keeping track of those values through the process (so that you could assoc them in at the end) would be clumsy compared to just tracking them in the map as I go.#2016-07-0802:34bbloomseems pretty sensible to me š#2016-07-0802:35bbloomer i meant: :grid m#2016-07-0802:35glv::start-coord is predefined; the other two emerge out of the process itself.#2016-07-0802:35bbloomgotcha#2016-07-0802:35bbloomiād use mutation š#2016-07-0802:37bbloom@alexmiller: one thing iāve discovered tinkering on language design: it turns out that language designers have to do the things that they tell other people not to do, heh#2016-07-0802:37Alex Miller (Clojure team)that is a thing#2016-07-0802:38glvcf. Stu's recent tweet about why clojure.core can't be idiomatic. #2016-07-0802:40glvhttps://twitter.com/stuarthalloway/status/741860271925432324#2016-07-0802:41bbloomanyway, @glv definitely wait to see what alex et al come up with. the emitting things during the middle is IMO almost a reason to abuse the key space š#2016-07-0802:41glv"abuse" :-)#2016-07-0802:42bbloomiām not above loaded language š#2016-07-0802:43bbloomthanks for the discussion! gotta run#2016-07-0802:43glvSure!#2016-07-0806:41Oliver George@alexmiller thanks for the suggestion. I would love to do that but don't know how. I presume you are suggesting a big try/catch around my code... my code is typically om/re-frame UI stuff which complicates that.#2016-07-0806:41Oliver GeorgePerhaps I'm missing something obvious#2016-07-0814:01donaldballJust to confirm, we donāt yet have a way to unite the specs for e.g. homogenous (`map-of`) and heterogenous (`keys`) maps?#2016-07-0814:06glvOnly by doing something like this: https://clojurians.slack.com/archives/clojure-spec/p1467387857001732#2016-07-0814:07glvWhich isn't bad at all, but could be made more expressive.#2016-07-0814:14Alex Miller (Clojure team)no, thatās bad :)#2016-07-0814:14Alex Miller (Clojure team)b/c it turns it from a declaration of truth into a process#2016-07-0814:15Alex Miller (Clojure team)there are a couple of ways to do it with existing tools#2016-07-0814:18Alex Miller (Clojure team)one other way is to treat the map as a coll-of s/or of s/tuples (where each tuple describes different kinds of map entries). For cases where the āhybridā-ness does not include registered attributes, this is probably a good choice. So something like a map that was either string->string or number->number.#2016-07-0814:19Alex Miller (Clojure team)you can handle registered attributes in the same way, but then you lose the goodness of s/keys and the attribute semantics#2016-07-0814:20Alex Miller (Clojure team)so in that case, you want to s/merge an s/keys (for the registered attributes) with something that more accurately states the truth of the map definition#2016-07-0814:20Alex Miller (Clojure team)sorry for not being detailed here - I hope to write a blog with the detail#2016-07-0814:21Alex Miller (Clojure team)and then Iām working on something new that will make this a little more palatable, but itās not quite done yet#2016-07-0814:23glvWell, compared to my attempt at a solution, itās not bad. š#2016-07-0817:29eggsyntaxIn 1.9, is there a way to disable namespaced-map output (eg for data that's going to be parsed by a process that's unaware of the idiom)? ie to get output like {:foo/bar 1}
rather than #:foo{:bar 1}
? [Edit:simplify example]#2016-07-0817:53eggsyntaxAnswer, if anyone's curious: CLJ-1967 will address this. For now ya gotta hack it if ya need it.
http://dev.clojure.org/jira/browse/CLJ-1967#2016-07-0818:12donaldballAre double-in
and friends macros so that clojure core doesnāt require test check or as a performance optimization?#2016-07-0819:56donaldballexercise-fn
is pretty rad, but correct me if Iām wrong, it looks like if the fn in question has an options map specified with s/keys
using :opt
or :opt-un
, the chances of getting a map with those keys isā¦ small#2016-07-0819:57donaldballShould the generator for s/keys
bias towards optional keysā presence?#2016-07-0820:08jrI think they are macros because they build on other macros (like spec
)#2016-07-0820:11stuarthalloway@donaldball: double-in
is a macro to capture the form for error reporting, IIRC#2016-07-0820:18Alex Miller (Clojure team)in general, all of the spec creators are macros to capture forms#2016-07-0820:20donaldballIām afraid the implication is not clear to me š#2016-07-0820:20donaldballI just wrote a decimal-in
spec creator fn: https://github.com/SparkFund/useful-specs/blob/master/src/specs/number.clj#L6#2016-07-0820:21donaldballWhat am I missing by (lazily) writing it as a fn instead of a macro?#2016-07-0820:26Alex Miller (Clojure team)(s/explain (s/double-in :min 0.0 :max 5.0) 20.0)
val: 20.0 fails predicate: (<= % 5.0)
=> nil
(s/explain (decimal-in :min 0.0 :max 5.0) 20.0)
nil nil 0.0 5.0
val: 20.0 fails predicate: pred
#2016-07-0820:26Alex Miller (Clojure team)that ^^#2016-07-0820:26Alex Miller (Clojure team)by using let from test.check, youāve also made test.check a production time dependency for you#2016-07-0820:27Alex Miller (Clojure team)(which is a bummer given how nice let is - I keep wanting it too)#2016-07-0820:29Alex Miller (Clojure team)but you can easily rewrite that with gen/fmap#2016-07-0820:29donaldballYeah, I plan to replace it with fmap
ā¦ jinx#2016-07-0820:30Alex Miller (Clojure team)double-in as a macro (and spec under it) allows the form of the predicate to be captured as a form as well as evaluated as code#2016-07-0820:31donaldballThanks, that makes good sense#2016-07-0820:31donaldballBtw any opinion on my optional keys generator observation?#2016-07-0820:34Alex Miller (Clojure team)did you maybe not generate enough to tell?#2016-07-0820:35Alex Miller (Clojure team)(gen/sample (s/gen (s/keys :req [::a ::b] :opt [::c ::d])))
=>
(#:spec.examples.guide
{:a 0, :b 0}
#:spec.examples.guide
{:a 0, :b 0}
#:spec.examples.guide
{:a 0, :b -2, :d -1, :c -1}
#:spec.examples.guide
{:a 0, :b -2, :c -2}
#:spec.examples.guide
{:a 3, :b -1}
#:spec.examples.guide
{:a 0, :b 3, :c 0}
#:spec.examples.guide
{:a 13, :b -29}
#:spec.examples.guide
{:a -17, :b -1}
#:spec.examples.guide
{:a -12, :b 4, :c 0}
#:spec.examples.guide
{:a 5, :b 5, :c -8, :d -9})
#2016-07-0820:35Alex Miller (Clojure team)I see optionals showing up pretty regularly there#2016-07-0820:37donaldballHmm, maybe Iām doing something else wrong. Thanks.#2016-07-0820:40Alex Miller (Clojure team)Iām not discounting the possibility of something wrong either :)#2016-07-0820:40Alex Miller (Clojure team)just that itās not obvious to me#2016-07-0820:40donaldballHmm, looks like keys*
and keys
have very different behavior#2016-07-0820:41donaldball(gen/sample (s/gen (s/keys* :opt-un [::foo])))
(() () (:_.c/*! [[{{} (Kk.B/?Q)}]]) (:AC!e._*0_/V ((({:*g #uuid "99a8b1d6-d3c3-40f7-a7c9-0fd7beaf14e8"})))) (:q?a.*kS.K*q?/?*k ()) (:x8x6?A.!/qU_m? {#uuid "5f81c97f-0e94-4a95-bdae-ff2adaba5378" 5} :T9y17p.!7p/!5a8 () :?x.d95?.*.OVnzN_/?3?J* [] :_-+U.z.F2JXV6.U070/+9a2? [()] :xQb ["<[ĆĀ©" .EM_N 1]) (:FtROj-1 [(19 :q0P:x5F:*T*41_G:H5_G:N-xN)] :?6IS.a+.AE4/vq2* [] :G.G+.d7Z!.w_+.a.C23/? () :?P*M/tj_Zg_ ()) (:XX [()] :F_Rv+.T+*n.Vp+2VR.w/+1tJQTg () :+R+9.r.per24*!9._NNt4*+.+.d5*T!.!9c/I {} :b.mw*bbbXT.*T+*-?g._h!!3_tg.Jvx.I.*++g14/+-S?Znvq () :K.H220.A9?g?h.s71-_+42.!9.-.z9iq4D*/jr83+S {} :x0w5+v.W!l6!s+/*5a? {0 -7}) (:Y!!D9w/J- [] :+!HS_l.XG+._XK*Bw_.UNW?.FPV20-3+_.*MYrk.Z1.pk-_!_/X*v {g._Y?!.QN+9g!/q.zC00Wj4 \space, Bj_t9- J?E**, true s!E-fSL4.R3Oj-A4._!vJ1!!.Vs5iz*n9a.-dM?5!9c.U.?7.*y/?-p+a!Lc, 1.0 :kP+b2Y:D*AA:74:U:-6yKx:phX!9??:I*Is, #uuid "f34904dd-a05a-4247-ad6d-1972baa0f58f" true, false -4.0, -5 false, \Ć¾ 5} :y*!G [:*_p?d?:-!:9f9:l0-3-Cw_5:2uT7f:46-_8P!h:C49*!*?gb:QJw:x8D \Åø 0.5 1/6] :mH.FH_PUY/Ik!3+z? ([{([]) [[*QN.]]}]) :ez0_.qeTiE_?.?-.d11.DrQDGZXu7.j9+7ou/N3M*T {} :!vdNK.-!!cb/Gv {ZS.K_d.fWR2/f+6.e?S. y4, v.w "|u\b"}) (:j65f.Z3r*.e8V7+6Y.L5_d.l+R8.dTp.L.l0n*9Z.AmW*_W8/c___ ({{} [()]})))
user> (gen/sample (s/gen (s/keys :opt-un [::foo])))
({} {} {} {:foo 1} {:foo 0.75} {:foo 0} {} {:foo -2} {:foo -8} {})
#2016-07-0820:42Alex Miller (Clojure team)oh, there was a bug in keys* gen that Rich fixed in master today#2016-07-0820:42Alex Miller (Clojure team)you said keys :)#2016-07-0820:43donaldballSorry, I just assumed keys*
was a pure wrapper macro#2016-07-0820:43Alex Miller (Clojure team)https://github.com/clojure/clojure/commit/1e236448104fc8a0fc51e26eae7cdb7e650b7ae9#2016-07-0820:44Alex Miller (Clojure team)itās a composition of &, a conformer, and keys#2016-07-0823:15ajssnot sure if others are interested, but I've figured out the nice way to do dependent types for spec/fdef arguments : use with-gen on the :args parameter
(defn sum [a b] (+ a b))
(spec/fdef sum {:args (spec/with-gen (spec/tuple int? int?) (constantly (gen/fmap (fn [x] [(* x 2) (* x 3)]) gen/int)))})
#2016-07-0901:23donaldballIām kinda wishing s/def
allowed a docstring. The generated docstring is okay, but I think Iād like to be able to communicate the intent also.#2016-07-0902:01donaldballWeirdly, I had a good test running in an earlier repl with stest/check
but now itās raising#2016-07-0902:01donaldballCaused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn
#2016-07-0914:35robert-stuttafordso we have s/merge
to make a single s/keys
out of two other s/keys
. i'm wondering, how do i spec a map that has some base set of req+opt keys, but also has an optional set of keys that must appear together as if speced with s/keys :req ...
?#2016-07-0919:38arohnerhas anyone seen java.lang.IllegalArgumentException: No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13361
#2016-07-0919:38arohnerIām doing something wrong, but I havenāt found it yet#2016-07-0919:40arohnerhrm, looks like I needed to clean
after upgrading clojure#2016-07-0920:50kendall.buchanan@donaldball: Iāve been experiencing the same with check-var
#2016-07-1012:17jjcomer@donaldball: I get that error when I execute in a deftest.#2016-07-1015:12donaldballI get it both when executing in a deftest and in cider, though it works fine from a repl#2016-07-1015:16donaldballI was just experimenting with ignoring Richās advice and expressing specs for strings as spec regexes. I find I actually quite like a couple of things that fall out: the form is easier to understand that the string regex, and you get a generator for free.#2016-07-1015:17donaldballhttps://gist.github.com/dball/e096fc7fb600fbc53fd94e4b367cf68f#2016-07-1015:19donaldballBut unless Iām missing something, you pretty much have to write one spec for the seq of chars and another spec for the string that unwraps it as a seq#2016-07-1015:20donaldballSetting aside the merits of the idea, is there a way I could do with s/and
and a conformed value?#2016-07-1018:25craigyhey all, I'm trying to figure out if I can use spec for parsing strings (sentences) into structured data. I'm struggling with whether or not I will need to split
around spaces, and how to handle things like phrases (sequences of words) if I use s/cat
. is this kind of use case intended at all, or am I better off with something else?#2016-07-1019:02mishanot-sure-fry#2016-07-1019:07xcthulhuHow do I write my own generator for cljs.spec
?#2016-07-1019:07xcthulhuI have data structures like {0 0.0, 1 0.1, 2 0.2, 3 0.3, 4 0.4, 5 1.0}
#2016-07-1019:07xcthulhu(maps of integers from 0 to 5, increasing in value as the index increases, where one of the values must be 1.0 and all of the values must be within [0,1])#2016-07-1019:08xcthulhuI can generate these things with this function:
(defn generate-random-map
"Return a random structured map"
[]
(let [values (sort (repeatedly 6 rand))
pivot-value (last values)]
(into {} (map-indexed vector (map #(/ % pivot-value) values)))))
#2016-07-1020:19gfrederickshuh.#2016-07-1020:23gfredericksxcthulhu: https://www.refheap.com/121363#2016-07-1020:27xcthulhuI'd rather not rope in test.check. Too bad that clojure.spec doesn't let you register generators separately from specs#2016-07-1020:28xcthulhuBecause I wouldn't mind associating a generator with my specs in my unit tests where I do use test.check#2016-07-1020:34xcthulhuI think you can do this with just just spec since it gives you 'cat' and 'fmap'#2016-07-1020:35gfredericksxcthulhu: I think clojure.spec is designed with the intention that you rope in test.check for non-trivial generator needs#2016-07-1020:35gfredericksis it just that you don't want test.check required in a non-test namespace?#2016-07-1020:37xcthulhuYup#2016-07-1020:38xcthulhuBut like I said, they do give you enough so you don't need test.check, it's just clumsy#2016-07-1020:38gfrederickswhere's fmap?#2016-07-1020:39gfredericksclojure.spec.gen/fmap?#2016-07-1020:39gfredericksyou could probably rewrite my code using clojure.spec.gen if that's what you really want#2016-07-1020:40gfrederickslooks like gen/shuffle isn't there#2016-07-1020:41henrytill@xcthulhu: arenāt you already pulling in test.check
as a transitive dep if yr using something the gen
namespace?#2016-07-1020:41gfredericksnot necessarily#2016-07-1020:42gfrederickssince it's lazy loaded you can get away with not having test.check available if you never run the generators/tests#2016-07-1020:42gfrederickswhich I think is the whole point of the lazy loading#2016-07-1020:42henrytillok, makes sense#2016-07-1020:43xcthulhuYeah#2016-07-1020:43xcthulhuOr cljs.spec.impl.gen for the brave#2016-07-1020:44gfredericksI wonder how they do lazy loading in cljs#2016-07-1020:46xcthulhuDynaload#2016-07-1020:46xcthulhuhttps://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/impl/gen.cljc#2016-07-1020:47gfredericksoh man#2016-07-1020:47gfredericksa custom var deftype#2016-07-1020:59xcthulhuYeah, it's pretty hacks. Anyway, if I read this right they lazily load vector from test.check. Sadly, clojureScript doesn't give you double*, only double#2016-07-1021:00xcthulhuAnyway, thank you I see how to do this crazy thing now#2016-07-1022:00xcthulhuHere's the generator I ended up going with BTW:
(gen/fmap
(fn [values]
(let [max-value (apply max values)
min-value (apply min values)]
(->> values
sort
(map #(-> % (- min-value) (/ (- max-value min-value))))
(drop 1)
(zipmap (range)))))
(gen/vector
(gen/such-that (complement #{infinity, (- infinity)}) (gen/double))
7))
#2016-07-1022:01xcthulhuThis hacks around the fact that clojurescript doesn't have double*
#2016-07-1022:02xcthulhuinfinity
is defined to be Double/INFINITY
and js/Number.INFINITY
in the jvm and js respectively#2016-07-1022:02gfredericksthat should still give you nans though#2016-07-1022:02xcthulhureally?#2016-07-1022:02gfredericksgen/double generates nans#2016-07-1022:03xcthulhuNot with high probability, but I'll go add a restriction on to the such-that
#2016-07-1022:03gfredericksshould be high enough to matter#2016-07-1022:03gfredericksunless I screwed something up#2016-07-1022:04gfredericksroughly 1% of the time#2016-07-1022:12xcthulhuHere we go:
(defn nan?
"Tests if a value is a NaN or not"
[x]
#?(:clj (and (double? x) (.isNaN x))
:cljs (js/isNaN x)))
;;;;;;;;;;;;;;;;
(gen/fmap
(fn [values]
(let [max-value (apply max values)
min-value (apply min values)]
(->> values
sort
(map #(-> % (- min-value) (/ (- max-value min-value))))
(drop 1)
(zipmap (range)))))
(gen/vector
(gen/such-that (complement (some-fn #{infinity, (- infinity)} nan?))
(gen/double))
7))
#2016-07-1022:12xcthulhuHopefully this yak has been shaved by now#2016-07-1022:17gfredericks:)#2016-07-1102:22Alex Miller (Clojure team)There is s/double-in with infinity and nan support#2016-07-1106:32robert-stuttafordforgive the repost, folks, but i'm a little stumped š#2016-07-1106:34robert-stuttafordhow do i specify that an optional set of keys must either appear together or not at all?#2016-07-1108:34mandragoran@robert-stuttaford: you can write your own predicate (-> (s/keys :opt [...]) (s/and #(every (partial contains? %) [...]))#2016-07-1109:04robert-stuttafordtrue, thank you. i wonder if it's a use-case that has been considered by Rich et al?#2016-07-1112:51Alex Miller (Clojure team)Have you looked at the and/or support in s/keys :req?#2016-07-1112:59Alex Miller (Clojure team)was thinking something like (s/def ::combined (s/keys :req [(or (and ::a ::b) (and ::a ::b ::c ::d))]))
#2016-07-1113:00Alex Miller (Clojure team)however b/c of open maps, I donāt think the latter two will fail as expected - for that you probably need and an additional pred#2016-07-1114:48robert-stuttafordthanks Alex#2016-07-1117:17semperosI wonder if thereās a place for an s/xor
within the clojure.spec space where one, and only one, of the provided specs may conform, or if this can be concisely expressed with a combination of the existing features#2016-07-1119:31semperosAlex or others, would appreciate your thoughts if/when you have time ^^#2016-07-1120:55arohnerHereās a fun one. If I start up a repl and clojure.test/run-tests, everything is fine. If I lein test
, I get java.lang.Exception: :clojure.spec/unknown is not a fn, expected predicate fn
#2016-07-1120:56arohnerAFAIK, Iām not doing anything weird. Everything is required in the proper order, not using load
, etc#2016-07-1121:02arohnerAlso AFAIK, Iām not referring to any spec that hasnāt been required#2016-07-1121:25arohnermore weirdness: I run my tests from a repl, everything is fine. I run c.spec.tests/instrument
, run the same tests, and I get No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13372
. Reload all my source namespaces, everything works again#2016-07-1122:44Alex Miller (Clojure team)I have seen some stuff like this with mixture of aot and non aot on classpath#2016-07-1215:04akielCan someone please comment on http://dev.clojure.org/jira/browse/CLJ-1966? I would like to know if I possibly understand something wrong.#2016-07-1216:37peejaI'm afraid I'm still failing to understand the difference between keys
and keys*
. I understand that only keys*
works in a cat
, but I don't understand why. Is this explained somewhere?#2016-07-1216:39Alex Miller (Clojure team)@akiel rich hasn't looked at it yet#2016-07-1216:41Alex Miller (Clojure team)@peeja keys is a spec for a map. keys* is a spec for a sequential list of alternating keys and vals#2016-07-1216:42peejaOhhhhh!#2016-07-1216:42peejaSo they both work in the same contexts, but match different things#2016-07-1216:43Alex Miller (Clojure team)I'd say they work in different contexts to match the same thing :)#2016-07-1216:43peejaBy "context" I mean where they go in a spec#2016-07-1216:44akiel@alexmiller itās ok, as long as itās on your list - thanks#2016-07-1216:44peejaYeah, I was misreading the guide. Thanks!#2016-07-1218:51donaldballRevisiting the āspec strings as char seqsā idea from the weekend, I just threw this together:#2016-07-1218:51donaldballhttps://gist.github.com/dball/d9f9bb7b6ea796e0de2ae49f6b21066e#2016-07-1218:51donaldballBad idea or simply misguided? š#2016-07-1219:16Alex Miller (Clojure team)@donaldball: from a spec perspective I think it's much better to use an actual regex for matching#2016-07-1219:17donaldballThe things Iām finding attractive about expressing specs on char seqs are that the spec regex forms are often more readable, but more to the point: you get working generators for free#2016-07-1219:18Alex Miller (Clojure team)There is a project for assembling readable regexes, can't remember the name#2016-07-1219:22arohnerthereās https://github.com/cgrand/regex#2016-07-1219:23arohnerthat doesnāt use java.util.regex.Pattern though#2016-07-1220:24Alex Miller (Clojure team)yeah, that one#2016-07-1220:24Alex Miller (Clojure team)and also test.chuck has support for generating strings based on a regex#2016-07-1221:23sparkofreasonWhen writing regex specs, is there a way to get at the bindings "so far"? For instance, say I want to validate a datom [e a v]
, where a
is keyword that represents a spec. I'd like to use that spec to validate v, and get "nice" output from explain
that the failing spec is the one represented by a
.#2016-07-1221:23danburtonoohh dependent specs, interesting#2016-07-1221:47Alex Miller (Clojure team)@dave.dixon custom predicate? (s/and <datom-spec> #(s/valid? (:a %) (:v %)))
#2016-07-1221:47Alex Miller (Clojure team)doesnāt integrate into explain of course#2016-07-1221:50bhaumanMulti spec using second as the dispatch?#2016-07-1221:51bhaumanThen use a confirmed#2016-07-1221:51Alex Miller (Clojure team)nice#2016-07-1221:51bhaumanConformer#2016-07-1222:04danielcomptonhttps://clojurians.slack.com/archives/clojure/p1468341869003436#2016-07-1222:04danielcompton@alexmiller: what is the thinking behind it being under consideration? What are the downsides of allowing docstrings on specs?#2016-07-1222:13sparkofreason@alexmiller: would be nice if there were a way to pass explain results out of custom predicates. Though perhaps not required in most cases. #2016-07-1222:24Alex Miller (Clojure team)@danielcompton asking the wrong person :)#2016-07-1222:25Alex Miller (Clojure team)@dave.dixon: while it's tempting, I don't think Rich is inclined to support that for uniformity.#2016-07-1222:28danielcomptonhttp://dev.clojure.org/jira/browse/CLJ-1965#2016-07-1300:00sparkofreason@alexmiller: Anyway, I'd rather see support for this specific case, which seems common in Clojure, where we have a sequence of keywords followed by associated values. Just supporting the case of having the value validated against the spec named by the keyword would cover a lot of ground, as opposed to having a general dependent spec scheme. There's a nice symmetry with the map behavior then.#2016-07-1300:02Alex Miller (Clojure team)Actually you could do this with keys* couldn't you?#2016-07-1300:07Alex Miller (Clojure team)(s/cat :e any? :av (s/keys* :opt-un [::a]))#2016-07-1300:11seancorfieldOne thing Iām finding with clojure.spec
as applied to user input is that I often seem to need a two-step process: the first is a very basic predicate and a conformer, the second is the real "spec" (and perhaps and conformer).#2016-07-1300:11seancorfieldExample, user inputs a country code and Iād like it to be case-insensitive.#2016-07-1300:12seancorfieldSo the first spec is that itās a two-character string and the conformer is clojure.string/upper-case
; then the second spec actually looks up the (now upper-case) two-character string and verifies itās a valid ISO country code (and could conform it to country information).#2016-07-1300:13seancorfieldIs that somehow a poor use of clojure.spec
? Or is there perhaps a better way to deal with input that needs a little "cleanup" before being validated/conformed?#2016-07-1300:17luke@seancorfield hm. If a non-uppercase string is to be considered āvalidā input, then Iād say itās on your country-code lookup function to handle case insensitive input. That seems like it would be easier than having two specs.#2016-07-1300:18lukeUnless you want to have a spec for your intermediate data format for some reason.#2016-07-1300:19seancorfieldWell, several of the inputs ā and country is actually possible here ā are sets of strings. For example, ::gender
is #{"male" "female"}
so it validates and generates nicely, but if you want it to be case-insensitive then you need to do "something" that isnāt necessarily easy with a single spec.#2016-07-1300:20seancorfield(`::country` could be (set (Locale/getISOCountries))
for example)#2016-07-1300:23seancorfieldI started leaning toward two specs because internally I wanted ::gender #{:male :female}
for the domain model and "appropriate strings" for the input and the conformer there would be (comp keyword str/lower-case)
ā¦ assuming you can validate the input strings with a different spec.#2016-07-1300:24seancorfieldPerhaps Iām making life more complicated for myself by trying to avoid with-gen
here...#2016-07-1300:24lukeYeahā¦ two specs isnāt bad if you do want separate models (one for the domain, one for the input) and are ok specāing them separately.#2016-07-1300:25lukeIt is also possible in theory to write a case-insensitive
helper, which given a set of strings returns a (generating) spec that accepts any capitalization.#2016-07-1300:30seancorfieldHmm, Iāll have to give it more thought. So far have specs for input fields that conform to the domain model values, and specs for the domain model itself is feeling like the right way to go, as it allows me to generate data for both layers: API (input) and "system" (domain model).#2016-07-1300:32seancorfield(weāre pushing very hard to tease apart what is some rather complected logic in our current codebase ā we currently do a sort of transform-validate-update operation on each field and weāve figured out a nice clean way to separate validate from update ā with a view to pushing the updates out to a queue and applying them asynchronously ā but that pre-validation transform on a few fields is proving problematic šø )#2016-07-1300:33seancorfieldThanks for the input so far @luke#2016-07-1300:34lukeyep if you want separate layers, and youāve fully reified both layers, then specs for each layer seems like the way to go!#2016-07-1300:34lukeno hard answers, just what comes off the top of my head when I hear the question š#2016-07-1301:05sparkofreason@alexmiller: Thanks, that works great. Knew there had to be a way to do that, but had my head stuck in regex-land...#2016-07-1301:15sparkofreasonSlightly shorter version: (s/cat :e any? :av (s/keys*))
#2016-07-1302:22Alex Miller (Clojure team)Yup#2016-07-1303:17lvhIs there a way to specify āstrings that match a particular regexā in a way that produces the correct generator?#2016-07-1303:18lvh(test.chuck I think has that)#2016-07-1303:20lvhit seems like test.check would have a real bad time generating strings until they accidentally match that regex#2016-07-1303:20lvh(that also seems like a general problem, which is why Iām assuming thereās a general soltuion)#2016-07-1303:23gfredericksthe test.chuck generator could be moved to test.check proper if somebody wants to rewrite the instaparse regex parser in pure clojure#2016-07-1303:24lvhWhat is it now? Bunch of Java?#2016-07-1303:24lvhAlso hi gfredericks burning the midnight Chicago oil too I see š#2016-07-1303:24lvhI havenāt really read the spec source code any maybe I should just go do that#2016-07-1303:25lvhIām presuming thereās a way to map predicates to generators#2016-07-1303:25gfrederickslvh: I'm saying the test.chuck code uses instaparse#2016-07-1303:25lvhand Iām guessing thatās why spec/and is not just every-pred#2016-07-1303:25gfrederickslvh: hi#2016-07-1303:25lvhoh, right, and test.check doesnāt get to depend on instaparse#2016-07-1303:25lvhgot it#2016-07-1303:26gfredericksexactly#2016-07-1303:26gfredericksso you'd have to rewrite this in clojure somehow: https://github.com/gfredericks/test.chuck/blob/master/resources/com/gfredericks/test/chuck/regex.bnf#2016-07-1303:26lvhthat doesnāt sound very fun#2016-07-1303:27lvhhow would you feel about clojure.spec-specific things being added to test.chuck?#2016-07-1303:27lvhsay, a spec that understands how to map to generators in test.chuck, like, say, hypothetically, if someone wanted to match a regex š#2016-07-1303:28gfredericksI keep thinking I'm going to want a similar clojure.spec utility library#2016-07-1303:29gfredericksclojure.schpec#2016-07-1303:29lvhšÆ#2016-07-1303:30lvhwould depend upon.#2016-07-1303:31lvhI guess Iām screwed anyway because this is a keyword, not a string, so unless thereās something that understands keywords, Iām going to manually assign a generator#2016-07-1303:32gfredericksI get the impression that manually assigning generators is not meant to be too rare#2016-07-1303:32lvhsame here#2016-07-1303:33lvhIāll defer to you since you obviously have a lot more experience here, but just generating samples based on only predicates is going to break really badly really quickly#2016-07-1303:33lvh(basically as soon as the probability of the predicate passing is not very big)#2016-07-2113:32gfredericksA 1-time dev utility or a runtime thing?#2016-07-2113:32mpenetwell I guess if you do the latter the former spawns in a couple of lines#2016-07-2113:33gfredericksThe former can be less robust#2016-07-2113:34mpenetIn my case I am mostly concerned about the ton of map schemas we have in our projects, would be nice to be able to port these without spending ages on it and laying a bug field in the process#2016-07-2113:34mpenetI could just not bother as well, and use both in parallel#2016-07-2113:37gfredericksJust punt on certain things and point the user to where it's broken#2016-07-2113:37mpenetthe example of nested maps is a good one I think, a very concise schema from plumatic Schema can end up being a tons of lines of mostly s/def's for k/v#2016-07-2113:38mpenetcould end up being a more concise way to write map specs as well#2016-07-2113:38gfredericksIt definitely would be#2016-07-2113:42EdHi ... I've just been looking at spec (alpha10) and am confused by some results I'm seeing ...#2016-07-2113:43Edthe docs from fdef suggest that it should instrument the function referred to#2016-07-2113:43Edbut it doesn't seem to#2016-07-2113:43gfredericksNot by itself#2016-07-2113:44gfredericksYou have to call c.s.test/instrument#2016-07-2113:44Edand that will permanently replace the var with a function that checks it's args?#2016-07-2113:45Edthe docs for fdef say:#2016-07-2113:45EdOnce registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests, and (if
a macro) used to explain errors during macroexpansion.#2016-07-2113:47gfredericks"Checked by instrument" is a reference to the function I mentioned above#2016-07-2113:48gfredericksThe wording is confusing though since that's not obvious#2016-07-2113:48bhauman@gfredericks: I have a strict-keys macro that uses a dynamic variable to set it's level of strictness#2016-07-2113:48gfredericks@bhauman: interesting#2016-07-2113:48Edok ... so I need also need to include clojure.spec.test in my production code to have specs checked at runtime?#2016-07-2113:48bhauman:ignore, :warn, :blowup#2016-07-2113:49gfredericks@bhauman: I'm in the middle of pushing more customization into plumatic/schema to support this kind of thing#2016-07-2113:49Ed@gfredericks: thanks#2016-07-2113:50bhaumanI think a better solution is to have both the dynamic variable with a spec level override to force a certain level for certain specs#2016-07-2113:50gfredericks@l0st3d: I think the intention is for you to do more explicit checks if you want production checking#2016-07-2113:51gfredericks@bhauman: cuz e.g. defproject is always open but certain submaps are more restrictive? #2016-07-2113:53Ed@gfredericks: fair enough ... just thought that the docs suggested it would do those checks for me if I included the fspecs ... just trying to work out the api atm i think š .. thanks for your help#2016-07-2113:53bhaumanjust that you would want certain maps to never allow extraneous members#2016-07-2113:53bhaumanthere is actually a really interesting case for misspellings#2016-07-2113:53bhaumanand still having an open map#2016-07-2113:54bhaumanhaving a level of verification that does a fuzzy-selection of the set of keys#2016-07-2113:55bhaumanso the provided keys are checked with a distance to the spec'ed keys#2016-07-2113:56bhaumanThis could frankly be done all the time for something like (s/keys ) where a warning will be generated#2016-07-2113:57bhauman@bbloom: ^#2016-07-2113:57bhaumanbut being able to set a dynamic var to adjust this level of checking seems appropriate#2016-07-2114:10sundbpiām struggling with a recursive map spec. what iām trying to do is something like this:#2016-07-2114:10sundbp(s/def ::node-content ::series)
(s/def ::node-inputs ::graph-seq)
(s/def ::graph-node (s/keys :req [::node-content ::node-inputs ::label ::stream]))
(s/def ::graph-seq (s/coll-of ::graph-node))
#2016-07-2114:10sundbpi donāt quite get how i can have a key spec referring to having a required key that is related to itself..#2016-07-2114:12sundbpI canāt find any examples of a recursive spec of a map#2016-07-2114:45Alex Miller (Clojure team)(s/def ::label keyword?)
(s/def ::children (s/coll-of ::node))
(s/def ::node (s/keys :req [::label] :opt [::children]))
(s/conform ::node {::label :a ::children [{::label :b} {::label :c}]})
=> #:user{:label :a, :children [#:user{:label :b} #:user{:label :c}]}
#2016-07-2114:45Alex Miller (Clojure team)thereās a simple example - I canāt quite work out what youāre trying to do#2016-07-2115:54sundbpthanks. iāll check it out#2016-07-2115:58sundbpfirst time i tried something like that i thought i got a complaint about the spec not existing.. think was because i didnāt have ::graph-seq and ::node-inputs in the right order.#2016-07-2115:58sundbpsimple mistake#2016-07-2115:58sundbpfine if i move those 2 around#2016-07-2116:12seancorfield@mpenet: My understanding is that, despite the docstring, you can use instrument
to turn on instrumentation for an entire namespace using that call.#2016-07-2116:13mpenetHmm I tried but it didn't seem to do anything.#2016-07-2116:13mpenetit returned [], which would seem to indicate it instrumented nothing#2016-07-2116:14mpenetI am not at work anymore, but I guess I ll check the souce next chance I get#2016-07-2116:21mpenetseems it might have changed here: https://github.com/clojure/clojure/commit/a4477453db5b195dd6d1041f1da31c75af21c939 at least that's what the docstring would suggest#2016-07-2116:22mpenet(didnt try it)#2016-07-2116:22seancorfieldYeah, something definitely isnāt working right now ā I changed an fdef spec and the tests still passā¦ investigating.#2016-07-2116:22seancorfieldIt definitely used to work.#2016-07-2116:23mpenet@alexmiller: would it be a bug or is it an intentional change?#2016-07-2116:23mpenet^ instrument
no longer working with a ns argument#2016-07-2116:24Alex Miller (Clojure team)thatās intentional - use st/enumerate-namespace to produce a list of syms in an ns#2016-07-2116:24mpenetok makes sense#2016-07-2116:27seancorfieldHmm, I missed that change in the release notes @alexmiller#2016-07-2116:27Alex Miller (Clojure team)alpha8#2016-07-2116:27seancorfieldAlthough now I canāt get my fdef
specs to work at all#2016-07-2116:27Alex Miller (Clojure team)iirc#2016-07-2116:28Alex Miller (Clojure team)for check
or instrument
?#2016-07-2116:28Alex Miller (Clojure team)I have just tracked down a problem with check
#2016-07-2116:28seancorfieldWhen I call fdef
, the spec does not subsequently show up on doc
...#2016-07-2116:29Alex Miller (Clojure team)how are you calling it? expects a fully-qualified symbol#2016-07-2116:29Alex Miller (Clojure team)actually, it resolves, so not necessarily fully-qualified#2016-07-2116:32seancorfieldI had (s/fdef drop-table-ddl ā¦)
after referring in :all
and that used to work but doesnāt now. I changed it to (s/fdef clojure.java.jdbc/drop-table-ddl ā¦)
and it works now.#2016-07-2116:41Alex Miller (Clojure team)yeah, if you use a bare symbol for def or fdef, it will treat that as <current-ns>/sym#2016-07-2116:41Alex Miller (Clojure team)so itās not going to pick up refer's#2016-07-2116:44seancorfieldThat changed at some point. This code used to work.#2016-07-2116:44seancorfield(not a big deal but the silent "failure" is disturbing)#2016-07-2116:51Alex Miller (Clojure team)fdef stuff changed around 6/7 - there were some major rewrites of it in there#2016-07-2116:52Alex Miller (Clojure team)you can use st/instrumentable-syms
to verify that things are speced maybe?#2016-07-2116:56sundbpwith these changes - if you want to run your clojure.test tests with specs instrumented and checked for fdefās, whatās the proposed setup?#2016-07-2116:59Alex Miller (Clojure team)youāll need to turn on instrumentation#2016-07-2117:00Alex Miller (Clojure team)so you can do that per-test, in a fixture, etc#2016-07-2117:02sundbpif one would like to turn on instrumentation for āeverythingā in a fixture - does one have to manually enumerate the NS to then do instrumentable-syms and finally instrument? i.e. thereās nothing that just turns on instrumentation for all NS in project?#2016-07-2117:06Alex Miller (Clojure team)just (st/instrument)
is supposed to do that#2016-07-2117:07Alex Miller (Clojure team)thatās like the old instrument-all#2016-07-2117:07Alex Miller (Clojure team)and macro fdefs are always instrumented in macroexpansion#2016-07-2117:16seancorfieldHereās what I ended up with in java.jdbc: (try
(require 'clojure.java.jdbc.spec)
(require 'clojure.spec.test)
(let [syms ((resolve 'clojure.spec.test/enumerate-namespace) 'clojure.java.jdbc)]
((resolve 'clojure.spec.test/instrument) syms))
(println "Instrumenting clojure.java.jdbc with clojure.spec")
(catch Exception _))
#2016-07-2117:16seancorfieldAnd I had to qualify all the symbols in clojure.java.jdbc.spec fdef
calls in order to get those working again.#2016-07-2117:16seancorfieldLife on the bleeding edge ā¦ š#2016-07-2117:17seancorfieldThanks @mpenet for the heads up on that ā I hadnāt noticed it was broken!#2016-07-2117:24fentonin my app is use GPS data. Normally my data structure looks like: {:lat 33.3 :lng -129.3}
. With spec things are looking like: {:pc.api/latitude 33.3 :pc.api/longitude -129.3}
. I'm finding it less fun to type that everywhere. I guess I could alias my namespace to [pc.api :as a]
and change latitude and longitude to lat/lng
then get {:a/lat 33.3 :a/lng -129.3}
. R others doing/finding something similar? I wonder if having keywords as short as lat/lng leads to being unclear? Thoughts?#2016-07-2117:29seancorfieldYou can use unqualified keywords in maps if you want @fenton#2016-07-2117:29mpenet:) @seancorfield i looked a bit at clj.jdbc to spec parts of alia, that s how i spotted this#2016-07-2117:30seancorfieldIām cutting 0.6.2-alpha2 with those updates.#2016-07-2117:30fenton@seancorfield: do you mean un-qualified?#2016-07-2117:33fenton@seancorfield: I guess I was suggesting the ns qualified keywords seem a bit of a pain to type everywhere...maybe I should look into using un-qualified keywords....not sure the implications off the top of my head... I'm thinking, how do you use unqualified keywords in other files? The way I'm using specs is to put them into a third *.cljc file that is shared by my front and backends. So dont i have to use qualified keywords in that case?#2016-07-2117:33seancorfieldYes, sorry, typo.#2016-07-2117:33seancorfieldFixed š#2016-07-2117:33fentonI keep my specs in an external library...does that make un-qualified an issue or can i still have them defined elsewhere?#2016-07-2117:33seancorfieldWell, you can do #::a{:lat 33.3 :lng -129.3}
#2016-07-2117:34fenton@seancorfield: okay that looks a bit better....#2016-07-2117:34seancorfieldA map does not need qualified keys in order to be used with spec.#2016-07-2117:35seancorfield(s/keys :req-un [::lat ::lng])
will conform {:lat 33.3 :lng -129.3}
#2016-07-2117:35fenton@seancorfield: oh really? how does spec use it then?#2016-07-2117:35fentonoh...really...cool!#2016-07-2117:35seancorfieldhttps://clojure.org/guides/spec#_entity_maps gives examples.#2016-07-2117:35fentoni guess i missed that, i'll go check it again...#2016-07-2117:36seancorfieldScroll down to where it shows :req-un
being used.#2016-07-2117:36fentonok...#2016-07-2117:37fentoni'll give that a try....that'll clean up my code nicely i think.#2016-07-2117:37fentonthanks!#2016-07-2118:25fenton@seancorfield: I'm unfamiliar with the #::a{:lat 3.3 :lng 3.3}
syntax. i.e. the pulling of the namespace out in front of the map. is there some documentation about that somewhere?#2016-07-2118:34seancorfieldhttp://dev.clojure.org/jira/browse/CLJ-1910#2016-07-2118:44Alex Miller (Clojure team)and more officially http://clojure.org/reference/reader#_maps#2016-07-2119:00Alex Miller (Clojure team)@jjcomer I can explain what youāre seeing now btw. spec.test/check
returns a lazy sequence of test results per sym. In your test, you are not realizing those results, then immediately uninstrumenting them, then later realizing (actually running the check on the uninstrumented functions). If you wrap a doall
around your two calls to check
, that addresses why youāre not seeing the instrumentation.#2016-07-2119:01Alex Miller (Clojure team)while check
does document this laziness, I admit it was a surprise to me.#2016-07-2119:02jjcomerGotcha. Thanks for the debug :)#2016-07-2119:02Alex Miller (Clojure team)and then the other thing was not including the sym youāre replacing in the instrument list#2016-07-2119:05jjcomerAwesome, I'll give it another go this afternoon. Thanks again#2016-07-2119:09Alex Miller (Clojure team)np, thx for the repro#2016-07-2122:50seancorfieldIf you define a spec with a custom generator (using s/with-gen
), does clojure.spec
filter the generated values using the spec itself, or does it trust that the generator will only ever produce conforming values?#2016-07-2122:51seancorfield(I ask because I have a spec with a very complex validation predicate and so I have to write a custom generator and Iām not certain whether the generator Iāve written is only going to produce conforming values by itselfā¦)#2016-07-2123:14Alex Miller (Clojure team)It does not trust and always re-checks the custom gen#2016-07-2123:15Alex Miller (Clojure team)It's not filtering though - it will error if the gen produces a bad value#2016-07-2123:19glv@alexmiller: in http://clojure.org/reference/reader#_maps, an example of the #::
behavior would be helpful. I think itās accurate as it is, but not particularly clear.#2016-07-2123:21seancorfieldThanks @alexmiller thatās good to know. I couldnāt produce a bad value in 10,000,000 generations so I think Iāll trust it as working for now š#2016-07-2123:21Alex Miller (Clojure team)@glv Will consider#2016-07-2202:20lvhWhatās the preferred way to check if a coll has only unique elements?#2016-07-2202:21lvhJust a pred? Iād like the generators to be efficient.#2016-07-2202:35codonnell@lvh: I think coll-of
and every
use clojure.test.check.generators/vector-distinct
under the hood#2016-07-2202:36lvhah; that solves part of the problem#2016-07-2202:36codonnellrather, they use vector-distinct if they get :distinct true
#2016-07-2202:37lvhah! I missed that opt#2016-07-2202:37lvhdoes coll-of take the same opts as every?#2016-07-2202:38codonnellyes#2016-07-2202:39codonnellthis is the entire body of the coll-of
macro: backtick before ( ---> (every ~pred ::conform-all true
#2016-07-2202:40codonnelldamn, how do you get a backtick in a code block -_-#2016-07-2203:12lvhah, awesome#2016-07-2203:12lvhI should really just read the source of that ns#2016-07-2203:33lvhI ran some test.check specs, but even with only 100 samples Iām getting actual: java.lang.OutOfMemoryError: GC overhead limit exceeded. I guess recursive data structures can grow big.#2016-07-2203:40lvhIs there something extra weird that goes on when you test against (for-all [ā¦] true)? I was expecting that to trivially pass.#2016-07-2203:40lvh(that seems like it would be the normal behavior once your code works.)#2016-07-2203:46glv@lvh is a generator in control of the size (depth or breadth) of the generated structure? The current integer generators in spec grow very quickly. By the 20th test, you're probably in 8-digit range.#2016-07-2204:06madstapIs there a way to say that if there are any numbers, the string also needs to be there, but if there aren't, it's optional?
(s/def ::xs (s/cat :str (s/? string?) :nums (s/* number?) :key keyword?))
#2016-07-2204:31madstap;; so this should be valid
(s/valid? ::xs ["s" 2 3 4 :k])
;; And this
(s/valid? ::xs ["s" :k])
;; but not this
(s/valid? ::xs [2 3 4 :k])
#2016-07-2205:24bfabry@madstap: (s/or :regex1 (s/cat ...) :refex2 (s/cat ...)) seems simplest to me#2016-07-2205:45Alex Miller (Clojure team)(s/def ::xs
(s/cat :pre (s/alt :opt1 (s/cat :str string? :nums (s/* number?))
:opt2 (s/? string?))
:key keyword?))
#2016-07-2207:52mpenetWhat do you think makes more sense for a lib with optional specs: ship with specs in a separate namespace or have specs in a separate repo even. former requires macro hackery to allow to run/use with clj1.9- latter is just a dependency.#2016-07-2213:00mpenetany idea why this fails when trying to validate with it:
(s/def :foo (s/fspec :args (s/cat :err #(instance? ExceptionInfo %))
:ret any?))
#2016-07-2213:00mpenet(s/valid? ::foo (fn [x] 1)) -> ExceptionInfo Unable to construct gen at: [:err] for: (instance? clojure.lang.ExceptionInfo %) clojure.core/ex-info (core.clj:4724)
#2016-07-2213:03mpenetoh I see on the guide#2016-07-2213:04codonnell@mpenet: regarding your earlier question, can you not use the port someone made of spec to clojure 1.8?#2016-07-2213:04mpenetit was more of a general question when publishing oss#2016-07-2213:05codonnelloops, I missed your "specs in a separate namespace" option#2016-07-2213:06mpenetabout my recent issue, I wonder why gen is coupled like this to preds#2016-07-2213:07mpenetI am mostly interested in instrumentation in that case, yet I have to specify a generator apparently#2016-07-2213:08codonnellI would guess that valid?
checks validity by generating some arguments using the :args
generator and then checking that the return values match your :ret
and :fn
predicates#2016-07-2213:12mpenetapparently#2016-07-2213:17mpenetso instrumentation works without having to specify more here. good#2016-07-2213:19codonnellthat's good to know#2016-07-2213:20mpenetwell actually no it doesn't#2016-07-2213:20mpenetdamnit#2016-07-2213:22codonnellit worked for me here:
=> (defn foo [& exs] (map str exs))
=> (s/fdef foo :args (s/cat :err #(instance? clojure.lang.ExceptionInfo %)) :ret any?)
=> (stest/instrument `foo)
=> (foo 1 2 3)
ExceptionInfo Call to #'user/foo did not conform to spec:
In: [0] val: 1 fails at: [:args :err] predicate: (instance? clojure.lang.ExceptionInfo %)
:clojure.spec/args (1 2 3)
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file "form-init8739029786060363099.clj", :line 990, :var-scope user/eval90084}
clojure.core/ex-info (core.clj:4724)
=> (foo (ex-info "test" {}))
("clojure.lang.ExceptionInfo: test {}")
#2016-07-2213:23codonnellhow did it fail for you?#2016-07-2213:23mpenetnot the same in my case, it's a function that takes another fn as argument, the fn passed fails to gen#2016-07-2213:24mpenetso no s/fdef but s/fspec#2016-07-2213:24codonnelloh, I see#2016-07-2213:24mpenet(s/def ::alia.execute-async/error
(s/fspec :args (s/cat :err (instance-pred clojure.lang.ExceptionInfo))
:ret any?))
#2016-07-2213:25mpenet(s/valid? ::alia.execute-async/error (fn [x] :meh))
#2016-07-2213:25codonnellit tries to call valid?
on the function returned rather than doing something like propogate the instrumentation to the returned function#2016-07-2213:26mpenetsame when it's on the test suite via instrumentation of the parent fn#2016-07-2213:26mpenethttps://github.com/mpenet/alia/blob/feature/specs/modules/alia-spec/src/qbits/alia/spec.clj#L311-L315#2016-07-2213:29mpenetI understand it's necessary when running valid? on the single spec, but via instrumentation not really#2016-07-2213:30codonnellI tend to agree.#2016-07-2213:30mpenetit probably is slow too#2016-07-2213:31mpenet@alexmiller: any thoughts on this?#2016-07-2213:35mpenetIn my mind instrumentation should "only" be pre/post assertions based on specs. no gen involved (same as plumatic/schema actually)#2016-07-2214:03lvh@glv: Yes, I suppose so.#2016-07-2214:04lvhHereās what I have now that blows up at 50: https://gist.github.com/lvh/243576d4e825d3792d4cd1bb8d8c39d7#2016-07-2214:04lvhat line 36, youāll see ::properties is a map-of keywords to ::schemas#2016-07-2214:04lvh(so recursive there)#2016-07-2214:04lvhitāll be worse when I add arrays, because those are also recursive json schemas#2016-07-2214:09glvOh ā¦ no, it doesnāt look like a generated value is in controlling the depth ā¦ in fact, nothing is controlling it except chance.#2016-07-2214:10glvSo on line 36, thereās a 1-in-7 chance that the generator will recur and add another level to the structure ā¦ but at line 32, thereās a 50% chance. Itās easy to imagine the process rolling the dice just right so that the structure grows until it fills the JVMās memory allocation, and that seems to be happening.#2016-07-2214:16glvThe problem for writing the specs is this: youāve accurately captured the semantics for the purposes of verification, but when generating for tests, you want to add some additional constraints. I ran into this when testing a 2D grid ā¦ in production code I donāt want to arbitrarily restrict the allowable sizes, but in my specs, I was ending up with grids that had many millions of cells, which was slow, failure-prone (because I also got out-of-memory errors) and also pointless (because a bunch of tests with 25x25 grids will catch any problems ā the size of the grid isnāt really the important factor).#2016-07-2214:16glvSo I ended up with this spec: ;; restrict grid sizes for testing, but allow larger grids in production.
(s/def ::grid-dimen (s/with-gen (s/and integer? #(>= % 2))
#(s/gen (s/int-in 2 25))))
#2016-07-2214:17glvIt doesnāt enforce an upper bound for validation, but it does for generation.#2016-07-2214:18glvYou might need something similar, but rather than limiting a size (because you donāt have one) you can alter the probabilities, so that the generator for ::additional-properties
only chooses to recur 10% of the time, say, instead of 50%.#2016-07-2214:52Alex Miller (Clojure team)when running check
etc you can supply generator overrides#2016-07-2214:53Alex Miller (Clojure team)same for exercise
, gen
, and instrument
#2016-07-2214:54Alex Miller (Clojure team)this allows you to override the generator with a more specific one at test time#2016-07-2214:54Alex Miller (Clojure team)also see s/*recursion-limit*
#2016-07-2214:56mpenetI saw this. But shouldn't it be possible to avoid generators usage for instrumentation?#2016-07-2214:57mpenetsince it's all predicates, just wrap args/ret#2016-07-2215:00Alex Miller (Clojure team)you can do so with the replace/stub functionality in instrument#2016-07-2215:00Alex Miller (Clojure team)or maybe Iām not understanding your question#2016-07-2215:01mpenetI don't know if you read my issue earlier#2016-07-2215:01Alex Miller (Clojure team)the fspec thing?#2016-07-2215:01mpenetyes#2016-07-2215:02mpenetI imagined that instrumentation would work without having the need to stub anything since it's all specified, I didn't expect it to run "gen" on arguments (hundreds of calls)#2016-07-2215:03Alex Miller (Clojure team)there is still some unfinished work in this area#2016-07-2215:03mpenetoki, glad to hear that, so in the final design "gen" wouldn't be required by instrumentation code#2016-07-2215:03Alex Miller (Clojure team)this is same as http://dev.clojure.org/jira/browse/CLJ-1936#2016-07-2215:04mpenetah indeed, I tried to look at jiras, I missed that one I guess#2016-07-2215:19glv@lvh: just to test my hypothesis, try changing ::additional-properties
to this and see if that helps: (s/def ::additional-properties
(s/with-gen
(s/or
:implicit-additional-properties boolean?
:explicit-additional-properties ::schema)
#(sg/fmap (fn [d10]
(sg/generate (s/gen (if (= d10 1)
::schema
boolean?))))
(sg/choose 1 10))))
#2016-07-2215:21glvYouāll get a nested schema as the generated value only 10% of the time.#2016-07-2215:21gfredericksalexmiller: ugh that "open intervals" email makes me wish I'd made the bounds opts on those numeric generators named #{:< :<= :> :>=}
:/#2016-07-2215:21glv(sg is aliased to clojure.spec.gen)#2016-07-2215:22gfredericksalexmiller: I'd consider deprecating the old opts and adding those four if you think that'd be useful for spec#2016-07-2215:24Alex Miller (Clojure team)I donāt think anything needs to change with it, itās fine#2016-07-2215:29gfredericksindependently I also halfway regret how :NaN? and :infinite? interact with :min and :max#2016-07-2215:30gfredericksor don't interact rather#2016-07-2215:30gfredericksbut that's harder to change without technically breaking#2016-07-2216:10rickmoynihanHmmm... any tips for debugging stackoverflow exceptions in specs?#2016-07-2216:16rickmoynihanalso is it possible to reset the spec registry?#2016-07-2216:16rickmoynihanpretty sure I've heard people ask about this before#2016-07-2216:32mpenetMaybe (reset! #'clojure.spec/registry-ref (atom {}))#2016-07-2216:32mpenet(Untested)#2016-07-2216:50rickmoynihanI've probably got a mistake in my specs...#2016-07-2218:39rickmoynihanI had a typo along the lines of by mistake (s/def foo ::foo)
#2016-07-2218:40rickmoynihanIs it possible to call an s/fdef'd
function and validate its :ret
spec?#2016-07-2218:40rickmoynihanI saw the guide said it wasn't supported because that's for testing#2016-07-2218:41Alex Miller (Clojure team)the only place thatās checked by spec is during check
#2016-07-2218:41Alex Miller (Clojure team)you can of course obtain and check it yourself#2016-07-2218:42Alex Miller (Clojure team)something like (s/valid? (:ret (s/get-spec 'user/foo)) ret)
#2016-07-2218:42rickmoynihanahh thanks#2016-07-2218:42Alex Miller (Clojure team)or s/assert
#2016-07-2218:43rickmoynihannice - I'd missed that one#2016-07-2218:53xcthulhu@alexmiller: As I opined on reddit, I don't see any deep reason for why you can't just have in clojure.spec
a schema transpiler:
(spec/def ::json-api
(spec/schema
{:id uuid?
:text str?
:rating (int-in 0 5)}))
This would make migrating a lot less annoying.#2016-07-2218:54Alex Miller (Clojure team)Clojure is not going to provide a schema transpiler#2016-07-2218:54xcthulhuWhy not?#2016-07-2218:54xcthulhuNIH?#2016-07-2218:54Alex Miller (Clojure team)b/c we have other things to do#2016-07-2218:54Alex Miller (Clojure team)people are more than welcome to make one#2016-07-2218:55Alex Miller (Clojure team)would we also create a truss transpiler?#2016-07-2218:55Alex Miller (Clojure team)and a herbert transpiler?#2016-07-2218:55Alex Miller (Clojure team)etc?#2016-07-2218:56Alex Miller (Clojure team)things in core are forever#2016-07-2218:56xcthulhuI suppose. IDK, clojure.spec.gen/fmap
is in there which is awkward.#2016-07-2218:57xcthulhuI don't really grok the logic behind what gets in and what doesn't#2016-07-2218:57Alex Miller (Clojure team)why?#2016-07-2218:57Alex Miller (Clojure team)gen is just a dynamically loaded skin around test.check#2016-07-2218:58xcthulhuWell, so there's tiny little monad in a framework where everyone did a great job of dodging that sort of thing.#2016-07-2218:58Alex Miller (Clojure team)you can ignore itās monadic nature if you like#2016-07-2218:58Alex Miller (Clojure team)it does what it does#2016-07-2218:59xcthulhuAll I'm saying is that great pains were made to wrap test.check
, but the lesser pain of wrapping schema
type syntax was ignored for some reason.#2016-07-2219:01xcthulhuIt's your product, everyone picks and chooses which wheels they want to reinvent I suppose.#2016-07-2219:05Alex Miller (Clojure team)test.check is a Clojure contrib library, following the same contributor agreement, license, and dev methodology as Clojure itself (to make things like this possible)#2016-07-2219:06Alex Miller (Clojure team)Schema is not (and I mean nothing negative towards Schema in this regard, they are just much different from a core perspective)#2016-07-2219:07xcthulhuOkay, so I guess we just need a clojure.contrib.spec.utils
library.#2016-07-2219:07Alex Miller (Clojure team)spec was designed to solve a set of problems, not to replace Schema (even though it solves an overlapping set of problems)#2016-07-2219:08xcthulhuWith schema
and a dynamically loaded test.check/let
#2016-07-2219:08xcthulhuSince that would be super nice#2016-07-2219:08Alex Miller (Clojure team)let
is tricky as itās actually a macro#2016-07-2219:08Alex Miller (Clojure team)Stu did the work on gen, but I suspect thatās the only reason itās not there#2016-07-2219:09Alex Miller (Clojure team)The Clojure contrib libs are primarily standalone libs without external deps (there are exceptions, but thatās the ideal). There is not going to be a contrib lib related to schema. But there is nothing stopping someone from creating a lib to do what you suggest.#2016-07-2219:10Alex Miller (Clojure team)if we think something is fit for spec, it will go into spec, not into a lib#2016-07-2219:12xcthulhuLooking here, it looks like you guys rope in bind
: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/gen.clj#L92#2016-07-2219:12xcthulhuI'll go try and write the little let
macro for you#2016-07-2219:13Alex Miller (Clojure team)bind is a function#2016-07-2219:13xcthulhuI know#2016-07-2219:13xcthulhuBut let
is just like the do
notation in Haskell, wrapping bind
#2016-07-2219:13xcthulhuI'll do it for you this weekend.#2016-07-2219:13xcthulhuOkay?#2016-07-2219:14Alex Miller (Clojure team)I think let
would basically have to be copied into gen rather than dynamically loaded#2016-07-2219:14xcthulhuYup.#2016-07-2219:14xcthulhuSo I'll write it and post it here for you.#2016-07-2219:14Alex Miller (Clojure team)you can file a jira if you like, but no guarantees for anything#2016-07-2219:15xcthulhuNah, I'll just post it here in a gist or whatever unless you need me to sign something#2016-07-2219:15xcthulhuIt's like 15 lines tops#2016-07-2219:15Alex Miller (Clojure team)we donāt take contributions via slack :)#2016-07-2219:16Alex Miller (Clojure team)sign the CA, file a jira, supply a patch#2016-07-2219:17xcthulhuOh man red tape my favorite#2016-07-2219:17Alex Miller (Clojure team)http://insideclojure.org/2015/05/01/contributing-clojure/#2016-07-2219:18xcthulhuOkay, I'll post it here for people to playtest and then do the patch, since that's a lot of tedium for 15 lines of code#2016-07-2220:12mpenetSeems like fmap appears twice in lazy-combinators call fyi#2016-07-2220:22glvJust for fun, Iāve been prototyping a little library to generate EBNF-ish syntax diagrams from specs. Delving into the spec internal representation has been ā¦ eeeenteresting. š#2016-07-2220:31xcthulhu@mpenet: Sounds like someone else is going to be signing something, issuing a JIRA ticket with a simple patch and waiting days and days for a fix#2016-07-2220:34xcthulhu@alexmiller: https://gist.github.com/ef92987c8160a796b62c95e207ee0ad4 (fixed a typo)#2016-07-2220:35xcthulhuIt's admittedly 16 lines....#2016-07-2220:42bbrinckAny ideas as to why clojure.test/check
would return the empty list?
user=> (require '[clojure.spec :as s])
nil
user=> (s/fdef my+
#_=> :args (s/tuple number? number?)
#_=> :ret symbol?
#_=> :fn #(= (:ret %) (apply + (:args %))))
user/my+
user=> (require '[clojure.spec.test :as stest])
nil
user=> (stest/check 'my+)
()
user=>
Am I missing a step here?#2016-07-2220:44bbrinckI have the following function defined (defn my+ [x y] (+ x y))
#2016-07-2220:45glvUse a backquote instead: (stest/check `my+)
#2016-07-2220:45glv(That yields a fully namespace-qualified symbol.)#2016-07-2220:45bbrinckah, thank you!#2016-07-2220:52Alex Miller (Clojure team)@mpenet thx, Iāll try to get that fixed the next time someone is paying attention#2016-07-2221:18xcthulhu> Just for fun, Iāve been prototyping a little library to generate EBNF-ish syntax diagrams from specs. Delving into the spec internal representation has been ā¦ eeeenteresting.
@glv: This is cool. Does it emit EBNF that instaparse can digest?
I would absolutely love if I could shake SQL tables out of specs. But that's super hard and probably will never happen.#2016-07-2221:22Alex Miller (Clojure team)why not model your data and shake specs and tables out of that?#2016-07-2221:25glvNo, but itās super basic right now. Iām more concerned with deciphering the structures and building code that can navigate them. Itāll be easy to play with the details of what gets generated later. (Plus, Alex said that those internals are still likely to change.)#2016-07-2223:26bsimacan I spec tagged literals?#2016-07-2223:26bsimaI'm trying to spec a config file, like this: https://github.com/juxt/aero#profile#2016-07-2223:38bsimanvm, the problem I'm having is that aero does its own eval of the config file, so I can't really get access to the data before the literals are processed (even if I do spec the config file before I pass it to aero, I don't have aero's data readers loaded, so #profile
and such is unrecognized). The solution is to spec the data after aero processes it and replaces the tagged literals#2016-07-2300:06codonnellI'm using clojure.spec to validate responses from a particularly poorly written API, and explain-data
is seriously my new best friend. :+1:#2016-07-2300:42lvhglv: Thanks! That did appear to make things better, but this is still pegging my CPU so Iāll work at it a little more š#2016-07-2300:59glvYeah ā¦ in my experiments it helped some, but not dramatically. #2016-07-2301:05lvhpart of the problem is that json schema is naturally extremely recursive#2016-07-2301:05lvhbut the ones that actually exist are wide, not deep#2016-07-2301:05lvhthe problem is that that implies statefulness in the recursion, IIUC#2016-07-2301:05lvhI want to recurse a lot high in the tree, less so at the bottom of he tree#2016-07-2301:07lvhIs there a shorthand for saying "no other keysā in a way thatās automatically generator efficient? I realize that these are normally intended to be open for extension, but this is an existing spec, and it is not open for extension#2016-07-2301:24glvI would expect test/check
to peg your CPU, though. For this kind of thing it's definitely CPU-bound. If you're not getting out-of-memory errors anymore, that's an improvement.#2016-07-2301:43Alex Miller (Clojure team)You can override generators at paths#2016-07-2301:43Alex Miller (Clojure team)Could let you use different gens at different levels#2016-07-2301:44glvWow, I've totally missed that. Doc pointer?#2016-07-2301:53Alex Miller (Clojure team)Was added recently, not sure there are any docs other than the api#2016-07-2301:54Alex Miller (Clojure team)Any place that takes a gen override map can take a vector of keys#2016-07-2301:54glvAh, interesting. Will check into that. Thanks!#2016-07-2301:54Alex Miller (Clojure team)Although I don't remember now if those are spec keys or path keys or both#2016-07-2301:55glvSome help you are. ;-)#2016-07-2301:55Alex Miller (Clojure team)Yeah#2016-07-2302:16Alex Miller (Clojure team)It has to be the path keys#2016-07-2321:39dominicm@bsima: You can access the reader literals in aero 0.4.0, but spec'ing after is okay too#2016-07-2322:56nhaIs it possible to coerce the content of an edn
file ?
My use case: coerce a configuration map (ex. coming from environ
where all values are strings to be able to override them from env vars).
It seems that edn/read-string does not accept namespaced keywords. Using load-file
seem to lead to problems when trying to aot
.#2016-07-2401:11Alex Miller (Clojure team)Namespaced keywords should be fine in edn#2016-07-2401:11Alex Miller (Clojure team)Autoresolved keywords are not part of edn though (as there is no namespace context)#2016-07-2401:12Alex Miller (Clojure team)Do you have an example?#2016-07-2401:34lvhWhat exactly does calling spec with a map do? map-spec-impl suggests I can do that, but specās docstring only mentions predicates and regex specs; I guess a map can behave like a predicate since itās an IFn?#2016-07-2401:56lvhIs there an easy way to get a keys thatās not open for extension? Iām digging into map-spec-impl and just extracting the generator bit isnāt super trivial.#2016-07-2402:01Alex Miller (Clojure team)in short, no#2016-07-2402:01Alex Miller (Clojure team)you can s/and with a restriction on the key set#2016-07-2402:03lvhyeah; I got that part; trying to fix the generator though#2016-07-2402:03lvhI guess I can fmap over the gen I get from keys, and then dissoc the noise it comes up with I donāt want#2016-07-2402:04lvh(I understand the general argument for having extra keys; it just doesnāt make sense for what Iām doing)#2016-07-2402:07lvh@alexmiller: Thanks! Much appreciated š#2016-07-2402:41Alex Miller (Clojure team)yeah, def leverage the existing specs as much as possible for gen#2016-07-2416:41lvhWhen using with-gen, is there some kind of automatic spec verification, or are generators assumed to be correct#2016-07-2416:48lvhMy excl-keys macro ostensibly works, but merging it doesnāt seem to.
(defmacro excl-keys
"Like [[s/keys]], but closed for extension."
[& {:keys [req-un opt-un req opt] :as keys-spec}]
(let [bare-un-keys (map (comp keyword name) (concat req-un opt-un))
all-keys (set (concat bare-un-keys req opt))]
`(let [ks# (s/keys
(s/def ::a (excl-keys :opt-un [::b] :req-un [::c]))
(s/def ::b string?)
(s/def ::c keyword?)
(gen/sample (s/gen ::a))
(s/def ::x (excl-keys :opt-un [::y] :req-un [::z]))
(s/def ::y integer?)
(s/def ::z rational?)
(gen/sample (s/gen ::x))
(s/def ::mix (s/merge ::a ::x))
(gen/sample (s/gen ::mix))
Samples for ::a
, ::x
work fine, merging less so. I guess Iāll dive into how s/merge builds generators š#2016-07-2418:45xcthulhuHey guys, I've been working on a patch to port test.check/let
to clojure.spec.gen
#2016-07-2418:46xcthulhuSince we all have fmap
blues from time to time.#2016-07-2418:46xcthulhuHere's the unit-test vector I want to push to clojure.test-clojure.spec
:#2016-07-2418:46xcthulhu(deftest gen-let-macro
(s/def ::a (s/spec keyword?))
(s/def ::b (s/int-in 7 42))
(let [t (s/tuple keyword? string?)
m (s/keys :req [::a ::b])]
(are [spec generator]
(s/valid? spec (gen/generate generator))
t (gen/let [x :abc, y "efg"] [x y])
t (gen/let [x ::a, y string?] [x y])
t (gen/let [[_ y] t] [::a y])
t (gen/let [] [keyword? string?])
t (gen/let [] t)
t (gen/let [[x y] (gen/let [] t)] [x y])
m (gen/let [{a ::a, b ::b} m] {::a a, ::b b})
m (gen/let [a keyword?] {::a a, ::b ::b})
m (gen/let [] {::a keyword?, ::b 7})
m (gen/let [] {::a keyword?, ::b #{7 8 9}})
int? (gen/let [i 'clojure.test.check.generators/int] i)
#{'abc} (gen/let [x 'abc] x))))
#2016-07-2418:47xcthulhuThat test vector should give a pretty good idea of how gen/let
would work.#2016-07-2418:47xcthulhuMy idea is that it's really annoying having to coerce specs, predicates and constants to generators so it just does it automagically#2016-07-2418:49xcthulhuI'll wait for @alexmiller to tell me if he likes it or not before I attempt a patch#2016-07-2419:25mike_ananevhi there! how to specify :ret value for fdef if function returns void?#2016-07-2419:26xcthulhunil?
#2016-07-2419:28mike_ananev:ret nil? or :ret #(nil? %) ?#2016-07-2419:30xcthulhuI think :ret nil?
but I'm not 100%#2016-07-2419:31xcthulhu#(nil? %)
is probably wrong#2016-07-2420:11xcthulhu@alexmiller: Since you're a category theory fan, I could generalize let
from generators to specs if we model specs as having type (a -> Bool, Gen a)
(Ć” la Haskell)#2016-07-2420:24Alex Miller (Clojure team)@mike1452 either will work, I'd use the shorter one#2016-07-2420:25Alex Miller (Clojure team)@lvh re with-gen, the generator is not trusted and all generated samples will be verified by checking the spec as well#2016-07-2420:28Alex Miller (Clojure team)@mike1452: sorry read further back - functions are never void in Clojure, only comes up in some interop cases#2016-07-2503:19lvhI guess thereās no way to override that? The issue Iām running into is that merge assumes the key specs it gets are open for extension; so keys-excl merged with anything else fails to produce anything, because the keys-excl part of the spec that checks that the map only contains specific keys will fail on all the keys being added by anything thatās merged in#2016-07-2507:35mike_ananev@alexmiller: last call in my clojure function is java (interop) fn call which returns void. So in clojure if java fn returns void we see nil, if i understand correctly.#2016-07-2511:41Alex Miller (Clojure team)Yep#2016-07-2512:56xcthulhu@lvh: Instead of merge, have you tried to make an intersection operation?#2016-07-2512:57xcthulhuSometimes the intersection of two keys-excl
won't work either of course, since you have certain mandatory keys.#2016-07-2513:01mpenet"closed" key sets seems like a common request, hopefully Rich reconsiders his choice here. I get that he doesn't want to make it the default, but simply providing a way to spec this can't hurt#2016-07-2513:22Alex Miller (Clojure team)There are no plans to revisit that afaik#2016-07-2513:24Alex Miller (Clojure team)@lvh you might try going down the path of treating your map as a coll-of tuple entries instead (make sure to use :into {}) #2016-07-2513:25Alex Miller (Clojure team)Then merge that with s/keys to pick up the attribute checks#2016-07-2513:28mpenetIt's super easy to please everybody with a system with "closed" keysets by default (you can just add a k/v spec with any?
if you want to "open" the map), but that apparently is out of the question. I personally do not understand the logic here. Never saw anybody complain about the "strictness" of prismatic/Schema for these things for instance.#2016-07-2513:31mpenetthen again, I don't consider the openness of records a good thing either#2016-07-2514:02lvh@xcthulhu: Iām not sure I understand. As in, remove keys?#2016-07-2514:03lvhI mean, I guess I could, but that seems very gauche. What would that look like? I have a base spec that everything conforms to, and some stuff thatās specific per type (i.e. multi-spec)#2016-07-2514:30xcthulhu@lvh:Yeah, you would remove keys. If a map-spec isn't open under extension, it may be open under specialization.#2016-07-2514:43xcthulhuSo provided that your map-specs s1
and s2
that have the form (keys-excl :opt [...] :opt-un [...])
then you could have
(defn intersection
[s1 s2]
(spec/with-gen (spec/and s1 s2)
#(fmap (fn [[a b]] (select-keys a (keys b)))
(spec/gen (spec/tuple s1 s2)))))
#2016-07-2514:44xcthulhuAnd you'd have the rule:
(valid? (intersection s1 s2) m)
; if and only if
(and (valid? s1 m) (valid? s2 m))
#2016-07-2516:38ghadi@mpenet: search slack history for @richhickey expounding on closed keysets making brittle systems
#2016-07-2517:54mpenetIt's kind of in the same class as static vs dynamic lang/systems imho: it depends, sometimes exactitude is necessary. My point was that allowing both would do no harm, it would be optional. #2016-07-2518:02mpenetI guess that ship has sailed anyway, and likely not open for debate as usual.#2016-07-2518:41ghadithat's a pessimistic attitude#2016-07-2518:41ghadiit's not open for debate (IMHO) because it's a bad idea, and the core team can justify why#2016-07-2518:43ghadiim guessing no one went back to the original discussion. This comes up often enough that it's probably worth documenting why it's not a good idea#2016-07-2518:48ghadihttps://clojurians-log.clojureverse.org/clojure-spec/2016-06-09.html#2016-07-2520:41glvThatās a pretty good discussion, and includes a not-too-bad way to accomplish it if you really feel you must.#2016-07-2603:50wildermuthnAnyone have experience using clojure.spec to spec out macros for use in clojurescript?#2016-07-2603:51wildermuthnI get odd errors relating to clojure.spec not being found when I run clojure 1.9.#2016-07-2603:51wildermuthnMight just be my setup.#2016-07-2604:08glvUse the cljs.spec
namespace in Clojurescript. #2016-07-2616:29nhaI have been trying to use clojure.spec to coerce edn (config) files. EDN prevents autonamespacing keywords ( I get that now, thanks alexmiller ), but allow custom #readers. Clojure (well read-string
and load-file
) allow autonamespacing but prevent custom readers. So it seems that my only option is to explicitly have the keyword namespace every time in my configuration file, is that correct ? I am asking because I find it a bit cumbersome/repetitive to have to have these in my file.#2016-07-2616:35Alex Miller (Clojure team)Yes, you'll need full namespaces in edn config files#2016-07-2616:36Alex Miller (Clojure team)You could also write specs on unqualified keys with req-un and opt-un#2016-07-2616:37nhaAh I see - not super familiar with spec yet. Will look into those keys, thanks š#2016-07-2616:37Alex Miller (Clojure team)That's in s/keys#2016-07-2616:39Alex Miller (Clojure team)Either the normal core reader fns or clojure.edn reader fns should be able to read tagged literals though with ##2016-07-2616:40Alex Miller (Clojure team)You just need to bind around the call to set up the readers#2016-07-2616:41nhaAh I see - I did not knew that. So all is fine š
I will go with clojure.spec (s/keys :req-un [...]) for now.#2016-07-2702:32lvhHm, I donāt really understand how to use retag#2016-07-2702:33lvhI mean, I can use it with a keyword, but now I have something a bit more complex: a schema is either a reference (`{:$ref āsome-string"}`) or a schema with a type#2016-07-2702:34lvhI was hoping to teach the multimethod about that, since s/or doesnāt seem to work#2016-07-2702:35lvh(when I s/or, it complains that it canāt find an appropriate method, which is true I guess; I wasnāt expecting it to even try that branch)#2016-07-2702:41lvhI tried doing with with s/or
this way:
(s/def ::schema
(s/or ::reference
(fn [m] (contains? m :$ref))
::direct-schema
(fn [m] (contains? m :type))))
#2016-07-2702:41lvhWhich doesnāt work because spec doesnāt know how to generate schemas from that pred.#2016-07-2702:43lvhI can (s/and map? ā¦) but that doesnāt really make it easier to generate maps#2016-07-2702:49lvhI guess I can just use map? and it can later specify with keys ā that appears to sorta work š#2016-07-2715:39xcthulhu(s/def ::$ref uuid?)
(s/def ::type keyword?)
(s/def ::schema (s/or ::reference (s/keys :req-un [::$ref]) ::direct-schema (s/keys :req-un [::type])))
#2016-07-2715:39xcthulhu@lvh: I think you are supposed to do it in the style above.#2016-07-2715:53michaeldrogalisHello. Is it possible to assert properties about keys and values in a map using s/keys
without using the registry?#2016-07-2716:12xcthulhuNot that I know of.#2016-07-2716:12codonnell@michaeldrogalis: You can put keys into s/keys
that aren't in the registry, and the spec will just check for their existence. It is not possible to provide inline specs to validate map values.#2016-07-2716:13michaeldrogalis@codonnell: Thanks for the confirmation.#2016-07-2716:14codonnellno problem#2016-07-2716:19Alex Miller (Clojure team)http://blog.cognitect.com/blog/2016/7/26/clojure-spec-screencast-testing#2016-07-2716:19Alex Miller (Clojure team)new screencast about check and instrument#2016-07-2716:34lvhAwesome, thank you!#2016-07-2716:39michaeldrogalis@alexmiller Neato š#2016-07-2717:51jonathanjAre there any thoughts on using spec for something like web form validation? Specifically having some way to generate user-readable messages from failed predicates, i.e. more fine-grained than "Name is wrong"#2016-07-2717:51jonathanj(I don't have much experience with spec, apologies if this is either a really frequent question or a really stupid one.)#2016-07-2718:15dominicm@jonathanj: IMO, the output would be okay for parsing for a web form.
However, spec doesn't play too well with cross validation, e.g. "is this email taken in the database?"#2016-07-2718:16dominicmI also found specifying equal fields difficult, with meaningful output, so you'd have a few special cases to handle outside of s/keys
#2016-07-2718:25jonathanjAre you referring to the error data when you say output, @dominicm? I don't see how the error data would be suitable for a user-facing form, you need a layer of translation since arbitrary predicate names are unlikely to be of value to users in most circumstances. #2016-07-2718:26dominicmMy apologies, suitable for translating to user information.#2016-07-2718:28jonathanj@dominicm: what about involving multiple values in a validation? We have a case where the SSN needs to be validated against the gender of the selected title (don't ask), would this be written as a separate spec that combines the others, with and
perhaps?#2016-07-2718:31jonathanjAnd perhaps more interestingly, would the error data allow me to determine all the participants in the failing validation?#2016-07-2718:31dominicmUser forms have plenty of awkward cases, domain logic runs rampant here.
I think and would be appropriate yes. You want to ensure the SSN, gender and selected title are all valid before doing the overall validation I assume? That would be s/and
.
I don't know the api too well though, my interest dwindled when I tried to use spec for web forms and found limitations in spec.#2016-07-2718:32dominicmI did find that the output error data was very thorough though, and you could derive a lot of information from it.#2016-07-2718:33jonathanj@dominicm: You're quite right about awkward cases for user forms, did you go another route in the end?#2016-07-2718:34dominicmI started writing my own validation library.#2016-07-2718:34jonathanjIs the code public?#2016-07-2718:34dominicmNot yet š#2016-07-2718:34dominicmI should open source my attempts.#2016-07-2718:34dominicmhttps://github.com/logaan/vlad I can highly recommend vlad though.#2016-07-2718:35jonathanjI've already written two (JS) validation libraries and they are plagued with reinventing existing predicates. I was hoping to find a better way of layering this on something else. #2016-07-2718:36dominicmvlad has a lot of useful predicates built in.
https://github.com/weavejester/valip/blob/master/src/valip/predicates.clj I sometimes take advantage.#2016-07-2718:37jonathanjMy experience was that a Boolean is not a good enough result for explaining errors to users, but having to decompose the data after the predicate failed was too much duplication of effort #2016-07-2718:38dominicmNope, you must attach additional semantics to it. That's where vlad comes in.#2016-07-2718:39dominicmI just realised that the README does not do it justice.#2016-07-2718:39dominicmhttps://github.com/logaan/vlad/blob/master/src/vlad/core.cljc#L117 this is the definition of the present?
checker#2016-07-2718:40jonathanjSkimming the readme, vlad seems worth further investigation, thanks!#2016-07-2718:41jonathanj@dominicm: Nice, reasonably succinct. #2016-07-2718:43dominicmIt's the result of much comparison, I'm glad you can benefit from my research š#2016-07-2719:32bsimadoes anyone know what this error means?#2016-07-2719:32bsimahttps://gist.github.com/bsima/042554a1a61ce18adad0ebaae60a833f#2016-07-2720:02seancorfield@bsima: Which version of Clojure? That works for me on Alpha 10.#2016-07-2720:02bsima1.9-alpha10#2016-07-2720:02bsimaĀÆ\(ć)/ĀÆ#2016-07-2720:05bsimaI think a lein clean
fixed it#2016-07-2720:12seancorfieldlein clean
solves so many evils š#2016-07-2720:52kendall.buchanan@gfredericks: Just following up real quick on that issue last week where explicit loading of test.check and clojure.spec.test/check
lead to MultiFn errors. You mentioned master
seemed to fix it. Any thoughts on when the next release will be cut?#2016-07-2721:11gfredericksProbably depends on whether it's still painful for @alexmiller #2016-07-2721:11Alex Miller (Clojure team)It is#2016-07-2721:21martinklepsch@dominicm: interesting, didn't know about vlad, looks cool#2016-07-2721:32gfredericks@kendall.buchanan: I'll prioritize getting a good release together ASAP#2016-07-2721:33kendall.buchanan@gfredericks: Thank you. I donāt mean to put pressure on you, or anything, but I do sure appreciate it.#2016-07-2721:34gfredericksNo worries#2016-07-2721:48Alex Miller (Clojure team)Iām willing to do a release, just would prefer to do as few as possible as it will take me at least an hour to figure out how again#2016-07-2721:49Alex Miller (Clojure team)@gfredericks: a good change log wrt to things likely to affect spec would also be helpful to have#2016-07-2721:50gfredericks@alexmiller: like to send to you folks or for permanent public consumption? #2016-07-2721:52Alex Miller (Clojure team)public, not looking for anything special for us, just to have a place to start analyzing impact#2016-07-2721:52gfredericksCool#2016-07-2807:43mpenet@gfredericks: chuck.gen/string-from-regex is nothing short of magic, impressive#2016-07-2809:42stammiHi! Maybe someone can spot my mistake in this:#2016-07-2809:42stammi(s/def ::number-between-one-and-zero (s/and number? #(< % 1) #(> % 0)))#2016-07-2809:42stammi(s/conform ::number-between-one-and-zero "-13")#2016-07-2809:43stammigives me valid#2016-07-2809:44stammioh forget that please...#2016-07-2809:44stammi...gnarf#2016-07-2812:56Alex Miller (Clojure team)Check out s/int-in or s/double-in#2016-07-2813:29mpenetsmall attempt at specing the latest ring specification -> https://github.com/mpenet/ring-spec/blob/master/src/clj/qbits/ring_spec.clj#2016-07-2813:49mike_ananevhi! if i have (s/def ::decimal #(instance? BigDecimal %)) how to make (s/exercise ::decimal) working? I suppose a should create custom generator in spec, but can't find how to do it. thanx#2016-07-2813:57mpenetI guess something like (gen/fmap #(BigDecimal. %) gen/nat)#2016-07-2813:58mpenet(s/def :decimal (s/spec #(...) :gen (constantly (gen/fmap #(BigDecimal. %) gen/nat)))) (untested)#2016-07-2813:59mpenet@mike1452: super handy cheat-cheat -> https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#2016-07-2814:30mike_ananev@mpenet: š doesn't work. I've tried similar ways and unfortunately can't get result. i'm dumb :((#2016-07-2814:31mpenetseems to work here#2016-07-2814:32mpenet(s/def ::decimal (s/spec #(instance? BigDecimal %)
:gen (constantly (gen/fmap #(BigDecimal. %) gen/nat))))
#2016-07-2814:32mpenetexercise returns -> ([0M 0M] [0M 0M] [1M 1M] [2M 2M] [1M 1M] [0M 0M] [4M 4M] [1M 1M] [2M 2M] [5M 5M])#2016-07-2814:32mpenetetc#2016-07-2814:33mpenetwell, you need to throw in the decimal part actually ex (str % ".0")#2016-07-2814:34mpenetyou could generate pairs for the part after .#2016-07-2814:35mike_ananevthanx. can you show your gen namespace from require section?#2016-07-2814:35mpenetgen is clojure.test.check.generators#2016-07-2814:36gfredericksMaybe it's a good time to get a good bigdec and bigint and ratio generator in test.check#2016-07-2814:36mpenetthere is ratio I think, not big* tho#2016-07-2814:37mike_ananevhave trioubles with gen namespace#2016-07-2814:37mike_ananevAlias gen already exists in namespace arina-aaa.atomix.spec, aliasing clojure.spec.gen#2016-07-2814:38mike_ananev(:require
[clojure.spec :as s]
[clojure.test.check.generators :as gen])#2016-07-2814:39mike_ananevAh! REPL restart helps!#2016-07-2814:39mike_ananevthanx!#2016-07-2814:39mpenetš#2016-07-2814:46gfredericksOh right. I meant a good ratio generator :)#2016-07-2815:53michaeldrogalisI'm a little caught up on s/def
being a macro. Is it possible to do something like:
(let [x :my/spec-name]
(s/def x string?))
#2016-07-2815:54michaeldrogalisThe problem seems to be that def
expands, and the first argument retains the value of the symbol x, not the value that it refers to (`:my/spec-name`)#2016-07-2816:20featalionyou can do it with another macro:
(let [x :user/the-x]
`(s/def ~x string?))
;;=> (clojure.spec/def :user/the-x clojure.core/string?)
#2016-07-2817:34stammithanks @alexmiller. that looks better indeed#2016-07-2909:50mpenethow can I spec the return spec of a function that depends on the args it received : ex ring-spec normal handler vs async-handler: (if args.length == 3 : ret then is nil else response map)?#2016-07-2909:50mpenetI guess I can hack something with :fn, but it doesn't feel right#2016-07-2909:53mpeneta "multiple arity" version of fspec could make sense here. it would feel more natural maybe#2016-07-2909:54mpenet(s/fspec (:args ... :ret) (:args :ret))
#2016-07-2909:56mpenetor maybe I can just write 2 fspec and just wrap them with (s/or ..)#2016-07-2912:22gfredericksYou could make the ret spec nilable and describe that relationship in the :fn as you mentioned. Doesn't seem like that much of a hack to me#2016-07-2912:23mpenetI think I prefer the s/or with the 2 fspecs, makes the relation between args/ret clearer#2016-07-2912:24mpenetI came up with this in the end https://github.com/mpenet/ring-spec/blob/master/src/clj/qbits/ring_spec.clj#L81-L111#2016-07-2913:49Alex Miller (Clojure team)@mpenet: the whole purpose of :fn is to describe relationships between args and ret#2016-07-2913:51mpenetgot it, so not really good to discern 2 arities of a function like I was doing (I had one "giant" fspec describing both arities/return types)#2016-07-2913:51Alex Miller (Clojure team)seems perfectly reasonable to cover each possibility in the :ret (just tell the truth about what it can be)#2016-07-2913:51Alex Miller (Clojure team)then in :fn you can declare that the :ret you got matched the :args that were sent#2016-07-2913:52Alex Miller (Clojure team)you get conformed values of :args and :ret in :fn, so it should be relatively easy to check#2016-07-2913:53mpenetYes I saw this, but wouldn't it be more work for gen down the road having all these branches? I mean vs. (s/or (s/fspec) (s/fspec))#2016-07-2913:53mpenetI guess it depends what kind of functions we re talking about#2016-07-2913:53Alex Miller (Clojure team)it just picks one or the other#2016-07-2913:53Alex Miller (Clojure team)thatās why I have a computer :)#2016-07-2915:00mpenetany teaser of what's coming in the next alpha?#2016-07-2915:14gfredericksa fully-featured dependent type system#2016-07-2916:30arohner@gfredericks: you joke, but: https://github.com/arohner/spectrum#2016-07-2916:31gfredericks@arohner: oh hey good!#2016-07-2916:32arohnerIt still needs a whole lot of polish, but Iām convinced its sound#2016-07-2916:33mpenetwow, impressive. will kick tires for sure#2016-07-2916:36arohneritās still basically at developer preview stage#2016-07-2916:36arohnerbasic tests pass, and simple files will check / not-check as appropriate, but nowhere near ready for production code#2016-07-2922:34Alex Miller (Clojure team)@mpenet I have some of the nastier core macro specs ready and some of them might land in the near future#2016-07-2923:30lvhgfredericks: Preferred schpec namespace for maps that are closed for extension and its generators? keys?#2016-07-2923:30lvhAlso, weighted-or#2016-07-2923:31lvh(weighted-or:
(defmacro weighted-or
"Like [[s/or]], but with weights."
[& weighted-options]
(let [[names specs ws] (apply map vector (partition 3 weighted-options))
sums (reductions + ws)]
`(s/with-gen
(s/or
)#2016-07-2923:58lvhIs there a pred for large integers?#2016-07-2923:58lvhLike biginteger or bigint#2016-07-3000:00lvhI canāt seem to find long?
either#2016-07-3000:01eggsyntaxint?
will capture all fixed-precision integers. It would still be nice to have a bigint? pred so that the generator specifically produces bigints.#2016-07-3000:02lvhah, gotcha#2016-07-3000:02lvhthanks š#2016-07-3000:02gfredericks@lvh I don't mind the top level namespace for things that are pretty useful/general#2016-07-3000:03lvhgfredericks: OK works for me#2016-07-3001:10Alex Miller (Clojure team)Doesn't integer? map to that?#2016-07-3001:10Alex Miller (Clojure team)Oh you want just bigints#2016-07-3001:13seancorfield(s/and int? (s/conformer bigint))
perhaps?#2016-07-3001:14seancorfieldboot.user=> (s/exercise (s/and int? (s/conformer bigint)))
([-1 -1N] [0 0N] [0 0N] [0 0N] [4 4N] [4 4N] [-12 -12N] [-45 -45N] [4 4N] [-63 -63N])
#2016-07-3001:15seancorfieldBTW @alexmiller I think the docstring on s/conformer
is very confusing about its intent...#2016-07-3001:15seancorfield"takes a predicate function with the semantics of conform i.e. it should return either a
(possibly converted) value or :clojure.spec/invalid, and returns a
spec that uses it as a predicate/conformer"#2016-07-3001:17Alex Miller (Clojure team)Because?#2016-07-3001:18seancorfieldBecause the function argument it takes isnāt really a "predicate function"ā¦ that makes it sound like it should be something like int?
...#2016-07-3001:19seancorfieldā¦perhaps if it said "takes a conforming function i.e., it should return either a (possibly converted) value or :clojure.spec/invalid"ā¦?#2016-07-3001:21Alex Miller (Clojure team)Maybe#2016-07-3001:24seancorfield(predicate is pretty much otherwise used for functions that return truthy / falsey values in the Spec docs I think?)#2016-07-3001:26seancorfield&
and and
take predicates, fdef
ās :ret
, merge
, spec
#2016-07-3001:28seancorfield(although there is mention in a couple of places that some of those "predicate" contexts may also conform the valueā¦ so itās not entirely clear whatās intended)#2016-07-3001:31seancorfield@lvh @eggsyntax I tested and it looks like you can use (s/and (s/double-in ā¦) (s/conformer bigdec))
if you wanted BigDecimals ā and (s/conformer rationalize)
if you wanted ratiosā¦ and the example above produces big ints#2016-07-3001:32seancorfieldOh, simpler: (s/exercise bigdec?)
ā didnāt realize there was a direct generator for that#2016-07-3001:33seancorfieldand (s/exercise ratio?)
...#2016-07-3001:33seancorfieldSo bigint?
is kind of the only missing one...#2016-07-3001:35eggsyntaxOh, interesting. I was thinking something roughly like (s/with-gen int? (gen/fmap #(bigint %) int?))
(that's just offhand & may have errors; I don't have a repl at hand).#2016-07-3001:48seancorfieldUsing s/conformer
seems simplerā¦ but now Alex has made me question whether thatās entirely valid šø#2016-07-3001:48seancorfieldj/k#2016-07-3001:50seancorfieldWe use s/conformer
as a way to map input field formats to domain model formats along with specs for input fields. Weāre now using specs as part of input validation and as part of our domain model testing.#2016-07-3001:52eggsyntaxNice. I haven't really played with s/conformer
yet (although I'm certainly making use of s/conform
).#2016-07-3001:53lvh@seancorfield: great, thanks š#2016-07-3001:54lvh@alexmiller: Iām OK with any integer; I didnāt realize integer? had been introduced and was using int?#2016-07-3002:02Alex Miller (Clojure team)integer? is old, int? is the new one#2016-07-3009:15dryewoIt seems that :ret
is not checked at all currently:
(defn foo [x] x)
(s/fdef foo
:args (s/cat :x int?)
:ret string?)
(stest/instrument `foo)
(foo 1)
#2016-07-3009:16dryewowhat am I doing wrong?#2016-07-3009:17dryewoif I put string?
into :args
, it throws as expected.#2016-07-3009:17dryewoIām using alpha10#2016-07-3009:28dryewoHowever, when I run (stest/check
foo)`, it complains about wrong :ret
. Should it be this way, only checked while testing?#2016-07-3009:30dryewoOh, it seems to be a feature: https://groups.google.com/forum/#!msg/clojure/RLQBFJ0vGG4/K-5Rp6QXCgAJ#2016-07-3009:30dryewookay, never mind#2016-07-3017:44seancorfield@dryewo: yes, instrument
is to test that functions are called correctly, check
is to test that functions are implemented correctly. #2016-07-3103:00gfredericksthat's a nice explanation#2016-07-3103:15seancorfield(Iām just repeating what Stu said in the latest Cognitect clojure.spec 'cast! š )#2016-07-3118:47gfredericks@alexmiller: the call to generate
here undermines test.check determinism (similar to CLJ-1949): https://github.com/clojure/clojure/blob/f374423053b75b7b484ffbc8b49b2ede9d92e406/src/clj/clojure/spec/test.clj#L155#2016-07-3118:47gfredericksin particular if you're running check
while instrumenting with stubs#2016-07-3118:48gfredericksI don't think there's as straightforward a fix as in CLJ-1949 though#2016-07-3118:49gfredericksprobably the cleanest thing would be a private dynamic var somewhere for registering quickcheck-driven generation fns#2016-07-3118:49gfredericksit could use gen/generate
as the default when you're not actually running a test.check test#2016-07-3118:54gfredericksshall I make a ticket?#2016-07-3119:02Alex Miller (Clojure team)Sure#2016-07-3119:16gfredericksACK#2016-07-3122:43lvh@gfredericks: FYI, I made a bunch of progress on the JSON Schema thing, knocking a few remaining ones down; the really really hard one is references (idk if youāre familiar with JSON schema, but basically you have a base URL and URL-y internal/external references, so you need a way to resolve things like #../../xyzzy/0 or whatever). Dunno if itās going to be of a size where itās interesting for spec, but weāll see#2016-07-3122:44lvhAre you interested in like, mostly-working-but-missing-stuff impls, or would you rather just pull in an entire impl when itās done, or rather have it live in its own package#2016-07-3122:44lvhItās currently about 250 lines of impl and 220 of test; thatāll probably go up to 300#2016-07-3122:45lvhit doesnāt have any particularly weird deps, but to do it Right(TM) you need test-chuckās regex generator#2016-08-0103:37gfredericks@lvh I was pondering having an "experimental" namespace, so we could probably start it there (c.g.schpec.expermental.json-schema)#2016-08-0103:38gfredericksschpec depending on chuck is fine and probably inevitable#2016-08-0109:09rickmoynihan@lvh: @gfredericks: Sorry, just seen the messages above about JSON Schema / spec... curious what you're doing exactly, it sounds interesting and useful!#2016-08-0109:46mpenetsame here, I'd love to see the work in progress#2016-08-0110:12mpenetI started to dump utils I am using in our projects in https://github.com/mpenet/spex, I might make a couple of PR if there's an agreed "spec utils" project in the works (and interest)#2016-08-0113:38lvhIām writing a fn that takes a JSON Schema and produces an equivalent spec#2016-08-0114:42dominicm@lvh: How possible would the reverse be do you think?#2016-08-0114:50gfredericksshould be possible for a subset of specs presumably#2016-08-0115:24lvh@dominicm: Trickier, but like gfredericks said possible for some subset of specs; the biggest problem is that you presumably need to statically determine what a particular spec pred means, and then see how much of that you can express in JSON Schema#2016-08-0115:25lvhitās not going to be very nice json schema (probably larger, decent amount of duplication) but as long as you treat it like a compilation product and donāt care about bidirectional access that seems fine#2016-08-0115:25dominicmYeah, that was my thoughts also.#2016-08-0115:25gfredericksthe fact the predicates are stored as symbols that can't be reliably mapped to vars seems problematic for doing this kind of thing robustly#2016-08-0115:25dominicm^^#2016-08-0115:26dominicmCould you say :foo/bar
is this part of JSON schema?#2016-08-0116:03rickmoynihana symbol is a namespace qualified value too though right? 'clojure.core/int
#2016-08-0116:06gfredericksNot necessarily#2016-08-0116:07gfredericksThe mapping from symbols to vars is entirely relative to the current namespace setup#2016-08-0116:08gfredericksYou can setup your namespace so that clojure.core/int
resolves to something entirely different#2016-08-0116:08gfredericksBut in any case I think spec would only store int?
generally#2016-08-0116:10rickmoynihansurely a macro could resolve it to its bound value though by inspecting ns-map etc... though#2016-08-0116:33arohner(s/form (s/spec int?))
clojure.core/int?
#2016-08-0116:34arohneryes, you could set up your namespace so that int? redirects to something else, but spec does store the ns#2016-08-0116:37eggsyntax@lvh: @seancorfield -- I just had occasion to write a spec for bigints, and was surprised to discover that (int? (bigint 1))
returns false
. So I went with:
(s/def ::bigint (s/with-gen #(instance? clojure.lang.BigInt %)
(fn [] (gen/fmap #(bigint %) (s/gen int?)))))
Just as an FYI followup.#2016-08-0116:38lvhWhat does integer? do?#2016-08-0116:39lvhI find the two pretty confusing still :)#2016-08-0116:41eggsyntaxint? has to be fixed-precision. integer? just has to represent an integer, I guess. So I may be able to simplify that š#2016-08-0116:42eggsyntaxYes indeed, this works:
(s/def ::bigint (s/with-gen integer?
(fn [] (gen/fmap #(bigint %) (s/gen int?)))))
#2016-08-0116:42eggsyntaxThanks @lvh, I would have missed that.#2016-08-0116:45seancorfield@eggsyntax: and why doesnāt (s/def ::bigint (s/and integer? (s/conformer bigint)))
work for you?#2016-08-0116:45seancorfield(all I changed from my original suggestion was to use integer?
instead of int?
there)#2016-08-0116:46seancorfieldI guess the question is should (s/conform ::bigint 1)
be 1N
or :clojure.spec/invalid
?#2016-08-0116:46eggsyntaxYeah, good question. For my purposes, sure. But I could imagine going the other way in another context.#2016-08-0116:47eggsyntaxThe conformer version works too, & is nicer actually, I'll go with that š#2016-08-0116:48eggsyntaxI mainly just meant to illustrate that it had to be integer?
rather than int?
#2016-08-0117:05seancorfieldYour predicate ā #(instance? clojure.lang.BigInt %)
ā is not the same as integer?
thoā. Your predicate rejects 1
. That was my question.#2016-08-0117:06seancorfield(and (s/exercise integer?)
doesnāt seem to generate bigints in my quick tests)#2016-08-0117:07seancorfieldclojure.spec
encourages us to be spec
ific š#2016-08-0117:56gfredericksw.r.t. generating bigints, I ran into a bug once that didn't surface until the bigints were larger than Double/MAX_VALUE#2016-08-0117:58gfrederickswhich makes it interesting to consider how large of a bigint a general bigint generator should test by default#2016-08-0117:58gfrederickspresumably at least bigger than Long/MAX_VALUE#2016-08-0117:58gfredericksbut Double/MAX_VALUE is quite large#2016-08-0118:02dominicm@gfredericks: I thought some of the automatic tests were around max/min values?#2016-08-0118:03gfredericksyou're asking about default generator behavior?#2016-08-2203:01Alex Miller (Clojure team)@lvh yes bug, and yes on core.typed#2016-08-2203:01Alex Miller (Clojure team)@lvh the with instrument is interesting#2016-08-2203:02lvhMaybe I just havenāt seen the other use cases; it just seems obvious that Iād use it in a deftest#2016-08-2203:03Alex Miller (Clojure team)You mean on instrument ? It's useful just to turn on instrumented specs at the repl#2016-08-2203:13lvhHm, OK, that makes sense#2016-08-2203:13lvhI filed that issue, Iāll file another underā¦ core.typed I guess#2016-08-2203:17lvhDone š#2016-08-2204:42seancorfield@lvh Have you seen how clojure.java.jdbc
deals with instrument
in the test namespace?#2016-08-2204:52seancorfieldFor stest/check
, you could look at stest/summarize-results
which prints a nice summary of the success/failure and produces a hash map you can easily assert about (using is
, for example).#2016-08-2204:54seancorfieldWe use Expectations, and with test.check
we (expect (more-of {:keys [...]} ...) (property-based checks go here))
#2016-08-2205:05seancorfield(or you can (map stest/abbrev-result (stest/check 'my-ns/foo))
and make assertions on each of those results)#2016-08-2212:07brabsterI'm having a bit of pain with namespaced keywords - I have code that defines specs as namespaced keywords and expects to see those keywords used in calls to its functions - but it's super easy for a client to accidentally typo a new keyword into my namespace instead of referring to one that I've defined, and I can't catch the mistake if the keyword I've defined is optional#2016-08-2212:09brabsterso I defined a map key as ::s3/bucket-name, and in the client code mistyped ::s3/ucket-name - it seems like a really obvious dumb error but I'm not sure I can do anything to catch it without starting to custom-code validation?#2016-08-2212:10brabsteris there anything I can do in clojure to prevent keywords being defined into my namespace from outside (i.e. making that an error) or in spec to say that only the keys I've defined are valid - or have I misused something?#2016-08-2212:27donaldballFor the latter, (s/and (s/keys ā¦) (s/map-of #{allowed-keys} any?))
is a possibility#2016-08-2212:45eraserhd@atroche I've always wanted a macro like this. Preferably allowing for with-test
as well.#2016-08-2213:33brabster@donaldball nice, hadn't thought of that - only problem being the need to duplicate union of :req and :opt into it to achieve the effect... but helpful, thanks!#2016-08-2213:34donaldballCould probably wrap it up in a nice macro#2016-08-2213:36brabsteryeah - it doesn't seem like it would be a terrible idea to just have a flag or something on s/keys
that made any keys that aren't in the union file#2016-08-2213:38brabsterI get the argument for the leniency about composing functions to build up maps from the rationale, but it seems like there's a case for the final step in such a process where I need to assert that I've produced a valid result.#2016-08-2214:41bbrinck@lvh @seancorfield Re: unit/generative tests, with my existing test.check tests, I've had luck using something like times
(https://github.com/gfredericks/test.chuck#times) to run a small percentage of generative tests in my normal, fast feedback loop and then run the full amount less often.#2016-08-2214:42bbrinckI haven't yet tried that approach with the clojure.test generative testing, but it's been a useful technique so far#2016-08-2218:49jstokesim trying to come up with a way to define a spec for a map, and also a spec for key/value pairs in that map#2016-08-2218:50jstokesit also seems like something like this doesnāt work, which makes me think Iām headed in the wrong direction#2016-08-2218:53angusiguessYou could define an entry as (s/or :a ::a :b ::b :c ::c)
#2016-08-2218:54angusiguessThat would get you the values only, you might have to do something with tuples.#2016-08-2218:55angusiguess(s/tuple #{::a} int?)
#2016-08-2218:55angusiguessMight start to get a little redundant#2016-08-2221:21Alex Miller (Clojure team)Entries are collections and maps can be treated as collections of entries#2016-08-2221:22Alex Miller (Clojure team)tuple is probably the best match for an entry#2016-08-2221:24Alex Miller (Clojure team)So you can spec a map as a coll-of an or of tuples#2016-08-2221:24Alex Miller (Clojure team)If you do so, you should also merge it with s/keys to get validation of values#2016-08-2303:10amacdougallI've been following http://clojure.org/guides/spec, and trying to apply it to some maze-building code I'm toying with. The core data structure is a grid (a two-dimensional vector) of cells (x, y, and a set of open exit directions). I have some specs which seem entirely reasonable to me:
(spec/def ::x integer?)
(spec/def ::y integer?)
(spec/def ::direction #{::n ::ne ::e ::se ::s ::sw ::w ::nw})
(spec/def ::exits (spec/coll-of ::direction :kind set?))
(spec/def ::cell (spec/keys :req [::x ::y ::exits]))
(spec/def ::row (spec/coll-of ::cell))
(spec/def ::grid (spec/coll-of ::row))
And then I have this pretty basic function, with a pretty basic fspec:
(defn grid-contains?
"True if the supplied x and y coordinates fall within the grid boundaries."
[grid x y]
(and (<= 0 y) (<= 0 x)
(< y (count grid))
(< x (count (nth grid y)))))
(spec/fdef grid-contains?
:args (spec/cat :grid ::grid :x ::x :y ::y)
:ret boolean?)
But when I take the guide's advice and do this:
(require '[clojure.spec.test :as stest])
(stest/check `grid-contains?)
...I use 350% CPU for fifteen minutes and counting. What gives? Is there some unbounded complexity in my specs? Or is test.check just being very thorough?#2016-08-2303:12amacdougallI did try this:
(stest/check `grid-contains? {:clojure.spec.test.check/opts {:num-tests 10}})
... which seems to be the right way to specify the options, according to https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec.test/check, but it didn't make a difference.#2016-08-2303:19amacdougall(spec/exercise ::grid)
does fine, so I want to think that Clojure is able to generate correct test data without any issues. ĀÆ\(ć)/ĀÆ#2016-08-2303:20amacdougallFinal note: when I first ran stest/check
, it hit null errors which revealed bugs (yay?). Once I fixed those bugs, I ended up in this situation.#2016-08-2303:58lvhCan I use spec to spec out a protocol? Just fdef the protocolās method separately??#2016-08-2304:23seancorfield@lvh I seem to recall a JIRA issue around protocol functions, but it may be only for instrument
...?#2016-08-2306:58brabster@amacdougall my first thought is maybe your spec is generating enormous grids under test which is chewing up cpu walking around it#2016-08-2307:13brabster@amacdougall just added a print to your fn and ran (stest/check `grid-contains? {:num-tests 1}), getting values like 32174406 809896496
-13491 -1
-3980 -2856
193086473 -53862035#2016-08-2307:15brabsterWould guess it's the (nth grid y) that's trying to walk along really long linked lists or something - out of time now but hope that helps!#2016-08-2310:45lvhhttp://dev.clojure.org/jira/browse/CLJ-1941 maybe#2016-08-2311:20magnetophonwhat's wrong with the above?#2016-08-2311:22magnetophonthe generator works, the s/def works if I comment out the with-gen parts, but the whole fails with: "Couldn't satisfy such-that predicate after 100 tries."#2016-08-2311:24minimalsort turns maps into lists of map entries#2016-08-2311:26magnetophon@minimal thanks.#2016-08-2311:26magnetophonwhat would be the proper way?#2016-08-2311:32minimalare you trying to gen a sorted-map? @magnetophon#2016-08-2311:32magnetophonyes#2016-08-2311:33minimali think this works (gen/sample (gen/fmap (fn [a] (apply sorted-map (flatten (sort a)))) (gen/map (s/gen int? ) (s/gen int?))))
#2016-08-2311:35minimal({} {0 -1} {0 -1} {-1 -1, 0 -4, 7 -3} {-1 -4, 0 -5, 1 -1} {-2 -2, -1 0, 0 3, 3 7} {-1 6, 0 -4, 24 -12} {-16 -21, -7 0, -4 16, -1 0, 0 -1, 1 -10} {-32 -7, 0 -23, 92 -13} {-13 1, -3 -1, -2 0, -1 1, 6 51, 14 11})
#2016-08-2311:37magnetophon@minimal It does! thanks! now I just need to decrypt that, but I'll manage.#2016-08-2311:37minimalJust turning it into what sorted map expects (sorted-map k1 v1 k2 v2 ā¦)
#2016-08-2311:40magnetophon@minimal I'm very new to clojure and lisp, so I'll have to study a bit to really get it, but I'll be fine.#2016-08-2312:41magnetophon@minimal Hmm, I sort of get what you're doing, and can use that with most types as the val
of the map, but when I use (the type I need )[https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79] I get: No value supplied for key: 0
#2016-08-2312:42magnetophonthe weird thing is: when I comment out the :src-index, it works. when I leave that in, but uncomment :length, it also works.#2016-08-2314:22amacdougall@brabster: Thanks for the pointerāmaybe if I refine my spec to declare that the grid should never be more than, say, 10000x10000, and also specify that it must be made of vectors (for faster indexed access), test.check
will be a bit less ambitious.
On the other hand, should we expect such a system to generate arbitrarily large collections up to the literal limit of the integer type? I certainly see the benefit, but if a single test takes such a long time to run, how could you run a whole suite without a Bitcoin-mining level of resources?#2016-08-2314:29amacdougallLast night before giving up, I did change the grid spec to include :kind vector?
:
(spec/def ::row (spec/coll-of ::cell :kind vector?))
(spec/def ::grid (spec/coll-of ::row :kind vector?))
But it didn't seem to make a difference. My understanding was that nth
on a vector is O(1), because it is effectively (v n)
, and although I don't understand all the voodoo at a glance, https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java seems to confirm that.#2016-08-2314:33amacdougallTo put it more generally: should developers expect to hand-tune specs to keep generative testing from trying to simulate the entire possible range of inputs? On reflection, the answer is yes: that's what generative testing is for.
But in practice, generative testing on strings generates a bunch of garbage strings, it doesn't generate million-character strings. So why is this test trying to generate a 193,086,473-column grid?
...end of musing.#2016-08-2314:48magnetophon@minimal ah, found the problem: flatten flattens recursively, so when I have an uneven number of elements I get an error.#2016-08-2314:49minimalcool, sorry was afk for a bit#2016-08-2314:50magnetophonnp, you've been very helpful.#2016-08-2314:51minimalnp#2016-08-2315:08amacdougallI'm afraid that setting explicit bounds on my specs doesn't make a difference. I updated the relevant specs to limit both coordinates and grid dimensions:
(spec/def ::x (spec/and integer? (partial > 1000)))
(spec/def ::y (spec/and integer? (partial > 1000)))
(spec/def ::row (spec/coll-of ::cell :kind vector? :max-count 1000))
(spec/def ::grid (spec/coll-of ::row :kind vector? :max-count 1000))
...and added a prn
to grid-contains?
to see what test.check is trying to do... and I get this:
#'user/grid-contains?
user=> (stest/check `grid-contains? {:num-tests 1})
"grid-contains? grid 0 0"
OutOfMemoryError GC overhead limit exceeded clojure.lang.PersistentVector$TransientVector.persistent (PersistentVector.java:568)
Perhaps there's something I could do to work around this, but I think for now I'll skip the generative testing.#2016-08-2315:14amacdougallThe other main use of fdef
is to instrument
a namespace during development/test, which forces spec checking of all function calls, right? That seems like it would still be quite beneficial even without full-spectrum generative testing. š#2016-08-2316:59raymcdermottHi guys, I want to do some checking of JSON data with spec, seems like it will be OK - any gotchas?#2016-08-2317:37magnetophon@minimal I found a solution ^^#2016-08-2317:40magnetophonwhen I s/conform
some ok values it works as expected. when I try to conform
values with a duplicate key, I don't get an error saying that, but instead I get: Unmatched delimiter: )
#2016-08-2317:41magnetophonshould I report that as a bug in clojure, or am I doing it wrong?#2016-08-2317:43magnetophonHere is the code, and some sample data,if someone wants to try it out: https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79#2016-08-2317:50minimal@magnetophon cool! Yep (into (sorted-map))
is better. Not sure about the error#2016-08-2317:54magnetophon@minimal a clearer example of the error is: (s/def ::test-type
(s/map-of int? int?))
(s/conform ::test-type {41 1, 42 2}) ; ok
(s/conform ::test-type {42 1, 42 2}) ; unmatched delimiter: )
#2016-08-2317:54bbrinck@amacdougall Keep in mind that instrument
will only check :args
, not :ret
or :fn
specs#2016-08-2317:57bfabry@magnetophon you should be getting an exception from the reader creating a literal map with duplicate keys#2016-08-2317:57minimal@magnetophon you canāt have map literals with duplicate keys anyway. The unmatched delimiter must be a symptom of that#2016-08-2318:01bfabryI mean.. you also can't have maps with duplicate keys either. if you try and construct one one of the keyval pairs will disappear#2016-08-2318:01magnetophon@bfabry @minimal thanks. It's just that I recently read the clojure spec page, and wanted to try out the promise of usefull error messages.#2016-08-2318:02bfabrytry (s/conform ::test-type {:a 2, 31 3})#2016-08-2318:03bfabrythe thing is spec is never coming into play when you're trying to detect duplicate keys in a map. the map data structure just doesn't support that, and the clojure reader also doesn't support that. so it gets chucked out way before spec ever checks it#2016-08-2318:04magnetophonI was hoping to get a nice sorry, the keys are not unique
message from spec. oh well... š#2016-08-2318:04bfabryI'm kind of surprised the reader didn't give you that error#2016-08-2318:04bfabrybecause that's what I get#2016-08-2318:05bfabryuser=> {1 1 1 1}
user=> clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
java.lang.IllegalArgumentException: Duplicate key: 1
#2016-08-2318:07magnetophon@bfabry yeah I get that too, but not when I do it in spec#2016-08-2318:07bfabryok but you're not doing it "in spec"#2016-08-2318:07bfabrythe error happens pre spec#2016-08-2318:08bfabryyou literally cannot create a map with duplicate keys, so there's no way for spec to look at a map that has duplicate keys and give you an error about it#2016-08-2318:09magnetophon@bfabry I mean I don't get a usefull error from the reader when the faulty map is inside a conform or explain function.#2016-08-2318:10minimalI get (s/conform ::test-type {42 1 42 2})
java.lang.IllegalArgumentException: Duplicate key: 42
clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 42
java.lang.RuntimeException: Unmatched delimiter: )
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )
#2016-08-2318:10bfabryyeah that's a bit odd, but I don't think it's related to spec#2016-08-2318:10bfabryuser=> (identity {1 1 1 1})
clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
java.lang.IllegalArgumentException: Duplicate key: 1
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )
java.lang.RuntimeException: Unmatched delimiter: )
user=>
#2016-08-2318:11minimalNo itās not spec#2016-08-2318:11bfabryit's just because you've blown up inside the reader, so there's unconsumed input#2016-08-2318:11minimalOnce the reader fails, all bets are off and it gets more errors#2016-08-2318:12magnetophon@minimal when I run that line, I get the )
error.#2016-08-2318:12bfabrysomething about your environment is eating the first error, I guess#2016-08-2318:12minimalYeah#2016-08-2318:13minimalItās a distracting error but the duplicate key error is the useful one. So it's confusing if itās missing from your output#2016-08-2318:14magnetophon@bfabry @minimal ah, I was looking in the wrong place. both errors are there. sorry for the noise.#2016-08-2318:15bfabrynp#2016-08-2318:15minimalno worries.#2016-08-2318:19magnetophonWhat is meant by By default map-of will validate but not conform keys because conformed keys might create key duplicates that would cause entries in the map to be overridden. If conformed keys are desired, pass the option `:conform-keys true'.
in the spec guide?#2016-08-2318:21magnetophonafaik conform means to add metadata like the keys of a map. What do they mean by conforming keys? And why would that create duplicates?#2016-08-2318:28bfabryconforming means modifying the data to fit the spec#2016-08-2318:28bfabryboot.user=> (s/conform (s/map-of int? (s/or :k keyword? :i int?)) {1 :foo 2 3})
{1 [:k :foo], 2 [:i 3]}
#2016-08-2318:28bfabryso you see there the values were conformed#2016-08-2318:28bfabryto tell you whether the value conformed to the :k branch or the :i branch of the spec#2016-08-2318:30bfabrybut you can use custom conformers, so hence the statement above#2016-08-2318:41Alex Miller (Clojure team)by default the keys in the conformed value of the map will be identical#2016-08-2318:42Alex Miller (Clojure team)but you can use :conform-keys true
in the call to map-of
to allow that to happen#2016-08-2318:44Alex Miller (Clojure team)user=> (s/conform (s/map-of (s/or :i int? :s string?) int?) {5 5, "a" 10})
{5 5, "a" 10}
user=> (s/conform (s/map-of (s/or :i int? :s string?) int? :conform-keys true) {5 5, "a" 10})
{[:i 5] 5, [:s "a"] 10}
#2016-08-2318:45Alex Miller (Clojure team)and the reason not to do that automatically is that when you are altering the keys of the map, you might end up producing the same key and effectively erasing some of your transformed input, so we err on the side of safety#2016-08-2318:56magnetophonthanks. so it only makes sense if you have a branch in your key spec, and you want to know which branch was taken?#2016-08-2318:56magnetophonthat is not clear (to me) from the guide.#2016-08-2321:56Alex Miller (Clojure team)It only matters if your key spec conforms to something other than the original value#2016-08-2321:57Alex Miller (Clojure team)Given the common map keys - strings, longs, keywords, symbols - those are all likely to conform the same and it's a nonissue#2016-08-2321:57Alex Miller (Clojure team)So in most cases you can ignore this entirely#2016-08-2406:42tgkIām sure my googling skills are just sub-par but I canāt seem to find an obvious way of running (clojure.spec.test/test)
from lein
e.g. as part CI integration#2016-08-2406:42tgkMaybe everyone has switched to boot
while I wasnāt looking š#2016-08-2407:01tgkThe approach Iām taking now is to have a spec-test
namespace loaded in dev
environments and run -main
using lein run -n spec-test/-main
(or an alias) and have -main
run clojure.spec.test/check
and analyse the result. I guess this might be the obvious way of doing this.#2016-08-2407:54magnetophonIs it expected behavior that s/unform
doesn't descend into vectors? IOW: (s/def ::test-type (s/coll-of (s/cat :test-val int? )))
(s/conform ::test-type (s/unform ::test-type (s/conform ::test-type (gen/generate (s/gen ::test-type )))))
gives an error because the outer conform gets passed :test-val
plus the actual value.#2016-08-2408:08magnetophon@alexmiller I just noticed your response. Thanks, all clear now.#2016-08-2512:11nicolaHi all, i think this question already was answered, but - how to restrict collection of keys in map, i.e. no extra keys?#2016-08-2512:18tgk@nicola: You could write a predicate to cover that case and do (s/and (s/keys ā¦) pred)
#2016-08-2512:19tgkI assume there isnāt a standard way of doing it to be liberal in what you should be able to receive as data#2016-08-2512:19tgk(seems to be the Clojure philosophy)#2016-08-2512:32nicolaI've already read this thread - https://clojurians-log.clojureverse.org/clojure-spec/2016-06-09.html š We want to use clojure.spec instead of json schema and we have many cases of miss-typed keys, so validating this would be helpful. :closed true
would be nice, but let's start from custom pred.#2016-08-2512:39nicolaThen another question: s/and report errors of first failed predicate, could i compose two predicates - so both be applied, even if first failed?#2016-08-2512:40minimalThere is a excl-keys in https://github.com/gfredericks/schpec#2016-08-2513:07nicola@minimal: same problem with excl-keys, it uses internally s/and š#2016-08-2513:08nicolaI want report both - extra-keys and ok-keys validation problems#2016-08-2513:09minimalYouāll probably have to come up with your own custom version#2016-08-2513:10nicolaDo you mean own version of s/keys? š#2016-08-2513:21minimal@nicola I guess so if it doesnāt report problems how you want#2016-08-2513:22minimalYou could use the clojure.set
namespace to give more interesting reporting#2016-08-2513:23minimal(defn keys-diff [expected provided]
(let [expected (set expected)
provided (set provided)
missing (set/difference expected provided)
extra (set/difference provided expected)]
{:missing-keys missing
:extra-keys extra}))
#2016-08-2513:25nicolathx @minimal , i've got it. It's not problem to write extra-keys predicate, but force validation and reporting both extra-keys and s/keys#2016-08-2513:26nicolai.e. error-greedy s/and#2016-08-2513:26minimalyeah could be useful#2016-08-2513:51xcthulhuAt one point someone here was trying to convert spec
s to BNF grammars - what came of that?#2016-08-2513:54xcthulhuThat could be incredibly valuable - it would be very useful if you could compile a spec
into a .proto
file, or even C++ code for a JNI interface.#2016-08-2514:01potetmHas anyone had experience setting up clojure.spec.test/check
to run automatically (e.g. via lein test
)?#2016-08-2514:03potetmI know you can wrap it in a (deftest foo (testing ...))
, but the output of an is
failure isn't as verbose as the output of check
.#2016-08-2514:35bbrinck@potetm I use something like this (might need to be updated for latest alpha): https://gist.github.com/alexanderkiel/931387c7a86c1879b2267ad064067af7#2016-08-2514:38potetm@bbrinck Thanks! That'll do!#2016-08-2517:51sattvikAre there any examples of non-trivial uses of instrument
, esp. showing how :spec
, :stub
, :gen
, and :replace
are used?#2016-08-2519:03Alex Miller (Clojure team)@sattvik have you read http://clojure.org/guides/spec ?#2016-08-2519:05sattvikYes, I have, and it has been really useful. Itās just weāre trying to figure out how to use instrument and check together for some functions that may not be specced or only partially specced.#2016-08-2519:07sattvikThe videos on ClojureTV have also been useful, but a more thorough example that shows how to use some of the arguments to instrument to deal with things that might otherwise be considered part of a ātest fixtureā.#2016-08-2519:09Alex Miller (Clojure team)http://clojure.org/guides/spec#_combining_code_check_code_and_code_instrument_code has one example and I will write a blog on it eventually (but prob not for a while)#2016-08-2519:09Alex Miller (Clojure team)I think Stu might have a screencast in mind for it, but not sure#2016-08-2519:10sparkofreasonI've been getting this exception a lot lately when running check
: "Additional data must be non-nil." Any thoughts on why?#2016-08-2519:11Alex Miller (Clojure team)that shouldnāt ever happen :)#2016-08-2519:12Alex Miller (Clojure team)it basically means that conform failed, but explain didnāt find any problems so ex-info was created with a nil map#2016-08-2519:13Alex Miller (Clojure team)unfortunately this exception happens while throwing the useful exception so itās hard to diagnose#2016-08-2519:13Alex Miller (Clojure team)but itās definitely a bug whatever it is#2016-08-2519:13Alex Miller (Clojure team)a bug in Clojure that is#2016-08-2519:13Alex Miller (Clojure team)and if you can figure out what it is, Iād love a jira on it#2016-08-2519:28Alex Miller (Clojure team)itās not too hard to hack Clojure itself to dump more info when this happens if you have the interest in doing so#2016-08-2519:31Alex Miller (Clojure team)FYI, Iām doing a half day spec workshop at the Conj this year - tickets available now https://ti.to/cognitect/clojure-conj-2016#2016-08-2520:19sparkofreasonIf I can come up with a consistent minimal repo, I'll file a ticket.#2016-08-2520:22Alex Miller (Clojure team)thx#2016-08-2520:41sparkofreasonI'm going to venture a guess, though, based on some recent debugging, which is that it seems to occur when I have a function that returns another spec'ed function, and the returned function throws an exception while spec is checking it. Just a guess, will try to see if that stick consistently when I have a little time.#2016-08-2520:59Alex Miller (Clojure team)hmm, well I guess thatās possible too (although Iād put that one on you :)#2016-08-2521:08sparkofreasonApache library š#2016-08-2521:20Alex Miller (Clojure team)ok, Iāll put it on Apache :)#2016-08-2521:36shaun-mahood@alexmiller: Any expectation of previous spec knowledge for your Conj workshop, or can we come in blind?#2016-08-2521:36Alex Miller (Clojure team)nope#2016-08-2521:36Alex Miller (Clojure team)I will expect basic Clojure knowledge :)#2016-08-2521:38shaun-mahoodPerfect, I should be able to handle that š#2016-08-2521:39Alex Miller (Clojure team)for sure#2016-08-2521:54sebastianpoeplauThis might be a stupid question, but can anyone give me a hint how to spec the following function:
(defn my-apply [f & args]
(apply f args))
(This is a simplified version of what I actually have.)
My problem is in expressing that (count args)
has to be equal to the number of parameters that f
takes...#2016-08-2600:38gfredericksThat sounds hard.#2016-08-2602:11Alex Miller (Clojure team)the hard part is that you donāt know how many params f takes#2016-08-2602:12Alex Miller (Clojure team)I donāt know that itās profitable to spec such a thing at that level#2016-08-2605:53rymndhngis there a way to take a map schema of un-namespaced keys, and promote them to namespace qualified ones? -- I'm imagining something like:
(s/def ::id int?)
(s/def ::config (s/keys :req-un [::id]))
(s/some-conform-like-function ::config {:id 10})
;; -> {::id 10}
#2016-08-2608:42Alex Miller (Clojure team)this is kind of an aside and I assume you did this really for readability, but youāll never see ::id printed - autoresolved kws are only read, never printed
but you could certainly use a conformer to do this#2016-08-2608:43Alex Miller (Clojure team)with conform#2016-08-2608:47Alex Miller (Clojure team)(defn qualify-keys [m]
(zipmap (map #(keyword (str *ns*) (name %)) (keys m)) (vals m)))
(s/conform (s/and ::config (s/conformer qualify-keys)) {:id 10})
;;=> #:user{:id 10}
#2016-08-2614:27vikeriThis has probably already been asked, but can I somehow pprint the error messages thrown by (spec.test/instument)
? They are pretty much impossible to read for what Iām doing right now.#2016-08-2617:54Alex Miller (Clojure team)Not yet#2016-08-2619:39sveriHi, seems I missed a few updates and instrument-all
is not in clojure.spec
anymore. What is the idiomatic approach to enable the specs during tests with the latest alpha release?#2016-08-2619:42Alex Miller (Clojure team)(clojure.spec.test/instrument)
#2016-08-2619:45sveri@alexmiller Ah, there it is and without arguments it instruments the ns, right?#2016-08-2619:45Alex Miller (Clojure team)it instruments everything in the registry#2016-08-2619:45Alex Miller (Clojure team)when you pass it no args#2016-08-2619:45sveriEven better, thank you very much#2016-08-2619:45Alex Miller (Clojure team)same thing for unstrument and check#2016-08-2620:32fenton(cross-posted from clojurescript...but no answer there): do others find it difficult to spec functions that use core-async for backend comms? This seems where stubbing would be very helpful, that is to stub out backend service calls. However since core-async based functions only return channels, not request responses...how do other people tackle this issue?#2016-08-2620:42dnolen@fenton thereās no story for that yet#2016-08-2620:43fenton@dnolen ok...#2016-08-2620:45fentonok, i'll nut it over with some clojure devs here at our Vancouver, Canada, meetup...see what we come up with.#2016-08-2621:26fentonmaybe this came up before, but wondering if this is being thought about. There is some overlap in clojure.spec'ing and datomic schema spec'ing. Could these be unified?#2016-08-2621:53Alex Miller (Clojure team)eventually I expect that spec will be more intimately involved in both core.async and Datomic#2016-08-2621:54Alex Miller (Clojure team)but thatās a ways off#2016-08-2621:58fenton@alexmiller cool. glad to hear that is the direction tho! š#2016-08-2622:38mjhamrickIs there a better way to do the following? I'm figuring the registry-ref is private for a reason, but I couldn't come up with another way to temporarily override the value of a spec. My usecase is that during runtime, it is okay for a spec to be nilable, but not during test time. Looking for any suggestions for this. I did try writing the specs separately, but that ended up with (quite a bit) of duplication since the spec I need to redef is nested a few levels down.
(defmacro with-spec-redefs
[& forms]
`(let [previous-registry# (s/registry)]
(let [result# (do
#2016-08-2715:58sattvik@fenton I believe exercise
returns tuples of [val conformed-val]
.#2016-08-2716:02sattvik@mjhamrick I think the way to do that is via instrument
. When you instrument
, you can pass something like (stest/instrument
some-ns/some-fn {:spec {`some-ns/some-fn (s/fspec ā¦)}})`#2016-08-2814:30lvhIām trying to use regex ops and running into some problems. I have two data structures naturally represented by a regex, the main one is of the shape: a*c
, in my map data structure in order to get the item at that location, the get-in
path is something like (ab)*c
ā it seems that the problem is that nested regex operators āmergeā; so now for a fn that takes a*c
and returns (ab)*c
, I canāt specify its :args
as (s/cat :a ::a)
ā does that make sense?#2016-08-2814:32lvhLooks like thatās intentional; found it in the guide š
> When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context. For example to describe a sequence like [:names ["a" "b"] :nums [1 2 3]], you need nested regular expressions to describe the inner sequential data:#2016-08-2814:50Alex Miller (Clojure team)Yes#2016-08-2814:50Alex Miller (Clojure team)That's intended#2016-08-2814:51lvhMakes sense ā youād want to be able to parse both, and break them out š#2016-08-2816:36sveriHi, how do I spec functions with optional arguments?#2016-08-2820:34lvhsveri: usually you use regex ops to describe the args as they would be given to apply#2016-08-2820:34lvhso .e.g you can use spec/?
#2016-08-2822:07potetmHow would you spec 2 arguments where one arg's validity depends on the other arg.#2016-08-2822:08potetmFor example: (fn [i-am-a-map i-am-a-key-in-the-map] ...)
#2016-08-2822:31gfrederickspotetm: you can use s/and
#2016-08-2822:32potetm@gfredericks perfect; just figured that out š#2016-08-2901:03potetmnext question: is there an ad hoc way to supply a :gen
for function :args
to stest/check
?#2016-08-2901:05potetmI see you can s/def
the args, then supply a gen. I'm not opposed to the notion that you must name something to use it elsewhere (even if that elsewhere is a test), but I thought there might be some sugar out there.#2016-08-2901:06sattvikI think you can do it with instrument
.#2016-08-2909:16sveri@lvh Thanks, I just took ? It works as expected š#2016-08-2913:17lvhIs there any chance of a with-instrument macro, and/or a clojure.test + clojure.spec.test.check deftest macro a la defspec with test.check? (Too many similarly named words; I feel like I screwed that up)#2016-08-2913:34sattvikWe just started really looking into clojure.spec last week, and while we are still figuring out how to best leverage it, I definitely agree that a clojure.test integration and a with-instrument macro sound like useful additions.#2016-08-2914:58sveriHm, I am not sure, but, so far I would want to have everything instrumented during my tests#2016-08-2914:58sveriThats the first thing I do in my tests, turn on instrumentation#2016-08-2915:08sattvikYes, in general, you want to have everything instrumented. However, if you are using stubs or overriding specs, you may only want to do that within the scope of a few checks.#2016-08-2916:30xcthulhuHey! I'm trying to cryptographically sign data structures in Clojure/ClojureScript. The approach I am taking is to serialize my structures into a canonical form - namely recursively sorting the keywords of every hash-map in the structure, prior to serializing with transit.
(1) Is there already something that does this for me?#2016-08-2916:30xcthulhu(2) Maybe this isn't the best practice, is there a better way?#2016-08-2916:31xcthulhu(3) If there isn't a better way, how do you enforce that a data structure is canonical in spec
?#2016-08-2916:36lvh@sveri +1 on also wanting instrumentation with stubs/overrides as sattvik mentioned#2016-08-2916:36lvh@xcthulhu Hello I am a cryptographer#2016-08-2916:36lvhSolving a similar problem, turns out#2016-08-2916:37lvhFortunately I donāt particularly care about canonicalization or deterministic signatures#2016-08-2916:37lvhIām personally using nippy, but I donāt think that has any particular canonical form guarantees#2016-08-2916:38lvh@xcthulhu How do you keep the hash-map structure in the serialized form whilst being sorted?#2016-08-2916:39xcthulhuSerialized form comes after the thing is canonical. Yeah, I've gone to pretty extreme lengths to keep all of my signatures deterministic.#2016-08-2916:40xcthulhuI'm doing ECC in ClojureScript and I don't believe I can get a reasonable secure RNG without resorting to RFC 6979#2016-08-2916:40lvhSure; when you say canonicalizing a data structure the first things that come to mind are ASN.1 C(X)ER#2016-08-2916:40xcthulhuYeah... I've implemented a fragment.#2016-08-2916:40xcthulhuI could go whole-hog. I was thinking of just using transit#2016-08-2916:41lvhFortunately RFC 6979 + ed25519 is how youād want to do it anyway regardless of how much you care about that determinism š#2016-08-2916:41lvhHow do you ship the ClojureScript?#2016-08-2916:42xcthulhuRight now, I'm just publishing a JAR - https://github.com/Sepia-Officinalis/secp256k1#2016-08-2916:42lvh(The devilās advocate argument against JS crypto usually goes by how you ship the source to begin with)#2016-08-2916:42lvhOh; OK ā why SECP256k1?#2016-08-2916:42xcthulhuBecause I started this when I was doing consulting for digital currency and it's what everyone uses because of BitCoin#2016-08-2916:42lvhEither way this is not a #spec discussion anymore so maybe we should move to #crypto š#2016-08-2916:43xcthulhuI should switch it to ed25519#2016-08-2919:57lfn3@fenton A thing Iāve done previously with schema is wrap a channel with a validation to check anything put into the channel conforms to the schema. This causes exceptions to be thrown where youāre putting the unexpected value onto the channel which is really useful. Sure you could do the same thing with spec if you wanted to.#2016-08-2919:58lfn3Is there a way to attach a doc-string to a spec?#2016-08-2919:58fenton@ifn3 that sounds like a great use of specs#2016-08-2920:19Alex Miller (Clojure team)@lfn3 re doc string, not currently but feel free to vote on http://dev.clojure.org/jira/browse/CLJ-1965#2016-08-2920:20Alex Miller (Clojure team)@lvh re with-instrument, if you wanted to log a jira, that would help us remember this idea (seems quite reasonable)#2016-08-2920:21lvhwill do š#2016-08-2920:50lfn3@fenton https://gist.github.com/lfn3/0aaf4c420df206484372e9b7bbbe6c6f <- demo of using specs to validate things put on channels.#2016-08-2922:01agif I have a vector of maps:
(def accounts
[{:name "Foo"} {:name "Bar"} {:name "Baz"}])
How do I define a spec that confirms to be one of the names, e.g.:
(s/def ::account-name ,,,)
: so when I do:
(gen/generate (s/gen ::account-name))
it would generate something like :account-name āBazā
#2016-08-2922:03bfabry@ag (s/def ::account-name (set (map :name accounts)))
#2016-08-2922:04agalright thanks!#2016-08-2922:09emrehanHello all. I'm stuck at defining a map with undefined keys with clojure.spec.
For instance, I'm defining a map with keyword keys and string values. I can define it as below with Schema library:
(require '[schema.core :as schema])
(def Data {schema/Keyword schema/Str})
(schema/validate Data {:x "str"})
> {:x "str"}
(schema/validate Data {:x 0})
> ExceptionInfo Value does not match schema: {:x (not (matches-some-precondition? 0))} schema.core/validator/fn--8583 (core.clj:155)
How can I define such a map with clojure.spec?#2016-08-2922:22bfabry@emrehan map-of
https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/map-of (s/def ::a-map (s/map-of keyword? string?))
#2016-08-2922:24emrehanthanks so much @bfabry š I must have been missed this map-of
function.#2016-08-2922:26bfabrynp#2016-08-2922:26agwhat if I have a collection and want a spec that satisfies for one of the items in the collection?
e.g.
(def accounts
[{:name "Foo" :type :foo}
{:name "Bar" :type :bar}
{:name "Baz" :type :baz}])
(s/def ::foo-or-bar-or-baz #_(any of those))
#2016-08-2922:28bfabry@ag not sure what you're saying, could you give a (s/valid ::account-name ...) => true example? also is accounts
known at spec definition time?#2016-08-2922:45agso when I do (gen/generate (s/gen ::foo-or-bar-or-baz))
it would give me one of the maps of accounts
vector#2016-08-2922:47bfabryassuming accounts
is known at spec definition time, and accounts
is reasonably small, then (s/def ::foo-or-bar-or-baz (set accounts))#2016-08-2922:59emrehanIs there a nice way to combine to clojure.spec definitions? I have a definition that must conform two defined schemas and I defined it as below:
(s/def ::spec3 (s/and #(s/valid? ::spec1 %) #(s/valid? ::spec2 %)))
Is there a better way to define this?#2016-08-2923:00bfabrys/and
accepts specs#2016-08-2923:01bfabryso (s/and ::spec1 ::spec2)
is fine#2016-08-2923:02bfabrythat's actually not that clear from the docstring is it? it just says "spec-forms"#2016-08-2923:09emrehanI didnāt look at the spec but I tried that form. It didnāt work, maybe Iām missing something.#2016-08-2923:10bfabryif ::spec1 conforms then the conformed value will be passed to ::spec2 instead of the original value#2016-08-2923:11bfabrymaybe that's your issue?#2016-08-2923:13bfabryif it is you could (s/and any? ::spec1 ::spec2) I suppose#2016-08-2923:15emrehanthanks for the tip. this behaviour is not obvious though. let me try that#2016-08-2923:17bfabryI think s/and conforming with the first spec has tripped a few people up so the docstrings are probably being worked I imagine. iirc the guide does cover it though#2016-08-2923:19agis there equivalent of clojure.test.check.generators/let
? in clojure.spec
?#2016-08-2923:25emrehanHereās a simplified example#2016-08-2923:25emrehan(s/def ::type (s/or :string string? :number number?))
(s/def ::a-map (s/map-of keyword? ::type))
(s/def ::id int?)
(s/def ::with-id (s/keys :req-un [::id]))
(s/def ::metadata (s/and #(s/valid? ::a-map %) #(s/valid? ::with-id %)))
(s/valid? ::metadata {:id 1 :n 1 :s "dsa"})
> true
(s/def ::metadata (s/and ::a-map ::with-id))
(s/valid? ::metadata {:id 1 :n 1 :s "dsa"})
> false
(s/def ::metadata (s/and any? ::a-map ::with-id))
(s/valid? ::metadata {:id 1 :n 1 :s "dsa"})
> false
#2016-08-2923:28emrehanIt seems to me that s/and passes the return value of (s/conform ::a-map {:id 1 :n 1 :s "dsa"})
to the second conform. However, the when s/or
is conformed the return value is different from the input.#2016-08-2923:31bfabryyes, conforming means changing the value to explain how it is true given the spec, ie if the spec has a branch#2016-08-2923:33bfabryI think maybe s/and has changed so it successively conforms the values now#2016-08-2923:34bfabryyes, it does#2016-08-2923:34bfabryboot.user=> (s/conform any? {:id 1 :n 1 :s "dsa"})
{:id 1, :n 1, :s "dsa"}
boot.user=> (s/conform (s/and any? ::a-map) {:id 1 :n 1 :s "dsa"})
{:id [:number 1], :n [:number 1], :s [:string "dsa"]}
boot.user=> (s/conform (s/and any? ::a-map ::with-id) {:id 1 :n 1 :s "dsa"})
:clojure.spec/invalid
#2016-08-2923:37bfabryanyway, you have options. if you want to preserve the conforming behaviour, I would just re-order your predicates so that ::with-id comes first. if you want to do away with it then creating a new predicate with #(s/valid? ::a-map %) just around the spec that you don't want to conform anymore seems sensible#2016-08-2923:38bfabryOH#2016-08-2923:38bfabryor you could use the new merge
function#2016-08-2923:38bfabrywhich is specifically for map specs#2016-08-2923:38bfabryboot.user=> (s/valid? (s/merge ::a-map ::with-id) {:id 1 :n 1 :s "dsa"})
true
boot.user=> (s/conform (s/merge ::a-map ::with-id) {:id 1 :n 1 :s "dsa"})
{:id 1, :n 1, :s "dsa"}#2016-08-2923:39bfabrytotally forgot about that#2016-08-2923:40bfabrymerge
appears to conform using the last spec. is that right @alexmiller ?#2016-08-2923:40bfabryboot.user=> (s/conform (s/merge ::with-id ::a-map) {:id 1 :n 1 :s "dsa"})
{:id [:number 1], :n [:number 1], :s [:string "dsa"]}
#2016-08-2923:41Alex Miller (Clojure team)Yes#2016-08-2923:42Alex Miller (Clojure team)Merge does not flow conformed values like and, it's more like union#2016-08-2923:43agis it possible to have āinlinedā spec in keys
I donāt think itās working for me. (s/def ::foo (s/keys :req [(s/def ::bar string?)]))
#2016-08-2923:43ag(g/generate (s/gen ::foo))
#2016-08-2923:44bfabryI think something named def
should always be at the top level#2016-08-2923:44bfabryand in that particular case I don't think it will work because keys
is a macro#2016-08-2923:45bfabryso the keys
macro is passed the form '[(s/def ::bar string?)] instead of [::bar]#2016-08-2923:46Alex Miller (Clojure team)@bfabry s/and has always flowed conformed values. We may change it to not flow and add s/and-> still as both are sometimes useful#2016-08-2923:47bfabryah, cheers, not sure where I ever got the idea that only the first did#2016-08-2923:52Alex Miller (Clojure team)@ag s/keys always requires registered keys - that's part of the design that is talked about at http://clojure.org/about/spec#2016-08-2923:52agokā¦ thanks..#2016-08-2923:56aghow do I create a spec for uuid? this doesnāt seem to work:
(s/def ::id g/uuid)
#2016-08-2923:57agI donāt want to use java.util.UUID/randomUUID
I may have to make it later work on cljs side too#2016-08-2923:59bfabry@ag there's a uuid?
predicate in core#2016-08-2923:59agoh, okā¦ thanks!#2016-08-3000:13agoh no.. I donāt think I know how to solve it with s/keys
ā¦ the problem where I need to have a spec for a map where values picked up from a vector and associated values are together#2016-08-3000:16agso this is how I am generating a map with test.check.generators
:
(g/let [acnt (g/elements ledger-accounts)]
(g/hash-map
:id (uuid-gen)
:account-name (-> acnt :account-name g/return)
:account-type (-> acnt :account-type g/return)
ā¦
how can I do the same thing with spec?#2016-08-3000:17aga map where related values pulled out of a vector?#2016-08-3000:21bfabry@ag are you using this spec for validation as well as generation?#2016-08-3000:21agright#2016-08-3000:21agI am planning#2016-08-3000:21agI donāt have a spec yet š#2016-08-3000:22agI am playing with clojure.spec, and I feel utterly stupid ;(#2016-08-3000:24agshould I be separating spec and generation parts?#2016-08-3000:24bfabrywell it depends#2016-08-3000:24bfabrybut it seems unlikely that you'll know all of the values of something like :account-name before runtime#2016-08-3000:26bfabryhave you read the guide?#2016-08-3000:26bfabryalso, I reckon the clojure spec screencasts by stuart holloway here https://www.youtube.com/channel/UCaLlzGqiPE2QRj6sSOawJRg might help. particularly the last one on customising generators#2016-08-3000:28agthatās the thing, account-names are predefined. and types are predefined. I need to have a map with bunch of keys along with name and type where name comes from that vector and type is the associated type (from the same vector)#2016-08-3000:35bfabryok, honestly it's strange enough that there's probably not a way to express it and still get generators so you'll probably need to specify the generator like you did above, and do the spec portion separate#2016-08-3000:39bfabryI'd probably write the spec as
(s/def ::account-name (set (map :account-name leger-accounts)))
(s/def ::account-type (set (map :account-type ledger-accounts)))
(s/def ::id uuid?)
(s/def ::your-map (s/and (s/keys :req-un [::account-name ::account-type ::id] :gen YOUR-GENERATOR) #((set (select-keys ledger-accounts :account-name :account-type)) (select-keys % :account-name :account-type))))
#2016-08-3000:49Alex Miller (Clojure team)@ag it sounds like you are expressing what have been called hybrid maps#2016-08-3000:49Alex Miller (Clojure team)Which you can spec with a merge of keys and every of tuples of key-value pairs#2016-08-3000:49Alex Miller (Clojure team)I've given some examples here in the past#2016-08-3000:51agYeah Iām watching Stuās screencast ācustomizing generatorsā, looking into gen/bind
, gen/tuple
etc.#2016-08-3001:25agso why for simple predicates generator wonāt work:
(s/def ::more-than-five #(< 5 %))
(s/conform ::more-than-five 6)
(s/exercise ::more-than-five)
clojure.lang.ExceptionInfo: Unable to construct gen at: [] for: :user/more-than-five
#2016-08-3001:26gfredericks@ag that would be a hard thing to support in general#2016-08-3001:28gfredericksif you do (s/and integer? #(< 5 %))
it will at least make an attempt#2016-08-3001:28gfredericksbecause clojure.spec has a registry of generators for some of the built-in predicates#2016-08-3001:29agoh, okā¦ thanks!#2016-08-3001:30gfredericksnp#2016-08-3001:47agonce again, how do I create s/def
with a custom generator? I think I saw it somewhere?#2016-08-3002:03sattvikI beleive it is (s/def ::foo (s/with-gen spec gen))
#2016-08-3002:22ag@sattvik Thanks!#2016-08-3007:24agerhmā¦ how do I refer to a spec s/def
fed in other namespace? I am trying to put spec in .cljc file, is that possible at all?#2016-08-3007:45agI think I found itā¦ so if spec is in foo.clj then require [foo :as f] ā¦ (s/exercise ::f/my-spec
#2016-08-3007:46agstill canāt figure out how to refer to spec (and use it without having to prefix it with namespace)#2016-08-3010:25emrehanthanks all; clojure.spec/merge
is exactly what I was looking for. I even read the whole API but I was quite sleepy.#2016-08-3010:26emrehanHow can I contribure to clojure.spec? docs/dev/testing/benchmarking?#2016-08-3012:16jannisHi. Using the latest ClojureScript version, I'd like to get all checkable syms (with cljs.spec.test/checkable-syms
, filter them and call clojure.spec.test/check
on each of them individually). However, if I do something like (doseq [sym syms] (st/check sym))
it tells me sym
is unresolvable. This works fine in Clojure but in ClojureScript, am I supposed to pass in something else than symbols?#2016-08-3015:57spinningtopsofdoomI'm looking to make a spec that validates nested maps (e.g.` {:value {:foo :bar :key :value}}`).#2016-08-3015:57spinningtopsofdoomSo far I'm only able to get one level of nesting via spec/keys
#2016-08-3015:59sattvikYou can use two different specs: one for the inner map and then use it in the outer map.#2016-08-3016:00sattvikThere are other options, but I think thatās one of the easiest.#2016-08-3016:00sattvik(s/def ::value :req-un [::foo ::key])
(s/def ::outer :req-un [::value])
#2016-08-3016:03spinningtopsofdoomI also have the need to have different specs for value (e.g. http://clojure.org/guides/spec#_multi_spec) . For example {:value {:foo :bar}}
and {:value {:key :value}}
#2016-08-3016:05sattvikWell, in that case, I think you could do with more specs:#2016-08-3016:06sattvik(s/def ::foo-value (s/keys :req-un [::foo]))
(s/def ::key-value (s/keys :req-un [::key]))
(s/def ::value (s/or :foo-value ::foo-value :key-value ::key-value))
(s/def ::outer (s/keys :req-un [::value]))
#2016-08-3016:07sattvikjava.jdbc has such an example: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#L27-L42#2016-08-3016:08sattvikHowever, not all of those specs are fully specified, e.g. ::user
.#2016-08-3017:12agso how come this
(s/def ::my-map (s/cat :id string?))
(gen/generate (s/gen ::my-map))
#2016-08-3017:12aggenerates: ("pn7zcHi1WDk830ZltM5L5Yl82E0Ulā)
#2016-08-3017:12agand not a map {:id āsomenonsenseā}
?#2016-08-3017:14djwhitts/cat specifies a sequence. ":id" in that code is specifying a label rather than a map key#2016-08-3017:15agso what should I use instead, if I donāt want to use s/keys?#2016-08-3017:17bfabryyou should use s/keys#2016-08-3017:18djwhitthmm... I'm not sure (just learning spec myself). any reason you can't use s/keys?#2016-08-3017:19agbecause it doesnāt allow āinline specsā e.g. s/keys (s/def :id string?)
#2016-08-3017:20djwhittI think spec is intentionally trying to steer people away from inline specs like that#2016-08-3017:21djwhittyou could make a separate spec for the key and use req-un to allow unnamespaced keys if that's what you're looking for#2016-08-3017:21bfabry@ag giving every spec a fully qualified name is one of the core design goals of spec, things are going to get difficult if you try not to follow it#2016-08-3017:22agmkay#2016-08-3017:34agguys, I need a good spec for values of milliseconds after the Unix epoch. i.e.:`clj-time.coerce/to-long`#2016-08-3017:34agcan you help?#2016-08-3017:49bfabry@ag (int-in? 0 Long/MAX_VALUE)
?#2016-08-3017:53agmmm thatās too simple, I need to make sure values are of reasonable datetime range, something like from 1900 to 2100 maybe#2016-08-3017:58bfabrywell unix epochs don't go down to 1900#2016-08-3017:58bfabryso that's out#2016-08-3017:58bfabry(int-in? 0 (c/to-long #inst "2100-01-01"))
#2016-08-3018:16Alex Miller (Clojure team)should really use int-in
if youāre defining a spec (not just the predicate int-in?
), as then you will get generator support#2016-08-3018:18Alex Miller (Clojure team)and there is a new fn for extracting the ms - inst-ms
in core#2016-08-3018:20Alex Miller (Clojure team)so I would say (s/int-in 0 (inst-ms #inst "2100-01-01"))
#2016-08-3018:23Alex Miller (Clojure team)assuming youāre cool with the year 2100 problem#2016-08-3018:24Alex Miller (Clojure team)inst-ms
calls into the Inst protocol, which is extended to both Date and (if you use JDK 8), Instant. Could also be extended to a JodaTime Instant.#2016-08-3018:27agI donāt understand why this is failing:
(s/def ::account-name string?)
(s/def ::account-type keyword?)
(s/def ::description string?)
(s/valid? (s/keys :req [::account-name ::description ::account-type])
{:account-name "pre-fund" :account-type :internal :description "Pre-Fundā})
#2016-08-3018:28sattvik:req
requires namespace-qualified keys.#2016-08-3018:28Alex Miller (Clojure team)it has them ^^#2016-08-3018:28sattvikYou may want to use :req-un
instead.#2016-08-3018:28Alex Miller (Clojure team):: will autoresolve and fully qualify#2016-08-3018:28Alex Miller (Clojure team)oh you mean in the data#2016-08-3018:28Alex Miller (Clojure team)yes#2016-08-3018:28Alex Miller (Clojure team)sorry! :)#2016-08-3018:29agoh, okā¦ thanks a lot!#2016-08-3018:35agSorry for bugging you with bunch of noob questions. Trying to solve real problem with spec. If I donāt get this right sooner, would be asked to stop my experiments.#2016-08-3018:36Alex Miller (Clojure team)noob questions are good#2016-08-3018:36Alex Miller (Clojure team)I asked Rich a lot of noob questions at the beginning too :)#2016-08-3018:37agis there a way to pass original value (I dunno of spec) to custom-generator function, when itās created with s/with-gen
?#2016-08-3018:38agsomething like (gen/generate (s/gen (s/with-gen ::account-balance-updated #(gen-account-balance-updated %))))
#2016-08-3018:39agI guess I can refer to ::account-balance-updated
from inside the gen-account-balance-updated
, yet thinking if thereās more āgenericā way#2016-08-3018:41agehā¦ I guess I can just pass it as is#2016-08-3018:41agnevermind#2016-08-3018:41sattvikWell, you can do something like use gen/fmap
or gen/bind
.#2016-08-3018:43sattvikThough, I am not sure that really answers your questionā¦#2016-08-3018:47Alex Miller (Clojure team)@ag in short, no#2016-08-3018:48Alex Miller (Clojure team)but you can define both the spec and the spec-with-custom-gen to do so#2016-08-3018:48agso I have a spec for a map structure, I need to use custom generator function where I would generate all the fields of that map, except of few selected, those should come from a predefined variable#2016-08-3018:49Alex Miller (Clojure team)itās easiest to use gen/fmap
and source it with (s/gen (s/keys ::a ::b))
, then in the function just merge with the constant map#2016-08-3018:50Alex Miller (Clojure team)where ::a and ::b are the variable parts#2016-08-3018:52eraserhdWe are having a weird problem: https://gist.github.com/eraserhd/0aca4172b34c3c64f2e17f8c9107174d#2016-08-3018:54Alex Miller (Clojure team)@ag (s/def ::a int?)
(s/def ::b int?)
(s/def ::c string?)
(s/def ::m (s/keys :req [::a ::b ::c]))
(s/def ::m (s/with-gen (s/keys :req [::a ::b ::c]) (fn [] (gen/fmap #(merge {::c "xyz"} %) (s/gen (s/keys :req [::a ::b]))))))
(gen/sample (s/gen ::m))
;; (#:user{:c "xyz", :a -1, :b 0} #:user{:c "xyz", :a 0, :b -1} #:user{:c "xyz", :a -1, :b 0} #:user{:c "xyz", :a -1, :b -1} #:user{:c "xyz", :a 0, :b 0} #:user{:c "xyz", :a -10, :b 1} #:user{:c "xyz", :a 3, :b -2} #:user{:c "xyz", :a -1, :b 40} #:user{:c "xyz", :a 45, :b -2} #:user{:c "xyz", :a -2, :b -7})
#2016-08-3018:56Alex Miller (Clojure team)@eraserhd whatās the point of that gen/tuple?#2016-08-3018:56sattvikCould it be that the tuple is adding an extra layer of being a collection, i.e. instead of returning [task]
it is returning [[task]]
?#2016-08-3018:56Alex Miller (Clojure team)seems like you just need vector
in that case#2016-08-3018:58Alex Miller (Clojure team)or you might just want to fmap with vector inside the fn#2016-08-3018:58eraserhd@alexmiller Er, we found out that spec conforms the generator's results to the spec.#2016-08-3018:59eraserhdWe had something with multiple values in the tuple, but deleted code until we found out where there was a problem.#2016-08-3018:59eraserhd(except that we didn't know that part)#2016-08-3019:02Alex Miller (Clojure team)yes, spec does not trust the generator#2016-08-3019:03Alex Miller (Clojure team)even custom gens must produce values valid for the spec#2016-08-3019:23sattvikHmmmā¦ so whatās the best way to create fixtures (for lack of a better term) with spec? This could range anywhere from setting up an environment within which to run the tests to testing that the output from the function under test matches a value generated from a replaced/stubbed function.
check
seems really useful for relatively pure functions, but having to orchestrate a lot via instrument
seems clunky. Does anyone have any recommendations?#2016-08-3019:32Alex Miller (Clojure team)if you want fixtures and assertions, use a testing framework which has them ?#2016-08-3019:32Alex Miller (Clojure team)that is, wrap a clojure.test around instrument+check#2016-08-3019:32Alex Miller (Clojure team)instrument could go in a fixture too if it applies broadly#2016-08-3019:36sattvikYes, Iāve done a bit of that. I have gotten it to work, but I was a bit unsatisfied with the result. I felt that the end result was a bit clunky and harder to understand/maintain compared to just using a traditional deftest.#2016-08-3019:37sattvikI will continue to experiment with it, but I was curious to see if anyone else had any experience with non-trivial testing using spec.#2016-08-3019:38Alex Miller (Clojure team)what was unsatisfying?#2016-08-3019:49sattvikWell, I am still a bit of a spec beginner (have been using it for less than a week), so there are things I may be missing. Some things that could help include:
1. More documentation around instrument
, especially with examples of using the various options.
2. It would be nice to be able to call instrument
with just options rather than (instrument (instrumentable-syms) opts)
.
3. The lack of a with-instrumentation
macro makes setting up āfixturesā harder. It was a simple one to write, but it would be nice to have it baked in.
4. Overall, specifying things through instrument
doesnāt read as well. When this sort of thing is written out long-hand in code, it is easier to understand how the flow works. Instrumentation is sort of like defining callbacks before you need to use them, which removes some of the context from them.
4. The last one it is a little bit harder to describe. Instrumenting is very good about modifying things that will happen during the execution of the function, but sometimes I want to do something around the execution of the function. I can do that by specifying a test function and running spec on that, but it would be nice if there was a more direct way of doing that with spec.#2016-08-3019:58Alex Miller (Clojure team)1. for sure :) more will be coming eventually.
2. did you find clojure.spec.test/enumerate-namespace? it helps with building sym lists.
3. there is a ticket for that that I was just looking at. there are questions - feel free to weigh in. http://dev.clojure.org/jira/browse/CLJ-2015
4. I can see what you mean, prob worth seeing more examples
4. thatās interesting. sounds like a check fixture? I assume you meant ārunning checkā, not ārunning specā above#2016-08-3019:59gfrederickswoah CLJ tickets have exactly caught up with the calendar year#2016-08-3020:00Alex Miller (Clojure team)now if we can just slow down to 1 per year, that will always be true#2016-08-3020:01gfrederickssounds doable#2016-08-3020:01Alex Miller (Clojure team)thx#2016-08-3020:08sattvikRegarding (4), you are right. I set up a method to handle (is (checks?
foo/bar {ā¦}))` where the last two arguments are just used to invoke stest/check
. I first started by putting my fixture in a separately defined test function that invoked the the function to test. It works, but adds a bit of noise (also there is not a way to automatically inherit the function spec of the original). In a case where the fixture is transparent, i.e. takes the same args/returns the same result, I was able to write a macro that too the var nameās symbol and a higher-order function that served as the test fixture. That macro dereferenced the var giving the old value a local binding, and with-redef
ed the var to invoke the fixture (which took the old binding as an argument and returned a new function that behaved like the old one plus the fixture logic).#2016-08-3020:08sattvikI had to do with-redef
because I couldnāt :replace
the function under test.#2016-08-3020:14sattvikThat technique works when the fixture is transparent, but perhaps I want to create a fixture that has a different signature than the function I am testing. For example, I might be testing that a function will eventually invoke a stubbed function that will return generated data. I want to check that the return value of the function under test satisfies a predicate that is partly dependent on the data that comes from the stubbed function, not just the inputs.#2016-08-3020:15sattvikThis is all doable right now by creating such a function and properly specifying the test function and instrumenting the third-party function.#2016-08-3020:15sattvikThis is all doable right now by creating such a function and properly specifying the test function and instrumenting the stubbed function.#2016-08-3020:16sattvikI suppose it might make more sense with an exampleā¦#2016-08-3022:08aghow can I create generate based on predefined vector? letās say I have a vector [{:name āAnnaā}{:name āDavid}] ..etc
, I need a generator that creates vector with the same amount of elements, with added fields letās say :age
and :height
?#2016-08-3022:15agI guess Iāll just wrap things into gen/return
and for
#2016-08-3022:30gfredericks@ag so you want a generator that completes the maps?#2016-08-3022:31gfredericksYou could use gen/vector with a fixed length and gen/hash-map#2016-08-3022:40agI need to create a bunch of accounts with some predefined fields and some randomly generated fields, and need to create bunch of other structures associated#2016-08-3022:44gfredericks(gen/fmap #(map merge % predefined) (gen/vector (gen/hash-map ...) (count predefined)))
@ag would something like ā that work?#2016-08-3022:50aglemme tryā¦ thanks!#2016-08-3100:12agerrrā¦ so I have spec like this:
#?(:clj (defn uuid-str-gen [] (gen/return (str (java.util.UUID/randomUUID))))
:cljs (defn uuid-str-gen [] (gen/return (str (random-uuid)))))
(s/def ::uuid (s/with-gen (s/and
string?
#(re-matches uuid-regex %)
uuid-str?)
uuid-str-gen))
for uuid vals. A bit bulky, I couldnāt find anything better. my problem now, if another spec includes it it always generates same uuids. How do I fix that?#2016-08-3100:13ag(gen/sample (s/gen ::uuid))
("06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2ā)
`#2016-08-3100:15agoopsieā¦ thereās gen/uuid
I guess I should be using that#2016-08-3100:15agI need only the sting though#2016-08-3100:22agso why this is not working:
(s/exercise (s/with-gen string? (gen/fmap str (gen/uuid))))
#2016-08-3100:28agnvmdā¦ got it (s/exercise (s/with-gen string? #(gen/fmap str (gen/uuid))))
#2016-08-3100:31gfredericks:)#2016-08-3112:00borkdudeWas there s/instrument-all
, which got removed later on?#2016-08-3112:01borkdudeI don't see it in my REPL, but I've seen it in a talk: https://www.dropbox.com/s/w1ktte022j5xy4o/Screenshot%202016-08-31%2014.00.08.png?dl=0#2016-08-3112:03minimal@borkdude just call instrument with no args#2016-08-3112:04borkdudeaah ok#2016-08-3112:04minimalit got changed at one point#2016-08-3112:33sveriAnybody experimented with using spec for form / input validation?#2016-08-3112:36martinklepsch@sveri I did#2016-08-3112:36martinklepsch@sveri very rough but maybe it helps https://gist.github.com/martinklepsch/e1366008c5a478b33c00d324314da4fd#2016-08-3112:43sveri@martinklepsch thanks, so basically its: 1. validate input with spec 2. try to parse result and display some useful text#2016-08-3112:44martinklepschright. Do all of that on a form level and do some basic dirty?
tracking#2016-08-3112:45sveriOk, thank you very much, I will try some stuff š#2016-08-3112:58Alex Miller (Clojure team)Rich checked in a round of perf improvements to master yesterday which fixed some hot spots#2016-08-3114:22mschmeleWhen I try to run check
on a function that Iāve specād with fdef
, it fails with the cause āUnable to construct gen at: [] for : clojure.spec$ā¦"#2016-08-3114:22mschmeleam I missing something obvious?#2016-08-3114:25mschmelehttps://github.com/mschmele/CSPOC#2016-08-3114:26mschmeleCode is up there for anybody who wants to read through it#2016-08-3114:29minimal@mschmele you should (generally) use the predicates on their own instead of wrapping in an anonymous function. Otherwise you won.t get generators for the functions that have them.#2016-08-3114:30minimale.g. instead of (s/def ::id #(string? %))
use (s/def ::id string?)
#2016-08-3114:31mschmeleOkay, I just saw an example that was formatted that way and it seemed to work. Changing that now#2016-08-3114:31minimalalso you should just use the specs inside fdef, no need for assert or valid which will probably give strange results#2016-08-3114:32mschmeleso something like#2016-08-3114:32minimaland you need to use s/cat for arg lists#2016-08-3114:32mschmele:args [::accounts ::id]
#2016-08-3114:33minimalno#2016-08-3114:33minimalmore like :args (s/cat :acc ::accounts :id ::id)
#2016-08-3114:34mschmeleah, thank you#2016-08-3114:35minimalhttp://clojure.org/guides/spec#_sequences#2016-08-3114:43mschmele@minimal thanks!#2016-08-3114:46minimalnp#2016-08-3121:48agis there a concise way to get a keyword?
s/def
but the one that always generates short keywords?
because this looks awful;
(s/def ::keyword (s/with-gen keyword? (fn [] (gen/fmap #(keyword %)
(gen/such-that #(and (> 10 (count %))
(< 0 (count %)))
(gen/string-alphanumeric) 1000)))))
#2016-08-3122:24sattvikCould you do as part of the spec itself?#2016-08-3122:24sattvikMaybe (s/and keyword? #(< (count (name %)) 10)
?#2016-08-3122:24sattvik@ag ^#2016-08-3122:28bfabry@ag
(s/def ::short-keyword (s/with-gen keyword? (fn [] (gen/fmap #(keyword %)
(gen/such-that #(and (> 10 (count %))
(< 0 (count %)))
(gen/string-alphanumeric) 1000)))))
(s/def ::keyword ::short-keyword)
(s/def ::keyword2 ::short-keyword)
#2016-08-3122:30bfabry@sattvik doing it that way makes the spec validate that the keyword is short, which you may not want, and also makes for slow generators#2016-08-3122:40sattvikWell, if the idea is to accept any keyword but only generate short ones during testing, then supplying the generator at check time is probably what you want. test.check has things like c.t.c.generators/resize
that can fix the generator to a manageable size, e.g. 10.#2016-08-3122:40bfabrythis is true#2016-08-3122:40sattvikUnfortunately, thatās not one of the functions clojure.spec mirrors.#2016-09-0100:26sattvikCurious. When working with higher-order functions, instrumentation will result in the execution of a function argument repeatedly. Which, isnāt great when the function has side effects.#2016-09-0100:29gfredericksyou probably need to stub things?#2016-09-0100:32sattvikEhā¦ itās kind of a weird situation. I have a function that performs some assertions for testing. It is invoked by the function under test. Under normal circumstances, my function with assertions is only invoked once per test. However, as the function under test has its parameters specified, the conformance checker ends up running my test function scores of times.#2016-09-0100:34sattvikI think one workaround is to instrument the function under test with less demanding specification (something like fn?
). In the end, I just use a mutable cell to hold the data I want to check and have my test code update that cell. I can count on the last invocation of the function to be the one I care about.#2016-09-0100:35gfredericksyou also get to override specs#2016-09-0100:35gfredericksso maybe you could test it in two ways#2016-09-0100:35gfredericksonce where you have the function doing the assertion and you make sure it gets called only once, and another test without that where the spec gets exercised#2016-09-0100:36sattvikWell, doesnāt overriding specs only be done when using check
? I am not doing that. I am only instrumenting for input argument validation.#2016-09-0100:39gfredericksthat might be true#2016-09-0101:40seancorfieldIs there any sort of standard pattern for specāing a function that curries its arguments? (defn my-fn
([a] (fn [b] (my-fn a b)))
([a b] ā¦ a ā¦ b ā¦))
#2016-09-0101:41seancorfieldSpecifically, how to handle the fact that :ret
depends on whether you provided one or two arguments...#2016-09-0101:45sattvikHmmā¦ good question. My guess is that you would use alt for the different return types and then an fn to tie them together to the passed args. Iām not sure if there is a better way, though.#2016-09-0102:17seancorfield@gfredericks all of a sudden, Iām getting this exception from test.check ā any pointers? clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)#2016-09-0102:19seancorfield@sattvik yeahā¦ that feels kinda ugly thoāā¦ Iām hoping thereās a cleaner way...#2016-09-0102:23sattvikHmmā¦ I wonder if it is possible to do (s/or :unary (s/fspec ā¦) :binary (s/fspec ā¦))
#2016-09-0102:28gfredericks@seancorfield that smells like this one thing hold on#2016-09-0102:29gfredericks@seancorfield http://dev.clojure.org/jira/browse/TCHECK-113#2016-09-0102:30seancorfieldPerfect! Thanksā¦ Iāll add that Leiningen setting!#2016-09-0102:31seancorfield(Iām expanding the clojure.spec
coverage for java.jdbc
and ran into that for the first time)#2016-09-0102:32gfredericksthis is all my fault for not having a new test.check release ready yet#2016-09-0104:49seancorfieldDon't sweat it. test.check is already great!#2016-09-0104:52seancorfieldMaking progress with spec'ing java.jdbc. Some good, some frustrating simple_smile #2016-09-0107:26sveriHi, I have trouble specing & args. What I want is (f [a1 a2 & ax]) where ax can be zero or more (s/*) and it must have an even number of elements.
This is what I thought should work: (s/* (s/and any? #(even? (count %))))
But it does not, instead it works for (f 1 2 3) but not for (f 1 2 3 4)#2016-09-0108:00sveriIs it because spec takes the args vector apart and checks every element of the vector?#2016-09-0112:43Alex Miller (Clojure team)The s/and there is in the spot for each element of the arg vector so that doesn't make sense#2016-09-0112:45Alex Miller (Clojure team)(s/& (s/* any?) #(even (count? %)))#2016-09-0112:45Alex Miller (Clojure team)Is one solution - s/& applies a predicate in addition to a regex match#2016-09-0112:46Alex Miller (Clojure team)Or you could make the pairs explicit in the regex if that makes sense#2016-09-0112:47Alex Miller (Clojure team)(s/* (s/cat :x1 any? :x2 any?))#2016-09-0112:48Alex Miller (Clojure team)Or actually since you require at least 2 args that should be + not *#2016-09-0112:49Alex Miller (Clojure team)That's probably preferred, esp if you can give better names to x1 and x2#2016-09-0112:49sveriZero / 2 / 4 / ...args are fine#2016-09-0112:50sveriI just wonder, if I look at: (s/ (s/cat :x1 any? :x2 any?)) I would assume it should have two arguments, not more, not less. But, I think its the s/ around it, that makes it work for more inputs, true?#2016-09-0112:53Alex Miller (Clojure team)Yes - it's just like regex#2016-09-0112:53Alex Miller (Clojure team)It's a repetition of pairs#2016-09-0112:54Alex Miller (Clojure team)@sattvik: you can override specs in instrument to use a simpler spec#2016-09-0112:55Alex Miller (Clojure team)@seancorfield re currying, you can use a :fn spec if you need to make an assertion about the relationship between :args and :ret#2016-09-0113:01sveri@alexmiller Thank you very much š#2016-09-0113:30mschmeleIs it generally considered bad practice to allow functions to have nilable return values?#2016-09-0113:30mschmeleIt makes having functions that pass spec tests quite a bit easier, but I feel like it kinda defeats the purpose to some extent#2016-09-0113:49sattvik@mschmele Itās perfectly reasonable, if nil means something like ānot foundā or āemptyā.#2016-09-0113:50sattvik@alexmiller Thatās true. Itās just that I was a little surprised (but perhaps I shouldnāt have been) that conforming a function argument involved repeated invocations of the function.#2016-09-0113:51mschmeleThatās what Iāve been using it for in simpler functions like get-from-id
for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer
which would take two accounts that canāt be nil#2016-09-0113:52mschmeleDespite having validation in the function, Iām having trouble getting it to pass spec check
#2016-09-0116:09seancorfield@alexmiller Yeah, I know I can deal with currying via :fn
but that still makes for a fairly complex spec ā Has there been any discussion of making it easier to spec multi-arity functions?#2016-09-0116:19Alex Miller (Clojure team)Not in regards to this#2016-09-0116:20Alex Miller (Clojure team)My experience has been that it is comparatively rare for the ret spec to rely on the args arity#2016-09-0116:21Alex Miller (Clojure team)And when it does :fn is the way to talk about that constraint#2016-09-0116:25seancorfieldWell, with a curried function, the :ret
will either be a function (of the remaining argument(s)) or a result...#2016-09-0116:26seancorfieldSo (defn foo
([a] (fn [b] (foo a b)))
([a b] (* a (inc b))))
=> :ret
would be int?
in the two arg case but some fspec
in the one arg case#2016-09-0116:27seancorfieldIf fspec
/ fdef
supported multi-arity, this could be neater...#2016-09-0116:30Alex Miller (Clojure team)Well I'd say curried functions are not idiomatic in Clojure :)#2016-09-0116:30seancorfieldš#2016-09-0116:31Alex Miller (Clojure team)But the main place I've run into this is with the core seq functions with transducer arity#2016-09-0116:31seancorfieldWhat about multi-arity functions in general?#2016-09-0116:32Alex Miller (Clojure team)Usually most arities call into a canonical arity and have the same ret spec#2016-09-0116:32seancorfieldTheyāre still pretty messy to spec out, even if :ret
is fixed.#2016-09-0116:32Alex Miller (Clojure team)I have not found multiple arities at all difficult to spec#2016-09-0116:32Alex Miller (Clojure team)They're often quite easy to talk about in regex #2016-09-0116:34seancorfieldAs long as the arities all extend the base case in order, yesā¦ but thereās quite a bit of real world code out there which doesnāt follow that model (or is that also non-idiomatic?).#2016-09-0116:36Alex Miller (Clojure team)Even if not in same order, regex can easily describe with ?#2016-09-0116:37Alex Miller (Clojure team)I'm not discounting what you're saying, but I have not found that to be an issue in my own experience#2016-09-0116:38Alex Miller (Clojure team)In general, I am more commonly surprised at how well regex specs work for args#2016-09-0116:41robert-stuttafordyou guys are having a ball, aren't you. can't wait to get stuck in myself!#2016-09-0116:41mschmele@sattvik copied from above, because I forgot to @ you earlier š
Thatās what Iāve been using it for in simpler functions like get-from-id
for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer
which would take two accounts that canāt be nil
Despite having validation in the function, Iām having trouble getting it to pass spec check
#2016-09-0116:43seancorfield@alexmiller Fair enough. I held off specāing some of the multi-arity java.jdbc
stuff at first since it didnāt seem "easy" (although it may yet prove to be "simple"). Iāll take another run at it soon (next week probably) and report back.#2016-09-0116:46seancorfield@alexmiller My last Q for the morning (I promise!): is there an idiomatic way to spec something that is treated as truthy / falsey, when it might not be strictly true
or false
. I tried #{true false nil}
before realizing wonāt work facepalm and (s/nilable #{true false})
"works" but seems a bit ā¦ I guess that treating an any?
argument as a pseudo-boolean is probably a bit sketchy but ...#2016-09-0117:03Alex Miller (Clojure team)I donāt think (s/nilable #{true false})
works either for false (for same reason as the set)#2016-09-0117:04Alex Miller (Clojure team)so you need some kind of fn to do it so pick your favorite function that matches those 3 values :)#2016-09-0117:05Alex Miller (Clojure team)I guess (s/nilable boolean?)
works#2016-09-0117:06Alex Miller (Clojure team)Iāll go with that :)#2016-09-0117:07donaldball(s/def ::boolish (s/or :truthy (complement #{false nil}) :falsey #{false nil}))
?#2016-09-0117:09Alex Miller (Clojure team)no set with falsey values in it is going to be useful :)#2016-09-0117:10Alex Miller (Clojure team)s/nilable has been made significantly better performing this week from Richās commits in master btw#2016-09-0117:12seancorfieldAh, yes, of course false
wonāt work either. facepalm again š#2016-09-0117:17seancorfield(it hadnāt failed in testing yet butā¦)#2016-09-0117:19seancorfieldThe specific case is for ::as-arrays?
in java.jdbc
where itās intended to be true / false or :cols-as-is
but in reality nil
is acceptable and common when passing defaulted options around (and, of course, it really accepts any?
and just treats it as a boolean).#2016-09-0117:20sattvikFor the multi-arity stuff, the following doesnāt work, but it might be nice if it did:
(defn foo
([a] (fn [b] (foo a b)))
([a b] (* a (inc b))))
(s/def foo
(s/or :unary (s/fspec :args (s/cat :arg int?)
:ret (s/fspec :args (s/cat :arg int?)
:ret int?))
:binary (s/fspec :args (s/cat :arg1 int?
:arg2 int?)
:ret int?)))
#2016-09-0117:26seancorfield(s/fdef foo
(:args (s/cat :a int?)
:ret (s/fspec :args (s/cat :b int?)
:ret int?))
(:args (s/cat :a int? :b int?)
:ret int?))
That was along the lines of what I was thinking...#2016-09-0117:26seancorfieldWhich matches defn
for multi-arity functions.#2016-09-0117:56Alex Miller (Clojure team)bleh#2016-09-0117:56Alex Miller (Clojure team):)#2016-09-0118:34Alex Miller (Clojure team)(s/fdef foo
:args (s/cat :a int? :b (s/? int?))
:ret (s/or :val int? :fun (s/fspec :args (s/cat :arg int?) :ret int?))
:fn (fn [m]
(= (-> m :ret key)
(if (-> m :args :b) :val :fun))))
#2016-09-0118:37Alex Miller (Clojure team)I think Rich would say about :ret here that it should simply state the truth - it can either be a number or a function#2016-09-0118:38Alex Miller (Clojure team)and :fn can add an arg-dependent constraint#2016-09-0118:39Alex Miller (Clojure team)I think itās unlikely we would extend to either of the two suggestions above#2016-09-0118:40Alex Miller (Clojure team)this kind of dependent constraint is the whole reason to have :fn#2016-09-0118:41Alex Miller (Clojure team)if you felt the need, I think you could create a macro that automatically created a spec for a curried fn#2016-09-0118:52seancorfieldHmm, yeah, that might well be worth doingā¦ Iāll have a think about that...#2016-09-0118:54seancorfieldA variant of fdef
for which you give :args
, :ret
, and :fn
-- and then also an indication of which curried variants you needā¦ and then it automatically generates the fdef
spec from that...#2016-09-0118:54Alex Miller (Clojure team)I guess youād need to specify the shape of every curried result#2016-09-0118:55seancorfieldThat would be the hard part since you canāt use s/?
for all those args, only for the last one.#2016-09-0118:56Alex Miller (Clojure team)you can stack em up#2016-09-0118:57seancorfield(s/? (s/cat ā¦))
#2016-09-0118:58Alex Miller (Clojure team)yeah, itās gross looking - youād want to generate it out of an s/cat or an s/cat with noted optional parts or something#2016-09-0118:58Alex Miller (Clojure team)I donāt think curried fns are common enough to do any of this :) but if youāre looking for a puzzle to play with ā¦#2016-09-0118:59seancorfieldI tend to curry functions quite a bit ā to avoid partial
all over the place ā but it is almost always currying just a two arg function.#2016-09-0119:02Alex Miller (Clojure team)well maybe thatās a simplifier#2016-09-0119:02Alex Miller (Clojure team)the shape I had above could be made generic#2016-09-0119:06seancorfieldI may just remove the currying in java.jdbc
since Iād be shocked if anyone actually leverages it in client code (given your comment about it being non-idiomatic). java.jdbc
doesnāt use the curried form of as-sql-name
internally and I donāt know why anyone would externally. And I wouldnāt expect the non-curried form of quoted
to be used by anyone either (the one-argument form is actually the common, useful arity).#2016-09-0119:06seancorfieldI hadnāt thought hard about either of those until I sat down to try to spec them out.#2016-09-0119:10seancorfieldclojure.spec
definitely makes you question your design choices š#2016-09-0119:32Alex Miller (Clojure team)indeed#2016-09-0119:33patrkrisAny ideas on how to organize a project's specs? For instance, if I have a domain model for my application that describes a customer entity, would I then create the spec in a com.example.customer
namespace and use ::name
for defining a spec for the customer's name? Or would I benefit from centralizing specs in a dedicated namespace, and in that spell out the fully namespaced keyword, i.e. com.example.customer/name
?#2016-09-0119:35Alex Miller (Clojure team)Iād say both are fine :)#2016-09-0119:38patrkrisYeah. That's what I thought you'd say. š But then I am imagining scenarios where a customer means different things in different contexts. It may be one thing in the domain model and another thing in a HTTP request handler. So it might be okay to have :com.example.customer/name
and :com.example.resources.customer/name
? Does that look wrong?#2016-09-0119:46Alex Miller (Clojure team)you can alias specs (s/def :com.example.customer/name :com.example.resources.customer/name)
#2016-09-0119:46Alex Miller (Clojure team)so you can have both#2016-09-0119:47Alex Miller (Clojure team)has to be done explicitly of course so ymmv#2016-09-0119:47Alex Miller (Clojure team)weāve talked about a version of s/keys that would separate the map keys from the specs rather than requiring them to be the same keyword, not sure if that will pan out#2016-09-0120:31manderson+1 to separating map keys from specs ^^^#2016-09-0120:55Alex Miller (Clojure team)the spec part would still be a qualified keyword (not inline specs), just to be clear#2016-09-0120:56Alex Miller (Clojure team)this was considered as an alternative to :req-un too#2016-09-0121:05seancorfieldIt took me a while to internalize that :req-un could have :my.foo/bar as a spec for the key :bar independent of the namespace you define the keys spec in. #2016-09-0121:07seancorfield(Because I didn't read the docs closely enough apparently)#2016-09-0121:08seancorfieldSo I'm not sure why you'd want to separate the specs from the keys at this point?#2016-09-0121:08seancorfield(Or am I still misunderstanding the issue?)#2016-09-0201:33seancorfield@gfredericks does the maven clojure plugin do the same monkey-patching of clojure.test that lein does? http://build.clojure.org/job/java.jdbc-test-matrix/453/CLOJURE_VERSION=1.9.0-alpha11,jdk=Sun%20JDK%201.6/console <ā seems to be the same exception as I got testing locally#2016-09-0201:55gfredericks@seancorfield um#2016-09-0201:56seancorfieldNot sure how to tell Mavenās plugin not to do the Leiningen naughtiness š#2016-09-0201:56gfrederickscom.theoryinpractise.clojure.testrunner
is your culprit#2016-09-0201:56gfredericksI don't know about maven clojure, but it smells like something similar#2016-09-0201:57gfredericksthis seems a lot harder to work around given the constraints#2016-09-0201:57seancorfieldhttps://github.com/talios/clojure-maven-plugin/blob/develop/src/main/resources/default_test_script.clj#L55#2016-09-0201:57seancorfieldIt rebinds the report
function#2016-09-0201:58gfredericksdo you get to choose an alternate version of the clojure-maven-plugin if you want?#2016-09-0201:59seancorfieldI could override it for my project I think, yes (this is for java.jdbc
)#2016-09-0201:59seancorfieldIs there a version that is compatible with test.checkās clojure_test stuff?#2016-09-0201:59gfredericksalternately, any tactic you can think of that lets you require test.check.clojure-test
prior to that line running will fix it#2016-09-0201:59gfredericksno I was just imagining forking it :)#2016-09-0201:59seancorfieldHahahaā¦ ok...#2016-09-0201:59gfredericksso a user.clj
could work if you can keep it out of the release jar#2016-09-0202:00seancorfieldWould that run with Maven?#2016-09-0202:00gfredericksit runs when clojure boots up#2016-09-0202:00gfredericksuser.clj
is a pretty reliably way to slip something in before just about anything else happens#2016-09-0202:01seancorfieldPretty sure that doesnāt work with Boot? (more an FYI butā¦)#2016-09-0202:01gfrederickswhy not?#2016-09-0202:03seancorfieldI donāt rememberā¦ but it was discussed in #boot a while back...#2016-09-0202:04gfredericksprobably something about their magical space age classloader thing#2016-09-0202:05seancorfieldAnd how do you get it to be loaded for Maven running Clojure?#2016-09-0202:05gfredericks@seancorfield heck for that matter adding a (:require clojure.test.check.clojure-test)
to any of your namespaces should also work#2016-09-0202:06gfredericksputting the user.clj
on the classpath means that clojure.core reads it at then end of its loading#2016-09-0202:06gfrederickse.g. src/test/resources/user.clj
or whatever#2016-09-0202:09seancorfieldAh, yeah, that worked...#2016-09-0202:15talios@seancorfield you can define/give your own test runner script to clojure-maven-plugin if you need to#2016-09-0202:16seancorfieldDynamically require
ing that namespace when my test namespace loads seems to do the trick ā and I can remove the Leiningen monkey-patch setting as well.#2016-09-0202:16talios<configuration>
<testScript>src/test/clojure/com/jobsheet/test.clj</testScript>
</configuration>
#2016-09-0202:16taliosin the pom#2016-09-0202:17seancorfieldI have this now in my test ns: (def with-spec? (try
(require 'clojure.java.jdbc.spec)
(require 'clojure.spec.test)
;; require this to workaround rebinding of report multi-fn
(require 'clojure.test.check.clojure-test)
(let [syms ((resolve 'clojure.spec.test/enumerate-namespace) 'clojure.java.jdbc)]
((resolve 'clojure.spec.test/instrument) syms))
(println "Instrumenting clojure.java.jdbc with clojure.spec")
true
(catch Exception _
false)))
#2016-09-0202:17seancorfieldWorks with Leiningen and Maven!#2016-09-0202:18taliosSweet - PRs welcome to improve that default test runner script as well.#2016-09-0202:18taliosif needed#2016-09-0215:08kurt-yagramI have a spec that looks like this:
(s/def ::value <???>)
(s/def ::type #{"type-date", "type-weirdo", "no-value"})
(s/def ::ent (s/keys :req-un [::type] :opt-un [::value]))
(s/def ::ents (s/coll-of ::ent))
Depending on the type, the value needs to be different, e.g., for type-date
, the value should be a date. I know there are multimethods that can, and probably should, be used, but I just can't get it right. What comes at the <???>
, so I can have different predicates for value depending on the value of type?#2016-09-0215:12Alex Miller (Clojure team)have you looked at s/multi-spec
?#2016-09-0215:12Alex Miller (Clojure team)there is an example in the guide http://clojure.org/guides/spec#2016-09-0215:19spinningtopsofdoomI ran into this problem two days ago you want to make N :value
specs. (e..g.`(s/def :my/value <???>)`, (s/def :other/value <???>)
, (s/def :one.more/value <???>)
and then dispatch those :value
's with a multimethod#2016-09-0215:20kurt-yagramyeah, I'm looking at it. but it seems to be somewhat different. So, I could do:
(defmulti ent-type ::type)
(defmethod ent-type "type-date" [_]
(s/keys :req-un [::value])
(s/def ent-type (s/multi-spec ent-type ::type))
So I day value is required for "type-date". But that still doesn't solve the problem. I must be missing something...#2016-09-0215:21kurt-yagram@spinningtopsofdoom Allright, and I don't need to care about the namespaces? - I mean, the map contains just 'value', no namespaces. I'll give it a few tries.#2016-09-0215:21Alex Miller (Clojure team)instead of using ::value, you could define many :foo1/value :foo2/value :foo3/value specs#2016-09-0215:22kurt-yagramš#2016-09-0215:22Alex Miller (Clojure team)each defmethod would use a different one in :req-un#2016-09-0215:22kurt-yagramoh, allright#2016-09-0215:25spinningtopsofdoom@alexmiller is there any movement on having spec/keys
take a map of keywords and specs (e.g.)
(spec/def :one-map (spec/keys :req-un {:value <one spec>})
(spec/def :other-map (spec/keys :req-un {:value <other spec>})
So that you don't have to have registered specs for spec/keys
#2016-09-0215:25Alex Miller (Clojure team)nothing yet#2016-09-0215:26Alex Miller (Clojure team)well, it will never take inline specs#2016-09-0215:27Alex Miller (Clojure team)we might possibly loosen the constraint between key name and spec name, but itās part of the design that s/keys
doesnāt use inline specs and I donāt expect that to change#2016-09-0215:27Alex Miller (Clojure team)the idea is to encourage defining semantics for attributes#2016-09-0215:29spinningtopsofdoomThe constraint between key name and spec name is what I would like to loosen. So then my example would be
(spec/def :one-map (spec/keys :req-un {:value ::one-spec})
(spec/def :other-map (spec/keys :req-un {:value ::other-spec})
Correct?#2016-09-0215:30Alex Miller (Clojure team)yeah, something like that has been mentioned, but I do not know whether weāll end up doing it or not#2016-09-0215:32spinningtopsofdoomWell I'll make a Jira ticket for that if it's not already there, then. Thanks for the clarification.#2016-09-0215:35Alex Miller (Clojure team)I do not know of a jira ticket for this#2016-09-0215:40gerstreeHi everybody, very new to specs, but have already a good part implemented. Now I need a little help on something that is probably very easy, but I'm stuck. I have a def that verifies a url to be a s3 url (s/def ::s3-url #(str/starts-with? % "s3")
. Using it to check a single argument works. Now I have a function with 2 arguments (from-url and to-url) of which 1 needs to conform to that spec. Can anyone hint me for the solution.#2016-09-0215:44gerstreeThis is what I have:
(s/def ::valid-url is-valid-url?)
(s/def ::s3-url #(str/starts-with? % "s3"))
(defn sync
"sync an s3 folder with a local folder, this works both ways"
[from-url to-url])
(s/fdef sync
:args (s/and (s/cat :from-url is-valid-url? :to-url is-valid-url?)
??? this is where I get lost ???))
#2016-09-0215:47Alex Miller (Clojure team)(s/fdef sync
:args (s/cat :from-url is-valid-url? :to-url is-valid-url?))
#2016-09-0215:48Alex Miller (Clojure team)I guess Iām also wondering what the difference is between is-valid-url?
, ::valid-url
and ::s3-url
#2016-09-0215:48Alex Miller (Clojure team)given that you have specs, I would actually use the specs in the sync fdef#2016-09-0215:49gerstreeGood one, I will have to clean that up#2016-09-0215:49Alex Miller (Clojure team)(s/fdef sync :args (s/cat :from-url ::s3-url :to-url ::s3-url))
#2016-09-0215:49Alex Miller (Clojure team)something like that#2016-09-0215:50gerstreeThe function will be called either (sync "" "")
or (sync "" "")
#2016-09-0215:51gerstreeWhat I am trying to spec is that either url is an s3 url#2016-09-0215:51Alex Miller (Clojure team)right so you could have something like:
(s/def ::file-url #(str/starts-with? % āfile://ā))
(s/def ::s3-url #(str/starts-with? % ās3://ā))
(s/def ::aws-url (s/or :file ::file-url :s3 ::s3-url))
ā¦ then use ::aws-url in the sync fdef
#2016-09-0215:52Alex Miller (Clojure team)or whatever is appropriate#2016-09-0215:52Alex Miller (Clojure team)oh, you want a constraint across the args!#2016-09-0215:52gerstreeYes, is that beyond what spec is meant for?#2016-09-0215:53Alex Miller (Clojure team)No, that's fine!#2016-09-0215:54Alex Miller (Clojure team)You can do it with s/and like you were or you can do that in the :fn spec too#2016-09-0215:54Alex Miller (Clojure team)fn is used for constraints between args and ret or also across args#2016-09-0215:55Alex Miller (Clojure team)Only the args spec is checked in instrumentation though so that might be what you want#2016-09-0215:55gerstreeAh, been reading about it all day and understood the args / ret, but of course that works for across args as well.#2016-09-0215:56Alex Miller (Clojure team)Yeah fn gets the conformed version of both args and ret#2016-09-0215:56gerstreeThat might be the easiest way to accomplish it.#2016-09-0216:16gerstreeAlso read about 10 times that stest/instrument only does :args, now I know that is true š#2016-09-0216:30gerstreeI have the :fn version working. I'm still curious about a solution in the :args part, where I got stuck to begin with. That way, if people use my sync function they can simply (stest/instrument `sync) and work from there.#2016-09-0216:39Alex Miller (Clojure team)(s/fdef :args (s/and (s/cat :from-url ::aws-url :to-url ::aws-url)
(fn [{:keys [from-url to-url]}]
(or (s3-url? from-url) (s3-url? to-url)))))
#2016-09-0216:39Alex Miller (Clojure team)something like that#2016-09-0216:40Alex Miller (Clojure team)the second function in the s/and receives the conformed version of the first part of the and#2016-09-0216:41Alex Miller (Clojure team)so that will be a map with :from-url and :to-url keys#2016-09-0216:43gerstreeWorks like a charm and is actually simple enough#2016-09-0216:43gerstreeLearned a lot today, thanks for helping me get through that last part!#2016-09-0216:49Alex Miller (Clojure team)np#2016-09-0217:44kurt-yagramis it possible to use `s/multi-spec' on different specs? Something like - but this doesn't work:
(defmulti m [::type ::op])
(defmethod m ["date-type" "do"] [_]
(s/keys ...))
...
(s/def .... (s/multi-spec m [::type ::op]))
#2016-09-0217:58Alex Miller (Clojure team)that defmulti definition looks wrong - it takes a dispatch function not a vector#2016-09-0218:07Alex Miller (Clojure team)(defmulti m #(vector (::type %) (::op %)))
maybe?#2016-09-0218:07Alex Miller (Clojure team)or itās a great opportunity to use juxt :)#2016-09-0218:08Alex Miller (Clojure team)(defmulti m (juxt ::type ::op))
#2016-09-0218:09kurt-yagramoh, cool... thx!#2016-09-0218:10Alex Miller (Clojure team)and then the retag value is not valid either - should either be a tag or a fn of generated value and dispatch-tag#2016-09-0222:34lvhhuh; are instrumentation exceptions supposed to escape clojure.test assertions?#2016-09-0222:35lvhoh, wait, never mind; only some of them are outside assertions š#2016-09-0300:44agcan anyone tell me ETA of 1.9 stable? I already have a piece of code that depends on clojure.spec, unfortunately my team did not approve my PR (bunch of alpha-phobics), that makes it a bit inconvenient for me personally.#2016-09-0300:45Alex Miller (Clojure team)No eta#2016-09-0300:47agnot even approximation? before Christmas, 2nd quarter next year etc.?#2016-09-0300:47Alex Miller (Clojure team)I'm sure we will try to hit some milestone before the conj but may just be beta (feature freeze)#2016-09-0300:48Alex Miller (Clojure team)We have a couple other things in flight that we haven't even talked about yet#2016-09-0300:49Alex Miller (Clojure team)We are not currently working towards any concrete deadline though#2016-09-0300:50agyeah, okā¦ I am so excited though. Spec turns out to be a massive feature of the platform. I really like it.#2016-09-0301:24Alex Miller (Clojure team)Thanks#2016-09-0311:47wagjo@ag there is a backport of specs for 1.8, https://github.com/tonsky/clojure-future-spec#2016-09-0319:10ag@wagjo yeah, I switched to backport. Worked as a charm.#2016-09-0322:34lvhAre you generally expected to add enough info to s/fdef such that running check on that fn works? Is that considered the ādefaultā state? Iām writing reasonably gnarly generators to make that work, because I have two args with related keys (and if you picked args at random, youād almost ceratinly end up with garbage)#2016-09-0322:47lvhHm. I wonder if itās OK for an fdefās :args to have a generator that uses the fn being fdefād (a lot of functions can clearly be run with a ābase caseā if you will; e.g. if you have something that conjs a bunch of stuff together, you might conj a bit ahead of time and assert that it doesnāt throw away previously conjād things)#2016-09-0322:47lvhI think so, but with the apparently suggested pattern of fdefing before the defn itself, you run into a compile error; so Iām wondering if I should just declare it and be done with it or what š#2016-09-0323:40lvhFYI, got a weird error when running clojure.spec.test/check + deftest through CIDER (and only CIDER): https://github.com/clojure-emacs/cider/issues/1841
Might be interesting for upstream to look into since just because CIDER ran into it (or rather; I only ran into it with CIDER) doesnāt mean itās necessarily CIDER-specific š#2016-09-0401:52hiredmanit likely is, cider is likely redefing the report function in clojure test with function that is not a multimethod#2016-09-0402:54Alex Miller (Clojure team)Yes this is the lein monkey patching#2016-09-0402:54Alex Miller (Clojure team)There are issues in both lein and test.check about it and it's fixed in test.check master#2016-09-0402:55Alex Miller (Clojure team)You can disable lein monkey patch to avoid#2016-09-0402:56Alex Miller (Clojure team)Well I guess it could be a cider variant of the same problem, not sure#2016-09-0403:14gfrederickssomebody filed a bug report a few months back, I think to test.chuck, wherein I convinced them that cider was monkeypatching the wrong way#2016-09-0403:16gfredericksno nevermind#2016-09-0403:16gfredericksit was probably fireplace too#2016-09-0417:53flyboarderHello, just a quick question with the new spec system. Where do I place my specs? In the same namespace as the things they specify are declared? And how do I go about making them compatible with older versions of clojure? I am imagining my lib being used in 1.7 and 1.8 as well.#2016-09-0418:03flyboarderAh I see there are back ports for spec#2016-09-0419:21gfredericksOther libs have put the specs in a separate ns so users on older versions can ignore them#2016-09-0420:42lvhmeh; stest/check now basically looks like it hangs forever and I donāt know which thing itās choking on#2016-09-0420:43lvhitād be nice if there was a debug mode or something where it told me what it was working on#2016-09-0420:47lvhItās definitely helpful in finding bugs though š#2016-09-0420:48lvhalso in making my laptop not very suitable for holding on a lap š#2016-09-0509:57ikitommi@alexmiller did you consider using Records instead of just doing reify
for the Spec protocol? For example and-spec-impl
could return clojure.spec.AndSpec
record? Would it have negative effect on performance? Records might be easier to understand and adding extensions would be easier. Trying to cover the JSON Schema generation for the core Specs. Extending records with a new protocol would be easy (the way we are doing the same with Schema).#2016-09-0510:02ikitommiPushed out a experimental version of the dynamic conformations (based on runtime data). Would like to get feedback on that, now using dynamic var as the conform
doesnāt take any optional parameters to do this. https://github.com/metosin/spec-tools#2016-09-0511:59mishagreetings, is there an equivalent to clojure.spec/def
, for defining many specs? like:
(s/def ::first-name string?)
(s/def ::last-name string?)
->>
(s/def-many
::first-name string?)
::last-name string?)
#2016-09-0512:08madstap(defmacro def-many [& key-spec-pairs]
(cons 'do (for [[k spec] (partition 2 key-spec-pairs)]
`(s/def ~k ~spec))))
#2016-09-0512:21misha@madstap š
is there any popular use case, of using return value of s/def
?#2016-09-0617:03nopromptare there any explanations of how to properly implement clojure.spec.Spec
?#2016-09-0617:13noprompti ask because i'm interested in integrating a small parser combinator library i wrote with clojure.spec
so i can hook into to the conform
, explain
, etc. goodness without having to do it manually post-hoc.#2016-09-0617:18nopromptit seems like a useful thing since, on occasion, one might be interested in validating/conforming nested (or unnested) textual data.#2016-09-0617:44Alex Miller (Clojure team)@noprompt for the moment, weāre considering all that to be implementation details#2016-09-0617:44Alex Miller (Clojure team)and subject to change without notice#2016-09-0617:45Alex Miller (Clojure team)in general, spec is designed to primarily be extensible via predicates (at the bottom) and wrapping with macros (at the top - particularly s/conformer and s/& are tools for this)#2016-09-0617:50noprompt@alexmiller i see. well, i'll continue to hold out then for things to stabilize. i've at least familiarized myself a bit more with the internals (implementing Spec
myself) so that can't be a bad thing. is there something similar to conformer
for explain
? that would be nice in my case because the parser combinators do produce failure data of where in the string the parse failed which could be appended to the path
.#2016-09-0617:51noprompttl;dr explainer
?#2016-09-0618:00Alex Miller (Clojure team)no, not currently#2016-09-0618:01Alex Miller (Clojure team)although in next alpha, the explain-out fn is a dynvar you can swap#2016-09-0618:01Alex Miller (Clojure team)thatās a bigger hammer though#2016-09-0618:03nopromptyeah. that sounds like too big of a hammer. some way to implement an explain
for a pred would be nice. i think i can get by with conformer
for now. i hope in the future the protocols will stabilize for this kind of thing.#2016-09-0700:25lvh+1; by the way; I have nothing to contribute to that other than āI can explain a problem better than my consumers can read Clojureā š#2016-09-0701:19aghow do I create an enum spec? Value that conforms that is in a given set?#2016-09-0701:20sparkofreason(s/def ::set-spec #{:value1 :value2 :value3})
#2016-09-0701:20sparkofreasonIt's just the set itself.#2016-09-0701:20agyeah but ohā¦ I got itā¦ (gen/generate (s/gen (s/and #{:foo :boo})))
#2016-09-0701:21agI was using s/exercise its output confused me#2016-09-0701:22agthanks#2016-09-0702:20bbrinck@ag I think (gen/generate (s/gen #{:foo :boo}))
is equivalent#2016-09-0715:18jannisOut of curiosity: Is there a way to say "in this s/cat
spec, please omit this item and this item from the map resulting from s/conform
"? One example (s/def ::myspec (s/cat :first-operand number? :_ #{:plus :minus} :second-operand number?))
-> (s/conform ::myspec [15 :plus 10]])
-> {:first-operand 15 :second-operand 10}
(with the :_
key indicating "please don't include this in the result")?#2016-09-0715:20jannisI'm asking because I'm parsing a small language with clojure.spec and there are some parts in the language that I don't care about later on. And the simpler the output of s/conform
the better - in this case.#2016-09-0715:26Alex Miller (Clojure team)there is no way to automatically omit syntactic elements like that, but you can use a conformer to get the same effect#2016-09-0715:27Alex Miller (Clojure team)something like (s/conform (s/and ::myspec (s/conformer #(dissoc % :_)) [15 :plus 10]])
#2016-09-0715:28Alex Miller (Clojure team)you can put the conformer inside the spec too of course, just wrapping here as an example#2016-09-0715:29jannisNice š#2016-09-0715:29jannisThat's pretty awesome.#2016-09-0715:29jannisI assume I can't have multiple :_
s in an s/cat
?#2016-09-0715:31Alex Miller (Clojure team)it probably wouldnāt throw an error, but I havenāt tried it#2016-09-0715:31Alex Miller (Clojure team)theyād probably all just override the prior as it parsed - you wouldnāt be able to unform that but doesnāt seem like you care about that anyways#2016-09-0715:32Alex Miller (Clojure team)(`conformer` can also take an unform function to support both directions)#2016-09-0715:33jannisYep, later :_
s override earlier occurences in the resulting map.#2016-09-0715:34jannisCool. I'm not sure I'm going to use this but it could make my life easier.#2016-09-0716:02joshgIs there an idiomatic way to define :args
and :ret
when defining a function without a separate fdef
declaration? It would be easy to write a macro to do this, but is it considered bad practice?#2016-09-0716:03joshg(similar to schemaās s/defn
)#2016-09-0716:04Alex Miller (Clojure team)spec does not provide this (and does not intend to)#2016-09-0716:05Alex Miller (Clojure team)but youāre welcome to do so :)#2016-09-0716:06joshgthanks#2016-09-0716:09joshg@alexmiller Iām curious why spec :args
and :ret
are not defined in defn
like pre and postconditions?#2016-09-0716:14Alex Miller (Clojure team)Several reasons#2016-09-0716:15Alex Miller (Clojure team)Rich talks about the notion of an independent registry at http://clojure.org/about/spec#2016-09-0716:16Alex Miller (Clojure team)That is, we don't need to hang everything off vars (and there are downsides to doing so, that affect bytecode size and startup time)#2016-09-0716:16Alex Miller (Clojure team)Also, from the purposes of API evolution, it is useful for the specs to be independent from the vars#2016-09-0716:17joshgGot it. Thanks for the explanation.#2016-09-0716:19joshgthe āAPI evolutionā sound interesting. I assume thatās yet to be announced?#2016-09-0716:27Alex Miller (Clojure team)Rich talked about some of that on the Cognicast episode he did#2016-09-0716:28joshgwhen he was talking about Postel's Law and API versioning?#2016-09-0716:28Alex Miller (Clojure team)But the idea is that if the specs are independent from the functions you can talk (programmatically) about whether one api subsumed another#2016-09-0716:29joshgthat makes sense#2016-09-0721:44Alex Miller (Clojure team)1.9.0-alpha12 is out, mostly spec related things https://groups.google.com/forum/#!topic/clojure/lQ5beZB6QYE#2016-09-0721:44Alex Miller (Clojure team)in particular, Rich did a big perf pass and improved times of most things#2016-09-0722:20noprompt@alexmiller have you talked to or has rich mentioned anything by the way of the explainer
concept we talked about yesterday? it'd be really nice to have something to pair with conformer
to produce explanation data for custom conformers!#2016-09-0723:08Alex Miller (Clojure team)No#2016-09-0802:36bbloomi just want to say: iāve been writing javascript with flowtype for the past ~2 weeks. Itās a very nice type system and itās very well implementedā¦. and I absolutely hate it. I miss clojure š#2016-09-0802:51seancorfield@bbloom No, no, you must be mistaken. Type systems are awesome š#2016-09-0802:52seancorfieldIām finding new things to use clojure.spec
for almost every day, I have to sayā¦#2016-09-0802:54bbloomi think iāve hit more false positives about NPEs from flowtype in two weeks than iāve had ACTUAL NPEs in a year of Go#2016-09-0802:59seancorfieldSo itās telling you "This will likely NPE" but the code wouldnāt?#2016-09-0803:03bbloombasically it says something like āpotentially null or undefinedā b/c itās doing a map lookup or something, but i know with 100% confidence by construction that value is in the map#2016-09-0803:04bbloomor similar#2016-09-0803:05bbloomyou wind up having to employ strategies all over the place:#2016-09-0803:05bbloom1) making tons of extra structure types for every possible state of things#2016-09-0803:05bbloomand 2) using invariant(whatever, āwhatever may not be nullā) which basically is just a glorified manual throw npe#2016-09-0803:36seancorfieldMust admit, when we were using core.typed at work, that was one of the problems we had: nil was contagious and then we kept getting complaints about code not accepting nil (when we knew nil wouldn't flow to that point),#2016-09-0803:37shaun-mahood@seancorfield: Have you run into anything like that with spec yet?#2016-09-0805:38seancorfield@shaun-mahood no, but thatās because spec isnāt "contagious" in the same way that type systems tend to be: specāing one piece of code doesnāt ripple out into other pieces of code.#2016-09-0805:42shaun-mahood@seancorfield: That's awesome - I ran into similar issues with schema in a very limited manner and it really bothered me. I know you'll put spec through a lot more than I ever will so I've been using your notes and comments about as sort of my real-world expectations. #2016-09-0805:45seancorfieldI'm surprised you hit that with Schema since it's really just runtime assertions. #2016-09-0806:22shaun-mahoodWell, it's more likely that I did something stupid or am remembering different problems then - I really didn't put a whole ton of effort into figuring out exactly what was going on and it was quite a while ago. I probably just didn't know what I was doing as well as I though I did :)#2016-09-0813:27jannisIs it a deliberate difference between Clojure and ClojureScript that ClojureScript's clojure.spec.test/check
only accepts a literal collection of syms instead of a code-generated value (e.g. (let [syms (filter in-my-namespaces? (st/checkable-syms))] (st/check syms))
will not work as it raises a Unable to resolve symbol: syms
error.#2016-09-0814:24jeroenvandijkQuestion about multi-spec
, is it meant/possible to use with something else than a map? I cannot find examples and after a lot of trying it seems the multi-spec implementation doesnāt call the multi-method like one would expect#2016-09-0814:32jeroenvandijkah ok, i had a clear moment. I was a victim of multimethod + multi-spec reloading. It is possible with the right dispatch function#2016-09-0815:09jeroenvandijkFYI https://gist.github.com/jeroenvandijk/2748b6af74c6ba6f8fd7dffaed5cc390#2016-09-0815:24Alex Miller (Clojure team)right#2016-09-0815:26Alex Miller (Clojure team)retagging is discussed in the docstring - itās for gen and can be a fn as well#2016-09-0815:27Alex Miller (Clojure team)@jannis prob a question @dnolen would have to answer re check
#2016-09-0815:28jannis@alexmiller Ok, I'll check with him#2016-09-0816:21jannisMight as well do it here. @dnolen: ping š#2016-09-0816:35Timis anyone using this: https://github.com/tonsky/clojure-future-spec ?#2016-09-0816:47jeroenvandijk@tmtwd yep, we use it. Not in something we deployed in production (yet). But I have used it quite a lot now and it works well#2016-09-0816:48Timhm#2016-09-0816:49jeroenvandijkIs anybody doing things with clojure.spec/unform
at the moment? I asked something on the mailinglist about using it for data migrations, but no answer yet#2016-09-0817:04Alex Miller (Clojure team)not many from my impression (given the number of bugs I am aware of but no one has mentioned yet :)#2016-09-0817:04Alex Miller (Clojure team)data migration is one interesting use case for it though#2016-09-0817:07jeroenvandijkthanks, for confirming the use case š Iāll see if I can make it work#2016-09-0818:21dmarjenburghHello. With multi-spec, is there a way to generate values conforming to only a specific dispatch-value? For example, in the :event/event example in http://clojure.org/guides/spec, the generator generates all events. What is the best way to generate events of only a certain type?#2016-09-0818:42bfabry@dmarjenburgh you can get the spec (and hence the generator) for a specific dispatch value by just invoking the multi-method, (event-type {:event/type :event/search})#2016-09-0818:51dmarjenburghAh, ofcourse. Thanks! š#2016-09-0818:52bfabrynp#2016-09-0913:03Alex Miller (Clojure team)The validation benchmark has been updated with alpha12 showing the spec improvements: https://muhuk.github.io/validation-benchmark#2016-09-0914:19ghadiPretty massive improvements#2016-09-0914:26Alex Miller (Clojure team)yes, should help a lot with runtime use cases#2016-09-0914:31Alex Miller (Clojure team)https://imgur.com/a/l9Hc3 shows the before#2016-09-0915:20otfromalexmiller: that's great. Means I'll use it to conform stuff in my sparkling spark jobs. :-D Thx!#2016-09-0915:20Alex Miller (Clojure team)generally, itās faster than schema, so if schema was fast enough for you, it should be sufficient#2016-09-0915:24otfromalexmiller: that was my alternative so I'm very happy. :-D#2016-09-0915:32seancorfieldI'm going to have to study the commit(s) that are responsible for that massive speed up -- very curious about how that was achieved! šø #2016-09-0915:34Alex Miller (Clojure team)lot of changes. one change of possible importance is that a spec using another registered spec will ācompileā in that definition, so changes to the upstream spec (during dev) now require the downstream spec to be reloaded#2016-09-0915:34Alex Miller (Clojure team)so itās a little less dynamic than prior#2016-09-0915:34Alex Miller (Clojure team)this does not affect recursive specs as delays are used#2016-09-0915:41otfrom> (s/conform (s/coll-of int? :type set?) #{0 1})
#{0 1}
> (s/conform (s/coll-of int? :type set?) [0 1])
[0 1]
(s/conform (s/coll-of int? :type set?) [0 "e"])
:clojure.spec/invalid
#2016-09-0915:42otfromnot sure why I'm not getting invalid on the 2nd one#2016-09-0915:43otfromah, I think I need :into#2016-09-0915:52otfrom> (s/conform (s/coll-of int? :into #{}) [0 1])
#{0 1}
#2016-09-0915:52otfromseems to work#2016-09-0915:52jannisI'm hitting odd behavior with clojure.spec and I'm not sure it's me or a bug in clojure.spec. Here's the example: https://gist.github.com/Jannis/f23a2ecf350b401745d190b02bbb619c
What I would expect is no surrounding []
around [:nested ...]
in the second case. It seems that ::link-value
spec is clashing with the ::query
spec. If I change ::link-value
to e.g. (s/tuple symbol? keyword?)
I don't get the extract []
. The odd thing is that despite the "clash", [:link ...]
never appears in the output... any ideas?#2016-09-0915:59Alex Miller (Clojure team)@otfrom :type isnāt a thing#2016-09-0915:59Alex Miller (Clojure team)itās :kind#2016-09-0916:00Alex Miller (Clojure team)if only you had the spec specs, it would tell you that :)#2016-09-0916:06Alex Miller (Clojure team)@jannis so the one youāre asking about is
(s/conform ::query '[a b [c d]])
;;=> [[:single [:simple a]]
;; [:nested {:parent [:simple b], :children [[:single [:simple c]]]}]]
#2016-09-0916:07Alex Miller (Clojure team)?#2016-09-0916:07Alex Miller (Clojure team)the outermost [ ] is from ::query#2016-09-0916:08Alex Miller (Clojure team)the [ ] around :single and :nested are due to the s/alt tagging#2016-09-0916:08otfrom@alexmiller thx. Mostly just flailing around in the library trying to learn it (and getting dumb things wrong) š#2016-09-0916:12jannis@alexmiller The one I'm asking about is [[:single [:simple a]] [[:nested ...]]]
- with the [[:nested ..]]
instead of [:nested ..]
.#2016-09-0916:16jannisWhat's puzzling me there is that the other two examples don't have the extra []
around [:nested ...]
- and that I can make it disappear in the second case if I change the :link-value
spec that should have no impact here.#2016-09-0916:16jannis(This is Clojure 1.9.0-alpha11 by the way)#2016-09-0916:17Alex Miller (Clojure team)oh, this reminds me of something#2016-09-0916:18Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2003?focusedCommentId=43552&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-43552#2016-09-0916:18Alex Miller (Clojure team)that example in the comment also has something similar#2016-09-0916:19Alex Miller (Clojure team)Iāve been looking at some regex conform results that seem off to me - although most of those relate to use of s/?#2016-09-0916:20jannisThe output certainly looks similar š#2016-09-0916:25jannisIn this case I think s/conform
may not know which of [::query :single :link]
and [::query :nested :children]
to pick. It decides on :nested
instead of :single
but surrounds it with the extra []
for some reason.#2016-09-0916:26jannisCould that be what happens in case of ambiguity?#2016-09-0916:30Alex Miller (Clojure team)I donāt think thatās whatās happening - I think it has to do with the guts of the regex derivatives and when it decides to create the nested context#2016-09-0916:36jannisYou know the internals, I don't. š Is there anything I can do to help investigate the problem (if it is one at all)?#2016-09-0917:06Alex Miller (Clojure team)no, Iām looking at some of this stuff already#2016-09-0917:06Alex Miller (Clojure team)if you wanted to boil it down to a simpler repro and file a jira, that would be helpful#2016-09-0917:06Alex Miller (Clojure team)just to make sure that it does get addressed and not just āin my headā :)#2016-09-0920:55fentonhow do I spec that a map should look like: {:db/id <something>}
. I don't own the db
namespace so am not sure how to spec for that.#2016-09-0920:56bfabry@fenton you don't need to own a namespace to refer to it#2016-09-0920:57Alex Miller (Clojure team)(s/keys :req [:db/id])
#2016-09-0920:57Alex Miller (Clojure team)and then separately (s/def :db/id <something>)
#2016-09-0920:58Alex Miller (Clojure team)which could come from a lib or could be something you build#2016-09-0920:59fenton(s/def :db/id (s/tuple #{:pc.api/name} string?))
(s/def ::kwm (s/keys req [:db/id]))
#2016-09-0921:00fenton{:db/id [:pc.api/name "JCZ4vAlwyUl6r37PeVJ"]}
is what i'm going for#2016-09-0921:00fentonrepl> (gen/generate (s/gen :pc.api/kwm))
{}
#2016-09-0921:01fentonpcbe.http> (s/valid? :pc.api/kwm {:db/id [:pc.api/name "orange"]})
true
#2016-09-0921:01fentoncool...#2016-09-1100:02ghufranIām going through the spec tutorial on http://clojure.org . Is there any difference between using spec/or
and using spec/alt
? It seems to work the same for the example from the tutorial:
`
(s/def ::config-alt (s/*
(s/cat :prop string?
:val (s/alt :s string?
:b boolean?))))
;;=> :testspec.core/config-alt
(s/conform ::config-alt [ "-server" "foo"
"-verbose" true
"-user" "joe"])
;;=> [{:prop "-server", :val [:s "foo"]} {:prop "-verbose", :val [:b true]} {:prop "-user", :val [:s "joe"]}]
(s/def ::config-or (s/*
(s/cat :prop string?
:val (s/or :s string?
:b boolean?))))
;;=> :testspec.core/config-or
(s/conform ::config-or [ "-server" "foo"
"-verbose" true
"-user" "joe"])
;;=> [{:prop "-server", :val [:s "foo"]} {:prop "-verbose", :val [:b true]} {:prop "-user", :val [:s "joe"]}]`#2016-09-1100:14Alex Miller (Clojure team)Yes (although there are situations where they can yield the same result)#2016-09-1100:15Alex Miller (Clojure team)in this particular example we are describing the structure of a sequential thing and regex ops are being used to specify that structure#2016-09-1100:15Alex Miller (Clojure team)s/alt is the proper choice here as it is the regex op version#2016-09-1100:16Alex Miller (Clojure team)it matters in the context where it is used - when used inside another regex op, it does not start a new nested context, rather it matches elements out of the current context#2016-09-1100:16Alex Miller (Clojure team)however s/or is effectively opaquely matching a value#2016-09-1100:16Alex Miller (Clojure team)so an example where alt works but or does not is:#2016-09-1100:20Alex Miller (Clojure team)user=> (s/def ::a (s/alt :i int? :s (s/cat :a string? :b string?)))
:user/a
user=> (s/conform ::a [1])
[:i 1]
user=> (s/conform ::a ["a" "b"])
[:s {:a "a", :b "bā}]
#2016-09-1100:20Alex Miller (Clojure team)here an alt by itself implies a sequential context but thatās not true with s/or#2016-09-1100:21Alex Miller (Clojure team)but the nested s/cat does (b/c also a regex op):#2016-09-1100:21Alex Miller (Clojure team)user=> (s/def ::o (s/or :i int? :s (s/cat :a string? :b string?)))
:user/o
user=> (s/conform ::o 5) ;; note 5 is a bare value, not in a collection
[:i 5]
user=> (s/conform ::o ["a" "b"])
[:s {:a "a", :b "bā}]
#2016-09-1100:23Alex Miller (Clojure team)and you can nest an alt inside another regex without requiring a new sequential context:#2016-09-1100:23Alex Miller (Clojure team)user=> (s/conform (s/cat :k keyword? :nested-a ::a) [:foo 5])
{:k :foo, :nested-a [:i 5]}
user=> (s/conform (s/cat :k keyword? :nested-a ::a) [:foo "a" "b"])
{:k :foo, :nested-a [:s {:a "a", :b "b"}]}
#2016-09-1100:23Alex Miller (Clojure team)here ::a is embedded inside something else but just describes more elements of the sequence#2016-09-1101:06ghufranThanks @alexmiller! The guide to spec is great so far, will keep working through it!#2016-09-1206:16bretIf I write a spec like:
(s/def :in/data (s/and
(s/keys :req-un [:in/id] :opt-un [:in/more])
(s/map-of #{:id :more} nil)))
My instinct is to not duplicate the keys and modify this to be:
(def req-keys [:in/id])
(def opt-keys [:in/more])
(defn unk
"Returns ns unqualified keys for (possibly) qualified ones."
[& ks]
(map #(-> % name keyword) ks))
(s/def :in2/data (s/and
(s/keys :req-un req-keys :opt-un opt-keys)
(s/map-of (set (apply unk (concat req-keys opt-keys))) nil)))
which fails due to the nature of the s/keys
macro. I understand the low level cause of the error.
However, I want to make sure Iām not missing something fundamental about the intention. I did find this on the google group https://groups.google.com/forum/#!searchin/clojure/s$2Fkeys%7Csort:relevance/clojure/mlMYUrPVdso/ATklLgpGBAAJ so, possibly, Iām not completely alone in my instincts. But there was no meaningful reply.#2016-09-1207:51jeroenvandijkIāve found a case where conform -> unform -> conform leads to an invalid result. This is the case with the clojure.core.specs/defn-args
spec. See https://gist.github.com/jeroenvandijk/28c6cdd867dbc9889565dca92673a531 Should I file a JIRA issue?#2016-09-1212:24jmglovSorry for asking such a basic question, but what is the recommended way to test spec'd functions in unit tests (i.e. by running lein test
)?#2016-09-1212:24jmglovI like the idea of combining unit and generative tests as per https://dpassen1.github.io/software/2016/09/07/from-repl-to-tests#a-better-way#2016-09-1212:26jmglovBut I'm not really sure how to get c.s.test/check
to hook into the (deftest ... (checking ...))
style.#2016-09-1212:26jmglovRTFM links welcome. š#2016-09-1212:29otfromjmglov: I'd be happy with figuring out that workflow too#2016-09-1212:32tgkYeah, I was puzzled with this as well. I ended up writing a very small namespace for it and creating a lein alias for running the specs#2016-09-1212:32jmglovUsing enumerate-namespace
?#2016-09-1212:34tgkUsing clojure.tools.namespace.repl
: https://gist.github.com/tgk/be0325e7b78bc692ad6c85ef6aca818d#2016-09-1212:34tgkThe file is only on dev path by the way š#2016-09-1212:37jmglovInteresting. I didn't realise that test/check
had a zero-arity form. Really useful!#2016-09-1212:38jmglovDoes it actually find all specs in all namespaces in your classpath, or something?#2016-09-1212:39jmglovI don't see you specifying any namespaces under test, or requiring them in.#2016-09-1212:39tgkYes, as long as they have been evaluated#2016-09-1212:39jmglovWow.#2016-09-1212:39tgkAh yes, thatās where refresh
comes into the picture š#2016-09-1212:39jmglovWould your approach actually run all the specs for dependencies as well?#2016-09-1212:40jmglovOr just stuff in your src
?#2016-09-1212:40jmglovThat's nifty!#2016-09-1212:41jmglovAll the same, it would be great to find an approach that would allow me to drop spec generative tests into my standard clojure.test
files.#2016-09-1212:41jmglovI haven't found anything with Google, but stemming is really fighting me on this one. š#2016-09-1212:44jmglovMaybe I need to roll my own checking
macro, like this guy did: http://blog.colinwilliams.name/blog/2015/01/26/alternative-clojure-dot-test-integration-with-test-dot-check/#the-alternative#2016-09-1212:49tgkHmm yes, I donāt think specs for dependencies would be run. I canāt see how they would š#2016-09-1212:49jmglovThat's good. š#2016-09-1212:49tgkI found it very hard to find anyone whoād hooked it into tests#2016-09-1212:49jmglovSo refresh
simply evals everything in your source directories?#2016-09-1212:54Alex Miller (Clojure team)using spec.test/check or spec.test/instrument will pick up any specāed fns that have been loaded and added to the registry, so it depends completely on what code youāve loaded#2016-09-1212:55jmglovThanks, Alex!#2016-09-1212:55jmglovAlso, thanks for the spec Guide. I finished reading it, and it is really excellent!#2016-09-1212:56Alex Miller (Clojure team)@bret I personally would prefer your first spec (although looks like you missing the kw namespaces on the map-of and you probably want s/merge instead of s/and)#2016-09-1212:57jmglovOK, switching gears for a second, I'm obviously doing something silly, but I'm not sure what. I'm trying to spec out the input coming in from some JSON, and I have some code like this:
(ns wtf
(:require [clojure.spec :as s]
[clojure.spec.test :as test]))
(s/def ::contract_type_id pos-int?)
(s/def ::product (s/keys :req-un [::contract_type_id]))
(s/fdef exclude-products
:args (s/cat :products (s/coll-of ::product)
:excluded-contracts (s/coll-of ::contract_type_id))
:ret (s/coll-of ::product)
:fn #(<= (-> % :args :products) (-> % :ret)))
(defn- exclude-products [products excluded-contracts]
(letfn [(excluded? [product]
(some #{(:contract_type_id product)} excluded-contracts))]
(remove excluded? products)))
#2016-09-1212:58jmglovMy exclude-products
function should do something this:
wtf> (exclude-products [{:contract_type_id 1} {:contract_type_id 2}] [1])
({:contract_type_id 2})
#2016-09-1212:59jmglovThings look good with exercise-fn
:
wtf> (s/exercise-fn `exclude-products 1)
([([{:contract_type_id 2} {:contract_type_id 2} {:contract_type_id 2} {:contract_type_id 1}] [2 2 1 1 1 1 1 2 1])
()])
#2016-09-1213:00jmglovBut check
completely rejects my entire worldview:
wtf> (test/check `exclude-products)
({:spec #object[clojure.spec$fspec_impl$reify__14244 0x2c221658 "
#2016-09-1213:01jmglovCan anyone shed light on what I'm doing wrong?#2016-09-1213:01Alex Miller (Clojure team)@jeroenvandijk yes, please file a jira. this is where we are hurting for an s/vcat which Rich and I have talked about several times.#2016-09-1213:05Alex Miller (Clojure team)@jmglov it looks to me like your :fn spec is wrong and should be comparing count
of each thing?#2016-09-1213:06jmglovdoh#2016-09-1213:06jmglov@alexmiller Of course you are right. Thanks for pointing out my idiocy! š#2016-09-1213:07Alex Miller (Clojure team)well I wouldnāt go that far. :) fwiw, Iāve done the same.#2016-09-1213:10jmglovOh, so much better! Now summarize-results
is showing me a bug in my code or spec. š#2016-09-1213:11bret@alexmiller I guess my point/observation is that the second one is not allowed at all. That was puzzling at first. I've missed s/merge
all this time. Iāll look at that and see if that changes anything.#2016-09-1213:16jeroenvandijk@alexmiller thank, will do#2016-09-1213:17Alex Miller (Clojure team)@bret not sure what you mean by your point/observation, sorry#2016-09-1213:23jeroenvandijk@alexmiller Iāve created the issue here http://dev.clojure.org/jira/browse/CLJ-2021?focusedCommentId=43842#comment-43842 Not sure what else to add#2016-09-1213:27bret@alexmiller I probably shouldn't start writing at midnight on Sunday night. :)
I guess it boils down to, is the reason this works
(s/keys :req-un [::k1 ::k2])
=> #object[clojure.spec$map_spec_impl$reify__13426 0x1b824394 "
and this doesnāt
(def rks [::k1 ::k2])
=> #'onenine.core/rks
(s/keys :req-un rks)
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(/Users/brety/dev/personal/onenine/src/onenine/core.clj:69:1)
merely a consequence of the s/keys
macro implementation or is this intended to not be valid?#2016-09-1213:27Alex Miller (Clojure team)well, both#2016-09-1213:28Alex Miller (Clojure team)s/keys is a macro and expect a concrete list of keys#2016-09-1213:28Alex Miller (Clojure team)so thatās as intended#2016-09-1213:28Alex Miller (Clojure team)and the reason most of the spec creating fns are macros is to capture forms for reporting#2016-09-1213:29jmglovDoes anyone know how to make test/check
work on private functions? I've tried #'full.namespace/foo
, but I get java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Var
.#2016-09-1213:29jmglovLeaving off the #'
tells me the function is not public, which I already know. š#2016-09-1213:30Alex Miller (Clojure team)we might in the future have a fn entry point for s/keys with the caveat that you may lose some of the explain reporting capability#2016-09-1213:30bretThat is what I suspected (the reporting aspect) but wanted to confirm.#2016-09-1213:31Alex Miller (Clojure team)@jmglov I donāt think that was considered in the design (and Iām not sure whether it should be)#2016-09-1213:32jmglovYeah, I'm a bit naughty, really.#2016-09-1213:32lvhOn a related note: I find myself regularly wanting validation of runtime-defined specs, where I learn e.g. the structure of the JSON in this particular REST API at runtime. This is for presumably obvious reasons, not very convenient right now.#2016-09-1213:32jmglovThis is always a tough call. I want to use private functions to communicate to my clients that they are not part of the interface, but I also want to be able to unit test them. š#2016-09-1213:33lvh(Everything ends up being evalād, which I guess is fine?)#2016-09-1213:33bretIt would be nice in some case, I think, to be able to define keys once and combine them in different ways ways when building specs. At least, as I think about repeating information in the spec(s) and trying to reduce that.#2016-09-1213:33bretBut I donāt have enough time in spec to be sure.#2016-09-1213:34jmglovI might just go the foo.bar.internal
route, where everything in the internal ns is public and can be tested, but is pretty clearly not for client consumption.#2016-09-1213:34Alex Miller (Clojure team)@lvh yeah, I understand that as a use case, not sure how common that will be in general though#2016-09-1213:34jmglovUsing a namespace-level docstring to warn off potential troublemakers, of course.#2016-09-1213:34Alex Miller (Clojure team)yeah#2016-09-1213:35Alex Miller (Clojure team)@bret well thatās exactly the point of having the registry#2016-09-1213:40lvhalexmiller: If you give Clojure programmers a feature it seems like a matter of time before theyāll try to express as much of it as data, and then itās not a long stretch until that data isnāt available at compile time š#2016-09-1213:40lvhI might be able to get around it and just move more stuff into compile-time-land#2016-09-1213:41Alex Miller (Clojure team)specs are data in s-expr form#2016-09-1213:42Alex Miller (Clojure team)we havenāt released it yet, but I have a spec for spec forms#2016-09-1213:43Alex Miller (Clojure team)(which revealed a lot of bugs in s/form :)#2016-09-1213:45lvhnice; I would very much like that#2016-09-1213:45lvhsince a hypothetical awful person might want to construct specs at runtime and have better feedback about why they donāt work š#2016-09-1213:46Alex Miller (Clojure team)oh, I donāt think youāre awful :)#2016-09-1213:46Alex Miller (Clojure team)itās reasonable#2016-09-1213:46Alex Miller (Clojure team)just not the primary use case we were working to support#2016-09-1213:47bret@alexmiller Ok, this will help me.
;; I want to check that an input map's keys are valid
;; where the keys are unqualified, some required, some optional,
;; and not allow keys outside that set.
; This is straight forward
(s/def :in/data (s/and
(s/keys :req-un [:in/id] :opt-un [:in/more])
(s/map-of #{:id :more} nil)))
; but I'm (kind of) repeating information.
;
; If I write
(def req-keys [:in/id])
(def opt-keys [:in/more])
(defn unk
"Returns ns unqualified keys for (possibly) qualified ones."
[& ks]
(map #(-> % name keyword) ks))
(s/def :in2/data (s/and
(s/keys :req-un req-keys :opt-un opt-keys)
(s/map-of (set (apply unk (concat req-keys opt-keys))) nil)))
; I have not repeated the key values but s/key doesn't allow it.
What is the proper way to write the spec where I not repeating information? I didnāt initially see a way to piece it together from āsmallerā specs since the args in map-of is really just a set used as a predicate.#2016-09-1213:48bretI could be missing something fundamental.#2016-09-1213:49Alex Miller (Clojure team)Iād say generally that Rich believes in open maps and thatās why this is not a feature provided out of the box#2016-09-1213:49Alex Miller (Clojure team)and that I think your first example is what I would do if I was doing it#2016-09-1213:49Alex Miller (Clojure team)(although nil is not a valid spec there - you want any?
)#2016-09-1213:51Alex Miller (Clojure team)and I would use s/merge
instead of s/and
#2016-09-1213:51Alex Miller (Clojure team)which I think would gen better#2016-09-1213:53bretSo, s/merge
can be used for combining more than s/keys
(`s/map-of` in this case)?#2016-09-1214:00Alex Miller (Clojure team)s/merge is used to combine (union) map specs#2016-09-1214:00Alex Miller (Clojure team)it differs in not flowing conformed results like s/and and also in being better at gen'ing#2016-09-1214:00bretI get the open map approach and generally like it. One thought I had, that really relates to the reporting aspect, is that s/keys
supports āandā/āorā combinations of key vectors. So, keeping the form used for reporting as close to literal boolean expressions of literal key vectors is not a bad thing. Ok, thanks, this helps.#2016-09-1214:22rickmoynihanwhat's the best way to spec that something satisfies?
a protocol?#2016-09-1214:27rickmoynihanobviously you can just use the (partial satisifies? Protocol)
predicate... but is there a way for implementers to hook in and extend the generator to generate types that satisfy it? I could imagine that if implementers spec'd their constructing functions, you could get this almost for free.#2016-09-1214:37Alex Miller (Clojure team)nothing built-in#2016-09-1214:54jmglovHere's another fun one. Using the fixed version of the same spec as previously, I can use stest/check
on it in my REPL:
kpcs.product-catalog.internal-test> (first (stest/check 'kpcs.product-catalog.internal/exclude-products))
{:spec #object[clojure.spec$fspec_impl$reify__14244 0x2c8e87a5 "
#2016-09-1214:56jmglovHowever, when I try to use it in a test, I get an exception. Here's what I'm trying to do:
(ns kpcs.product-catalog.internal-test
(:require [clojure.spec.test :as stest]
[clojure.test :refer [deftest is testing]]
[kpcs.product-catalog.internal :as internal]))
(deftest exclude-products
(testing "Specs"
(let [res (first (stest/check 'kpcs.product-catalog.internal/exclude-products))]
(is (= java.lang.String (type res)))))
#2016-09-1214:57jmglovAnd I get this:
java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
at java.util.concurrent.FutureTask.report (FutureTask.java:122)
java.util.concurrent.FutureTask.get (FutureTask.java:192)
clojure.core$deref_future.invokeStatic (core.clj:2290)
clojure.core$future_call$reify__9352.deref (core.clj:6847)
clojure.core$deref.invokeStatic (core.clj:2310)
clojure.core$deref.invoke (core.clj:2296)
clojure.core$map$fn__6856.invoke (core.clj:2728)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:56)
clojure.lang.LazySeq.first (LazySeq.java:71)
clojure.lang.RT.first (RT.java:682)
clojure.core$first__6379.invokeStatic (core.clj:55)
clojure.core/first (core.clj:55)
...
#2016-09-1214:59jmglovAny ideas?#2016-09-1215:00jmglovTo be clear, if I REPL into my kpcs.product-catalog.internal-test
and run the code above, it works. If I run lein test
, I get the exception.#2016-09-1215:00jmglovI don't see what should be different between the two.#2016-09-1215:09Alex Miller (Clojure team)thatās a problem with lein testās monkeypatching#2016-09-1215:09Alex Miller (Clojure team)itās fixed in (not yet released) next version of test.check#2016-09-1215:09Alex Miller (Clojure team)but you can disable lein monkeypatching to fix#2016-09-1215:10Alex Miller (Clojure team):monkeypatch-clojure-test false
#2016-09-1215:10jmglovGreat, thanks!#2016-09-1215:10Alex Miller (Clojure team)that will disable lein retest
but otherwise should not affect what youāre doing#2016-09-1215:11jmglovJust tried it, and it works perfectly.#2016-09-1215:18Alex Miller (Clojure team)you are not the first person to encounter it :)#2016-09-1215:22jmglovThank goodness for that!#2016-09-1215:27jmglov@otfrom @tgk Here's a cheap hack to fit check
into my standard tests:
(deftest exclude-products
(testing "Specs"
(let [result (-> (stest/check 'kpcs.product-catalog.internal/exclude-products)
first
:clojure.spec.test.check/ret
:result)]
(if (true? result)
(is result)
(is (= {} (ex-data result)))))))
#2016-09-1215:28otfromjmglov: thx!#2016-09-1215:28jmglovI'll make a checking
function out of it and throw it in a test lib. The output is decent enough with the humane-test-output plugin. š#2016-09-1215:29jmglovIf it's useful, you're welcome. Otherwise, I'm sorry for such a disgusting kludge! š#2016-09-1215:48mike_ananevHi there! Sorry for very dumb question. How to define spec for function with variable args?#2016-09-1215:56bahulneelHi guys, I've just started using clojure.spec and was wondering how to use clojure.spec.test/check with clojure.test#2016-09-1215:56bahulneelsorry, missed the msg from @jmglov#2016-09-1216:38Alex Miller (Clojure team)@mike1452 (s/fdef myf :args (s/cat :map-params (s/? map?)))
will take both 0 and 1 (map) arg#2016-09-1216:38Alex Miller (Clojure team)you can replace map?
with something more specific too of course#2016-09-1217:20bfabryit seems awkward that core/defn has a special syntax for arity dispatch but spec/fdef doesn't provide one for speccing/testing#2016-09-1217:21bfabrydoesn't really affect me as I never use arity dispatch but I could see it sucking if you previously used it a lot#2016-09-1217:25Alex Miller (Clojure team)they are doing different things. most multi-arity functions share param definitions across arities and merging them works very nicely for this in most cases.#2016-09-1217:28bfabrysure, for the :args, but it doesn't make the :fn for say map
less readable?#2016-09-1217:45Alex Miller (Clojure team)sure, although I think thatās an unusual case#2016-09-1217:55bfabrytrue. I guess the common case would be reduce
. smaller arity providing default value#2016-09-1218:57ikitommiIs there a reason why if-let
and when-let
canāt return :clojure.spec/invalid
?#2016-09-1219:01hiredmanwell that is problematic#2016-09-1219:04hiredman (if-let [a 1] '::s/invalid)
works if you need a work around#2016-09-1219:05ikitommi@hiredman cool, thanks. didnāt know that works too.#2016-09-1220:26Alex Miller (Clojure team)@ikitommi heh, thatās fun#2016-09-1220:28Alex Miller (Clojure team)thereās actually a ticket related to this#2016-09-1220:28Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-1966#2016-09-1221:07seancorfieldI added the if-let
example above to that ticket. I suspect people will run into this in more and more situations as they try to write conformers.#2016-09-1221:13Alex Miller (Clojure team)and as there are more specāed things#2016-09-1221:21agI need a spec that would generate vectors of values taking random elements from a predefined list, e.g: [:foo :bar]
[:foo]
[:baz :bar] []
ā¦ etc.#2016-09-1221:21aghow?#2016-09-1221:24Alex Miller (Clojure team)(s/coll-of #{:foo :bar :baz} :kind vector?)
#2016-09-1221:27Alex Miller (Clojure team)you can also use the other options on coll-of
to set :min-count, :max-count, :count constraints on the spec or :gen-max to cap what the generator will produce#2016-09-1221:28agrightā¦ http://clojure.org/guides/spec#_collections#2016-09-1221:28agthanks!#2016-09-1306:55jmglov@bahulneel I'm working on upgrading a production project to Clojure 1.9 and spec right now, so I'll keep this channel informed of any new stuff I figure out with the clojure.test
integration.#2016-09-1306:56jmglovGiven that clojure.spec.test/check
takes a quoted symbol, I shouldn't even need a macro to accomplish what I want. A plain 'ol function should do the trick. š#2016-09-1310:00bahulneel@jmglov thanks, I'll keep my ear to the ground#2016-09-1311:18jmglovI'm having trouble getting clojure.spec.test/check
to run a non-default number of tests. From reading the docs and the code, it looks like I should be able to do this:
(stest/check 'kpcs.event.processor/schema-version {:num-tests 1})
But it always runs 1000 tests:
({:spec #object[clojure.spec$fspec_impl$reify__14244 0x7b0d682d "
#2016-09-1311:18jmglovWhat am I missing?#2016-09-1311:22jmglovHrm... I realise that I'm dealing with a namespaced key. This doesn't work, either:
(stest/check 'kpcs.event.processor/schema-version {:clojure.test.check.stc/opts {:num-tests 1}})
#2016-09-1311:24jmglovOK, finally figured it out:
(stest/check 'kpcs.event.processor/schema-version {:clojure.spec.test.check/opts {:num-tests 1}})
({:spec #object[clojure.spec$fspec_impl$reify__14244 0x7b0d682d "
#2016-09-1311:24jmglovI've never seen the ::foo/bar
-style keywords before, only ::bar
.#2016-09-1311:25jmglovWhy does ::stc/opts
in the clojure.spec.test
namespace expand into :clojure.spec.test.check/opts
?#2016-09-1311:49Alex Miller (Clojure team)clojure.spec.test sets up an alias of āstc to āclojure.spec.test.check#2016-09-1311:50Alex Miller (Clojure team)the ::stc/opts just says to create a fully-qualified keyword using the local alias stc so that expands#2016-09-1312:46jmglov@alexmiller Thanks for the explanation!#2016-09-1317:42uwocould someone point me to the reasoning behind fspec
requiring :ret
and fdef
not?#2016-09-1317:47Alex Miller (Clojure team)I think they both used to and fdef was relaxed#2016-09-1317:47Alex Miller (Clojure team)fspec prob should be too#2016-09-1317:47Alex Miller (Clojure team)if you file a jira enhancement weāll look at it#2016-09-1317:47Alex Miller (Clojure team)(in other words, I think no reason :)#2016-09-1317:48uwoš thanks#2016-09-1317:54esp1fyi :ret
is apparently required for fspec
in alpha12#2016-09-1317:56esp1(s/explain (s/fspec :args empty?) (fn [] nil))
Success!
=> nil
(s/valid? (s/fspec :args empty?) (fn [] nil))
=> false
#2016-09-1318:16Alex Miller (Clojure team)just to be sure, thatās the same thing @uwo said above right?#2016-09-1318:16uwoyes#2016-09-1318:17Alex Miller (Clojure team)ok, Iām agreeing thatās wrong if it wasnāt clear :)#2016-09-1318:19esp1yup just pointing out that itās required now. in previous alphas :ret
was actually not required on fspec
, but in alpha12 that restriction is enforced#2016-09-1318:20jannisIs there a way to refer to a spec that is defined later in the same file? E.g. (s/def ::foo (s/with-gen ::bar ...)) ... (s/def ::bar ...)
?#2016-09-1318:22jannisI may be getting an unspecified behavior, where sometimes the spec is found and sometimes it isn't, resulting in conform
returning a tagged, conformed value and sometimes returning the untagged original value.#2016-09-1318:22jannisGranted, the second spec is more complex than in this example but the first is pretty much that.#2016-09-1318:24jannis(describe ::foo)
says :clojure.spec/unknown
.#2016-09-1318:29darwinclojurescript related question, Iām working on a library which uses clojure.spec to describe a data structure using s/*
, the data structure can be used statically during compilation in macros, or during runtime in cljs.
all is nice and shiny on clj side[1], but on cljs side, I want native js arrays to be treated as collections for the purpose of speccing, but unfortunatly s/*
does not treat them such. Ended up writing custom predicates for native array case[2], just wondering if there is a better way to teach s/*
to walk native js arrays the same way as cljs vectors.
[1] https://github.com/binaryage/cljs-oops/blob/master/src/lib/oops/sdefs.clj
[2] https://github.com/binaryage/cljs-oops/blob/master/src/lib/oops/sdefs.cljs#2016-09-1318:33jannisFunny. (s/def ::foo (s/with-gen (s/and ::bar) ..))
makes it work at all times.#2016-09-1318:37Alex Miller (Clojure team)@jannis I donāt know of any issue with that. however, one of the perf changes in alpha12 means that one spec gets compiled into anotherās definition (via delay) so when you change (re s/def) a spec at the repl, downstream specs need to be re-s/defāed as well#2016-09-1318:37Alex Miller (Clojure team)is there any chance you are seeing this behavior at the repl?#2016-09-1318:37Alex Miller (Clojure team)maybe when you made it āworkā you just caused a new s/def that was needed#2016-09-1318:38Alex Miller (Clojure team)@esp1 I donāt know of any change that would have altered this behavior in alpha12, but thatās possible (or that it changed at some point during the alphas)#2016-09-1318:40esp1actually previously i was using tonskyās future-spec port of spec to 1.8, so possibly it was something specific to that, not sure#2016-09-1318:44jannis@alexmiller: If I load the namespace that these specs are in I get it in the REPL. If I reload the same namespace, it disappears. And I get it when running tests outside the REPL (using boot test
).#2016-09-1318:45jannisI also get it when defining them in the REPL: https://gist.github.com/Jannis/d249d6d21e92f6bca7736c79eb008b49#2016-09-1318:45jannisThis is alpha11 though. I can't use alpha12 at the moment because it fails requiring DataScript due to an invalid ns
expression in it.#2016-09-1318:50Alex Miller (Clojure team)I canāt even run the first line of that with a bare alpha12 repl, not sure about earlier#2016-09-1318:51Alex Miller (Clojure team)I realize this is slimmed down, but I donāt think you should expect to with-gen over a spec that doesnāt exist yet#2016-09-1318:52jannisThese specs are for a language that is recursive (think: Om Next query expressions). How would I include the recursive nature in the spec other than refering to the top-level spec in a part of it?#2016-09-1318:53jannisThe top-level spec needs the sub-spec, the sub-spec needs the top-level spec. Hmm...#2016-09-1319:04Alex Miller (Clojure team)yeah, thatās fine#2016-09-1319:04Alex Miller (Clojure team)Iām talking about with-gen in particular#2016-09-1319:05Alex Miller (Clojure team)you can provide gen overrides separately too of course. gen with recursion is generally hard.#2016-09-1319:05jannisHow do I do it separately?#2016-09-1319:16Alex Miller (Clojure team)fns like instrument
take an override map#2016-09-1319:16Alex Miller (Clojure team)from name (or path) to generator#2016-09-1319:31jannisAh, yes. So that would also work with clojure.spec.test/check
. I'd prefer to have all generators defined along with the actual spec so it's as reusable as possible (e.g. in combination with fdef
args in functions other people may write). I'll see what the best solution is. Thanks!#2016-09-1319:37bhaganyI'm trying to understand why nested calls s/cat
don't operate the way I expect - to illustrate:
cljs.user> (s/explain (s/cat :x number? :y number? :z number?) [0 0 0])
Success!
nil
cljs.user> (s/explain (s/cat :coords (s/cat :x number? :y number? :z number?)) [[0 0 0]])
In: [0] val: [0 0 0] fails at: [:coords :x] predicate: number?
nil
In my actual situation, I'm trying to reuse the inner s/cat
as both the return value of a fn, and an argument to another fn. Also, it does work the way I expect if I use (s/tuple number? number? number?)
, but I'd like to be able to conform it and get a map. Am I missing something?#2016-09-1319:40seancorfield(s/explain (s/cat :coords (s/spec (s/cat :x number? :y number? :z number?))) [[0 0 0]])
=> Success!#2016-09-1319:40bfabry@bhagany the regex operators assume concatenation by default, you need to wrap nested calls in s/spec#2016-09-1319:41bhaganyaha, thanks @seancorfield and @bfabry!#2016-09-1319:41bfabrycovered towards the bottom of this section http://clojure.org/guides/spec#_sequences#2016-09-1319:43bhaganyI'd read it, but I think I have not yet internalized that cat
is a regex operator. thanks again#2016-09-1320:05Alex Miller (Clojure team)any set of nested regex ops (`cat`, alt
, ?
, +
, *
, &
, keys*
) describes the structure of a single sequential context#2016-09-1320:06Alex Miller (Clojure team)which is just like regex on a string#2016-09-1320:07Alex Miller (Clojure team)strings are different in that a single character in a string canāt also be a string (but that is true of elements in a Clojure sequential collection)#2016-09-1320:08hiredmanthe video of dnolen's talk on parsing with derivatives / spec is up https://www.youtube.com/watch?v=FKiEsJiTMtI#2016-09-1322:14gfredericks@alexmiller: test.check already supports collections distinct by custom key-fns#2016-09-1322:15gfredericksE.g. gen/vector-distinct-by
#2016-09-1322:21gfredericks@jmglov: https://github.com/gfredericks/schpec exists in case you don't want to bother managing a whole library for that function#2016-09-1323:07Alex Miller (Clojure team)@gfredericks nice#2016-09-1323:40wilkesAre there any conventions around where to put specs? In the namespace, in a special whatever.spec namespace, etc..#2016-09-1323:46seancorfield@wilkes "it depends" is the answer to that.#2016-09-1323:46wilkesš#2016-09-1323:47seancorfieldAt World Singles, our specs are very data centric so they mostly live in their own namespace and we pull them into other namespaces as needed, and then our tests pull them in to instrument / check the code.#2016-09-1323:48seancorfieldIn clojure.java.jdbc all the fdefās are in a separate namespace so the code still works with pre-1.9.0 versions of Clojure.#2016-09-1323:49seancorfieldBut if you know youāre only going to use 1.9.0 onward and youāre mostly specāing functions, rather than data, Iād imagine it would be convenient to put fdefās next to their defnās in the same namespace.#2016-09-1323:53wilkes@seancorfield Thanks, that helps. I was expecting this to be at least in flux until people have spent some time with specs in their projects.#2016-09-1323:56Alex Miller (Clojure team)Clojure itself is putting specs in clojure.core.specs#2016-09-1400:17hiredmanit seems like that could end up mixing testing code in with, production code (for lack of a better description)#2016-09-1400:17hiredmanis the idea that people will do similar lazying loading kind of stuff like clojure.spec does for test.check?#2016-09-1400:24seancorfield"mixing testing code in with production code" ā¦ how? Itās not the case with how WS is using spec at the moment.#2016-09-1400:25seancorfieldOur spec namespaces have just specs in them ā which our (production) code uses for explicit validation and conforming of data at each "system" boundary (input; domain; persistence).#2016-09-1400:26seancorfieldIn addition, our test code can pull in the specs and instrument
and/or check
code. Any test-specific code would live in the test (expectations) namespace, not the main (production) namespace.#2016-09-1400:27seancorfield(itās certainly possible Iāve missed some subtlety here butā¦)#2016-09-1400:27hiredmando you have specs with custom generators?#2016-09-1400:30hiredmanor, I guess I should say, do you define your own predicates with generators? it seems like that would depend at least on test.check, so you would either need to do lazy loading or have a runtime dependency on test.check#2016-09-1400:31seancorfieldAh, gotcha...#2016-09-1400:31hiredmanI haven't used spec in anger, so I dunno how it plays out, just something I wonder about#2016-09-1400:32bfabry@hiredman you can specify the generators in the spec.test/check or instrument call, instead of with the spec itself#2016-09-1400:32seancorfieldWe lazy-load via the functions that return generators (and do so in a separate ns): ;; copyright (c) 2016 world singles llc
(ns ws.spec.generators
"Helpers for our specs that need custom generators.
Dynamically loads the namespaces as needed so we can compiler
this without test dependencies.")
(defn fn-string-from-regex
"Return a function that produces a generator for the given
regular expression string."
[regex]
(fn []
(require '[com.gfredericks.test.chuck.generators :as xgen])
(let [string-from-regex (resolve 'xgen/string-from-regex)]
(string-from-regex regex))))
#2016-09-1400:32seancorfieldOoh, typo. compiler
instead of compile
...#2016-09-1400:32hiredmanah#2016-09-1400:35seancorfieldWe have a few with-gen
s that leverage stuff in clojure.spec
or code thatās already available in production, otherwise we defer to our ws.spec.generators
"proxy" namespace.#2016-09-1400:36seancorfieldBut, yeah, ensuring those sort of test artifacts only get loaded when weāre testing was something we had to navigate at first...#2016-09-1400:36seancorfieldIn our build.boot
we have (deftask testing-context
"Provide main testing context."
[]
(merge-env! :dependencies '[[com.gfredericks/test.chuck "0.2.7" :scope "test"
:exclusions [*/*]]
[javax.servlet/servlet-api "2.5" :scope "test"]
[org.clojure/test.check "0.9.0" :scope "test"]])
identity)
#2016-09-1400:37seancorfieldand thatās only used by our test tasks#2016-09-1400:39bnoguchiI have been having trouble specāing an function arg to another function where the former can be either a function with arity 0 or a function with arity 1 (both not variadic functions). Any ideas?
(s/explain (s/fspec :args (s/alt :arity-0 (s/cat) :arity-1 (s/cat :arg1 any?)) :ret any?) (fn []))
; val: (nil) fails predicate: (apply fn), Wrong number of args (1) passed to: user/eval16844/fn--16845
#2016-09-1400:44bfabry@bnoguchi your spec seems to be working. you said the function should accept 1 or 0, your function only accepts 1#2016-09-1400:45hiredmanI think you need to pull the alt out#2016-09-1400:45bfabry@bnoguchi fwiw there's a briefer way to describe that spec: (s/cat :arg1 (s/? any?))
#2016-09-1400:46bnoguchi@bfabry Itās written to alternatively accept 1 or 0.#2016-09-1400:46bfabry@bnoguchi your spec is, but the anonymous function (fn [] ) only accepts 0 arguments#2016-09-1400:46hiredman(s/explain (s/alt :a0 (s/fspec :args (s/cat) :ret any?) :a1 (s/cat :a1 any?)) (fn []))#2016-09-1400:46bfabryboot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) (fn []))
val: (nil) fails predicate: (apply fn), Wrong number of args (1) passed to: user/eval2135/fn--2136
nil
boot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) (fn [a]))
val: () fails predicate: (apply fn), Wrong number of args (0) passed to: user/eval2260/fn--2261
nil
boot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) (fn [& a]))
Success!
nil
#2016-09-1400:47bnoguchi@bfabry That is not what Iām after, however#2016-09-1400:47hiredmanoh, whoops#2016-09-1400:47bnoguchii.e., I donāt want to spec a variadic function. Perhaps this form will help#2016-09-1400:47hiredman(s/explain (s/alt :a0 (s/fspec :args (s/cat) :ret any?) :a1 (s/fspec :args (s/cat :a1 any?) :ret any?)) (fn []))
#2016-09-1400:48hiredmanif you have two fspecs, with the alt outside, then it seems like you get he behavior you want#2016-09-1400:49hiredmanI think that makes sense, but it is hard to write the distinction in prose#2016-09-1400:49hiredmana function that takes 0 or 1 arg vs a function that takes 0 args, or a function that takes 1 arg#2016-09-1400:49bfabryit's not that it's variadic, it's that a variadic function satisfies the spec. so does a function that is arity 1 or 0
boot.user=> (defn foo
#_=> ([] nil)
#_=> ([a] nil))
#'boot.user/foo
boot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) foo)
Success!
nil
#2016-09-1400:51hiredmanoh, am I am just wrong, and not paying close enough attention#2016-09-1400:56hiredmanuser=> (s/explain (s/or :a0 (s/fspec :args (s/cat) :ret any?) :a1 (s/fspec :args (s/cat :a1 any?) :ret any?)) (fn []))
Success!
nil
user=>
#2016-09-1400:56hiredmanthere we go, of course it needs to be s/or not s/alt#2016-09-1400:56hiredman(outside of matching the arglist#2016-09-1400:56hiredman)#2016-09-1400:57bfabryoh I'm sorry you wanted to spec either one or the other, not a function that expects both. in that case yes you need two fspec's (two different fiunction specs) not one fspec (a single function that does two things)#2016-09-1400:57bnoguchi(defn arity [f] (-> f class .getDeclaredMethods first .getParameterTypes alength))
(defn fn-expecting-a-fn-arg-either-0-or-1-arity [func] (if (zero? (arity func)) (func) (func 1)))
(s/fdef fn-expecting-a-fn-arg-either-0-or-1-arity :args (s/cat :fn (s/fspec :args (s/or :arity-0 (s/cat) :arity-1 (s/cat :arg1 any?)) :ret any?)) :ret any?)
(require '[clojure.spec.test :as stest])
(stest/instrument `fn-expecting-a-fn-arg-either-0-or-1-arity)
(fn-expecting-a-fn-arg-either-0-or-1-arity (fn []))
throws
ExceptionInfo Call to #'user/fn-expecting-a-fn-arg-either-0-or-1-arity did not conform to spec:
val: (#object[user$eval16875$fn__16876 0x5fa44d5a "
#2016-09-1400:58hiredmanyou can't put the choice (alt or or) in the args for the function, because that means the function needs to satisfy both#2016-09-1400:58bfabry@hiredman's example is correct, you want to s/or two s/fspec's#2016-09-1400:59hiredmanyou have to spec the possible arguments distinctly in different f/specs#2016-09-1401:03bnoguchi@hiredman Yeah I think youāre right#2016-09-1401:04bnoguchiThis works
(s/fdef fn-expecting-a-fn-arg-either-0-or-1-arity :args (s/alt :arity-0 (s/fspec :args (s/cat) :ret any?) :arity-1 (s/fspec :args (s/cat :arg1 any?) :ret any?)) :ret any?)
#2016-09-1411:53jetzajacHello here! Iām trying to use spec with a DataScript app. And I want to validate several different things:
1) tx-data in a map form
2) results of a pull
3) entities
In these 3 flavours of data keywords may have different sorts of values. like itās ok to have an int as a value of ref attr in tx-data, not in a pull. Itās ok to miss some attrs in pull result, not in a tx-data with negative id (creating a new entity, we have a notion of required attrs).
And the question is if we can have different spaces of specs? like separate registries and somehow refer them in spec and make it narrow down to this space. Say, space of tx-data and pulls that have to be validated differently. Not sure if I described the demand clear, bit hope I could find somebody who does these kind of things or be clarified why itās impossible or is a bad idea. Thanx.#2016-09-1412:04jetzajacnot sure if I like the idea to have one (private) atom for registry. why not a dynamic binding?#2016-09-1412:16jmglov@bahulneel @otfrom This is what I'm using in tests so far:
(defn- check [function num-tests]
(if num-tests
(stest/check function {:clojure.spec.test.check/opts {:num-tests num-tests}})
(stest/check function)))
(defn checking
([function]
(checking function nil))
([function num-tests]
(testing "Schema"
(let [result (-> (check function num-tests)
first
:clojure.spec.test.check/ret
:result)]
(if (true? result)
(is result)
(is (= {} (ex-data result))))))))
(defn fails-spec [f & args]
(try
(let [res (apply f args)]
(is (instance? Exception res)))
(catch Exception e
(is (contains? (ex-data e) :clojure.spec/problems)))))
#2016-09-1412:20jmglovAnd then tests can look like this:
(deftest plus
(checking 'foo/plus)
(testing "Adding 0"
(is (= 1 (foo/plus 1 0))))
(testing "Invalid input"
(fails-spec foo/plus 1 :forty-two)))
#2016-09-1412:27jmglovIs there an "any" spec?#2016-09-1412:27bahulneel@jmglov I've found that i need to extend the (if (true? result) ...
to
(cond
(true? result) (t/is result)
(nil? (ex-data result)) (t/is (= {} result))
:else (t/is (= {} (ex-data result))))
#2016-09-1412:28jmglov@bahulneel Thanks!#2016-09-1412:29jmglovI see that @esp1 pointed out that :ret
is required for fdef
now, and I'm trying to spec a function for which the return value is effectively Unit.#2016-09-1412:30jmglovSo I don't care what it returns, only that spec is happy with it. š#2016-09-1412:31bahulneel@jmglov What about a predicate like (constantly true)#2016-09-1412:31bahulneel(def any? (constantly true))
#2016-09-1412:35jmglovProbably shouldn't call it any?
, though. š#2016-09-1412:35jmglovuser> any?
#function[clojure.core/any?]
#2016-09-1412:35bahulneelyeah, that was more for illustration, best to make it explicit and not hide behind a name#2016-09-1412:36jmglovI actually like naming it. I think that makes the intention quite clear. I'll probably just call it unit
.#2016-09-1412:36jmglovClear enough to me.#2016-09-1412:36bahulneelsure#2016-09-1412:38jmglovThanks for that! It's a nice, simple solution.#2016-09-1412:41bahulneel@jmglov no problem#2016-09-1412:42bahulneel@jmglov any plans to throw that testing code into a little library to house test helpers?#2016-09-1412:43jmglov@bahulneel I put it in its own namespace, but I'm not planning to lib it up, no.#2016-09-1412:43jmglovNot sure the Clojure community needs yet another library of utility functions. š#2016-09-1412:44jmglovI figure best practises will emerge as more people start mixing specs into their tests.#2016-09-1412:44bahulneelfair enough. Maybe a GIST would do the trick, at least then it's updatable and discoverable#2016-09-1412:45jmglovI'll stick it in a Gist.#2016-09-1412:45jmglovHaha, great minds think alike.#2016-09-1412:45jmglovI'll do that.#2016-09-1412:47bahulneelalso if you put a namespace declaration at the top then, in theory, you could include it as a git submodule in your source tree#2016-09-1412:55jmglov@bahulneel Here it is: https://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29#2016-09-1413:08bahulneel@jmglov thanks, I've added it as a submodule and then added the directory to my test-paths#2016-09-1413:08jmglovNinja!#2016-09-1413:08jmglovPlease ping me as you find improvements.#2016-09-1413:09bahulneelwill do#2016-09-1413:21madstapThere is already a spec utils library that @gfredericks made... https://github.com/gfredericks/schpec#2016-09-1413:22madstapJust throwing that out there, maybe useful to collect things like these in one place#2016-09-1413:40jmglov@madstap Thanks for the heads up!#2016-09-1413:40jmglovhttps://github.com/gfredericks/schpec/issues/14#2016-09-1414:52bahulneel@jmglov I noticed that you need to require [clojure.test.check.clojure-test] so that it's loded as some useful multimethods are added#2016-09-1416:01kkruitIs there any reason to not use instrument for validation if the behavior I want is for the function to throw an error if the data passed is invalid?#2016-09-1416:01kkruit(in production)#2016-09-1416:02Alex Miller (Clojure team)no (as long as it meets your perf concerns)#2016-09-1416:04kkruitI'm thinking it wouldn't be less performant than doing a verify?, throw, and explain, would it?#2016-09-1416:06kkruitI can run some tests...#2016-09-1416:09Alex Miller (Clojure team)with alpha12, itās pretty fast but test it and see#2016-09-1416:09kkruitWill do. Thanks!#2016-09-1416:20kkruitand by verify? i meant valid?...#2016-09-1421:40kkruitSo I'm looking at using spec in place of schema for a rest api and am running into a question about the best way to handle something. I have something like this:#2016-09-1421:43kkruitI like using :ns/map-key in combination with :req-un like that but I don't like how long the namespace is/could get any thoughts? Am i shoehorning :ns/map-key somewhere where it shouldn't go?#2016-09-1421:44arohner@kkruit you can alias namespaces using :require#2016-09-1421:45arohner(:require [my.controller.spec.update :as c]) (s/def ::c/body ā¦))
#2016-09-1421:46kkruitdoh! I was trying to do:#2016-09-1421:46kkruit(:require [my.controller.spec.update :as c]) (s/def :c/body ā¦))#2016-09-1421:47kkruitThanks @arohner#2016-09-1421:48arohnerthat would look for an unaliased namespace named āc, rather than resolving the alias āc#2016-09-1421:48arohnernp#2016-09-1422:44stuartmitchellIs this a known issue
(def common-attributes [::id ::type ::from-date ::to-date ::style])
(s/def ::common
;; defines the common parts of an activity
(s/keys :req-un common-attributes))
Gives this compiler error
---- Could not Analyze src/day8/plan8/spec/activity.cljs line:24 column:3 ----
Don't know how to create ISeq from: clojure.lang.Symbol
22 (s/def ::common
23 ;; defines the common parts of an activity
24 (s/keys :req-un common-attributes))
^--- Don't know how to create ISeq from: clojure.lang.Symbol
25
#2016-09-1422:45seancorfields/keys
is a macro so it doesnāt evaluate its arguments (so this is by design)#2016-09-1422:46stuartmitchellstrangely this works
(def common-attributes [::id ::type ::from-date ::to-date ::style])
(s/def ::common
;; defines the common parts of an activity
(s/keys :req-un (concat common-attributes)))
#2016-09-1422:46stuartmitchellbut will it fail later on?#2016-09-1422:46seancorfieldIt does not do what you think it does.#2016-09-1422:47seancorfield(Iām a bit surprised it conforms to specās specs ā are you trying this on Alpha 12 or an earlier version?)#2016-09-1422:49stuartmitchell1.8.0
I'll bump it up#2016-09-1422:51stuartmitchellbut how should we compose specs i.e. I want spec ::b
to be spec ::a
with some extra keys#2016-09-1422:51stuartmitchellthis is what I wanted
(def common-attributes [::id ::type ::from-date ::to-date ::style])
(s/def ::common
;; defines the common parts of an activity
(s/keys :req-un common-attributes))
(s/def ::tv
;; defines the attributes for a tv activity
(s/keys :req-un (concat common-attributes [::daymask ::tarp ::cpt ::rate
::market])))
#2016-09-1422:54seancorfieldOh, so youāre using the backport of clojure.spec
rather than the official version?#2016-09-1422:55seancorfieldYou want clojure.spec/merge
for merging key specs.#2016-09-1422:55stuartmitchellahh thank you#2016-09-1422:55seancorfieldboot.user=> (doc s/merge)
-------------------------
clojure.spec/merge
([& pred-forms])
Macro
Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs. Unlike 'and', merge can generate maps satisfying the
union of the predicates.
#2016-09-1422:56seancorfield(note: also a macro)#2016-09-1423:01stuartmitchellThx, I also tried the concat
version on 1.9.0-alpha12
and it didn't give a compiler error however I am using clojurescript as well so someone should probably try it on vanilla java, if indeed it should fail spec's specs#2016-09-1423:12Alex Miller (Clojure team)there are no included spec specs#2016-09-1423:13Alex Miller (Clojure team)Iāve made some but they are a bit out of date and unclear whether we will release them. there are some recursion issues with checking specs in spec too. :)#2016-09-1423:32seancorfield@alexmiller Am I right that (s/keys :req-un (concat common-attributes))
is going to create a spec where the two required keys are the symbols concat
and common-attributes
? And that will error out when usedā¦?#2016-09-1500:21kennyCan you set the :default
method for a multi-spec? Just adding a multimethod with :default
for the dispatch value results in no method
when calling explain
. Small example:
(defmulti event-type :event/type)
(defmethod event-type :event/search [_]
(s/keys :req [:event/type :event/timestamp :search/url]))
(defmethod event-type :default [_]
(s/map-of any? any?))
(s/def :event/event (s/multi-spec event-type :event/type))
(s/explain-data :event/event
{:event/type :foo
:event/timestamp 1463970123000
:search/url ""})
=> #:clojure.spec{:problems [{:path [:foo], :pred event-type, :val {:event/type :foo, :event/timestamp 1463970123000, :search/url ""}, :reason "no method", :via [:event/event], :in []}]}
#2016-09-1500:30hiredmanit looks like no, because multi-spec re-implements the multimethod dispatch logic without :default#2016-09-1500:34hiredmanI think https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L888 would need a (or (.-defaultDispatchValue mm) ...)
#2016-09-1500:38hiredmanI suppose the default stuff would be tricky to handle well for generative testing#2016-09-1500:51kennyIs there reasoning behind this?#2016-09-1500:54kennyBehind not handling :default
in defmethod
^#2016-09-1500:54kennyOr has it just not been discussed?#2016-09-1501:09Alex Miller (Clojure team)@seancorfield that would be my guess too#2016-09-1501:10Alex Miller (Clojure team)@kenny if :default doesnāt work, it should be made to#2016-09-1501:10Alex Miller (Clojure team)at least as far as my thinking atm, so jira appreciated#2016-09-1501:10Alex Miller (Clojure team)I did put a patch in alpha12 to make hierarchies work again, but I donāt recall testing :default stuff#2016-09-1501:12kennyOh shoot, I'm still on alpha11 because of the hash changes. I'll test it out in alpha12 and if it doesn't work I'll file a jira issue.#2016-09-1501:15kenny@alexmiller Yes it is fixed in alpha12. Thanks!#2016-09-1506:11curlyfryHow "safe" am I in using spec in ClojureScript? Is it subject to any large changes in the future?#2016-09-1509:43jmglovAny tips for writing a spec for the string version of a positive int? Here's what I'm doing now:
(defn- pos-int-string? [s]
(try
(pos-int? (Integer/parseInt s))
(catch Exception _
false)))
(s/def ::id pos-int-string?)
#2016-09-1509:43jmglovI assume that will need a custom generator though, right?#2016-09-1509:43jmglovAny more idiomatic way to do it?#2016-09-1511:08flipmokidCould spec be used to parse a vector such as '[1 + 3 * 4] and conform the input in such a way so that operator precedence is preserved?#2016-09-1514:16jmglovThe first time you see a function spec actually catching invalid input data is absolutely wonderful!#2016-09-1514:33Alex Miller (Clojure team)@curlyfry spec is in alpha and subject to change (but probably unlikely to change dramatically, more additive)#2016-09-1514:38jmglovIs there a way to use conform to drop all map keys not explicitly mentioned in an s/keys
spec?#2016-09-1514:39jmglovLet's say I have something like this:
(s/def ::foo string?)
(s/def ::bar int?)
(s/def ::thingy
(s/keys :req-un [::foo]
:opt-un [::bar])
#2016-09-1514:41Alex Miller (Clojure team)you could use something like (s/and (s/keys ā¦) (s/conformer #(select-keys % #{::foo ::bar}))
#2016-09-1514:41jmglovI'd like to do this:
> (s/conform ::thingy {:foo "yup", :bar 42, :baz [1, 2, 3]})
{:foo "yup", :bar 42}
#2016-09-1514:42curlyfry@alexmiller: Thanks!#2016-09-1514:43jmglov@alexmiller Wow! You guys thought of everything!#2016-09-1514:43jmglovBTW, it is possible to run spec.test/check
on a private function; my failure to do so was operator error. š#2016-09-1514:46Alex Miller (Clojure team)No#2016-09-1514:46Alex Miller (Clojure team)Right now at least#2016-09-1514:46Alex Miller (Clojure team)No one has filed an enhancement for that yeah afaik#2016-09-1514:47Alex Miller (Clojure team)I'm not sure whether it should be supported or not#2016-09-1514:47jmglovWell, it works. š#2016-09-1514:49jmglov(stest/check 'foo.bar/baz)
is just fine, even if baz
is defn-
'd.#2016-09-1514:50jmglovI personally think it should not be disallowed, since Clojure generally doesn't try too hard to save you from yourself with respect to "private" stuff in a namespace.#2016-09-1514:50jmglove.g. you can happily call a private function as long as you grab its var.#2016-09-1518:02uwohow might I spec a map whose keys I do not know, but whose values all have the same shape?#2016-09-1518:03bfabry@uwo map-of
#2016-09-1518:03uwodoh. thanks#2016-09-1518:03bfabrynp#2016-09-1518:52richiardiandreaI have a little problem with this small test:
(defn all-divisions []
{:test nil})
(s/fdef all-divisions
:args empty?
:ret #(do (log/error "here") (not (data-format/nil-keys? %))))
:: repl
(test/instrument `all-divisions) ;; => [app.db.queries/all-divisions]
(all-divisions) ;; => {:test nil}
Am I spec-ing it right? (I don't see "here" in the logs)#2016-09-1519:17bfabry@richiardiandrea instrument
does not check :ret or :fn#2016-09-1519:17richiardiandreaoh#2016-09-1519:18richiardiandreaonly :args
?#2016-09-1519:18bfabryyup#2016-09-1519:18bfabry:ret and :fn are checked during clojure.spec.test/check checks though#2016-09-1519:18richiardiandreaand what do I need to use to check :ret
at runtime while developing ?#2016-09-1519:18richiardiandreaah ok#2016-09-1519:18richiardiandreatnx @bfabry I did not know that#2016-09-1519:19bfabrynp#2016-09-1520:32richiardiandreaso if I want to instrument all the instrumentable symbols in my project, should I run!
on instrumentable-syms
the function instrument
?#2016-09-1520:33Alex Miller (Clojure team)No just call instrument with no args#2016-09-1520:34Alex Miller (Clojure team)Also be aware of enumerate-ns#2016-09-1520:34richiardiandrea@alexmiller cool, sorry it was right there in the docs#2016-09-1520:35richiardiandreareading through it now#2016-09-1522:17smwoh my godā¦ s/exercise is so cool.#2016-09-1609:40jmglovI have a couple of timestamp string specs that require custom generators. I'm using a function to make the generators with the following function spec:
(s/fdef make-timestamp-gen
:args (s/cat :min-year pos-int?
:max-year pos-int?
:formatter #(instance? DateTimeFormatter))
:ret clojure.test.check.generators/generator?)
I'd really like to spec the function, but that would require adding org.clojure/test.check
to my non-dev dependencies, which feels a little yucky. Is there another way to ensure that the return value of the function is a generator? clojure.spec.gen
doesn't have a generator?
function.#2016-09-1611:34jmglovI have a data structure like this:
{:schema {:version "v1"}
:payload {:type "v1" ...}}
The schema version can either be "v1"
or "v2"
, and the contents of the payload vary based on this. Is there any good way to write a custom generator that will match up version and payload? I can write the conformer reasonably easily, but I can't think of a good way to do the generator.#2016-09-1611:57jmglovOK, I came up with something that feels only mildly hackish:
(defn- basic-gen []
(->> (s/gen (s/keys :req-un [::schema]))
(gen/fmap (fn [{{:keys [version]} :schema :as event}]
(let [payload-spec (payload-spec-for-schema version)]
(assoc event :payload (gen/generate (s/gen payload-spec))))))))
(s/def ::event (s/with-gen
(s/keys :req-un [::schema
::payload])
basic-gen))
#2016-09-1611:58jmglovIs this OK, or is there a better way?#2016-09-1613:17jmglovWriting a custom generator / conformer pair has led me down a merry path of learning a lot more about spec.#2016-09-1613:17jmglovI have what I hope is the ultimate question in this particular journey:#2016-09-1613:20jmglovGiven a custom conformer, how to I make the s/explain
output meaningful? At the moment, my conformer just says:
user> (s/conform ::event intentionally-bogus-event)
:clojure.spec/invalid
user> (s/explain ::event intentionally-bogus-event)
val: :kpcs.spec.event/basic fails predicate: :clojure.spec/unknown
#2016-09-1616:09hiredmanre: the v1 vs v2 thing, that sounds sort of like what multi-spec does#2016-09-1616:21hiredmanre: :clojure.spec/unknown, my guess is it is something related to the Specize protocol, the fall through Object case for that protocol uses unknown as the name for the predicate#2016-09-1616:38jmglov@hiredman I remember reading about multi-spec now that you bring it up. I think you're right there!#2016-09-1616:39jmglovAny clue what I would need to do to solve the :clojure.spec/unknown
problem?#2016-09-1616:40jmglovOr maybe multi-spec
would sort that for me anyway. š#2016-09-1616:40hiredmanyeah, maybe#2016-09-1616:42Alex Miller (Clojure team)@jmglov thatās a known issue that Rich and I are working on#2016-09-1616:42Alex Miller (Clojure team)just hasnāt been finished yet#2016-09-1616:42jmglovNice!#2016-09-1617:17uwoIāve instrumented the following spec and it catches errors except for the optional parameter. Any advice?
(s/def ::skip integer?)
(s/def ::limit integer?)
(s/def ::request-options (s/keys :un-opt [::skip ::limit]))
(s/fdef request!
:args (s/cat :field-path ::field-path
:query-data ::query-data
:options (s/? ::request-options)))
#2016-09-1617:18uwoI can provide any value for skip/limit and it wonāt complain#2016-09-1617:20bfabry@uwo :opt-un not :un-opt#2016-09-1617:20uwojeeze, Iām on a roll with stupid questions. thanks!#2016-09-1617:21bfabryhaha it's all good. it'd be nice if spec caught those, but I imagine having spec spec'd would actually be a rather difficult problem#2016-09-1618:57Alex Miller (Clojure team)I have spec specs but using them presents some problems#2016-09-1619:46liamddoes anyone know of any guides that go beyond explaining how spec works and show how to integrate it into a real project?#2016-09-1619:53hiredmanyou know, there isn't a non-alpha version of clojure released with spec yet#2016-09-1619:54liamdi know, but how is it intended to be used? what do i do with my specs in an actual codebase?#2016-09-1619:55hiredmanlike where to put specs in relation to what they are specing?#2016-09-1619:56liamdyeah i found this
(defn person-name
[person]
{:pre [(s/valid? ::person person)]
:post [(s/valid? string? %)]}
(str (::first-name person) " " (::last-name person)))
#2016-09-1619:57liamdand i see the instrument
function#2016-09-1619:58liamdis the idea that you have them as like accessories to your test suite or that they would run on deployed code?#2016-09-1619:58hiredmanI think spec can be split in to 1. descriptions of data 2. ways to use those descriptions#2016-09-1619:58liamdyeah so 2 is where i'm foggy#2016-09-1619:59hiredmanright#2016-09-1619:59hiredman#2 can be basically anything, but spec provides a few things out of the box#2016-09-1620:00hiredmana. generating data that matches the description b. checking if data matches the description#2016-09-1620:00hiredmanand it actually does b. in a few different ways#2016-09-1620:01hiredmanone way it can check that data matches is you can call instrument#2016-09-1620:01hiredmanand instrument will instrument your functions with checks (and some of those checks rely on generating data, so you really shouldn't instrument out side of testing)#2016-09-1620:02hiredmananother way is using valid? like in that person-name function#2016-09-1620:07hiredmaninstrument is very similar to sort of the classic idea of what you can do with assertions, assert all these expensive pre and post condition checks in development, then compile with assertions turned off for production#2016-09-1620:07hiredman(not that I have ever seen a clojure build that turns off assertions, but I am sure they exist)#2016-09-1620:08seancorfield@liamd I think itās mostly too early for standard patterns of usage to have settled down (and I donāt think thereās One True Way to use specs anyway). Have you read the http://clojure.org Guide? And watched Stu Hallowayās webcast demos of it? Also the Cognicast with Rich talking about the motivations for it is good listening.#2016-09-1620:08liamdi've listened to the cognicast and the generative testing was what sounded most useful to me#2016-09-1620:08liamdand yeah i've read the guide#2016-09-1620:08liamdi'll check out the webcast#2016-09-1620:08bfabryas an aside, instrument
doesn't do what would normally be considered :post checks. It only checks the :args section of an fdef#2016-09-1620:08liamdmaybe it's because i'm thinking of spec as an answer to the lack of typing? are those related concerns?#2016-09-1620:09seancorfieldAt World Singles, weāre specāing data primarily and using conform
as part of our validation and transformation across application layer boundaries. Weāre using instrument
to augment our testing. Weāre starting to work with the generative side more at this point but we havenāt settled into a good flow for it yet.#2016-09-1620:10hiredmanmaybe more like a database schema than what you would typically think of as a type#2016-09-1620:10seancorfield@liamd Rich is pretty clear that spec != type system and should not be viewed the same way#2016-09-1620:12liamdso basically my thinking is:
- in a statically typed system you can make some assumptions about your passed in args since they wouldn't compile otherwise
- without static typing we can just make those assumptions anyway but we might have to debug and get ugly errors down the line
- or we could do some defensive programming and handle incorrect args gracefully (by some standard, a nice error or whatever)#2016-09-1620:12richiardiandreaA side question: how folks use exercise
?#2016-09-1620:12bfabryit's not a type system, but it's clearly geared at solving some of the same problems in a different way, as well as solving some problems that type systems don't. documentation, correctness, better errors#2016-09-1620:12liamdso does spec just make that third point easier? do a conform and then handle malformed inputs in every function without have to rewrite ugly validation logic everywhere#2016-09-1620:13shaun-mahood@liamd: 2 interesting resources for the intersection of spec and typing
https://www.indiegogo.com/projects/typed-clojure-clojure-spec-auto-annotations#/
https://github.com/arohner/spectrum#2016-09-1620:13bfabry@liamd instrument
and clojure.spec.test
are attempts to make us more confident about point 2#2016-09-1620:13hiredmanI would say no#2016-09-1620:14hiredmanif you have a system where you are calling conform in every function on all your arguments, something has gone way wrong#2016-09-1620:14seancorfield@liamd Weāre finding spec to be a better way to write our existing validation and transformation code since it abstracts out the specification part "as data". We donāt view it as "defensive programming".#2016-09-1620:14liamdsomething like:
(let [my-x (s/conform my-x-spec x)]
(if (= my-x :clojure.spec/invalid)
(do-something)
(proceed)))
#2016-09-1620:15liamd@shaun-mahood i'll take a look#2016-09-1620:15liamdthanks!#2016-09-1620:15hiredmanlike, where are your functions getting passed all these malformed inputs from?#2016-09-1620:15liamdit's a wild world out there#2016-09-1620:15hiredmanat an edge, like a rest api, sure#2016-09-1620:16hiredmanif all your functions are exposed an all passed arbitrary input from the outside world, sure#2016-09-1620:16liamdi was just thinking "couldn't we try to analyze the specs at compile time" and then i clicked spectrum#2016-09-1620:17liamdlooks neat#2016-09-1620:17liamdbut i mean isn't that the strength of type systems#2016-09-1620:17hiredmantype systems are going to need help at the edge too#2016-09-1620:17liamdwe can make assumptions about the data we're handling#2016-09-1620:18liamdright, parsing json in any type system is sticky#2016-09-1620:18hiredmantype systems can only analyze data they see, random data coming from the edge isn't going to be seen at compile time#2016-09-1620:19arohner@liamd no, spectrum canāt handle that. It can prove that you havenāt validated the data though#2016-09-1620:19seancorfieldSpec can also enforce data "structure" in a way that type systems cannot. It can specify valid ranges of numbers, relationships between parts of a data structure, patterns in strings and so on.#2016-09-1620:20arohner@seancorfield there are workarounds for that in spectrum, in some cases#2016-09-1620:20hiredman@seancorfield: that is just opening the door and waiting for someone to step in and start talking about dependent types#2016-09-1620:20arohnerbecause spectrum can assert that the spec gets validated at runtime#2016-09-1620:20bfabry@liamd spec isn't going to start down the path of compile time checking other than for macros. at the end of the day clojure is a dynamic language. spec is an attempt to give us stronger confidence about what we're passing around at test and dev time using instrument
and check
. It also includes tools to validating the shape of things at runtime in production but imo I think that's more meant for things that are at high risk of being invalid (boundaries) rather than things that are likely fine#2016-09-1620:20seancorfield@hiredman Hahahaā¦ trueā¦ but we donāt have those in very many languages and almost none that are in common production usage!#2016-09-1620:21seancorfieldBesides, if folks want a type system, then go use a statically typed language. Clojure is not that language.#2016-09-1620:21hiredmanI think the big thing about spec is the language of terms (regular clojure expressions) and the language of descriptions (specs) are the same, so you can easily get access to specs at runtime, and write some interpreter for them to do whatever you want#2016-09-1620:22liamdi see. so it should be though about as "a set of tools that helps mitigate the downsides of a dynamic language during your dev work flow"#2016-09-1620:22seancorfieldI donāt see those as "downsides" š#2016-09-1620:22hiredmancalling them downsides reflects a particular stance that is not universal#2016-09-1620:23liamdwell surely spec was developed as a solution to some perceived problem?#2016-09-1620:23liamdmaybe i'm wrong in thinking that that problem is rooted in lack of typing#2016-09-1620:23bfabryit also gives you more confidence than any static type systems I'm aware of because predicates can be arbitrary#2016-09-1620:23richiardiandreaA very nice talk about this was the last one by Felleisen at Clojure/West#2016-09-1620:24hiredman@liamd: static typing is a solution to a similar set of problems#2016-09-1620:25hiredmanbut it is sort of like saying you have a problem P, and solutions A and B, and saying solution B is there to fix a lack of solution A#2016-09-1620:25hiredmanno, B is there to fix P#2016-09-1620:25arohnerthereās significant overlap between spec and static types, but neither completely subsumes the other#2016-09-1620:26arohnerthere are bugs that can be caught by static types that are hard to catch w/ spec, and vice versa#2016-09-1620:26bfabrystatic type systems give a lot of confidence about correct calls (probably more than spec) but provide no tools for regular data validation. they're also limited in what they can describe regarding correctness#2016-09-1620:28hiredmanin most typed languages, the language of terms and the language of types are distinct#2016-09-1620:29liamdi see#2016-09-1620:29hiredmantypes end up being kind of their own langauge that exists largely at compile time#2016-09-1620:29hiredmanspecs are more like a database schema, they exist in the system and you can poke and fiddle with them live#2016-09-1620:29liamdso due to them being just plain old clojure we can ingest the specs and do something with them#2016-09-1620:30liamdis there a way to get the spec of something at run time?#2016-09-1620:30hiredmanthings don't inherently have a spec#2016-09-1620:30liamdright#2016-09-1620:30hiredman(another different from types)#2016-09-1620:30arohneryes, you can get spec definitions at runtime#2016-09-1620:30liamdwe just have things and specs and we can smash them into each otehr if we want#2016-09-1620:31arohner(s/form (s/spec ::foo))#2016-09-1620:31liamdwould it make sense to use specs as part of your logic? like if this conforms to my Person spec do one thing else do some other thing#2016-09-1620:31bfabrytotally#2016-09-1620:32liamdor i could see in web dev using them to validate incoming json#2016-09-1620:32bfabryconvenient dsl for expressing that kind of logic#2016-09-1620:32liamdit'd be nice if explain
could map back to a json error#2016-09-1620:33liamdi don't know if api consumers want to deal with ;; val: 42 fails spec: ::suit predicate: #{:spade :heart :diamond :club}
#2016-09-1620:34arohners/explain-data
#2016-09-1620:34liamdah, there it is#2016-09-1620:34liamdvery nice#2016-09-1620:50seancorfield@liamd As I noted above, at World Singles, we are using spec to replace our custom validation and transformation logic so we can have separate, comprehensive specifications we can show to and discuss with product owners / stakeholders, and it removes a lot of bespoke code.#2016-09-1620:51seancorfieldWe are actually using spec for three layers: input, domain, persistence and conforming between them. The validate-and-conform aspect of spec is very nice.#2016-09-1620:53liamdis that running only during dev or when you deploy?#2016-09-1621:06seancorfieldItās a core part of our production code, calling conform
.#2016-09-1621:08seancorfieldWe also use instrument
in testing, and weāre using some generative testing too (`check`). As I said above, we havenāt yet settled onto a final workflow for the latter since generative testing can be slow and we want "unit tests" to be fast (so we can run them easily while writing code).#2016-09-1714:26petterikTwo talks on clojure.spec have just been uploaded to youtube:
Strange Loop 2016: Stuart Halloway - "Agility & Robustness: Clojure spec"
https://www.youtube.com/watch?v=VNTQ-M_uSo8
ClojuTRE 2016: Arne Brasseur - "Introduction to clojure.spec"
https://www.youtube.com/watch?v=-MeOPF94LhI#2016-09-1720:04surreal.analysisChicago Clojure also recently had a meeting on Spec, which was posted to Youtube a couple weeks ago - https://vimeo.com/181231654#2016-09-1814:01wilkesCan someone post the slides for Chelimskyās intro?#2016-09-1819:11lvhHm; are there any specs somewhere that someone is collecting for datomic/datascript datoms?#2016-09-1822:47Oliver GeorgeHere's a quick script which generates simple clojure.spec definitions for a database. https://gist.github.com/olivergeorge/468464ce82b8da486736fe725a4b6ff8#2016-09-1822:48Oliver GeorgeNothing particularly magic in there. JDBC makes it easy to inspect types for table columns.#2016-09-1822:49Oliver GeorgeAn interesting extension would be to interpret relations and generate optional keys#2016-09-1822:50Oliver George(My use case is generating dummy data for a re-frame UI which is fed from a pull api (which in turn mirrors a db schema)#2016-09-1906:08jasonjcknis there an inverse of s/conform ?#2016-09-1906:10jasonjcknfound it#2016-09-1908:21jeroenvandijk@jasonjckn yes, itās s/unform
#2016-09-1908:22jeroenvandijk(itās not perfect yet, see http://dev.clojure.org/jira/browse/CLJ-2021?focusedCommentId=43842#comment-43842)#2016-09-1917:28jasonjckn@jeroenvandijk thanks! I ran head smack right into that issue yesterday š#2016-09-1917:28jasonjckn@jeroenvandijk but used s/conformer to get around it#2016-09-1917:28Alex Miller (Clojure team)btw, that issue is imo a problem with conform, not unform :)#2016-09-1917:29jasonjcknaside from that issue is making macro writing a joy#2016-09-1917:29Alex Miller (Clojure team)are you using conform to destructure input?#2016-09-1917:29jasonjcknyes#2016-09-1917:29Alex Miller (Clojure team)I havenāt seen too many people doing that yet but our presumption has been that it would remove a lot of fragile gross code#2016-09-1917:29jasonjckn(let [ast (s/conform ::defui forms)
ast2 (ast-transform ast)
out (s/unform ::defui ast2)
#2016-09-1917:29jasonjckntotal joy š#2016-09-1917:30Alex Miller (Clojure team)cool, thatās great to hear about#2016-09-1917:30jasonjcknhere's my ast transformer still toying with this code (defn ast-transform [ast]
(letfn [(walk-fn [xf]
(-> xf
(match->
{:protocol (_ :guard #(static-symbs (name %)))}
(assoc :static? 'static)
{:method-name 'render :args [this om-props] :body body}
(assoc :args [this]
:body `[(let [~(s/unform ::cs/binding-form om-props)
(om.next/props ~(s/unform ::cs/binding-form this))]
#2016-09-1917:30jasonjcknthe pattern is walking + match->#2016-09-1917:30jasonjcknI implemented match->#2016-09-1917:32Alex Miller (Clojure team)cool#2016-09-1917:38patrkrisIs clojure.spec a suitable tool for validating JSON formatted HTTP request bodies? I am asking because the pervasive use of namespaced keywords specs seems to conflict with that#2016-09-1917:39patrkrisI mean using namespaced keywords as object keys in JSON seems unnatural#2016-09-1917:50Alex Miller (Clojure team)you can use s/keys
with :req-un
and :opt-un
for that#2016-09-1917:51Alex Miller (Clojure team)many people are and itās been discussed a number of times here - you might find it in the back chat#2016-09-1917:52patrkrisThat was what I thought about doing myself. But then I started thinking about nested maps and whether that would end up posing a problem. Probably not š
You mentioned some time ago that you were thinking about adding a way to "rename" keys in s/keys. Any development on that?#2016-09-1917:54Alex Miller (Clojure team)nope. up to Rich whether that idea comes back, so Iām not sure if it will.#2016-09-1921:28liamdiām not able to import clojure.spec.gen into my clojurescript project? any idea why?#2016-09-1921:29liamdNo such namespace: clojure.spec.gen, could not locate clojure/spec/gen.cljs, clojure/spec/gen.cljc, or Closure namespace āclojure.spec.genā
#2016-09-1921:32ggaillardHi @liamd, according to the current clojure.spec guide, it seems you need to add [org.clojure/test.check "0.9.0"]
to your :dev
profile in your project.clj
:profiles {:dev {:dependencies [[org.clojure/test.check "0.9.0"]]}}
Or with boot
(set-env!
:dependencies '[[org.clojure/test.check "0.9.0" :scope "test"]])
Did it solve your problem ?#2016-09-1921:32hiredmanlooking at the clojurescript repo, that namespace doesn't exist#2016-09-1921:33hiredmanit doesn't seem to have been ported#2016-09-1921:33liamdi did add test.check to my dependencies#2016-09-1921:34liamdnot under dev for now, but that shouldnāt matter right?#2016-09-1921:34ggaillardHo my bad, sorry. Try with [cljs.spec.impl.gen :as gen]
#2016-09-1921:37ggaillardMaybe more information here : http://dev.clojure.org/jira/browse/CLJS-1696#2016-09-1921:38liamdthat seems to have gotten rid of the import error, but iām still not sure which things to use to get spec generation working#2016-09-1921:39liamdnow itās
clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required
#2016-09-1921:39liamdso looks like i need to require clojure.test.check.generators#2016-09-1921:39liamdah got it!#2016-09-1921:43ggaillardYes same for me, sometime I need cljs.spec.impl.gen
, sometime clojure.test.check.generators
. But this code
(gen/sample (s/gen int?))
effectively require clojure.test.check.generators :as gen
and cljs.spec :as s
#2016-09-2000:27Oliver GeorgeThis is fun. Using a pull-query to generate specs means I can generate test data for UI views based on the pull query they use.
https://gist.github.com/olivergeorge/da6487fc60c67b79ba082bc807072e43
This still requires table and field specs based on something like this:
https://gist.github.com/olivergeorge/468464ce82b8da486736fe725a4b6ff8#2016-09-2000:29danielcomptonIs there a JIRA issue open for spec coercions (in the same way that schema does them)? Iām aware of conform, but that isnāt quite the same#2016-09-2001:20Alex Miller (Clojure team)there is no plan to add that#2016-09-2001:20Alex Miller (Clojure team)Rich sees them as separate concerns#2016-09-2008:00otfromI'm presuming I'm doing something wrong and that there is a function in core that will do this, but this bit of code is working for me to integrate spec tests into my normal clojure.test flow#2016-09-2008:00otfrom(defn passes? [sym]
(-> sym
stest/check
first
:clojure.spec.test.check/ret
:result))
(t/deftest conformer-testing
(t/testing "Checking the conformer"
(t/is (true? (passes? `sut/my-specced-func)))))
#2016-09-2008:01otfromI've elided the bit where I define the :args :ret and :fn for my-specced-func#2016-09-2008:02otfromis there something that does the same as passes?
and is true?
really the right way of putting this into t/is ? Success and failure both return truthy values which is why I'm using it atm#2016-09-2008:11jmglov@otfrom Here's what @bahulneel and I came up with: https://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29#2016-09-2008:22jmglovI have a record that implements a protocol. How do I spec this?#2016-09-2008:53jmglovWhen I try to gen/sample
some deeply nested spec, I get a "Couldn't satisfy such-that predicate after 100 tries." exception with no mention of the spec or predicate in question. Is there a good way to figure out which spec caused this?#2016-09-2008:56jmglovSo far, my only hope is a binary search by running gen/sample
on the sub-specs until I find the offender. š#2016-09-2009:04bahulneel@jmglov I had this problem also. I had to eyeball my specs and try exercising individual ones. In the end, it's usually ones that use and
and limit the valid input to some "shape". To fix these I use with gen, an example I stole is here: https://github.com/nwjsmith/datomic-spec/blob/master/src/datomic_spec.clj#L6#2016-09-2009:05jmglov@bahulneel Yes, that was exactly it. My offending one turned out to be (s/and map? empty?)
. I changed it to #{{}}
. š#2016-09-2009:06bahulneelnice#2016-09-2009:09bahulneel@jmglov it's worth noting that, in that case (s/and empty? map?)
will also work fine#2016-09-2009:09jmglovOf course! That's actually nicer to read, I think.#2016-09-2009:09jmglovThanks!#2016-09-2009:10bahulneelthe only issue is that the explanation would fail empty? before map? so the message may not be as useful#2016-09-2009:11bahulneeluser> (clojure.spec/explain (clojure.spec/and empty? map?) [1])
val: [1] fails predicate: empty?
nil
user> (clojure.spec/explain (clojure.spec/and empty? map?) [])
val: [] fails predicate: map?
nil
user> (clojure.spec/explain (clojure.spec/and empty? map?) {1 2})
val: {1 2} fails predicate: empty?
nil
user>
#2016-09-2009:12bahulneelBut it's better than the alternative:
user> (clojure.spec/explain #{{}} {1 2})
val: {1 2} fails predicate: :clojure.spec/unknown
nil
user> (clojure.spec/explain #{{}} [])
val: [] fails predicate: :clojure.spec/unknown
nil
user>
#2016-09-2009:14bahulneelThis works better:
user> (clojure.spec/def ::empty-map #{{}})
:user/empty-map
user> (clojure.spec/explain ::empty-map {1 2})
val: {1 2} fails spec: :user/empty-map predicate: #{{}}
nil
user>
#2016-09-2009:15bahulneel(also naming (s/and empty? map?)
gives a similar level of clarity)#2016-09-2009:23mpenetany known attempt to generate json-schema from specs?#2016-09-2009:24mpenetthere's a ring-swagger fork that tried, but it's not there yet#2016-09-2009:29ikitommi@mpenet would we easy with records, https://groups.google.com/forum/m/#!topic/clojure-dev/9EjTvxPPaGQ#2016-09-2009:39mpenetI am not really interested in the conform part right now#2016-09-2009:39mpenetI just need generating decent json-schemas from specs#2016-09-2009:42jmglov@bahulneel Nice!#2016-09-2010:03ikitommiok, not there yet. It would be easy to cook up a working prototype of the conversion from existing spikes, but still gaps to make it robust and extendable. Good thing that, there are smarter people here :)#2016-09-2010:05ikitommiSpectrums parse-spec might be a good way for it.#2016-09-2010:06mpenetIsnt it possible just to gen from s/form?#2016-09-2010:06mpenetIt s a one time thing so perf doesnt matter#2016-09-2010:11ikitommiHmm... there was one version with s/form & core.match, can't recall what was the problem with it. Maybe just not finished.#2016-09-2010:14ikitommiactually, it could be extended too, with multimethod on the spec symbol. Ping @miikka. #2016-09-2011:40mpenetI have something along these lines already, even tho the multimethod is kinda useless ultimately, it can be closed#2016-09-2012:09bhagany@jmglov: regarding protocol implementations - I read (on the mailing list, I think), that directly specāing protocol fns wasnāt supported, and the recommendation was to make the protocol fns āprivateā, provide wrapper fns, and spec those. This worked fine for me, especially because I had some common logic that fit nicely in the wrapper fns.#2016-09-2012:19jmglov@bhagany I'm not trying the spec the protocol functions themselves, but rather an argument to a function that should implement a certain protocol. i.e.
(defprotocol Database
(read-thing [this id])
(store-thing [this thing]))
...
(defn do-stuff [db updates]
(let [thing (database/read-thing (:id updates))]
(-> thing
(merge updates)
database/store-thing)))
I want to spec do-stuff
something like this:
(s/fdef do-stuff
:args (s/cat :db #(implements? Database %)
:updates (s/keys :req [::thing/id]
:opt [::thing/name]))
:ret any?)
Of course implements?
is not a real function, but this illustrates what I'm trying to do, I hope.#2016-09-2012:26mpenetyou can use satisfies?
#2016-09-2012:32jmglov@mpenet Exactly what I needed! Thanks!#2016-09-2012:37mpenetit gets more tricky if you want to add gen support#2016-09-2012:38jmglovNo need for now, thank goodness.#2016-09-2015:29bhaganyIām trying to run āfreeā generative tests in cljs, using doo, but it seems as though the registry is not being populated in my test build. Iām importing a namespace that defines some named specs, and I can run the functions in that namespace, but clojure.spec.test/check
and checkable-syms
are both empty. However, in my figwheel repl, the registry is populated as I would expect. Any ideas what I might be missing?#2016-09-2015:33bhaganyI should also add that clojure.spec.test/enumerate-namespace
does see the fns that are specād in my test build#2016-09-2015:34bhaganyand also that I can (s/valid? ::some/data-spec data)
in my test build. It seems like only fn specs are missing.#2016-09-2015:43husain.mohssenPossibly stupid question: why is s/fdef
not called s/defn
?#2016-09-2016:05Alex Miller (Clojure team)because thatās not what Rich named it#2016-09-2016:17husain.mohssengood enough for me š all hail Master Rich š
(I was asking in case there was some subtlety Iām missing)#2016-09-2019:40liamdhaving a hard time wrapping my head around writing generators, how would i write a generator for a set of distinct strings?#2016-09-2019:40hiredmanthere is a function, it might be called one-of#2016-09-2019:41hiredmanhttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L272-L289#2016-09-2019:41hiredmanoh, sorry, no wrong one#2016-09-2019:42hiredmanhttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L337-L348#2016-09-2019:42hiredmanelements#2016-09-2019:42liamdi tried (gen/fmap set (gen/vector gen/string))
#2016-09-2019:43hiredmanoh, I misunderstood#2016-09-2019:43hiredmanthere is a generator for sets#2016-09-2019:43hiredmanso (gen/set gen/string)
#2016-09-2019:45liamdexcellent that did it#2016-09-2019:45hiredmanyou may need to do some voodoo connect that to spec, I haven't looked too much at how test.check and spec are tied together#2016-09-2019:45liamdi just had to do with-gen
#2016-09-2019:45liamdreally easy#2016-09-2019:48liamdhereās another one: any idea how to generate long strings?#2016-09-2019:48liamdsay > 200 chars?#2016-09-2019:53hiredmanif you run more generative test iterations, the size of the strings will get larger, but it would likely be easier to create a special purpose larger string generator#2016-09-2019:54hiredmanwhich maybe some combination of resized and the string generator would help you do#2016-09-2019:54hiredmanhttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L233-L240#2016-09-2019:55hiredman(gen/set (gen/resized 200 gen/string))
#2016-09-2020:03liamdresized looks good#2016-09-2020:05bfabrythere's sooooo much stuff in the clojure.test.check.generators namespace, trying to internalise all that is the one thing I'm not looking forward to when we start using spec#2016-09-2020:08liamdthe docs donāt mention everything so you gotta dig into the sources#2016-09-2020:08liamdthat said, using spec with devcards is kicking ass right now for building om ui's#2016-09-2020:22Alex Miller (Clojure team)in many cases itās easier to just gen from a spec#2016-09-2020:22Alex Miller (Clojure team)(s/gen (s/coll-of string? :kind set?))
#2016-09-2020:24Alex Miller (Clojure team)@liamd ^^#2016-09-2020:24liamdfor some specs i get {:message "Couldn't satisfy such-that predicate after 100 tries.", :data {}}
#2016-09-2020:24Alex Miller (Clojure team)if you are andāing specs with fairly restrictive filters, youāre likely to run into this#2016-09-2020:25Alex Miller (Clojure team)this is discussed (plus making custom generators) in http://clojure.org/guides/spec if you havenāt seen that yet#2016-09-2020:25liamdyeah iāve read that over, seems the solution is to write your own generators and then use with-gen
which works great#2016-09-2020:26liamdiām already re-using a good amount of generators for things like certain numbers and strings with various properties#2016-09-2020:26liamdone thing is that error wonāt tell me which generator itās talking about#2016-09-2020:26liamdwhich isā¦ confusing#2016-09-2020:26Alex Miller (Clojure team)yes, I think thatās worth improving#2016-09-2020:27Alex Miller (Clojure team)I donāt think anyone has actually filed a ticket about it, but I would be in favor of improving that#2016-09-2020:28liamdis that specific to the cljs implentation or spec in general?#2016-09-2020:29liamdor i guess thatās test.check#2016-09-2020:29Alex Miller (Clojure team)it is, but I suspect it could be wrapped somehow in spec#2016-09-2020:30Alex Miller (Clojure team)havenāt looked at it#2016-09-2020:31liamdyeah i suppose youād want to tie it back to your namespaced key#2016-09-2020:51madstapI'm having a problem with generators.
(def col-gen
(gen/fmap #(->> % (apply str) str/upper-case keyword)
(gen/vector gen/char-alpha)))
(s/def :cell/col (s/and simple-keyword? (comp (set util/alphabet) name)))
This throws an error `java.lang.AssertionError
Assert failed: Arg to vector must be a generator (generator?
generator)`
How can I generate a vector of characters from the alphabet?#2016-09-2020:54madstapOther question, how can I make a generator from a set?#2016-09-2020:56hiredmanwhat namespace is gen?#2016-09-2020:56madstapclojure.spec.gen#2016-09-2020:56hiredmanI bet you need to invoke gen/char-alpha#2016-09-2020:57hiredmanthe things in clojure.spec.gen aren't actually generators, they are 0 argument functions that you invoke to get a generator#2016-09-2020:57madstapThanks, that worked#2016-09-2021:01madstapAnd how can I make a generator from a set?#2016-09-2021:02hiredmanyou have a set of elements and you want a generator to pick elements from it?#2016-09-2021:03madstapYeah#2016-09-2021:03hiredmangen/elements#2016-09-2021:03Alex Miller (Clojure team)or of course just (s/gen #{:a :b :c})
#2016-09-2021:04Alex Miller (Clojure team)people do not leverage gens from specs enough imo#2016-09-2021:04Alex Miller (Clojure team)that should be your first choice#2016-09-2021:04Alex Miller (Clojure team)then stuff in spec.gen, then stuff in test.check#2016-09-2021:05Alex Miller (Clojure team)or test.chuck etc#2016-09-2021:06hiredmanso (s/gen (s/spec number?))
instead of gen/number
or whatever?#2016-09-2021:06Alex Miller (Clojure team)just (s/gen number?)
#2016-09-2021:07Alex Miller (Clojure team)but yes#2016-09-2021:07ggaillard@alexmiller about that case, is it normal for
(clojure.test.check.generators/sample (cljs.spec/gen #{true false}))
to always be
(true true true true true true true true true true)
?#2016-09-2021:07Alex Miller (Clojure team)well any set with falsey elements is going to be a bad time#2016-09-2021:08Alex Miller (Clojure team)thatās kind of a special case - it will return falsey elements which is the signal that the value doesnāt match the spec#2016-09-2021:08Alex Miller (Clojure team)in this case (s/gen boolean?)
will be better#2016-09-2021:08ggaillardOh, I get it !#2016-09-2021:09Alex Miller (Clojure team)and if you want to support nil,true,false then (s/gen (s/nilable boolean?))
#2016-09-2021:12ggaillardThank you !
I had some bad time with (s/gen boolean?)
in ClojureScript, throwing an error. I figured it out with with-gen and (gen/elements #{true false})
. I don't know if it's a known bug or not btwā¦#2016-09-2021:13Alex Miller (Clojure team)sounds like a bug#2016-09-2021:14Alex Miller (Clojure team)you might ask in #clojurescript to see what @dnolen thinks#2016-09-2021:17dnolen@ggaillard it should work, make sure you are on the latest 1.9.229, if it still doesnāt work, file an issue in JIRA#2016-09-2021:19ggaillardYou are quick! I didn't had time to post in #clojurescript š
Thank you, I'll check that.#2016-09-2021:34ggaillardI was using 1.9.216, 1.9.229 fixed it. Sorry for that š#2016-09-2022:56devnThe clojure.spec/nilable
docs say it takes a pred, but it also seems to work if it's passed a spec. Is that behavior I can rely on?#2016-09-2102:22gardnervickersHey folks, if I had a key in a map that I want to constrain to be a function that conforms to a spec, how would I go about that? I thought I would be able to pass the results of s/fdef
to (s/def ::my-key ā¦)
but that does not seem to be the case.#2016-09-2102:25hiredmanfspec#2016-09-2102:27gardnervickersOh jeeze, missed that one thanks#2016-09-2102:27hiredmanI am sort of surprised that the result of fdef didn't work though#2016-09-2102:49Alex Miller (Clojure team)It returns the result of def not fspec#2016-09-2102:49Alex Miller (Clojure team)Fdef is def + fspec#2016-09-2102:49bfabryah, so you could use (s/spec ::my-key) @alexmiller ?#2016-09-2102:50bfabrywait#2016-09-2102:50bfabrythat makes no sense#2016-09-2102:50bfabryso what does fdef define?#2016-09-2102:50Alex Miller (Clojure team)Same as def but the key is a symbol not a keyword#2016-09-2102:51Alex Miller (Clojure team)It registers, not defines#2016-09-2102:51Alex Miller (Clojure team)Or at least that's how I think about it#2016-09-2102:51bfabryoh, so you could use the symbol 'my-function in place of the spec? or at least (s/spec 'my-function)#2016-09-2102:51Alex Miller (Clojure team)Yes - fully qualified though#2016-09-2102:52bfabrymakes sense#2016-09-2102:52Alex Miller (Clojure team)So back tick #2016-09-2102:52hiredmanuser=> (defn foo [x] x)
#'user/foo
user=> (s/fdef foo :args (s/cat :arg any?) :ret any?)
user/foo
user=> (s/def :foo/bar user/foo)
:foo/bar
user=> (s/valid? :foo/bar foo)
true
user=>
#2016-09-2102:52bfabry@hiredman I think that will just use foo as a predicate#2016-09-2102:53hiredmanoh, did I flip that around?#2016-09-2102:53hiredmanno, I think that is right#2016-09-2102:53bfabrylike, (s/def :foo/bar user/foo) == (s/def :foo/bar clojure.core/identity)#2016-09-2102:54hiredmanOh#2016-09-2102:55bfabrybuuut I think (s/def :foo/bar 'user/foo) would use the spec created by fdef#2016-09-2102:56hiredmanright#2016-09-2102:56hiredmanlike regular def the thing being def'ed(registered) is evaluated#2016-09-2102:57hiredmanhah, I can tell that changed it because valid? takes longer to return because of the generative checking of functions#2016-09-2102:59bfabrydamn, so a lot of the core functions take a single arity it turns out, but anyway yes
oot.user=> (s/def :foo/bar 'boot.user/foo)
:foo/bar
boot.user=> (s/valid? :foo/bar foo)
true
boot.user=> (s/valid? :foo/bar clojure.string/replace-first)
false
#2016-09-2113:31kestrel7@alexmiller some feedback. I created a generator that did not conform to the spec (doh!). The generator contained the such-that predicate. When I tried creating a sample from the generator I got this error:
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
I assumed that it referred to my custom generator but that was a red herring because in fact spec must be using such-that to ensure that the generated value conforms to the spec, and it was this such-that that generated the failure, not the one in my custom generator.
Code (with the problem corrected but showing the such-that in my generator:
(defn mod11-checkdigit
"Calculate the checkdigit see "
[n]
(let [x (->> (map #(Integer/parseInt (str %)) (take 9 n))
(map * (range 10 1 -1))
(reduce +))
y (mod x 11)
c (- 11 y)]
(cond (== 10 c) nil
(== 11 c) 0
:else c)))
(def nhs-number-gen
"Generates a valid NHS number"
(gen/fmap #(str (+ (* 10 %) (mod11-checkdigit (str %))))
(gen/such-that #(mod11-checkdigit (str %))
(gen/choose 100000000 999999999))))
(defn nhs-number?
"Returns true if passed a valid nhs number else returns false"
[n]
(and (string? n) (= 10 (count n)) (= (str (mod11-checkdigit n)) (str (last n)))))
(s/def ::nhs-number (s/with-gen nhs-number?
(fn [] nhs-number-gen)))
#2016-09-2113:33kestrel7It would be nicer if the error thrown due to the generated value being non-conformant with the spec stated this.#2016-09-2113:46patrkrisSay I have a namespace with two functions create-component-a
and create-component-b
. Each function takes a map with options. How would I go about spec'ing those option maps in the same namespace as the functions? Let's say that each map takes an :id
(or something else) attribute, but they're semantically different. If I want to use the ::keyword
shortcut to create a namespaced keyword, I can't call them both ::id
. I could create a namespace for each type of option map, alias them and use something like ::a-options/id
and ::b-options/id
, but that seems wrong.#2016-09-2114:10Alex Miller (Clojure team)@kestrel7 you are correct in your assessment - spec doesnāt trust custom generators and re-confirms they match the spec. Agreed that it would be nice to know the cause here more explicitly (kind of a special case of the general desire to know more about which part of a generator failed to generate). Happy to see a jira for enhancement.#2016-09-2114:11Alex Miller (Clojure team)@patrkris your last suggestion is what I would do there (you donāt have to alias of course - you can just use fully-qualified too)#2016-09-2114:11Alex Miller (Clojure team)why does that seem wrong?#2016-09-2114:13patrkris@alexmiller: I don't know. Maybe it shouldn't š. It's probably just from being used to define a schema (with herbert) in the same namespace as the function relying on it. But using the fully qualified version also feels a little wrong, because then you have hardcoded the namespace. It possibly just something I need to get used to#2016-09-2114:15patrkrisI thought about whether allowing the use of "subnamespaces" in aliases would be useful. For instance ::a-options/id
without having an ::a-options
alias. But I can imagine that it would turn out to be problematic.#2016-09-2114:16patrkrisEhr... by "subnamespaces" I meant being able to say ::order.delivery/type
without having a order.delivery
alias created#2016-09-2114:19Alex Miller (Clojure team)well, I donāt think weāll do that :)#2016-09-2114:21patrkrisBut it's not definite no? š I'm kiddin'#2016-09-2114:21kestrel7@alexmiller as requested http://dev.clojure.org/jira/browse/CLJ-2025 - thanks for the great work.#2016-09-2114:22Alex Miller (Clojure team)thanks!#2016-09-2115:21kkruitIs there a way to require that in {:a "a" š "b" :c "c"} :a is required and š or :c is required and still have the other be optional? I've tried this using s/keys but cant figure out how i'd do it and I can't see how i could use s/map-of and still require :a...#2016-09-2115:24minimal(s/def ::mymap (s/keys :req-un [::a (or ::b ::c) ]))
#2016-09-2115:27kkruitoh man! I tried s/or... Thanks!#2016-09-2115:28minimalthe or
is syntax in the keys macro rather than clojure.core/or or spec/or#2016-09-2115:28kkruitoh, I was just thinking. how is that working as clojure.core/or ... guess it's not haha#2016-09-2115:29kkruitthanks again!#2016-09-2115:29minimalI didnāt notice it at first either#2016-09-2118:48sparkofreasonSome functions, like exercise
, allow you to override generators by name. Is there any mechanism to do the same with conformers?#2016-09-2118:49leongrapenthin@alexmiller I think a good solution to aliasing keyword and symbol namespaces not existing in the codebase like :human.programmer/language
would be (defalias hp human.programmer)
. So :hp
would still be :hp
, but ::hp/language
would resolve to :human.programmer/language
. This would overwrite aliases created via require
. Should I write a tickent?#2016-09-2118:49Alex Miller (Clojure team)@dave.dixon no#2016-09-2118:50Alex Miller (Clojure team)@leongrapenthin alias
already exists#2016-09-2118:50Alex Miller (Clojure team)the problem with it is that it requires the ns you are aliasing to exist#2016-09-2118:51leongrapenthinRight#2016-09-2118:51Alex Miller (Clojure team)we are already considering loosening that constraint or auto-creating the ns#2016-09-2118:51Alex Miller (Clojure team)I donāt need a ticket for it#2016-09-2118:51leongrapenthininteresting. auto-creating seems like a good strategy#2016-09-2118:52Alex Miller (Clojure team)that is by far easier given the existing implementation of Namespace etc#2016-09-2118:52leongrapenthinyeah#2016-09-2118:52leongrapenthinI just realized that I could use create-ns in combination with alias#2016-09-2118:53leongrapenthinSo there is a good workaround#2016-09-2118:53Alex Miller (Clojure team)yes that works fine now#2016-09-2118:54leongrapenthinDK about Cljs though#2016-09-2119:17Alex Miller (Clojure team)have you looked at multi-spec?#2016-09-2119:17Alex Miller (Clojure team)has that kind of a feel to me#2016-09-2119:58mandersonI keep running into the same problem where I have two maps that have the same named key, but with different specs. For example: two maps that both have a key called name
, where in the first it is a string?
, but the second is int?
because it is more of an ID. How do I specify this so that the specs can be different, but the field name the same in both maps? Only solution I see is to change name
to something else (like person-name
) and just living with the name change, but that doesn't work especially if modeling a domain I don't control. Thoughts?#2016-09-2120:11bfabry@manderson :opt-un and :req-un in s/keys
#2016-09-2120:13mandersonNot sure I'm following... I still have to define a spec called ::name
in two different ways. Yes, I could use or
, but that muddies the definition since they are actually separate specs.#2016-09-2120:17bfabry@manderson sorry I should've been clearer, define a spec called :my.app.person/name and a spec called :my.app.service/name or whatever makes semantic sense#2016-09-2120:20mandersonOk, so using the namespacing. That makes sense and should work for most of my cases (often translating from a format like JSON where there aren't namespaces). I suppose in the case where namespaces are expected, it would probably already be modeled correctly from the beginning (hopefully). Thanks @bfabry#2016-09-2120:20bfabryno worries#2016-09-2121:40seancorfield@manderson To (hopefully) clarify: if youāre specāing with :req-un
and :opt-un
then you are using simple (non-namespaced) keys in your maps, but you can use a namespaced spec name / key name there to distinguish the two specs.#2016-09-2121:42seancorfield(s/def ::person (s/keys :req-un [:person/name]))
(s/def ::service (s/keys :req-un [:service/name]))
Both person and service specs are maps with a required :name
key, but the specs used to validate those keysā values will be different for ::person
and ::service
ā does that help?#2016-09-2121:42seancorfield(it was something that I did not get on the first two or three reads of the Guide and it was Alex that set me straight on that)#2016-09-2123:19stuartmitchellHi I am beginning a large multi-developer project that will use spec extensively I was wondering if there is any advice for how to place specs in namespaces.
I was thinking of having a namespace structure like
project-name.spec
-common
-thing1
-thing2
...
and then requiring these namespaces in the actual code, probably when doing s/fdef
below the defn
this will require lots of map access to be ::thing1/id
or ::thing1/date
etc
is this a good idea or should I be trying to keep all specs within the namespaces they deal with so ::id
can be used when possible#2016-09-2123:48hiredmantying your data model (names of attributes) to the way your code is currently organized seems like a bad idea#2016-09-2123:49hiredmanI would recommend picking good names for things in your maps, and sure namespace them, but don't think you have to use the same namespace names as your code#2016-09-2123:50hiredman::
is a real pain when moving code around#2016-09-2200:03stuartmitchellso you are saying :thing1/id
(non aliased namespace) used everywhere we were thinking that, but it doesn't seem to look like the examples (which I assume are best practice)#2016-09-2200:14stuartmitchellI guess you loose the uniqueness of the keywords though.#2016-09-2200:14hiredmanwhat uniqueness?#2016-09-2200:16hiredmanin five years on a large clojure project we had several shakeups where namespaces were changed or renamed in mass, a long with regular day to day code churn#2016-09-2200:16stuartmitchellfor instance if i define :spec/thing1
in library code it could conflict with someone else defining :spec/thing1
#2016-09-2200:17hiredmanuse a better namespace then#2016-09-2200:17hiredmanmy.orgs.spec/thing1#2016-09-2200:18stuartmitchellyeah good point, I was just pondering what the differences would be#2016-09-2200:27hiredmandata is something that can be exchanged with other systems, stored long term, etc, so using names that reflect the organization of code in one codebase isn't the best#2016-09-2212:50jmglov@skjutare Hi!#2016-09-2212:50jmglovJust noticed you here. š#2016-09-2212:58jmglovI think I'm running into an infinite loop with a custom generator. I need an atom holding a map with specific keys, which seems pretty straightforward:
(s/def ::foo (s/map-of ::foo/id ::foo/event))
(s/def ::bar (s/map-of ::bar/country ::bar/event))
(s/def ::db (s/keys :req [::foo
::bar]))
;; Give these predicates names so spec's error messages are clear
(defn- atom? [x] (instance? clojure.lang.Atom x))
(defn- deref-db? [x] (s/valid? ::db @x))
(s/def ::db-atom (s/and atom? deref-db?))
So far, so good:
user> (s/explain ::db-atom {::foo {}, ::bar {}})
val: #:user{:foo {}, :bar {}} fails spec: :user/db-atom predicate: atom?
nil
user> (s/explain ::db-atom (atom 42))
val: #atom[42 0x702dec31] fails spec: :user/db-atom predicate: deref-db?
nil
user> (s/explain ::db-atom (atom {::foo {}, ::bar {}}))
Success!
nil
#2016-09-2213:01jmglovHowever, using a custom generator is getting me into a world of trouble:
(s/def ::db-atom (s/with-gen
(s/and atom? deref-db?)
(fn [] (gen/fmap atom (s/gen ::db)))))
When I try to generate a value, my REPL becomes unresponsive for awhile and then eventually crashes with a really odd stacktrace:
user> (gen/generate (s/gen ::db-atom))
...
#2016-09-2213:03jmglovCan anyone spot my error?#2016-09-2213:36mandersonThanks @seancorfield, just saw your reply. That does make sense. I think I need to rethink the way I organize my specs. Right now, I have everything in a single namespace and now I'm wondering if I should break these into sub-namespaces. I know that I can still collect my specs in 1 namespace and just add different namespaces to the keys. Not sure which is the best approach. How do you group your specs? I'm still trying to get a feel for the proper balance of the right amount of spec without becoming cumbersome.#2016-09-2213:41mandersonHa, funny. I just actually read back through the rest of the messages and see @hiredman making some good suggestions to my very question... sounds like best practice is to de-couple top level namespace of a file from namespace of a keyword. So, all could be grouped in a file, but with explicit namespaces for keys instead of ::
.#2016-09-2213:50patrkris@manderson: @hiredman talked about specs for data that might be exchanged between systems. If you want to spec data used only inside your system, creating namespaces for that may be feasible. For example, if you have a component in a com.example.payments-processor
with a factory function that takes an options map, you could potentially spec it inside com.example.payments-processor.options
, and alias that namespace wherever you need to create the options map. Hope I make sense.#2016-09-2213:52mandersonYep, that makes sense. I suppose there is no one right answer. Like I said, I'm still getting the feel for what works and what doesn't. We're building our first real app with this (as I imagine most are!) and running into pain points with name conflicts and a large spec file, so I've been thinking of how to best maintain it.#2016-09-2213:55patrkrisYes. I have been thinking a lot about this too. The problem is that you can't use aliased namespaces in specs if there isn't actually a file containing that namespace. In those cases you need to spell out the whole namespace of the spec, which can be verbose.#2016-09-2213:56mandersonHm. that's a good point.#2016-09-2213:59mandersonIt's kind of similar to the question of lots of fn's in a single namespace so only 1 to require, or fn's spread over ns's, which is more modular, but adds complexity. With spec, you would have to keep up with more ns's, but then you could alias them for simpler inline usage... It seems to encourage simplifying your model domain as much as possible in terms of defining it, which is probably a good thing.#2016-09-2216:12seancorfieldThere are workarounds for the alias issue that let you introduce aliases for non-existent namespaces. #2016-09-2216:15seancorfield@manderson: we group our data specs into files organized for what they spec, primarily, with almost no code in those files. Then we require them into namespaces that use those specs (i.e., where the code / functions are).#2016-09-2216:16seancorfieldThat's probably not a very helpful answer without you knowing a lot more about our codebase šø #2016-09-2217:03mandersonThanks @seancorfield, that's similar to what we're doing now, but it's a single spec file for a new app with a changing domain so it's been a bit of a pain. I think it's just going to take some trial and error to work out the kinks.#2016-09-2217:13seancorfieldYeah, weāre still evolving our approach. One of the issues for us, particularly, is that our domain model spec is the most extensive part and weāre working to automate the validation and conformance from our API spec to our domain model spec, so weāre moving toward dynamic derivation of API specs from domain model specs, which I hadnāt considered at firstā¦ so thatās changing the dependencies between our spec namespaces as well as how we use the specs in the first place. And we havenāt even gotten very far into specāing functions at this point (specāing the domain model and doing data validation and transformation has proved much more valuable for our initial focus).#2016-09-2217:21kkruitI've noticed that, for specs that are specific to your application, clojure src is using them in a sub-namespace e.g. all the specs for clojure.core are in clojure.core.specs. This has seemed to work okay for me.#2016-09-2217:25kkruit@patrkris I've been using create-ns to make namespaces for that issue.#2016-09-2217:28seancorfieldThatās nicer than the workaround I was using, from Alex. I think Iāll switch to that @kkruit ā thanks!#2016-09-2217:33kkruitno problem#2016-09-2219:00mpenetI have something somewhat similar used there: https://github.com/mpenet/alia/blob/master/modules/alia-spec/src/qbits/alia/spec.clj#2016-09-2219:01mpenetit's just an util fn, but maybe that's what you're refering to @seancorfield#2016-09-2219:02seancorfieldThatās a big file @mpenet ā are you referring to a particular part of it?#2016-09-2219:03mpenetright: https://github.com/mpenet/alia/blob/master/modules/alia-spec/src/qbits/alia/spec.clj#L152-L153#2016-09-2219:03mpenetit's just a wrapper for create-ns+alias#2016-09-2219:12seancorfieldAh, from another project...#2016-09-2219:13mpenetit's a 3 line fn, as i said, create-ns + alias#2016-09-2219:13seancorfieldSince your alias is so long, it took me a while to realize it even was an aliasā¦ and the args are the opposite way round to alias
which made it look even stranger: (x/ns-as 'qbits.alia.cluster-options.ssl-options
'cluster-options.ssl-options)
as opposed to (alias 'cluster-options.ssl-options (create-ns 'qbits.alia.cluster-options.ssl-options))
#2016-09-2219:14seancorfieldGiven that alias
will probably be modified to auto-create the ns, Iām preferring alias
directly since I can probably just remove the create-ns
call at some point.#2016-09-2219:14mpenetyou have a point#2016-09-2219:16mpenetI just wanted to get rid of "qbits.alia" if I recall, since I used this single file for the whole project I still like to keep some ns'ing#2016-09-2219:46nicolaWe had discussion on clojure-russia hangout about clojure.spec & namespaced keys and there is an idea by @prepor: we have reader macro to do ::key => :my.ns/key
, we have sugar to do (require ... :as a) ::a/key => :my.required.ns/key
. But there are a lot of cases where we need something like ::.subns/key => my.ns.subns/key
. For real world deeply nested data-structures create a lot of namespaces just for fully qualified keys for spec looks like overhead...#2016-09-2221:12patrkris@seancorfield: what was that workaround you mentioned earlier?
> Thatās nicer than the workaround I was using, from Alex. I think Iāll switch to that @kkruit ā thanks!#2016-09-2221:13seancorfield@patrkris: https://clojurians.slack.com/archives/clojure-spec/p1474565192002064#2016-09-2221:14patrkris@seancorfield: I mean the one from Alex#2016-09-2221:16seancorfieldOh, that was to use in-ns
twice which is what something in core does if I recall. #2016-09-2221:17seancorfieldI doubt it's still available in the scroll back and I don't remember what code he linked to. #2016-09-2221:21seancorfieldOk you made me curious so I searched the source code: https://github.com/clojure/clojure/blob/a1c3dafec01ab02fb10d91f98b9ffd3241e860c0/src/clj/clojure/spec/test.clj#L17 @patrkris #2016-09-2223:55noprompthas anyone been able to bend spec to parse and conform strings?#2016-09-2300:03devn@noprompt: what mad science are you up to now? :)#2016-09-2300:09noprompt@devn my problem is that i want to conform nested string data so i don't need to do it in two steps. it's possible to use conformer
but when there's a problem the error message is really, really bad (and there's nothing available in the public api for custom explanations). i'd like to simply write a custom spec against the protocols but alexmiller, in a previous discussion, has cautioned against that.#2016-09-2300:10nopromptbasically i'm at a crossroads where it's either write the data portion of the parser by hand, forgoing spec, or bend spec to do my bidding.#2016-09-2300:56kennyHow do you spec a function with optional args that should be conformed to a map?
(defn foo
[& {::keys [bar]}]
;...
)
#2016-09-2300:59kidpollohmmm sounds interesting and actually useful @noprompt It might actually also help my use case#2016-09-2301:00kidpollowhen an input param for example is a string but the string contains an elastic search query#2016-09-2301:00kidpolloan extra ste would need to be made to validate the syntax of the string#2016-09-2301:05kennyThis is a more concrete example:
(s/conform (s/and #(even? (count %))
(s/conformer (partial apply hash-map))
(s/keys :req [::foo]))
[::foo true])
There must be a better way of handling this for optional map args in fns#2016-09-2301:09bfabry@kenny keys*
is for this#2016-09-2301:09bfabryhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/keys*#2016-09-2301:09kennyAh! Perfect!#2016-09-2301:09kennyThanks!#2016-09-2301:09bfabryI knew I'd seen a specific function for it but it took me ages to find that again#2016-09-2301:42Alex Miller (Clojure team)@noprompt I will stick with my caution in trying to use spec regex to parse strings - that's not what it's designed for and I a normal regex is going to be a better choice. Why cant you use a regex predicate?#2016-09-2314:18djpowellwhat is the recommended way to make a spec for a constant string?#2016-09-2314:22donaldballI put it in a set#2016-09-2314:23djpowellooh - gen/return maybe?#2016-09-2314:26djpowellah yeah, a set seems to work ok#2016-09-2315:15Alex Miller (Clojure team)yes, a set #{āfooā}
#2016-09-2317:10mishagood day!
is there a way to spec a function, which I expect to get as an argument?
like "I expect function of 2 arguments: int and string"#2016-09-2317:12mishaso far I see only s/fdef
, which expects a symbol identical to existing function name.#2016-09-2317:13mishas/fspec
!#2016-09-2317:24bfabry^ yes š#2016-09-2317:26bfabryalso, something which I think is a bit not clear from the docs, fdef
adds an fspec to the registry under the fully qualified name of the function, so if you've written an fdef for foo
, you can use the symbol `foo in place of a spec#2016-09-2318:07spiedenanyone know how to spec a map by predicates over its keys and values? really expected this to work: (s/explain (s/+ (s/cat :foo string? :bar keyword?)) {"hi" :bye})
In: [0] val: ["hi" :bye] fails at: [:foo] predicate: string?
#2016-09-2318:08bfabry@spieden map-of
#2016-09-2318:09spieden@bfabry thanks! ok, now just for academic interest: (s/explain (s/+ (s/cat :foo string? :bar keyword?)) [["hi" :bye]])
#2016-09-2318:10spiedenah looks like i want āevery"#2016-09-2318:10bfabrya map is also I believe a sequence of tuples, not a single long sequence of key,val,key,val#2016-09-2318:11spiedenyeah i guess i was expecting the regular expression to match multiple instances of [āhiā :bye] within the sequence#2016-09-2318:11bfabryso this works
boot.user=> (s/explain (s/+ (s/spec (s/cat :foo string? :bar keyword?))) {"hi" :bye})
Success!
nil
#2016-09-2318:11spiedenooo#2016-09-2318:12bfabryas does this
boot.user=> (s/explain (s/+ (s/tuple string? keyword?)) {"hi" :bye})
Success!
nil
#2016-09-2318:12bfabrybut like I said, the simplest way to spec it is
boot.user=> (s/explain (s/map-of string? keyword?) {"hi" :bye})
Success!
nil
#2016-09-2318:13spiedenthatās what iām going with. thanks#2016-09-2321:11noprompt@alexmiller i can't use regex in my case because the strings i'm parsing are context-free.#2016-09-2321:12Alex Miller (Clojure team)so use a parser like instaparse?#2016-09-2321:13noprompt@alexmiller i probably would but unfortunately the story for clojurescript isn't solidified yet on that account.#2016-09-2321:15Alex Miller (Clojure team)@bfabry I would not use s/+
for that example above but (s/every (s/tuple string? keyword?))
- thatās basically (minus things like :kind
and :into
) what map-of
translates to#2016-09-2321:15noprompteven still, the story for plugging into explainability with conformer
is just not good enough. yes, the result does tell what string is wrong but i'm not able to say where in the string something is wrong.#2016-09-2321:15nopromptto be sure, i'm not complaining about spec. it's already saved me a huge amount of work. if i could tap into explainability it'd be a closed case for me.#2016-09-2321:16Alex Miller (Clojure team)not sure what to tell ya - youāre asking for things spec is unlikely to provide ĀÆ\(ć)/ĀÆ#2016-09-2321:17nopromptwhy wouldn't spec want to expose hooks to explainability?#2016-09-2321:17Alex Miller (Clojure team)because one of the ideas in spec is to make explainability generic#2016-09-2321:18nopromptthat presupposes that such an idea is possible.#2016-09-2321:18Alex Miller (Clojure team)I havenāt talked with Rich about this specifically so this is just my read on it#2016-09-2321:19noprompt(not saying it's not)#2016-09-2321:19Alex Miller (Clojure team)when you get to a predicate the explanation is: the predicate failed#2016-09-2321:19Alex Miller (Clojure team)if you want more detail, make finer-grained predicates and combine them#2016-09-2321:20Alex Miller (Clojure team)if you want that + some sort of string parsing, I think youāre into the territory of custom specs#2016-09-2321:20nopromptright, however, sometimes the context is a little more granular e.g. failure alone is not enough for reporting. sometimes there's additional context for why the predicate failed.#2016-09-2321:20nopromptcan we get a promise to stabilize the protocols? š#2016-09-2321:21Alex Miller (Clojure team)now? no#2016-09-2321:21noprompthahah#2016-09-2321:21Alex Miller (Clojure team)certainly not in alpha#2016-09-2321:21noprompti meant when 1.9 is stable.#2016-09-2321:21noprompthonestly, that would be phenomenal for my use case.#2016-09-2321:21Alex Miller (Clojure team)I donāt have an answer on that#2016-09-2321:22Alex Miller (Clojure team)we can provide guidance when things move past alpha#2016-09-2321:22nopromptchances are most folks won't be interested in implementing the protocols and will just use "stock" spec. a handful of people, like myself, would probably greatly appreciate and benefit from the protocols being stable.#2016-09-2321:23nopromptawesome.#2016-09-2321:23nopromptwould this conversation be better on the dev mailing list?#2016-09-2321:23Alex Miller (Clojure team)doesnāt make any difference to me#2016-09-2321:23nopromptfair enough.#2016-09-2321:24Alex Miller (Clojure team)I read both :)#2016-09-2400:35seancorfield@alexmiller I was just looking at https://github.com/clojure/clojure/commit/7ff4c70b13d29cf0a1fc7f19ba53413a38bb03d5 and noticed nonconforming
which I had not previously seen in the APIā¦ and now itās deliberately undocumentedā¦ I have several places where I have to conform
with or
and then use (conformer second)
to "undo" the conformed result to get back what I need. Is there any likelihood that nonconforming
would be "restored" to the documented API as something usable by developers? Or was it always intended to be an internal implementation detail and it just accidentally leaked?#2016-09-2400:36Alex Miller (Clojure team)It was accidentally public and stands a good chance of being removed#2016-09-2400:36seancorfield(I note that there actually is a test added in that commit for the nonconforming
function, even thoā that commit also removes it from the documentation)#2016-09-2400:36seancorfieldhttps://github.com/clojure/clojure/blob/7ff4c70b13d29cf0a1fc7f19ba53413a38bb03d5/test/clojure/test_clojure/spec.clj#L166#2016-09-2400:36Alex Miller (Clojure team)Well in my diligent way I added a test for the thing I fixed#2016-09-2400:37agguysā¦ why when I have something like this:
(s/def ::mgen (s/with-gen
(s/and not-empty map?)
#(gen/map (s/gen ::keyword-with-question-mark) (s/gen boolean?))))
(gen/generate (s/gen ::mgen))
it would generate a map with multiple keysā¦ but when used in prop/for-all
always just a single key-value?#2016-09-2400:37seancorfieldDo you think "flattening" out an or
is a good use case for having something like nonconforming
thoā?#2016-09-2400:38Alex Miller (Clojure team)But now that nilable doesn't use nonconforming Rich is considering removing it#2016-09-2400:38Alex Miller (Clojure team)You can still easily do that with a conformer on val#2016-09-2400:38seancorfieldTrue, for a simple spec. It gets harder with more complex specs.#2016-09-2400:39Alex Miller (Clojure team)Well, rely on it at your peril :)#2016-09-2400:40seancorfieldSo far thatās the only case Iāve run into thoā, where I need s/or
in order to combine specs but I really donāt want that branch in the result. (s/and (s/or :a ::some-pred :b ::other-pred) (conformer second))
seems a bit of a mouthful šø#2016-09-2400:41Alex Miller (Clojure team) @ag test.check is a bit different from spec in that spec wraps generators in a no-arg function - maybe that's causing the disconnect?#2016-09-2400:42Alex Miller (Clojure team)@seancorfield: advice from us is: use conformers very sparingly#2016-09-2400:47aghmmā¦ I still have no clue how to deal with that.#2016-09-2400:47agwrap into something like gen/fmap?#2016-09-2400:50agusing overrides? second param of s/gen ?#2016-09-2400:51Alex Miller (Clojure team)I don't totally understand what you're trying to do #2016-09-2400:53agI have a spec that generates a map (with gen/map
), when used in gen/generate
- it generates maps with multiple keys. but when used with test-checkās prop-all
and quick-check
it always generates maps with single key/value pair#2016-09-2400:57Alex Miller (Clojure team)Hmm, not sure#2016-09-2401:03seancorfield@alexmiller My use case ā and itās cropped up in a few specs Iāve needed at work ā is that we have situations where we have some data that is either an X or a string that converts to an X (partly because of legacy constraints) but we want to conform the data to X in both cases and donāt care whether it was initially an X or a string.#2016-09-2401:05seancorfieldNow, you can certainly argue "Donāt do that!" but having it baked into the spec and being able to just (s/conform ::my-spec data)
is nicer than having (if (string? data) ā¦)
everywhere that uses it.#2016-09-2406:14plexusis this a bug or a feature?#2016-09-2406:14plexus(s/valid? #{:clojure.spec/foo} :clojure.spec/foo) ;;=> true
(s/valid? qualified-keyword? :clojure.spec/foo) ;;=> true
(s/valid? #{:clojure.spec/invalid} :clojure.spec/invalid) ;;=> false
(s/valid? qualified-keyword? :clojure.spec/invalid) ;;=> false
#2016-09-2408:24seancorfieldIt's a feature. :clojure.spec/invalid
is special -- it only ever satisfies s/invalid?
.#2016-09-2408:56mpenetOdd, seems like an impl. quirk#2016-09-2416:17borkdudeIt would be nice if t/instrument
also gave a warning when the :ret
spec doesnāt conform. I know spec only tests this with t/check
, but is there a reason the first one isnāt in spec (optionally) right now? Seems useful for dev purposes.#2016-09-2416:25borkdudeIf I could programmatically insert a post-condition on a function that would get me far#2016-09-2417:23borkdudelittle hack: https://gist.github.com/borkdude/38336a9ef2eba8ee0b3c9536fda28233#2016-09-2420:03Alex Miller (Clojure team)@borkdude: you can use s/assert to state postconditions if you like#2016-09-2420:04Alex Miller (Clojure team)They can then be turned off and even optionally compiled out#2016-09-2420:22borkdude@alexmiller ah yes, like in the snippet#2016-09-2421:18jfntnIf I have defined a spec for a value like :datomic/tempid
, how can I reuse that in a map spec thatād say that the :db/id
key should have a value that conforms to :datomic/tempid
?#2016-09-2421:54jfntnI think I got something usable! The point is that I want to define a map spec for an entity, and reuse that spec in describing the inputs to a tx and the outputs from a read. The trick is that the read and write specs should be identical with the exception that :db/id
takes on different values#2016-09-2421:55jfntnNot sure thatās a good way to do this, would appreciate some feedback#2016-09-2422:56mishawhat am i doing wrong?
;; [org.clojure/clojure "1.9.0-alpha12"]
;; [org.clojure/clojurescript "1.9.229"]
=> (s/def :foo/bar
(s/fspec :args
(s/cat
:id number?
:type keyword?)))
:foo/bar
=> (s/spec :foo/bar)
#object[cljs.spec.t_cljs$spec32342]
=> (s/conform :foo/bar #(identity {}))
#object[TypeError TypeError: Cannot read property 'call' of undefined]
=> (s/explain-data :foo/bar '#(identity {}))
{:cljs.spec/problems [{:path [], :pred ifn?, :val (fn* [] (identity {})), :via [:foo/bar], :in []}]}
#2016-09-2422:58misha(I want to register a function spec to a keyword, to be able to "say" later:
this map contains :foo/bar key, and its value is a function of 2 args: number and keyword)#2016-09-2422:59mishaor just to conform any other function to this function signature#2016-09-2423:10misha=> (defn yoyo [f] (s/explain-data :foo/bar f))
=> (yoyo map)
#object[TypeError TypeError: Cannot read property 'call' of undefined]
#2016-09-2500:00mishahowever in clojure it works:
(s/explain-data :foo/bar #(identity {}))
=> #:clojure.spec{:problems [{:path [], :pred (apply fn), :val (0 :A), :reason "Wrong number of args (2) passed to: sandbox/eval7829/fn--7830", :via [:foo/bar], :in []}]}
#2016-09-2505:48Oliver GeorgeAnyone here use devcards? I'm experimenting with how cards could be initialised based on spec/gen data. Would love any opinions/insights. Ticket: https://github.com/bhauman/devcards/issues/112 and gist https://gist.github.com/olivergeorge/82a20dd03fd86e82ab9b0f3959590f3f#2016-09-2505:50Oliver GeorgePerhaps my use case is special but it seems like something which might be useful to others.#2016-09-2505:55Oliver GeorgeThe other fiddly bit was working with the function sym. I needed to resolve the sym to the current local var value but also keep the sym to call (s/get-spec ...). In the end I used (var). Not sure if there's downsides to that I'm missing.#2016-09-2509:33stathissideris@olivergeorge itās been done before š https://juxt.pro/blog/posts/generative-ui-clojure-spec.html#2016-09-2510:09Oliver GeorgeThanks @stathissideris #2016-09-2510:43Oliver GeorgeI think that was my inspiration. I've added a bit more finessing about showing interesting random data (overrides) and controls (seed & size) which might make for a more useful UI dev tool. #2016-09-2510:46Oliver GeorgeNeeds some helper generators. I never leave home without my lorem ipsum generator. #2016-09-2613:13Oliver Georgehttps://github.com/olivergeorge/devcards-vs-clojure-spec#2016-09-2614:22kestrel7Is it possible to define a spec that depends on multiple keys within a map? e.g. ::date-1 must be greater that ::date-2#2016-09-2614:45bcbradleyi'm having issues with setting up clojure.spec#2016-09-2614:47bcbradleyi'm getting "could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on class path"#2016-09-2614:47bcbradleywhat does that mean?#2016-09-2614:54thegeez@bcbradley that might be due to this: http://clojure.org/guides/spec#_project_setup#2016-09-2614:54bcbradleythankyou!#2016-09-2614:58bcbradleyi'm still getting the error š#2016-09-2615:01bcbradleyhere is my lein#2016-09-2615:01bcbradley(defproject sand "0.0.1-SNAPSHOT"
:description "FIXME: write description"
:profiles {:dev {:dependencies [[org.clojure/test.check "0.9.0"]]}}
:dependencies [[org.clojure/clojure "1.9.0-alpha12"]]
:javac-options ["-target" "1.6" "-source" "1.6" "-Xlint:-options"]
:aot [sand.core]
:main sand.core)#2016-09-2615:02thegeezIf you are using Leiningen, you can try to put the dependency at the top level. So put it next to [org.clojure/clojure ...] in :dependencies#2016-09-2615:02bcbradleyyou mean don't use a dev profile?#2016-09-2615:03misha@bcbradley clj or cljs ?#2016-09-2615:03bcbradleyclojure#2016-09-2615:03bcbradleythe function causing the issue is exercise-fn#2016-09-2615:03thegeezYeah, just to check if that works. You might be using the test.check function in a way that does not fall under leiningen dev profile#2016-09-2615:04thegeezor in a place, rather than a way#2016-09-2615:04bcbradley(defproject sand "0.0.1-SNAPSHOT"
:description "FIXME: write description"
:dependencies [[org.clojure/clojure "1.9.0-alpha12"]
[org.clojure/test.check "0.9.0"]]
:javac-options ["-target" "1.6" "-source" "1.6" "-Xlint:-options"]
:aot [sand.core]
:main sand.core)#2016-09-2615:04bcbradleystill no luck#2016-09-2615:05bcbradleyi can use clojure.spec/fdef#2016-09-2615:05bcbradleybut can't use clojure.spec/exercise-fn#2016-09-2615:15thegeezhmm curious, it works on my machine š#2016-09-2615:17bcbradleyhere is what i'm looking at#2016-09-2615:22seancorfield@bcbradley Youāre AOTāing that namespace so youāre forcing all those specs ā including exercise-fn
ā to run at compile time.#2016-09-2615:23bcbradleyi see#2016-09-2615:23bcbradleyi'm sorry for being so new at this š#2016-09-2615:23bcbradleythe AOT is an acronym i assume#2016-09-2615:23bcbradleywhat does it stand for?#2016-09-2615:23seancorfieldAhead Of Time (compilation) ā relating to the :aot
in your project.clj
#2016-09-2615:25seancorfieldStill, having the exercise-fn
call at the top-level of your code means it will be executed whenever that ns is run ā put it inside -main
so it only runs when you run the -main
...#2016-09-2615:25seancorfieldSame with generate
.#2016-09-2615:25bcbradleyok, i was wondering though#2016-09-2615:26bcbradleyif i wanted to make a library how would i test the code? Would i need a main?#2016-09-2615:26seancorfieldYouād normally put test code in a separate ns and have it run via one of the test runners.#2016-09-2615:27seancorfieldIn your project tree, youāll see a test
folder next to src
.#2016-09-2615:27seancorfieldHmm, no, in your case you wonāt ā did you create that project manually?#2016-09-2615:28seancorfield(Iām not familiar with how NightCode creates projectsā¦ but normally lein new app myapp
will create a test
folder as well as a src
folder)#2016-09-2615:28bcbradleyi just chose "create new console project"#2016-09-2615:28bcbradleyi've used lein before#2016-09-2615:28bcbradleybut importing a lein project into nightcode gave me some problems#2016-09-2615:29seancorfieldIām surprised (and a bit disappointed) that NightCode doesnāt create a test
folder tree for all projects š#2016-09-2615:29thegeezI think this is an issue with nightcode. A top level exercise-fn with :gen-class and aot would works#2016-09-2615:29bcbradleyi've had trouble finding a good ide for clojure that i like#2016-09-2615:30seancorfield@thegeez Well, if test.check
is part of the regular :dependencies
yes, but you donāt want that code running at compile time or ns load time...#2016-09-2615:30bcbradleyi don't wanna use emacs because its 2016, i don't like eclipse because its sluggish and smells of enterprise, i don't like intellij + cursive because i don't want to pay for it, I don't like netbeans because netbeans. Light table isn't working with clojure 1.9-alpha12#2016-09-2615:30bcbradleyit looks like nightcode is my only choice unless i want to rock it in vim#2016-09-2615:31seancorfieldThe majority of Clojure devs use a form of Emacs according to the annual "State of Clojure" surveys š#2016-09-2615:31bcbradleyi'd prefer not to develop carpal tunnel before i turn 30#2016-09-2615:32bcbradleyjabbing aside, even after removing the AOT and pulling it into main i get some issues#2016-09-2615:34thegeez@seancorfield yes that setup is not ideal, but more meant to isolate the problem#2016-09-2615:35thegeez@bcbradley now you have the name my-index-of and a spec for my-index-of2#2016-09-2615:36seancorfield@bcbradley Your spec has my-index-of2
but the function ā¦ yeah, what @thegeez said š#2016-09-2615:36bcbradleyyou are right!#2016-09-2615:36bcbradleythat fixed it xd#2016-09-2615:37bcbradleyi know i'm probably being a pest, but can you explain the difference between clojure.spec and any normal library like clojure.set?#2016-09-2615:37bcbradleyi mean in terms of pitfalls,#2016-09-2615:37bcbradleylike for instance, i didn't know that i "shouldn't" invoke certain methods or macros at the top level#2016-09-2615:37bcbradleynormally i wouldn't invoke anything at the top level in an application, but i figure heck#2016-09-2615:37bcbradleyif it works on a repl#2016-09-2615:37bcbradleyi'm just trying to learn#2016-09-2615:38bcbradleyi need to know what NOT to do#2016-09-2615:44seancorfieldIn general you donāt want any executable code at the top-level of your namespace.#2016-09-2615:46seancorfieldWhatās your language background, prior to Clojure? (so weāll have a frame of reference for what youāre familiar with)#2016-09-2615:48seancorfieldAnd regarding the difference between clojure.spec
and "normal" libraries, probably the big one is that thereās a bunch of "testing" stuff in clojure.spec
that belongs in test
code, where test.check
gets brought in via a :dev
profile (for REPL and running tests, as opposed to "production" line code). Does that help?#2016-09-2615:49seancorfieldThe specs themselves ā and valid?
and conform
ā are intended for production code, but the generative testing stuff (`generate`, exercise
, check
) is intended for either interactive use or for running as part of test code.#2016-09-2616:02bcbradleysorry i was away, my language background: C, C++, Java, Python, Javascript and then Clojure#2016-09-2616:03bcbradleyi've got 8 years in C, 6 years in C++, 3 years in python, 2 years in javascript and 1 year in clojure#2016-09-2616:04bcbradleyi'm in my senior year for software engineering and would like to get a job where the primary language is clojure, or at the very least some other functional oriented job with scala or erlang or whatever#2016-09-2616:04bcbradleyi've seen enough of stateful programs to know they are a pitfall#2016-09-2616:04seancorfieldš#2016-09-2616:04bcbradleyand i'm not just espousing things i've heard#2016-09-2616:05bcbradleyi've had time to think about it for myself#2016-09-2616:06bcbradleymy experience with clojure is probaby summarized best as follows:#2016-09-2616:06bcbradleyi know how to use map, reduce, apply partial blah blah blah, i've been doing it for a while#2016-09-2616:06bcbradleywhen i moved from javascript to clojure i did it for elegance#2016-09-2616:06seancorfield'k so a Clojure namespace produces a class, and all the top-level forms become static initializers in that classā¦ if you think of it in Java / C++ terms, those static initializers get run when a class is loaded, long before any class instance is createdā¦ so the same caveats apply.#2016-09-2616:07bcbradleyah i see#2016-09-2616:07bcbradleyi don't think i'm actually familiar with static initializers#2016-09-2616:08bcbradleywell a search on stack overflow was fast enough#2016-09-2616:08bcbradleygood to know thats how clojure namespaces work#2016-09-2616:10bcbradleyso basically what you are saying is that if I have a bunch of top level forms that depend on some other class already being loaded, but it hasn't been loaded, i'll get a "class nil" error or some such#2016-09-2616:11bcbradleythat makes perfect sense#2016-09-2616:50bfabry@bcbradley cursive is free for non-commercial use, fyi#2016-09-2616:52bfabryand in a commercial setting, I can't imagine any company would care about the cost#2016-09-2617:05bfabryit's also rapidly catching up with the the perpetual yak emacs š#2016-09-2617:20seancorfieldSome of us really like yaks...#2016-09-2617:33dominicmMy vim shaving is unmatched in fun vim #2016-09-2619:10Alex Miller (Clojure team)1.9.0-alpha13 is out https://groups.google.com/d/msg/clojure/QWPUWG9BwbE/9a7ymJb9AQAJ#2016-09-2619:11Alex Miller (Clojure team)which is mostly fixes and improvements to nilable, which is also probably a little faster#2016-09-2621:57bcbradleyi really like nightcode for some reason#2016-09-2621:57bcbradleyidk its just not that pretentious#2016-09-2621:57bcbradleyits not one thing pretending to be another. Its just a clojure ide. Thats it.#2016-09-2712:48kurt-yagramhow to s/fdef
functions with optional named args?
(s/fdef myfun
:args (s/cat :first string? <???>))
(defn myfun "" [first & {:keys [named]}]
...)
so what should be at <???> to add a spec for the named argument?#2016-09-2712:54madstap(s/? (s/cat :named-k #{:named} :named boolean?))
#2016-09-2712:54kurt-yagramGot it. Makes sense.#2016-09-2714:51Alex Miller (Clojure team)no, donāt do that - this is what s/keys*
is for#2016-09-2714:52Alex Miller (Clojure team)(s/def ::named boolean?)
(s/fdef myfun :args (s/cat :first string? :opts (s/keys* :req-un [::named])))
#2016-09-2714:53Alex Miller (Clojure team)this allows you to specify any number of kwarg options at the end in any order#2016-09-2715:42madstapRight, I had missed s/keys*
, thanks for the clarification. Sometimes it seems like you guys think of everything, probably all that hammock time.#2016-09-2715:45kurt-yagramah, ok... thanks!#2016-09-2800:07jfntnI canāt find a good solution to write different specs that all have a :db/id
key but with different values#2016-09-2800:08danielcompton@jfntn what do you mean by ādifferent valuesā?#2016-09-2800:09jfntnWell one would be say a ::temp-entity
spec thatād check that :db/id
is a datomic/tempid
, and the other would be an ::entity
spec that would check that :db/id
is an entity id (long)#2016-09-2800:10jfntnSo I need to attach multiple specs to the same namespaced key, but s/or
wonāt do since the point is to be able to assert whether an entity is temporary or not...#2016-09-2800:11jfntnI found a weird trick using multiple slashes in the spec name and using :req-un
in the spec, but those donāt seem to really be supported since Iām getting intermitent analyzer errors and the keys wonāt compile in cljs#2016-09-2800:15bhaganyI think in this case, I would use s/or
, and use the conformed value to say whether itās a temp id or not#2016-09-2800:16bhaganyitās an extra step, but I donāt see another way to make it work#2016-09-2800:18bhaganyif youāre spec-ing a function that needs to take a temp entity, or something like that, you could always make a predicate function, temp-entity?
or some such, that conforms its argument and checks which branch s/or
took#2016-09-2800:32jfntn@bhagany thatās a good idea!#2016-09-2800:33jfntnNot as direct, but Iām able to express what I wanted, which is a reusable keys
spec and combine it with the temp and final specs#2016-09-2806:17misha@jfntn
-(s/def :datomic.tempid/idx int?)
+(s/def :datomic.tempid/idx neg-int?)
#2016-09-2806:42przemekHi, while trying to migrate from schema to spec I've come across an issue. Following the guide I was able to define specs for maps, but I think I have a case that is not covered. Basically the same key i.e. :id or :name may be an attribute of different entities. If I define ::name then the only way to use a different type definition for :name is to use namespaces which puts me in sort of OO style when I keep module per "noun".... Is there not a way to decouple key from value so that I can define i.e. ::plan-name type, but use :name in my map?#2016-09-2806:50misha@przemek what about (s/keys :req-un [...] :opt-un [...]
?#2016-09-2806:55mishabasically, you'd have multiple namespaced specs for name
: :foo/name, :bar/name; and then use them in map specs, but inside :req-un (or :opt-un), with the only requirement of keeping (name kw)
identical#2016-09-2806:59misha(s/def :foo/id int?)
(s/def :bar/id string?)
(s/def :foo/thing (s/keys :req-un [:foo/id]))
(s/def :bar/thing (s/keys :req-un [:bar/id]))
(s/valid? :foo/thing {:id 1})
=> true
(s/valid? :foo/thing {:id "one"})
=> false
(s/valid? :bar/thing {:id "one"})
=> true
(s/valid? :bar/thing {:id 1})
=> false
#2016-09-2807:02przemekhell yes š That looks like what I was looking for. I makes sense to document more how namespacing looks in action. I am gonna try this one in a moment. thanks#2016-09-2807:11przemekit works. I guess for Datomic that :xxx/yy pattern is heavily used (I have not used it though). thanks again!#2016-09-2807:53jmglovIs there anyone here who can shed any light on my customer generator for an atom holding a map? https://clojurians.slack.com/archives/clojure-spec/p1474549130002028#2016-09-2807:53jmglovOr is this something better posted to the mailing list?#2016-09-2808:22przemekAnother question. Given this definition (s/def ::email (s/and string? #(re-matches email-regex %))) I am not able to generate a sample getting "Couldn't satisfy such-that predicate after 100 tries".#2016-09-2808:25przemekOk. Solved. Found a paragraph in the guide on custom generators.#2016-09-2810:42madstap@przemek https://github.com/gfredericks/test.chuck has a string-from-regex
generator#2016-09-2810:50przemek@madstap thx, will look at that. I managed to build a custom gen, but this is more elegant#2016-09-2813:16jfntn@misha good catch, thanks!#2016-09-2901:52richiardiandreaHello all! A question, let's say I have:
(s/def ::event-common (s/keys :req [::nonce ::payload]))
I would like to declare an event that has fixed ::type
:
(s/def ::command-event (s/merge ::event-common -- what do I put here? --))
#2016-09-2901:53richiardiandrea(I was also wondering if I can use s/and
with my ::event-common
def)#2016-09-2903:59Alex Miller (Clojure team)@richiardiandrea not entirely clear what you want to do. maybe:
(s/def ::command-event (s/merge ::event-common (s/keys :req [::type])))
?#2016-09-2904:01Alex Miller (Clojure team)or are you asking because they canāt both be satisfied that way? I suspect this is the kind of thing where youāre happier using s/multi-spec
#2016-09-2904:12richiardiandrea@alexmiller tnx for answering, I have a multi-spec
in place so that I can check if the event conforms to my expected :type
s. Re-reading my question I see that maybe I am looking at it from the wrong angle..I will clarify my own ideas first and ask again#2016-09-2904:17richiardiandreaI think I wanted to be sure that a function returns only a certain :type
of events. So I need to combine the ::common-event
spec with a predicate...probably an s/and
will suffice.#2016-09-2904:29Alex Miller (Clojure team)Yeah s/and is fine#2016-09-2906:32seancorfieldSo I still keep running into situations where I have a spec for a domain level entity that is, say, a set of keywords and then the spec for the input level entity is a set of strings that are the names of the keywords... so my gut says to spec the input as (s/and string? (s/conformer (comp keyword str/lower-case)) ::domain-entity)
with a generator of (fn [] (g/fmap name (s/gen ::domain-entity)))
#2016-09-2906:33seancorfieldBut @alexmiller has said to avoid s/conformer
so I suspect this is an anti-pattern, even tho' it's cropping up over and over again.#2016-09-2906:35seancorfieldThe other (anti-)pattern I hit is a domain spec for an int (int-in or whatever) and the input spec is string? and parse to long and the domain spec, again with a generator of fmap str over the domain spec generator...#2016-09-2907:26mpenetThinking about using a set of specs just for json/http params coercion here, then pass the result to domain specs, I think I prefer to keep both clearly separate.#2016-09-2907:28mpenetmakes it a bit easier to reuse also maybe#2016-09-2907:29mpenetanyway, still have a ton of Schema code in our codebases, and still undecided on if/how to port it so far.#2016-09-2911:03mpenetis there an equivalent of describe/form
that will resolve specs recursively?#2016-09-2911:04mpenetI mean resolve specs from the registry when it's composed from these#2016-09-2911:05mpenet-> apparently not#2016-09-2911:33mpenet(probably another anti-)pattern I encountered: setting a default value if nil when conforming, it's useful at input level too#2016-09-2911:37mpenet(defn default [spec default]
(s/conformer
#(-> spec s/nilable (s/conform %) (or default))))
#2016-09-2911:53mpenetanother annoying quirk, the following won't compile on alpha13:
(defn foo [x] :clojure.spec/invalid)
ExceptionInfo Call to clojure.core/defn did not conform to spec:
:clojure.spec/args (foo [x] :clojure.spec/invalid)
clojure.core/ex-info (core.clj:4725)
#2016-09-2911:58mpenetmakes it impossible to write a declared conformer ...#2016-09-2912:01mpenetwell you can create an alias first (def invalid :clojure.spec/invalid)
and use it instead of :clojure.spec/invalid
in the code, but that's kind of ugly#2016-09-2916:49jeroenvandijkQuestion, do people add docs to specs? Iām making specs for some configuration data and I want to add meta data/documentation to individual specs to give more background information about the specific options#2016-09-2918:18seancorfieldI donāt think you can add docs to specsā¦?#2016-09-2918:55Alex Miller (Clojure team)you canāt#2016-09-2918:56Alex Miller (Clojure team)but feel free to vote on the idea here http://dev.clojure.org/jira/browse/CLJ-1965#2016-09-2922:28jrheardfirst time spec user here, trying to wrap my head around the best way of using namespaced keywords, never used āem before
whatās the more correct pattern: having my keywords have namespaces that match my projectās namespaces, like :myapp.system.collision/foo?
or having my keywords have namespaces that correspond to arbitrary concepts, unrelated to my appās file/namespace structure, like :collision/foo?#2016-09-2922:38jrheardso far iām going with the latter, and it feels pretty good, but iād just like to confirm with the experts š#2016-09-2922:48jrheardalso - do you folks have any favorite example github repos that use spec, like, completely idiomatically? iām switching my project over from schema, and some parts of the transition are straightforward, but other parts just feel like they could be done in a variety of different ways and iād love to have a few canonical examples to look at and learn from#2016-09-2922:52bfabryit's probably too early for that to exist =/#2016-09-2922:53bfabryfwiw I'm leaning toward the latter of your two options as wel#2016-09-2923:08seancorfield@jrheard Weāre creating new namespaces specifically for the data structure specs, and then requiring them into code namespaces as we need them. In a few places we have both data/specs and code. Itās whatever seems most appropriate.#2016-09-2923:09seancorfieldBTW, if folks have any thoughts on my comments posted late last night https://clojurians.slack.com/archives/clojure-spec/p1475130747003048 (specifically about the pros and cons of using s/conformer
)#2016-09-2923:10seancorfield(if you click that link, youāll see two more follow-on comments)#2016-09-3001:20aghey guysā¦ anyone here using spec and prop. based testing in clojuresript? I have encountered strange problem. (s/gen)
within a test that works perfectly on my machine, on CI machine (same java, lein, node and phantomjs versions) says ""Couldn't satisfy such-that predicate after 100 tries.ā,#2016-09-3001:21agis it possible that a certain spec would work on one machine and fail on other?#2016-09-3001:22agitās not like random. I run it many times. it passes on my machine and it ALWAYS fails on CI machine#2016-09-3001:22agother tests work fine though#2016-09-3001:22agĀÆ\(ć)/ĀÆ#2016-09-3001:56danielcomptonwhat kind of spec is it?#2016-09-3002:15agohā¦ ohā¦ itās a bit nastyā¦ I need to generate iso-date strings and I did this:
(s/def :date/iso (s/with-gen (s/and string?
#(re-matches iso-date-regex %))
#(gen'/for [d (gen'/datetime)]
#?(:clj (time-f/unparse local-date-formatter d)
:cljs (.format (js/moment (js/Date. d)))))))
#2016-09-3002:15agusing test.chuck#2016-09-3002:38bfabry@ag my first guess would be a different timezone setting between CI and your pc#2016-09-3006:26danielstocktonI know this has probably been asked 1000 times but what is the prevailing preference for where to put specs? In their own parallel namespace or inline?#2016-09-3006:27stathissiderismy impression is inline for function specs, different namespace for ādomainā specs#2016-09-3006:27seancorfieldIt depends.#2016-09-3006:29seancorfieldBut yeah, I think @stathissideris comment reflects what is becoming the most likely best practice at this point. It's what we're doing at World Singles so far...#2016-09-3006:29danielstocktonand "domain" spec means specs which are likely to impact more than one namespace and be reused a lot?#2016-09-3006:30seancorfieldData structure specs, as opposed to function specs.#2016-09-3006:36danielstocktonKnow of any open source projects that are already using it, that I could take a look at?#2016-09-3006:40seancorfieldclojure.java.jdbc
? š#2016-09-3006:42danielstocktonThanks. Looks like they follow the some.namespace and some.namespace.spec convention instead#2016-09-3006:43danielstocktonI was thinking an advantage of in-lining is that it provides some documentation, but am I right that spec adds something to the docstring anyway?#2016-09-3006:45danielstocktons/they/yourself/ i see#2016-09-3006:45bfabryfdef
specs are added to the output of doc
ya#2016-09-3006:46danielstocktonOk, I might be leaning towards a separate namespace in that case#2016-09-3006:47seancorfieldInlining fdef makes sense for documentation, but not for portability across versions. def for data structures in a separate namespace makes sense regardless.#2016-09-3006:48seancorfieldWe've generally found if we start to def a data structure spec in a namespace with functions, it's gets messy ... keeping the data structure specs separate is more flexible...#2016-09-3006:50danielstocktonI'm worried things will be messy with function specs too, it seems the documentation advantage is somewhat negated by the docstring#2016-09-3006:55seancorfieldNot sure what you mean...? Specs just add to the docstring...#2016-09-3007:08danielstocktonI mean that if it's added to the docstring then it isn't much of an advantage having it inline as well.#2016-09-3007:09seancorfieldRight, yes, but the docstring is only updated if you've loaded the ns with the spec so you might as well have the fdef inline (IMO)#2016-09-3007:10bfabryyeah.. different documentation requirements. documentation for people reading/maintaining the code, documentation for people using the function#2016-09-3007:20yendaHi, I am trying to use spec to conform the arguments of a function: there is at most 4 arguments with 4 distinct types. I want to conform them into a map but the solution I came up with is not fully compliant because it allows 2 arguments of the same type. Is there a way to make sure there isn't 2 arguments of the same type ?#2016-09-3007:20seancorfieldNot sure I follow... can you share some code?#2016-09-3007:21yenda@seancorfield yes sorry you are too quick š#2016-09-3007:21seancorfieldš#2016-09-3007:27seancorfieldYour coll-of
is a map so each :message
item overwrites any prior item before :distinct
can be checked.#2016-09-3007:28seancorfieldAnd if you used :into []
you would still accept those two :message
arguments because they are different values so they are distinct.#2016-09-3007:28seancorfieldThe :distinct
flag says that all the conformed values must be distinct -- which they are.#2016-09-3007:29seancorfieldYou'd need your ::argument
spec to conform to just the type, not the type and value.#2016-09-3007:34seancorfieldHmm, that doesn't quite work either: (s/def ::argument (s/and (s/or :message string? :app-data map? :type keyword? :variables vector?) (s/conformer first)))
(s/def ::arguments (s/coll-of ::argument :distinct true :into [] :max-count 4))
#2016-09-3007:34seancorfield(s/conform ::arguments ["helo {a} hello" "hello" ::random-log {:c '(inc c)}])
[:message :message :type :app-data]
Not what I expected.#2016-09-3007:35yenda@seancorfield if I have 2 identical arguments the predicate distinct? is triggered so the distinct check happens before the into {}#2016-09-3007:36yendahowever this not really what I want I could remove it, what I would like is to have distinct types for the arguments not distinct arguments#2016-09-3007:37seancorfieldYeah, I think you're right about the :distinct
check... it would check distinct values before the conforming.#2016-09-3007:37bfabrythere's something from a very long time ago whispering in my ear that this is an irregular grammar, and so you're going to need a custom predicate to validate it. but I could be wrong#2016-09-3007:38bfabrys/irregular/not context free/#2016-09-3007:38seancorfieldSo you need s/and
on ::arguments
to force it to be distinct types.#2016-09-3007:39seancorfield(s/def ::arguments (s/and (s/coll-of ::argument :into [] :max-count 4) (s/coll-of keyword? :distinct true)))
#2016-09-3007:39seancorfield(s/conform ::arguments ["helo {a} hello" "hello" ::random-log {:c '(inc c)}])
:clojure.spec/invalid
#2016-09-3007:40seancorfield(s/conform ::arguments ["helo {a} hello" ['b] ::random-log {:c '(inc c)}])
[:message :variables :type :app-data]
#2016-09-3007:40seancorfieldAnd that's with (s/def ::argument (s/and (s/or :message string? :app-data map? :type keyword? :variables vector?) (s/conformer first)))
#2016-09-3007:41seancorfieldSo each argument needs to conform to its type (throwing away the value) and then the arguments collection needs to flow the conformed arguments into a distinct collection#2016-09-3007:43yendawow thank you it works but it also makes me realize I still have a lot of work to do on spec š#2016-09-3007:48seancorfieldYeah, it's radically changing how we approach several types of problems. #2016-09-3007:49yendaoh ok I understand now because I didn't realize that I don't get the values anymore while conforming#2016-09-3007:51yendathe function I don't get is conformer#2016-09-3007:55seancorfieldTook me a while too. conformer
uses its function argument to transform the data rather than just being a true/false predicate. #2016-09-3007:55seancorfieldBut Alex says it's an anti-pattern so be cautious. #2016-09-3008:20yendawell I'll play a bit to see if I can keep my conformed map and still have the validation on distinct keywords#2016-09-3008:21yendabtw do I misunderstand unform ? I thought this call (s/unform ::x (s/conform ::x data-to-conform)) would return data-to-conform but it just returns the conformed data#2016-09-3008:30yenda@seancorfield is the explanation on why it is an anti-pattern available somewhere ?#2016-09-3008:32yendabelow is the final spec I made thanks to your input that matches my needs, I could eventually just do the map transform outside of the spec iff conformer is an anti-pattern#2016-09-3008:39jmglov@ag Here's how I generate date strings:
(require '[clj-time.core :as t]
'[clj-time.format :as f])
(import '(org.joda.time.format DateTimeFormatter))
(defn parseable-timestamp
"Returns a spec for a timestamp string can be parsed with the specified
datetime formatter."
[formatter]
(s/and string?
(fn [timestamp-str]
(try
(f/parse formatter timestamp-str)
true
(catch Exception _
false)))))
(defn make-timestamp-gen
"Returns a generator for a timestamp string between the minimum year and
maximum year (exclusive) that can be parsed with the specified datetime
formatter."
[min-year max-year formatter]
(fn []
(let [year-gen (s/gen (s/int-in min-year max-year))
month-gen (s/gen (s/int-in 1 12))
day-gen (s/gen (s/int-in 1 28))
hour-gen (s/gen (s/int-in 0 23))
m-s-gen (s/gen (s/int-in 0 59))]
(->> [year-gen month-gen day-gen hour-gen m-s-gen m-s-gen m-s-gen]
(map #(gen/fmap vector %))
(apply gen/cat)
(gen/fmap #(->> (apply t/date-time %)
(f/unparse formatter)))))))
(def timestamp-formatter (f/formatters :date-time-no-ms))
(s/def ::timestamp
(s/with-gen
(parseable-timestamp timestamp-formatter)
(make-timestamp-gen 2000 2050 timestamp-formatter)))
#2016-09-3013:26mpenetasking again: I'd like to generate documentation from specs, is there a way to resolve/expand specs from (s/form ...) (or other) ?#2016-09-3013:28mpenetI guess I can hack this walking the (s/registry) + (s/form ...), but that seems a bit "hairy"#2016-09-3013:41mpenet(it's for front-end team, so I cant just spit keywords, i need to resolve the leafs of specs to stuff they understand)#2016-09-3013:53mpenetmight be a good fit for some specter juggling#2016-09-3014:02Alex Miller (Clojure team)are you talking about a ādeepā version of s/form?#2016-09-3014:03mpenetyes#2016-09-3014:03Alex Miller (Clojure team)yeah, weāve talked about providing that but it doesnāt exist right now#2016-09-3014:04mpenetthat'd be nice, it's should be easy enough to write, but that's something one would expect out of the box imho#2016-09-3014:04Alex Miller (Clojure team)I have specs for spec forms (kind of, modulo a number of bugs in s/form) which could help a lot#2016-09-3014:04mpenetsounds good#2016-09-3014:04Alex Miller (Clojure team)because you could conform the spec and then pick the parts per form that you need#2016-09-3014:05Alex Miller (Clojure team)I will probably get around to this soon-ish as I need it for other things#2016-09-3014:05mpenetI didn't actually think about approaching it that way, neat#2016-09-3014:05Alex Miller (Clojure team)having specs for specs opens up all sorts of things#2016-09-3014:06Alex Miller (Clojure team)for example, automated testing of spec by generating specs from spec specs, then checking conformance on data generated for the generated spec#2016-09-3014:06mpenetthat's spec inception#2016-09-3014:06mpenetoh since you're here, I have another question about the conformer + :clojure.spec/invalid issue I mentioned yesterday#2016-09-3014:07Alex Miller (Clojure team)yes, it feels like that :)#2016-09-3014:07mpenethttps://clojurians.slack.com/archives/clojure-spec/p1475150010003090#2016-09-3014:07Alex Miller (Clojure team)can you repeat? I canāt keep up the slacks#2016-09-3014:07mpenetit used to compile (I think with alpha10)#2016-09-3014:07Alex Miller (Clojure team)yeah, thatās been logged#2016-09-3014:07mpenetprobably a result of the recent optimisations#2016-09-3014:07mpenet(guess)#2016-09-3014:07Alex Miller (Clojure team)would have changed as of alpha 11 or 12#2016-09-3014:07mpenetoki, good to hear#2016-09-3014:08Alex Miller (Clojure team)I mean changed as in āstarted failingā#2016-09-3014:08mpenetI couldn't find a ticket about it in jira#2016-09-3014:08Alex Miller (Clojure team)Iām not sure there is a good solution to it#2016-09-3014:08mpenetoh, isn't this considered a bug?#2016-09-3014:09Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-1966 is basically the same probelm#2016-09-3014:09mpenetit makes writing conformers a bit tricky, I basically have to (def invalid :clojure.spec/invalid) and use this in the body of the functions I am using#2016-09-3014:12Alex Miller (Clojure team)yeah, but wait for a spec on def to break that too :)#2016-09-3014:12mpenetš#2016-09-3014:12Alex Miller (Clojure team)itās a broader problem#2016-09-3014:12Alex Miller (Clojure team)one possible solution would be to do something like is done in the reader where a reader is handed a token that it can return to indicate a special case#2016-09-3014:13Alex Miller (Clojure team)so rather than being expected to return ::s/invalid, you are given a token that you can return (and that can be a per-instance (Object.)
)#2016-09-3014:13Alex Miller (Clojure team)something like that#2016-09-3014:14mpenetok#2016-09-3014:15Alex Miller (Clojure team)that still doesnāt solve the problems around core specs though#2016-09-3014:15Alex Miller (Clojure team)it needs a longer conversation with Rich and he hasnāt had the time to have it#2016-09-3014:15mpenetany hint of what might come in the next alpha?#2016-09-3014:28Alex Miller (Clojure team)whatever we fix or add next :)#2016-09-3014:30Alex Miller (Clojure team)CLJ-2024, CLJ-2027, and CLJ-2026 all have patches that are ready for Stu and Rich to look at so I expect those to be in the next alpha#2016-09-3014:30Alex Miller (Clojure team)(presuming they like them)#2016-09-3014:51mpenetActually walking the specs isn't really possible in my case: if you do (s/def ::foo ::bar)
(s/form ::foo) will have expanded ::bar to a predicate so I cannot check my stop point (since I don't want to expand down to the last bits, I have a set of specs that are the surface I want to expose to the front-end people)#2016-09-3014:53mpenet(s/def ::foo any?)
(s/def ::bar ::foo)
(s/form ::bar)
=> clojure.core/any?
#2016-09-3014:54mpenet((s/registry) ::bar)
works in that case actually#2016-09-3014:54mpenetnevermind#2016-09-3019:01jrheardiāve got a somewhat complex map spec that iām trying to s/exercise, and iām getting the dreaded "Couldn't satisfy such-that predicate after 100 tries.ā error.
what are the primary things i should be looking for when tracking down the root cause?#2016-09-3019:01jrheardiām familiar with s/and and how you should eg do int? first, followed by even?, etc#2016-09-3019:02jrheardis naive s/and
usage the primary trigger for this error, or are there other categories of spec misuse that often cause it?#2016-09-3019:02jrheard(if itās helpful, the spec iām trying to exercise is https://github.com/jrheard/voke/blob/spec/src/voke/specs.cljs#L74 )#2016-09-3019:02jrheardiāve only got one s/and in here, and itās a (s/and number? pos?)
, so i feel like there must be some other category of thing iām doing totally incorrectly#2016-09-3019:14jrheardhm, seems to be specific to :component/input#2016-09-3019:15jrheardthis guyās the culprit: https://github.com/jrheard/voke/blob/spec/src/voke/specs.cljs#L22#2016-09-3019:17jrheardthe documentation on :kind says: "Note that if :kind is specified and :into is not, this pred must generate in order for every to generate.ā; and adding :into #{}
fixes the issue. guess i need to sit down and take some time to understand the semantics of :kind
and :into
. thanks all ā¤ļø#2016-09-3019:26jrheard(if you click on those links after coming back from lunch, pretend that the (s/coll-of)
calls with :kind
specified do not have a corresponding :into
; thatās the state the file was in when i linked it, and thatās what the problem was š )#2016-09-3019:32seancorfield@jrheard Late to your spec party but, yeah, naĆÆve s/and
has been my primary trigger for that error ā Iāve taken to using s/with-gen
quite a lot to "help" clojure.spec exercise stuff.#2016-09-3020:04danielstocktonI s/fdefed a function but the doc string doesn't contain my spec. Did I miss something?#2016-09-3021:28kennyIs this the expected behavior?
(require '[clojure.core.specs :as clj-specs])
(def c (s/conform ::clj-specs/defn-args '(t
[x y]
x)))
=> #'boot.user/c
(s/unform ::clj-specs/defn-args c)
=> (t (x y) x)
Shouldnāt the args be unconformed to a vector, not a list?#2016-09-3021:36kenny(let [args '(t
[x y]
x)
args-s ::clj-specs/defn-args
c (s/conform args-s args)]
(s/conform args-s (s/unform args-s c)))
=> :clojure.spec/invalid
#2016-09-3021:42bfabryhmmmmm#2016-09-3021:43bfabry::arg-list seems to use s/and
not s/coll-of
#2016-09-3021:43kennyYeah: https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/specs.clj#L67#2016-09-3021:44bfabrystill, wouldn't flowing back through vector?
cause it to be turned into a vector? shrug#2016-10-0100:09jrheardwhatās the recommended way of actually running stest/check tests?#2016-10-0100:09jrheardiām in a cljs project, using lein-doo and cljs.test#2016-10-0100:10jrheardshould i be using (stest/summarize-results (stest/check `foo/bar)), and verifying that thereās no :check-failed key in the returned map? or is there a better way?#2016-10-0100:15jrheardeg https://github.com/pieterbreed/ansible-inventory-clj/blob/9857f22ccd92e21b629d5d79907e4c266bdcbc52/test/ansible_inventory_clj/core_test.clj#L79#2016-10-0109:31decoursinHow can I duplicate a spec?
(s/def ::my-spec number?)
(s/def ::a ::my-spec)
(s/def ::b ::my-spec)
This fails with:
=> #error {
:cause "Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k))"
:via
[{:type clojure.lang.Compiler$CompilerException
:message "java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k)), compiling:(ann.clj:63:1)"
:at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3657]}
{:type java.lang.AssertionError
:message "Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k))"
:at [clojure.spec$def_impl invokeStatic "spec.clj" 315]}]
:trace
[[clojure.spec$def_impl invokeStatic "spec.clj" 315]
[clojure.spec$def_impl invoke "spec.clj" 312]
[clojure.lang.AFn applyToHelper "AFn.java" 160]
[clojure.lang.AFn applyTo "AFn.java" 144]
...
#2016-10-0112:52Alex Miller (Clojure team)I canāt reproduce that - that should certainly work#2016-10-0112:52Alex Miller (Clojure team)what version are you using?#2016-10-0113:14puzzlerunform is not properly reversing vectors that have been conformed, e.g. (spec/unform :clojure.core.specs/defn-args (spec/conform :clojure.core.specs/defn-args '(f [x] x)))
=> (f (x) x)
#2016-10-0113:14puzzlerI checked that the spec "knows" that the args must be a vector.#2016-10-0113:15puzzlerI'm trying to make a defn-like macro by using spec to conform with the defn spec, then manipulate, then unform.#2016-10-0113:25Alex Miller (Clojure team)yeah, this is a known issue with the spec as itās not enforcing the vector part#2016-10-0113:26Alex Miller (Clojure team)we are talking about creating an s/vcat
as doing regex + vector is tedious right now#2016-10-0113:27Alex Miller (Clojure team)and itās relatively common in macro syntax#2016-10-0113:27Alex Miller (Clojure team)this is a good example of it#2016-10-0113:28Alex Miller (Clojure team)you can make it work using something like (s/and (s/cat ā¦) (s/conformer vec))
but like I said, itās tedious#2016-10-0113:30puzzlerSo you mean I should write my own more verbose version of the defn-args spec rather than use the one built into core?#2016-10-0113:31Alex Miller (Clojure team)well you could wrap defn-args too#2016-10-0113:33Alex Miller (Clojure team)part of being in alpha means things arenāt done#2016-10-0113:34puzzlerThe problematic piece is :arg-list
, which is a component of the :defn-args
spec: (s/def ::arg-list
(s/and
vector?
(s/cat :args (s/* ::binding-form)
:varargs (s/? (s/cat :amp #{'&} :form ::binding-form))))
#2016-10-0113:34puzzlerSo could I just "patch" arg-list?#2016-10-0113:37puzzlerI guess if I s/def :clojure.core.specs/arg-list
in my library, that will overwrite the spec for other consumers of my library as well, so maybe I shouldn't do that.#2016-10-0113:55Alex Miller (Clojure team)maybe you could make the patching a separate (temporary) step#2016-10-0115:27decoursin@alexmiller Thanks for having a look. It's working now. I'm not sure, if it comes up again, I'll let you know#2016-10-0120:12jrheardiām having a lot of trouble getting specās generative testing to work#2016-10-0120:12jrheardwhen i call (stest/check `foo/bar), and the output is [], how should i interpret that empty list?#2016-10-0120:14jrheardi get [] from calling (stest/check) on each fdefād function in this file, eg https://github.com/jrheard/voke/blob/spec/src/voke/input.cljs#L85 , and donāt really know where to go from here. has anyone seen this behavior? is there something obvious that iām doing wrong?#2016-10-0120:15jrheardexample code:#2016-10-0120:16jrheard(deftest generative
(let [output (stest/check `input/intended-directions->angle)]
(print "yo")
(print output)
(print (stest/summarize-results output))))
;; lein doo phantom test output:
Testing voke.input-test
yo
[]
{:total 0}
#2016-10-0120:19jrheardrelevant specs are in https://github.com/jrheard/voke/blob/spec/src/voke/specs.cljs#2016-10-0122:52lvhHm, going from -alpha11 to -alpha12/-alpha13 I get a pretty unhelpful NPE when calling explain on some data. clojure.spec portion of the stacktrace:
spec.clj: 864 clojure.spec/spec-impl/reify
spec.clj: 150 clojure.spec/conform
spec.clj: 782 clojure.spec/map-spec-impl/reify
spec.clj: 150 clojure.spec/conform
spec.clj: 731 clojure.spec/dt
spec.clj: 727 clojure.spec/dt
spec.clj: 1081 clojure.spec/explain-pred-list
spec.clj: 1118 clojure.spec/and-spec-impl/reify
spec.clj: 198 clojure.spec/explain-data*
spec.clj: 209 clojure.spec/explain-data
spec.clj: 202 clojure.spec/explain-data
#2016-10-0122:52lvhI dunno if anyoneās seen anything like that or if itās worth producing a minimal sample#2016-10-0122:53lvhwhich suggests that pred in spec-impl
is nil
#2016-10-0122:59lvhDid anything change about how registry-ref works?#2016-10-0123:04lvh(The weirdest thing that code path does is (with-redefs [s/registry-ref (atom @@#'s/registry-ref)] ā¦)
, which it needs to because itās generating specs so I donāt want to have tests fail because of stale state)#2016-10-0123:19lvhThat does not look like itās part of the problem.#2016-10-0123:36agI am dealing with strangest bug I cannot reproduce anywhere else: it works everywhere except Circle CI. Basically this spec:
(require '[com.gfredericks.test.chuck.generators :as gen']
'[clojure.spec :as s])
(def iso-date-regex #"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}-\d{2}:\d{2}$")
(s/def ::date-iso (s/with-gen (s/and #(re-find iso-date-regex %) string?)
#(gen'/for [d (gen'/datetime)]
(.format (js/moment (js/Date. d))))))
It works everywhere but on CI it says:
ERROR in (test-datetime-string) (:)
Uncaught exception, not in assertion.
expected: nil
actual: #error {:message "Couldn't satisfy such-that predicate after 100 tries.", :data {}}
#2016-10-0123:38agfirst I thought itās due to incompatible version of phantomjs. and then version of Leineingen, and then I disabled any parallelism on CI. and still canāt figure it out. I thought maybe I could replicate in a Docker container - still nothing#2016-10-0123:39agAnyone, any ideas?#2016-10-0123:39lvhhm, thatās weird: it would only make sense if the generator portion is being ignored#2016-10-0123:40agbut why would that happen only in Circle CI and nowhere else?#2016-10-0123:40lvhmaybe you should create a test PR that s/gen + gen/samples that spec#2016-10-0123:40lvhthat could be a few reasons, including profile.clj files or whatever#2016-10-0123:40lvhs/gen + gen/samples will tell you what it thinks the generator is for that spec#2016-10-0123:41lvhhow did you determine it wasnāt an incompatible version of phantomjs?#2016-10-0123:42agI am replacing phantom with the right version#2016-10-0123:43gfredericksLEIN_NO_USER_PROFILES=1 can be helpful for debugging things that only happen in ci#2016-10-0123:45lvhIāve seen a lot of problems with Phantom where phantom versions insisting they were the same, although that regex does look innocuous#2016-10-0123:45lvh1.9.8 is like seventeen different webkits#2016-10-0123:45gfredericks Also have you rerun the ci? That "such-that" error could also just be highly improbable#2016-10-0123:46lvhWouldnāt the such-that error never happen with with-gen? I thought that with-gen did not check that generated values match the spec?#2016-10-0123:48lvh(that may have changed in recent alphas, or not be true in JS, or I might be misremembering entirely of course)#2016-10-0123:49gfredericksI don't know where the such-that came from actually#2016-10-0123:50lvhwell, Iām thinking of the spec by itself#2016-10-0123:50lvh(s/and #(re-find iso-date-regex %) string?)
#2016-10-0123:50lvhIIRC s/and uses the first arg to determine type#2016-10-0123:50lvhso it wonāt even be able to guess āstringā let alone āstring that incidentally matches this regex"#2016-10-0123:51lvh(Sorry, to be less obtuse: I think the such-that is coming from the s/and
)#2016-10-0123:52gfredericksOh I didn't see an and#2016-10-0123:53gfredericksIt shouldn't matter though since it's wrapped in with-gen#2016-10-0123:55lvhexactly#2016-10-0123:56lvh@ag: Can you please dump the full test ns somewhere#2016-10-0123:56lvhideally a reproducible sample#2016-10-0200:02jrheard@gfredericks - have you ever seen s/check output the empty list like i mentioned above? sorry to bug you directly, iāve just seen your name around test.check-related things, figured you might have an answer off the top of your head š#2016-10-0200:02jrheardiām a complete newbie to generative testing, so for all i know this is normal well-understood behavior, but iām having trouble making heads or tails of it#2016-10-0200:03gfredericksI expect it's a clojure.spec-related thing#2016-10-0200:04gfredericksI've seen that happen when I've used it but can't remember why#2016-10-0200:04jrheardcool, iāll keep poking at it#2016-10-0218:36lvhI found my problem from yesterday: between -11 and -12, a nil
spec changed meaning#2016-10-0218:36lvhostensibly from meaning any? to silent failure#2016-10-0218:37lvhI appreciate that there might be a bootstrapping failure, but that seems like it could be found at spec definition time and not with an NPE thrown from a reify#2016-10-0218:43bfabrygenerated values are still checked against the spec, I'm guessing there's something in the environment causing .format to return a differently formatted date#2016-10-0218:52lvhoh, huh; I wonder if thatās now or Iām just horribly misremembering#2016-10-0218:52lvh(s/def ::a integer?)
(s/def ::b (s/with-gen string? #(s/gen ::a)))
(sg/sample (s/gen ::a))
(sg/sample (s/gen ::b))
.. confirms#2016-10-0218:52lvhI guess .format isnāt awfully specific#2016-10-0218:53bfabrythere's probably some dumb version of the JS engine that returns "Monday the 2nd of Feburary 10:13PM Australian eastern daylight savings time"#2016-10-0218:57lvhright#2016-10-0218:57lvhMoment.js takes explicit format strings though, so hopefully easy to recover from#2016-10-0218:57lvh@ag ^#2016-10-0223:11Alex Miller (Clojure team)@lvh There was a bug introduced with conforming nilable in 12 that is fixed in 13#2016-10-0223:13lvh@alexmiller In -alpha11, a nil
spec was effectively any?
afaict#2016-10-0223:13lvhthatās gone in -12 and -13, which broke my tests; but really it was just exposing something that was a bug anyway#2016-10-0223:15lvhthis is not a problem unless youāre generating specs, which I guess is the part of clojure.spec Iām exercising a little more that most folks š#2016-10-0223:15lvh(my generated specs included nils by accident; that was actually a bug)#2016-10-0223:54jrheard@alexmiller - have you seen situations where (stest/check `foo/bar) returns []? what type of mistake does that return value usually indicate?
iām having a hard time figuring out what causes it - it seems to return [] sometimes and eg [{:spec #object[cljs.spec.t_cljs$spec8685], :clojure.test.check/ret {:result true, :num-tests 1000, :seed 1475452376922}, :sym voke.input/intended-directions->angle}] other times, the behavior varying from run to run when none of the code has changed#2016-10-0300:10gfredericksthat sounds pretty weird#2016-10-0301:27jrheardi think tomorrow iām just gonna have to learn how to use checkout dependencies and start adding printfs to spec.test code so i can see whatās going on in there#2016-10-0302:21bhagany@jrheard fwiw, Iām eager to hear what you learn. I tried getting doo to run my cljs spec tests a few weeks ago, and hit a wall.#2016-10-0304:02jrheardwell, for starters i think iāve found a small issue with the cljs.spec documentation#2016-10-0304:03jrheardhttps://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L210 says that options maps passed to (s/check) should look like {:clojure.spec.test.check/opts {:num-tests 100}}
#2016-10-0304:03jrheardbut actually, they should look like {:clojure.test.check/opts {:num-tests 100}}
#2016-10-0304:36jrheardi havenāt managed to reproduce this [] return value issue using my local checkout of clojurescript, so maybe this behavior has been fixed on master? will poke at it some more in the morning#2016-10-0304:55seancorfieldAre you sure about the options map @jrheard ? When I was trying that on Clojure, it definitely needed to be :clojure.spec.test.check/opts
(even thoā no such namespace exists).#2016-10-0304:56seancorfieldIf it really is the latter, as you say, then thatās a bug in my opinion ā ClojureScript should follow Clojure there I think?#2016-10-0306:56decoursinHow to override keys in (s/merge ..)
? My failing attempt:
(s/def :my-ns/a string?)
(s/def :my-other-ns/a int?)
(gen/generate (s/gen (s/merge (s/keys :req-un [:my-ns/a])
(s/keys :req-un [:my-other-ns/a]))))
=> ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
#2016-10-0307:18misha@decoursin: what do you expect merged spec to be?#2016-10-0307:20misha"Int or string"?#2016-10-0307:23decoursinEither one, make a choice. Always take the first or always take the second.#2016-10-0307:25decoursinprobably the same way clojure.core/merge
works which is the second (or last one.)#2016-10-0307:25decoursin(merge {:a 3} {:a 4})
=> {:a 4}
#2016-10-0307:27mishaTakes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs. Unlike 'and', merge can generate maps satisfying the
union of the predicates.
does not sound like it can satisfy int?
and string?
with a same value on a single key#2016-10-0307:29bfabryI think in terms of usefulness the way it's behaving atm is probably best, though I could see an argument for it behaving like core.merge#2016-10-0307:30decoursinYeah I think it should behave like core.merge. I'd rather see that what @misha described to be called like union or something#2016-10-0308:22decoursinHow could I do this then? Have one key override another#2016-10-0308:42mishawhat is your use case? or rather "problem"#2016-10-0311:31mpenetis there a way to validate against a (dynamic) subset of keys in a map? ex I define an exhaustive spec for a map (with mixed req/opt), and at runtime I want to be able to validate against a subset of it#2016-10-0311:32mpeneta bit in the flavor of s/merge
, I'd like s/select-keys
(kind of)#2016-10-0311:33mishayou can give names to those subsets, and just merge those in uber-map-spec. no?#2016-10-0311:34mpenetI dont want to define as many subsets are they are key combos no#2016-10-0311:34mpenetex with Schema you can just do (since map specs are just Maps) :
(defn validate-subset
[schema value]
(-> (select-keys schema (keys value))
(s/validate value)))
#2016-10-0311:35mpenetseems like s/select-keys
could be a nice addition actually, thoughts @alexmiller ?#2016-10-0311:38mpenetjust putting every key as optional is not an option either#2016-10-0311:42mishagiven spec's "global" nature, I'd go and name all the combos.
also multi-spec
might be a good fit too#2016-10-0311:42mpenetthat's a lot of combos for a 20 key map for instance ... it'd be horrible#2016-10-0311:43mpenetI'll check multi-spec, but my gut feeling it doesn't fit that problem either#2016-10-0311:43mishadepending on how you'd define combos out of those 20#2016-10-0311:44mishayou might have only 2-3 "domain-valid" combos.#2016-10-0311:44mpenetdoesn't matter#2016-10-0311:44mpenetin my case the user can update any of them and I cannot pass all of it#2016-10-0311:45mpenet(there are concurrent update issues for the "pass the whole thing" case)#2016-10-0311:45mishanot enough information then#2016-10-0311:46mpenetit's a common enough problem imho, any REST api would hit it for partial update for instance#2016-10-0311:47mpenetthe least horrible (yet it sucks) imho would be to call valid? against every key in the set but yuk. Or just have a s/keys
spec with all as optional for updates, but that's a lot of duplication I'd like to avoid#2016-10-0311:47mishathen probably I'd have 20 different specs defined, and constructed map spec on demand based on incoming map's keys#2016-10-0311:48misha20 for possible keys, + maybe some for key combinations, like "if that one present, those 2 are required".#2016-10-0311:51mishaa collection of specs for possible keys should not necessarily be a ready to use map spec.#2016-10-0311:51mpenetnot going to do that, that's truly awful, not to mention for 20 fields (the example here but some of our records have more than this) that's a large number of specs (a lot more than 20)#2016-10-0311:52mishanot everything can be solved with oneliners kappa#2016-10-0311:52mpenetit's not the point, you don't have a solution to this, no biggie#2016-10-0311:53mpenetAt the moment there's no elegant way to solve this with spec.#2016-10-0312:29misha@mpenet how about this?
(s/def :foo/bar int?)
(s/def :foo/baz string?)
(def schema #{:foo/bar :foo/baz})
(defn validate-subset
[schema m]
(let [ks (filterv schema (keys m))]
(s/valid? (s/keys :req `[
(validate-subset schema {:foo/bar 1 :foo/baz "y"})
=> true
(validate-subset schema {:foo/baz "y"})
=> true
(validate-subset schema {:foo/baz 1})
=> false
#2016-10-0312:35mpenetThat's more or less what I mentioned earlier, but I dont like it. Having 1 separate spec with all opt keys is probably nicer#2016-10-0313:46danielstocktonIs there a way to instrument everything at once, rather than individual functions?#2016-10-0313:47Alex Miller (Clojure team)Just call instrument with no args#2016-10-0313:52danielstocktonIs there a good way to hook this into the test runner? I imagine it can be quite helpful to know not only what tests failed but which functions got unexpected data.#2016-10-0313:55mpenetI think you can just call instrument at top level of your test#2016-10-0313:55danielstocktonI suppose, i'd have to do it in every namespace#2016-10-0313:56danielstocktonMight also be useful to turn it on whenever starting lein repl or in a dev environment#2016-10-0314:09mpenet@alexmiller (s/valid? #{false} false) => false
that's what you mention on github?#2016-10-0314:09mpenetit does look like a bug from the outside (without considering/knowing how it's implemented under the hood)#2016-10-0314:14mpenetI guess its (#{false true} false)
#2016-10-0314:14Alex Miller (Clojure team)Yes that's what I meant. It's not a bug, just a consequence of using sets as specs#2016-10-0314:14mpenetok#2016-10-0314:14Alex Miller (Clojure team)Sets with falsey values won't work #2016-10-0314:15Alex Miller (Clojure team)So don't do that#2016-10-0314:15Alex Miller (Clojure team)You've used a function that returns false for what you consider to be a valid value #2016-10-0314:16mpenetyep, got it#2016-10-0314:26mpenetactually about my s/select-keys
proposal earlier, why not making s/keys spec an c.l.Associative instance and allow us to compose this stuff with normal core fn?#2016-10-0314:27mpenetmaybe it's crazy talk#2016-10-0315:35Alex Miller (Clojure team)the problem with that is that we capture the key set for describe and form, so you would lose that ability#2016-10-0315:36Alex Miller (Clojure team)thatās the reason itās a macro and not a function that takes data now#2016-10-0315:42Alex Miller (Clojure team)@jrheard no, havenāt seen that#2016-10-0315:59mlimottehi. I'm just starting to play with clojure.spec. I'm ok with some basic specs and validation that I've tried. But having trouble with even a trivial example of specing a higher-order fn. Here's what I'm seeing:
(def my-spec (s/fspec :ret string?))
=> #'user/my-spec
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:568)
#2016-10-0316:04Alex Miller (Clojure team)(s/fdef my-spec :ret string?)
will be better for you right now#2016-10-0316:05Alex Miller (Clojure team)sorry, I misread that first def as s/def, let me read again#2016-10-0316:06Alex Miller (Clojure team)so the issue here is that to verify that the function youāve passed is valid, it will generate args based on the :args spec for the function and invoke it#2016-10-0316:06Alex Miller (Clojure team)but youāve passed no args spec#2016-10-0316:08mlimottetrue. I was going with a simple example -- just validating that it is a fn that returns a string. I did try with args before, and was getting different errors, so I was trying to simplify the example.#2016-10-0316:08mlimottei'll try w/ args again#2016-10-0316:08Alex Miller (Clojure team)although I am seeing some weird stuff on this path#2016-10-0316:09mlimotte(s/def my-spec (s/fspec :args (s/tuple integer?) :ret string?))
=> user/my-spec
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:568)#2016-10-0316:09mlimotteš same thing, even with args.#2016-10-0316:14Alex Miller (Clojure team)ah, so conform takes a qualified keyword or symbol#2016-10-0316:14Alex Miller (Clojure team)(s/conform `my-spec (fn [j] (str j)))
#2016-10-0316:14Alex Miller (Clojure team)will fully-qualify my-spec#2016-10-0316:14jrheard@seancorfield yeah, very sure- compare clojureās https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L19 vs cljsās https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljs#L19 [and also iāve verified interactively]#2016-10-0316:15Alex Miller (Clojure team)@jrheard the idea in both of those is to pass a map of options through to test.check - I think the underlying option keys differ in clj vs cljs test.check#2016-10-0316:16Alex Miller (Clojure team)so there may be a disconnect between clj vs cljs and docs here#2016-10-0316:16jrheardyeah, i think the cljs docs need to be updated to reflect the disconnect - https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L211#2016-10-0316:16jrheardi was using :clojure.spec.test.check/opts as the docs recommend, but nothing was happening#2016-10-0316:16Alex Miller (Clojure team)I suspect so - prob should file a jira (and if you like a patch!)#2016-10-0316:16mlimotteahh! it does say that in the spec guide! Maybe it would be helpful for the conform and valid? docstring(s) could also highlight that requirement.#2016-10-0316:16jrheardsounds good, itāll be my first of both of those š time to go read the contributing guidelines again#2016-10-0316:17Alex Miller (Clojure team)@mlimotte seems like there should be a better error message in this case#2016-10-0316:17Alex Miller (Clojure team)I will log that and add a patc hfor it#2016-10-0316:19mlimotteyep. that would help. here's another question. What's a recommended way to spec the requirement that the fn should be a one-arg fn? I tried:
(s/def my-spec (s/fspec :args (s/tuple identity) :ret string?))
(s/def my-spec (s/fspec :args #(= (count %) 1) :ret string?))
#2016-10-0316:20mlimottewhich results in
ExceptionInfo Unable to construct gen at: [0] for: identity clojure.core/ex-info (core.clj:4617)
ExceptionInfo Unable to construct gen at: [] for:
#2016-10-0316:21Alex Miller (Clojure team)(s/def my-spec (s/fspec :args (s/cat :j any?) :ret string?)
#2016-10-0316:21jrheard(iāve made a jira account, username jrheard - alex, are you the right person to bug to add whatever necessary permissions to that account, or should i go bother someone else?)#2016-10-0316:22Alex Miller (Clojure team)you can add tickets with the starting permissions#2016-10-0316:22jrheardawesome#2016-10-0316:24mlimottewhat is any?
, AFAICT that is not in clojure.core nor clojure.spec#2016-10-0316:24Alex Miller (Clojure team)itās in core#2016-10-0316:24lvhItās new in Clojure#2016-10-0316:24Alex Miller (Clojure team)itās just (constantly true)#2016-10-0316:25lvhhm; I figured it might take only 1 arg#2016-10-0316:25Alex Miller (Clojure team)actually it does#2016-10-0316:25Alex Miller (Clojure team)I was more sketching than saying the actual def :)#2016-10-0316:25Alex Miller (Clojure team)(defn any?
"Returns true given any argument."
{:tag Boolean
:added "1.9"}
[x] true)
#2016-10-0316:26lvhah, gotcha š#2016-10-0316:27mlimottei see. I'm using clojure-future-spec w/ clojure 1.8, so this works:
(s/def my-spec (s/fspec :args (s/cat :j clojure.future/any?) :ret string?))
#2016-10-0316:28Alex Miller (Clojure team)ah#2016-10-0316:28mlimotteas does:
(s/def my-spec (s/fspec :args (s/tuple clojure.future/any?) :ret string?))
Was your choice of s/cat over s/tuple just stylistic, or is there something more significant?#2016-10-0316:28Alex Miller (Clojure team)@jrheard I added edit permissions for you as well in jira#2016-10-0316:29jrheardthanks!#2016-10-0316:31jrheardopened CLJS-1808 , will submit a patch later today#2016-10-0316:31Alex Miller (Clojure team)thanks#2016-10-0316:32Alex Miller (Clojure team)@mlimotte tuple is fine too. we usually use regex ops to describe args. Args are effectively syntax and thatās what regex ops in spec are the best match for.#2016-10-0316:33Alex Miller (Clojure team)one benefit of cat is that you will get named parts for conforming the parts whereas you will get just indexes for tuple#2016-10-0316:34Alex Miller (Clojure team)so that affects both the conform result (which youāll see in the :fn spec) and the explain errors#2016-10-0316:34Alex Miller (Clojure team)generally I find being able to tie the spec to the arg name helpful#2016-10-0316:35mlimottegot it. thanks for the help.#2016-10-0317:32Alex Miller (Clojure team)@mlimotte fyi, http://dev.clojure.org/jira/browse/CLJ-2032#2016-10-0317:32mlimottethx#2016-10-0317:43jrheard@alexmiller - i just added a couple patches to the ticket. is there anything else i need to do, or should i just wait and expect to get a response at some point in the future?
iām asking you this not because iām in an incredible hurry to have someone immediately look at this jira issue this very second; i just want to make sure iām not missing some ānotify the maintainersā step before i close all these tabs š thanks!#2016-10-0317:47Alex Miller (Clojure team)@dnolen: ^^#2016-10-0317:48Alex Miller (Clojure team)For cljs-1808#2016-10-0317:48mlimotteCould there be a problem with s/tuple? I have the code below, which bombs w/ the error below, but if I change the '(s/tuple integer?)` to (s/cat :a integer?)
than it works.
(defn myfn [a] (str a))
(s/fdef myfn :args (s/tuple integer?) :ret string?)
(stest/instrument `myfn)
(myfn 1)
I get
=> #'user/myfn
=> user/myfn
=> [user/myfn]
ExceptionInfo Call to #'user/myfn did not conform to spec:
val: (1) fails at: [:args] predicate: vector?
:clojure.spec/args (1)
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file "form-init1342783443404146057.clj", :line 4, :var-scope user/eval30918}
clojure.core/ex-info (core.clj:4617)
#2016-10-0317:50bfabry@mlimotte s/tuple
specifically expects its input to be a vector, the :args list is not a vector#2016-10-0317:51bfabrys/cat
just validates a sequence, which the :args list is#2016-10-0317:53mlimottei see. in retrospect, kind of obvious from the error message. i should have gotten that.#2016-10-0317:55dnolen@jrheard just assign it to me I will look at it when I have time#2016-10-0317:55jrheardgreat, will do, thanks!#2016-10-0321:15tokenshiftCan s/conform handle coercion? For example, could I have an s/def that defines a ::uuid to be either a (valid) string or a java.util.UUID, and always conforms it to a java.util.UUID?#2016-10-0321:28lvhtokenshift Check out āconformer"#2016-10-0321:29tokenshiftIād pass a conformer (the result of s/conformer) to s/def, e.g. (s/def ::foo (s/conformer #(do-stuff %)))?#2016-10-0321:31tokenshiftAnswered my own question; yep, that works#2016-10-0321:31tokenshiftThanks!#2016-10-0321:33tokenshiftNow I have to decide if thatās a good pattern, or if itād be better to keep input coercion completely separate from specs#2016-10-0321:38lvhI think itās fine to conform to a specific object#2016-10-0321:38lvhconforming is not something that you want in your hot loop though IIUC; itās a measurable performance impac#2016-10-0321:38lvht#2016-10-0321:39lvhbut at the edges? sure; youāre gonna do that anyway, might as well have it be declarative#2016-10-0321:39tokenshiftHmm, thatās a good point#2016-10-0321:39tokenshiftIf itās only at the edges, then I can assume the input will be a string (for a web service, at least)#2016-10-0321:40tokenshiftThough if the backend requires a UUID, Iād probably still want to coerce it as early as possible.#2016-10-0321:40tokenshiftI guess if the coercion is lossless, itās safe; but if itās lossy, Iād want to have an explicit coercion step.#2016-10-0321:48tokenshiftHeading out#2016-10-0321:48tokenshiftThanks for the help!#2016-10-0400:10jrheardheh, enabling (s/instrument) on all instrumentable vars does not do wonders for my gameās performance#2016-10-0400:10jrheardinvaluable for finding the 7 random spots where iād goofed when converting to namespaced keywords, though š#2016-10-0400:11jrheardiām not complaining, i absolutely expected this degree of perf degradation when instrumenting like every function in this tight loop, it was just funny to go from 60fps to 0.5 š#2016-10-0400:15jrheardinstrumenting only at the edges brings me back up to good perf, nice#2016-10-0400:21agguys, you remember me struggling with this:
https://clojurians.slack.com/archives/clojure-spec/p1475364998003541#2016-10-0400:22jrheardyeah!#2016-10-0400:22agso it turned out that my regex is bad#2016-10-0400:22agit does not take into account timezones#2016-10-0400:22agthat minus sign sometimes is a plus sign#2016-10-0400:23agalthough I think it would be helpful if spec told me why exactly it couldnāt satisfy the predicate#2016-10-0400:25jrheardi worked with a guy once whose most famous quote around the office was:
<jlatt> dates suck
<jlatt> f&@# time#2016-10-0400:25jrheardiāve had similar feelings when running into the āCouldnāt satisfy such-that predicate after 100 triesā error - i wish there was some way it could tell me how iāve messed up#2016-10-0400:25agalso I think test.chuck generator does not generate values with random timezones. maybe it should#2016-10-0400:25jrheardiām not sure how it could do that though#2016-10-0413:07minimal@danielstockton :args (s/cat :arg1 (s/and ::query (s/+ (s/or :db map? :coll vector?)))
#2016-10-0413:20danielstocktonHas this changed? http://clojure.org/guides/spec#_spec_ing_functions#2016-10-0413:20danielstocktonArgh, I think I want s/cat
instead of s/and
actually#2016-10-0413:21danielstocktonnow i see that s/and
is adding two specs#2016-10-0413:21danielstocktonyep, thanks @minimal, somehow i was blind to that#2016-10-0422:23seancorfieldWe officially have clojure.spec-based code in production now ā Clojure 1.9.0 Alpha 13 and all of our REST API search code relies on spec for validation and conformation!#2016-10-0423:08jcsims@seancorfield any plan to write about it?#2016-10-0423:35seancorfield@jcsims I submitted a proposal to Clojure/conj about itā¦ so if that gets accepted, Iāll have to write about it. If it doesnāt get accepted, Iāll probably still write it up for my blog, but in a very different format I expect.#2016-10-0423:36seancorfieldPart of my concern is that core to a lot of what weāre doing with spec is something that @alexmiller said is essentially an anti-pattern (heavy use of s/conformer
).#2016-10-0423:36seancorfieldThat, and we are still at times struggling mightily with how to apply spec to some situations...#2016-10-0423:36jcsimswell, looking forward to it either way :+1:#2016-10-0507:02mpenetIs there a workaround to http://dev.clojure.org/jira/browse/CLJ-2033 for un-namespaced keys ?#2016-10-0507:02mpenetI just hit this too#2016-10-0507:04mpenetit makes s/merge a bit useless tbh, not sure why it's not considered a bug#2016-10-0507:05mpenetI guess a workaround would be to do the composition at a higher level, and not rely on spec/merge for merging specs like this, which is... odd#2016-10-0507:14mpenetand the fact that it's macros all the way down makes this workaround also a bit hairy#2016-10-0508:43jmglovIs it possible to spec protocols in any way?#2016-10-0508:43jmglovOr objects that satisfy them?#2016-10-0511:08mpenetI think it's encouraged to wrap protocol fn invocation, so you can instrument/spec the wrapper fn. otherwise if you mean spec predicate it's just #(satisfies? %), possibly with (s/spec pred :gen ... ) if you want to have the gen part#2016-10-0511:43odinodinAnyone used spec with DataScript? Need tips on how to deal with the fact that entities are not maps, and specifically how to get the same functionality as clojure.spec/keys but for Datascript entities#2016-10-0511:53misha@odinodin what do you mean by "not maps"? pull
pretty much returns maps.#2016-10-0511:54mishawhat exactly do you want to cover with spec?#2016-10-0511:54odinodinI have a function that takes DataScript entities#2016-10-0511:55odinodinI'd rather avoid creating maps from entities, and just use entities directly#2016-10-0511:56mishaoh, you mean thing returned by d/entity
? no idea : )#2016-10-0511:57odinodinš#2016-10-0512:29Yehonathan SharvitIs it possible to define a spec
that references itself in the definition - like in Context-Free Grammars?#2016-10-0512:30Alex Miller (Clojure team)Sure, via it's registered keyword#2016-10-0512:30Alex Miller (Clojure team)Recursive and mutually recursive specs are fine#2016-10-0512:31Yehonathan SharvitCould you share an example?#2016-10-0512:32Alex Miller (Clojure team)There are several in clojure.core.specs - destructuring is recursive#2016-10-0512:33Alex Miller (Clojure team)Some of the ns stuff with prefix lists , etc#2016-10-0512:36Alex Miller (Clojure team)::prefix-list there is self-recursive#2016-10-0512:37Alex Miller (Clojure team)::binding-form can be ::seq-binding-form or ::map-binding-form, which can both include ::binding-form#2016-10-0512:38Yehonathan Sharvitthx#2016-10-0512:38Yehonathan SharvitIām looking at it#2016-10-0512:40Alex Miller (Clojure team)A really good example to try is just a simple tree of leaf and branch#2016-10-0512:41Alex Miller (Clojure team)Where branches can contain either leaf or branch#2016-10-0513:49mlimotteWhat's the best way to trigger clojure.spec tests with lein test? I.e. I have a namesapce with (stest/check (stest/enumerate-namespace 'wwai.common.util.instant))
, I want it run those checks when I do lein test
.
I'm really using expectations
, but if I see how to make it work with standard test, I should be able to adapt from their.#2016-10-0516:48ag@mlimotte clojure.test.checkās defscpec
worked for me#2016-10-0516:52jrheard@ag iād love to see an example of that on github if you have one, np if not, just figured iād check š#2016-10-0518:08mlimotteI came up with this hack, which kind of works, although the output in the error case, isn't very friendly:
(defmacro is-result-ok?
[result]
`(is (not (:failure (first ~result)))))
(deftest auto
(doseq [spec-fn-sym (stest/enumerate-namespace 'wwai.common.util.instant)]
(is-result-ok? (stest/check spec-fn-sym))))
#2016-10-0518:09mlimotteI looked at defspec, but didn't see how to make it work with stest/check.#2016-10-0518:20jrheardme neither!#2016-10-0518:51luxbockwhat's the idiomatic way to express lack of arguments for a function with fdef
?#2016-10-0518:52luxbockjust empty?
should do I guess#2016-10-0518:53bfabry@luxbock (s/cat)#2016-10-0518:54luxbockwhat's the benefit over just using empty?
#2016-10-0518:55bfabrynothing afaik, it's just that s/cat
is the normal way to start specifying :args#2016-10-0518:56luxbockalright#2016-10-0519:07luxbockI would've expected this to throw: https://gist.github.com/luxbock/532a19f75553ae938c0a998e3be47851#2016-10-0519:11bfabryinstrument does not check return values#2016-10-0519:13luxbockI see#2016-10-0519:15bfabryyou're probably looking for clojure.spec.test/check#2016-10-0519:16bfabryit runs the function a bunch of times with generated args (in this case always nothing) and checks the result conforms to :ret and that any supplied :fn relationship between :args and :ret is true#2016-10-0519:17bfabryboot.user=> (s/def ::number int?)
:boot.user/number
boot.user=> (s/def ::number-list (s/coll-of ::number :min-count 1))
:boot.user/number-list
boot.user=> (s/fdef foobar
#_=> :args (s/cat)
#_=> :ret ::number-list)
boot.user/foobar
boot.user=> (defn foobar [] [])
#'boot.user/foobar
boot.user=> (clojure.spec.test/check `foobar)
({:spec #object[clojure.spec$fspec_impl$reify__13891 0x222df61e "
#2016-10-0519:20luxbockthanks, yeah the function which I simplified here probably won't need a spec at all, but it's good to know this for the future#2016-10-0520:02jasonjcknhow do I specify that ::start_date must start before ::end_date in a map#2016-10-0520:02jasonjcknso far I have (s/keys :req-un [::start-date ::end-date]) but I need to add in a predicate#2016-10-0520:02jasonjcknthe relationship between these two#2016-10-0520:07jrheardperhaps you could write a constructor function rather than making instances of that map by hand, and itās got an (s/fdef) that encodes that information?#2016-10-0520:08jrhearddoesnāt 100% solve the problem though#2016-10-0520:09mlimottemaybe you could use s/and and a predicate, something like:
(s/and (s/keys :req-un [::start-date ::end-date]) #( date-before? (:start-date %) (:end-date %)))
#2016-10-0520:09jrheardoh interesting#2016-10-0520:09jrheardi like that better#2016-10-0520:10jasonjckncool thanks, testing now#2016-10-0520:38luxbockwhat is the use case of the optional argument unf
to conformer
?#2016-10-0520:40bfabry@luxbock I'm guessing it's for a custom unformer#2016-10-0520:42luxbockah right, yeah I get it now#2016-10-0520:44bfabryboot.user=> (s/def ::thing (s/conformer {1 "foo"}))
:boot.user/thing
boot.user=> (s/unform ::thing (s/conform ::thing 1))
java.lang.IllegalStateException: no unform fn for conformer
boot.user=> (s/def ::thing (s/conformer {1 "foo"} {"foo" 1}))
:boot.user/thing
boot.user=> (s/unform ::thing (s/conform ::thing 1))
1
#2016-10-0521:47jasonjcknis there a way to include doc string with the spec?#2016-10-0521:47jasonjckne.g. (s/and (s/keys :req-un [::start-date ::end-date]) "start date must come before end date" #( date-before? (:start-date %) (:end-date %)))
#2016-10-0521:48jasonjckn(and retrieve the doc string in explain-data)#2016-10-0522:05jrheardhas anyone else had trouble doing eg (stest/instrument (stest/enumerate-namespace āmy.ns))
in cljs?#2016-10-0522:06jrheardi see a stacktrace with error messages like java.lang.RuntimeException: No such namespace: stest, compiling:(/private/var/folders/zl/bh7pbyz95rg7pmcvcc_f_kdm0000gn/T/form-init2230077429322923281.clj:6:19)
; gonna dig into it a bit, just curious if this is known / expected / if anyone else has dealt with this#2016-10-0522:17jrheardif i do eg
(let [syms (stest/enumerate-namespace 'voke.events)]
(stest/instrument syms))
then i get
Caused by: clojure.lang.ExceptionInfo: java.lang.RuntimeException: Unable to resolve symbol: syms in this context
feels like i must be doing something wrong#2016-10-0522:18jrheardperhaps related to the (eval) call in https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L110 ?#2016-10-0522:22jrheard@dnolen am i doing something wrong / is this known/expected?#2016-10-0522:23dnolen@jrheard yes wonāt work, not intended to work#2016-10-0522:23jrheardcool, good to know#2016-10-0522:23jrheardshould i just instrument vars by hand, naming one at a time, or is there some better way?#2016-10-0522:23dnoleninstrument
is a macro - must take symbols that reference vars directly#2016-10-0522:23dnolenjust skip the locals stuff#2016-10-0522:23jrheardi donāt follow#2016-10-0522:24dnolenremove the let binding and it will work#2016-10-0522:24dnoleni.e. do this inline#2016-10-0522:24jrheardwhen i do
(stest/instrument (stest/enumerate-namespace 'my.ns))
, i get an error message complaining that stest doesnāt exist#2016-10-0522:24jrheardis that what you mean by doing this inline, or am i misunderstanding?#2016-10-0522:24dnolenso thatās a bug, file an issue in JIRA#2016-10-0522:24jrheardgotcha, will do! thanks!#2016-10-0522:25dnolenthanks, to be clear, do this in the CLJS JIRA#2016-10-0522:25jrheardyou got it#2016-10-0522:30jrheard( CLJS-1811 )#2016-10-0523:58Oliver GeorgeToday's random spec experiment: Can you define an SQL schema from a clojure.spec definition?#2016-10-0523:58Oliver GeorgeHere's what I came up with: https://gist.github.com/olivergeorge/27e9fced404f5ddef371ea3376456f2f#2016-10-0523:59Oliver George(do
(s/def ::name string?)
(s/def ::address (varchar 256))
(s/def ::age integer?)
(s/def ::gender #{"M" "F"})
(s/def ::example-table (s/keys :req-un [::name ::age ::gender]
:opt-un [::address]))
(spec->sql-table ::example-table))
#2016-10-0523:59bfabryoooh I should bookmark this. I'm definitely going to need spec->avro schema at some point#2016-10-0523:59Oliver GeorgeProduces
[["name" "varchar2(4000)" "not null"]
["age" "int" "not null"]
["gender" "varchar2(1)" "not null"]
["address" "varchar2(256)"]]
#2016-10-0600:00Oliver George@bfabry no promises that it's the most elegant approach. I'm looking forward to seeing how others work with spec forms.#2016-10-0600:16mattlyheh, I'm actually working on something to define graphql forms from specs definitions#2016-10-0600:17jrheardunrelated: youāre using graphql? dāyou like it?#2016-10-0600:17jrheardi havenāt used it myself but the talks iāve seen on it are pretty compelling#2016-10-0600:20Oliver George@mattly that's on my radar too. We are replacing our simple "datomic style" pull api with graphql soon. Here's what I came up with generating recursive pull specs from clojure.spec forms: https://gist.github.com/olivergeorge/9d822447e4838092d07138ea51af5782#2016-10-0600:21mattlyoh neat#2016-10-0600:21Oliver George(again, no promises that it's a good approach... really want to see other peoples code and learn some effective techniques)#2016-10-0600:22mattlyI'm going to keep my pull-style api, but aim to hookup a graphql parser to it#2016-10-0600:23Oliver GeorgeThat makes sense.#2016-10-0600:24mattlymostly because some of my query endpoints, I need expressive arguments#2016-10-0600:26Oliver GeorgeWe bolted a "pull" arg to our REST endpoints which was a practical step forward at the time. Same old REST params etc but more flexibility around what data we return. Now that GraphQL is looking fairly stable we're intrested in standardising on it's interface. That decouples our frontend from the backend which should allow us some flexibility.#2016-10-0600:27mattlyI'm fairly lucky in that mine is a greenfield project#2016-10-0600:28Oliver GeorgeNice. We're in the enterprise space so typically the backend tech stack is dictated to us.#2016-10-0600:29mattlyI may try to abstract out my resolver into a library#2016-10-0600:30mattlyit has a separate planning stage to help with the n+1 problem#2016-10-0600:30Oliver George@mattly I'd love to see what you come up with.#2016-10-0600:33Oliver GeorgeI'm not sure it's related but I did some thinking around translating graph queries into flat sql queries.#2016-10-0600:34Oliver GeorgeThis paper gave me the idea: http://homepages.inf.ed.ac.uk/slindley/papers/shredding.pdf#2016-10-0600:37mattlyeh, removed the link, there's a bug in that#2016-10-0600:37mattlyI need to properly extract it#2016-10-0600:38Oliver Georgeš#2016-10-0607:35mikeb@olivergeorge any thoughts about reversing direction and generating spec definitions from a sql schema?#2016-10-0609:30Oliver George@mikeb I've had some fun with that actually. It's easy to produce something for each column in a database but the relations are trickier.#2016-10-0609:31Oliver GeorgeHere's some related code. https://gist.github.com/olivergeorge/468464ce82b8da486736fe725a4b6ff8#2016-10-0609:32Oliver GeorgeThe slightly fiddly bit is that you can't really do it dynamically. At least I can't work out how to have my script generate and run macros. In the end I generated some forms and committed them as code.#2016-10-0615:04Yehonathan SharvitI was looking into clojurescript
code and I saw that clojure/core/specs.clj
from clojure
is not there.
Why clojure/core/specs.clj
hasnāt been ported into clojurescript
?#2016-10-0615:10Yehonathan Sharvit@dnolen is it on purpose?#2016-10-0615:11dnolenJust haven't gotten to it yet#2016-10-0615:14Yehonathan Sharvitwould you be interested in a patch?#2016-10-0615:22mlimotteI see this in the spec guide:
"check also takes a number of options that can be passed to test.check to influence the test run, as well as the option to override generators for parts of the spec, by either name or path."
Any examples of the "path" option for generator overrides?#2016-10-0615:25jrheardiād love to see an example of generator override usage, path or no#2016-10-0615:28mlimotte@jrheard here's a basic example with override by name:
(defn foo [x] (println "FOO" x) (inc x))
(s/fdef foo :args (s/cat :x ::x))
(s/def ::x clojure.future/int?)
(stest/check `foo
{:gen {::x #(gen/return 10)}
::stestc/opts {:num-tests 1}})
#2016-10-0615:29jrheardthanks!#2016-10-0615:29jrheardgen comes from clojure.test.check, not from spec, right?#2016-10-0615:30mlimotteBut not sure how to do it when the spec is defined like this:
(s/fdef foo :args (s/cat :x clojure.future/int?))
#2016-10-0615:30mlimotte[clojure.spec.gen :as gen]#2016-10-0615:31jrheardnice, thx#2016-10-0615:43dpiatek@mlimotte I think you need to do (s/fdef foo :args (s/cat :x (s/spec clojure.future/int?)))
#2016-10-0615:46mlimotteThe foo spec above seems to work even w/out (s/spec ...). I'm still not sure how to do a generator override for this case where the spec is not named, though.#2016-10-0615:48dpiatekOh, I see - sorry, misread the conversation!#2016-10-0615:49mlimottenp#2016-10-0618:34naomariki'm sorry about this naive question: if i were to spec a map that has strings as keys or even another type, i would have to convert them to keywords first before passing them into spec?#2016-10-0618:42mlimottei don't think so, do you have an example of what you're trying to do?#2016-10-0618:43bfabry@naomarik no. you need keywords for keys to use s/keys
but there's still lots of other predicates that you can use to validate maps, including writing your own#2016-10-0618:46naomariki've been struggling to find an example for this, let's say just validating something simple like {"type" "tweet" "user" {"type" "registered"}}
. I would know how to go about doing this from reading the official guide if these were keywords, but not like this#2016-10-0618:48mlimottei think you're saying that those are required keys. So, like bfabry said, you can't use s/keys. you have to write your own predicates to do that.#2016-10-0618:49bfabry@naomarik it depends on what you want to validate. if you know what keys you're going to get in advance and what their semantics are and which ones are required then I'd recommend converting them to keywords. it just makes sense. if the keys are dynamic then I would leave them as strings and validating other properties about the map#2016-10-0618:55mlimotte@naomarik if it's just required keys, a simple predicate isn't too bad:
(s/def ::my-map (s/and map? #(every? (partial contains? %) #{"type" "user"})))
(s/valid? ::my-map {"type" "tweet" "user" {"type" "registered"}})
=> true
(s/valid? ::my-map {"type" "tweet" "NotUser" {"type" "registered"}})
=> false
If it's more complicated and you need all the features of s/keys, than writing your could be a hassle.#2016-10-0618:57naomarikah i see#2016-10-0618:57naomariks/keys is a lot more convenient at that point#2016-10-0618:58bfabryyeah like I said, if you know what keys your map contains I would convert them to keywords. for lots and lots of good reasons including that you get nice spec tools for working with them#2016-10-0619:00naomariki always write all my maps as keywords, but is there a discussion online why namespaced keywords are enforced so i can be enlightened?#2016-10-0619:01bfabrythere's been a few, I think maybe the cognicast podcast with rich hickey talking about spec might be the best bet. they're not enforced though#2016-10-0619:02bfabrys/keys
has corresponding :req-un and :opt-un options that you can use for un-namespaced keywords#2016-10-0619:05mpenetYeah but some stuff is brittle with un- keys ex broken conforming with s/merge#2016-10-0619:06bfabryeh?#2016-10-0619:06mpenetSee recent discussion about it in history#2016-10-0619:07mpenetClj-1981#2016-10-0619:08bfabryalex's comments seem to indicate that's confusion over how conforming works with merge, and isn't related to namespaced keys#2016-10-0619:09mpenetWrong ticket, my google fu failed me#2016-10-0619:09bfabryhttp://dev.clojure.org/jira/browse/CLJ-2033#2016-10-0619:10bfabryhe's saying only the last spec in the merge conforms#2016-10-0619:10bfabryohh wait no I understand#2016-10-0619:10bfabryok, but there's no way to fix that#2016-10-0619:11mpenetNever saw the point in ns keys personally. #2016-10-0619:11bfabryyou get behaviour for free when using namespaced keys. but it's just not possible if they're not namespaced#2016-10-0619:11bfabrywell. this would be one of the points. you get conforming for free#2016-10-0619:12bfabryI think perhaps alex's example hasn't explained what's happening properly. this might help
boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (defn convert [n] (if (double? n) n (double n)))
#'boot.user/convert
boot.user=> (s/def ::value (s/conformer convert))
:boot.user/value
boot.user=> (s/conform (s/keys) {::value 5})
#:boot.user{:value 5.0}
boot.user=>
#2016-10-0619:13bfabryso even though s/keys doesn't specify ::value as req or opt, because ::value is namespaced and has a spec associated it automatically gets conformed and validated#2016-10-0619:15bfabryso it's not that merge behaves differently with namespaced keys, it's that s/keys behaves differently with namespaced keys (it gives you free stuff if they're namespaced)#2016-10-0619:19Yehonathan SharvitSomething weird about s/and
:
user=> (s/def ::ff (s/cat :start integer? :end integer?))
:user/ff
user=> (s/def ::gg (s/* ::ff))
:user/gg
user=> (s/explain-str ::gg [4 7 2 1])
"Success!\n"
user=> (s/def ::ff (s/and (s/cat :start integer? :end integer?)))
:user/ff
user=> (s/explain-str ::gg [4 7 2 1])
"In: [0] val: 4 fails spec: :user/ff predicate: (cat :start integer? :end integer?)\n"
#2016-10-0619:20Yehonathan SharvitWhy in the second case - after adding s/and
the vector doesnāt conform?#2016-10-0619:31seancorfieldRegex.#2016-10-0619:32seancorfieldYour ::gg
spec is zero or more integer-followed-by-integer patterns.#2016-10-0619:32seancorfieldBecause s/*
and s/cat
combine as a regex sequence.#2016-10-0619:33seancorfieldWhen you insert s/and
you isolate the the pair spec so ::gg
becomes zero or more pairs#2016-10-0619:33seancorfieldi.e., [[4 7] [2 1]]
#2016-10-0619:36Yehonathan SharvitWhat is the way not to isolate the pair?#2016-10-0619:36Yehonathan SharvitMy use case is that I need extra-validation#2016-10-0619:36Yehonathan Sharvit(s/def ::ff (s/and (s/cat :start integer? :end integer?)
#(< (:start %) (:end %))))
(s/def ::gg (s/* ::ff))
#2016-10-0619:37Yehonathan SharvitMakes sense?#2016-10-0619:37Yehonathan Sharvit@seancorfield#2016-10-0619:40seancorfieldSo you want a sequence of integers that is even in length and when you partition that sequence into pairs, each pair is orderedā¦?#2016-10-0619:40Yehonathan Sharvitexactly!#2016-10-0619:42seancorfieldSo (s/and seq-of-ints length-is-even every-partitioned-pair-is-ordered)
#2016-10-0619:43mpenet@bfabry Yep, thats what I meant. The fact that ns keys are so deeply rooted in spec makes this stuff (merge for instance) counter intuitive. #2016-10-0619:46Yehonathan Sharvit@seancorfield But I want to do it in an idiomatic clojure.spec
way#2016-10-0619:46bfabryagain, I'd say it's (s/keys) that's counter-intuitive, not s/merge, but yeah possibly the automatic conform/validate nature of namespaced keywords could be called out more#2016-10-0619:46Yehonathan SharvitActually, my real need is to build a spec for sequence of integers that are palindromes...#2016-10-0619:48Yehonathan SharvitI tried:
(s/def ::short-palindrome
(s/and (s/cat :a integer?
:rest ::palindrome
:a integer?)
#(= (:a1 %) (:a2 %))))
(s/def ::palindrome (s/* ::short-palindrome))
#2016-10-0619:48mpenetI know, I never said merged was bugged, but that s one of the places where this behavior shows through in a bad way#2016-10-0619:49Yehonathan SharvitBut it only accepts nested sequences like [[1 [ 2 2] 1]]
but not flat sequences like that: [1 2 2 1]
#2016-10-0619:53Yehonathan Sharvithow would you write the spec for palindrome @seancorfield ?#2016-10-0619:56seancorfieldFor a sequence that is a palindrome? (s/and (s/coll-of integer?) #(= % (reverse %)))
#2016-10-0619:56seancorfield(untested)#2016-10-0619:56Yehonathan Sharvittesting it...#2016-10-0619:58Yehonathan SharvitIt works (except for empty sequences)#2016-10-0619:58Yehonathan SharvitBut...#2016-10-0619:58Yehonathan SharvitI would like to write it like a context-free grammar#2016-10-0619:59Yehonathan SharvitMaybe thatās not the idea with clojure.spec
@seancorfield ?#2016-10-0619:59seancorfieldYou probably want to ask @alexmiller ...#2016-10-0620:08Yehonathan SharvitNow, Iām trying to implement the grammar for algebraic expressions - as described here https://en.wikipedia.org/wiki/Context-free_grammar#Algebraic_expressions:
Here is a context-free grammar for syntactically correct infix algebraic expressions in the variables x, y and z:
S ā x
S ā y
S ā z
S ā S + S
S ā S - S
S ā S * S
S ā S / S
S ā ( S )
#2016-10-0620:08Yehonathan SharvitDo I have a chance to succeed @seancorfield and @alexmiller ?#2016-10-0620:10Alex Miller (Clojure team)I would have written something like this for palindromes (& is much better than and here as it stays in regex)#2016-10-0620:10Alex Miller (Clojure team)(s/def ::pal
(s/alt :0 (s/cat)
:1 int?
:n (s/& (s/cat :a int? :b ::pal :c int?) (fn [{:keys [a c]}] (= a c)))))
#2016-10-0620:12Alex Miller (Clojure team)you can use the same approach for your algebraic expressions#2016-10-0620:12Yehonathan Sharvitgreat#2016-10-0620:12Yehonathan SharvitIām trying it now#2016-10-0620:13Yehonathan Sharvit::pal
works well#2016-10-0620:13Yehonathan Sharvitworking on ::algebraic-expression
...#2016-10-0620:16Yehonathan Sharvit@alexmiller whatās the exact meaning of (cat)
with no arguments?#2016-10-0620:16Yehonathan SharvitIs it like empty?
?#2016-10-0620:17Alex Miller (Clojure team)yes#2016-10-0620:17Alex Miller (Clojure team)itās a sequential collection with no contents#2016-10-0620:17Alex Miller (Clojure team)but itās a regex so will compose better with other regex ops#2016-10-0620:18Alex Miller (Clojure team)in particular, it will be treated in the same sequential context here, not be a separate (nested) collection#2016-10-0620:19Yehonathan Sharvitthx#2016-10-0620:21seancorfieldAh, yes, I still forget about s/&
...#2016-10-0620:22Yehonathan Sharvit@alexmiller Now, Iām having stackoverflow issues#2016-10-0620:22Yehonathan Sharvitš#2016-10-0620:23Yehonathan Sharvit(s/def ::arithmetic
(s/alt
:var int?
:plus (s/cat :a ::arithmetic :op #{"+"} :b ::arithmetic)))
(s/explain-str ::arithmetic [1 "+" 1])
#2016-10-0620:24Alex Miller (Clojure team)yeah, that's not going to work#2016-10-0620:24Alex Miller (Clojure team)there are other typical approaches to writing left-recursive grammars like this#2016-10-0620:26Yehonathan Sharvitleft-recursion is not (yet) supported in clojure.spec
?#2016-10-0620:28Alex Miller (Clojure team)this is just a consequence of how the regex logic works#2016-10-0620:31Yehonathan Sharvityou mean the deriv
and accept-nil?
functions#2016-10-0620:31Yehonathan Sharvit- that implement the ideas of the āParsing with derivativesā paper ?#2016-10-0620:36Alex Miller (Clojure team)spec regex walks all viable alternatives in parallel - in this case, that becomes an ever-growing set#2016-10-0620:36Alex Miller (Clojure team)but you can rewrite those kinds of grammars like#2016-10-0620:36Alex Miller (Clojure team)(s/def ::ar (s/cat :a int? :r (s/? (s/cat :op #{"+ā} :b (s/alt :i int? :e ::ar)))))
#2016-10-0620:38Alex Miller (Clojure team)in reality, a) these problems donāt come up that often and b) prefix-factoring is an option#2016-10-0620:39Yehonathan SharvitBy āprefix-factoringā you mean that the recursion appears last or that it doesnāt appear first?#2016-10-0621:01seancorfield@viebel you can see why I deferred to Alex on the "context-free grammar" aspect of clojure.spec, eh? š#2016-10-0621:02Yehonathan SharvitYeah! Itās hard stuff#2016-10-0621:02Yehonathan SharvitNow, Iām struggling with the parentheses#2016-10-0621:02Yehonathan Sharvit(s/def ::my-int (s/* #{\0 \1 \2 \3 \4 \5 \6 \7 \8 \9 \x \y \z}))
(s/def ::ar (s/alt :operation (s/cat :a ::my-int
:r (s/? (s/cat :op #{\+ \* \- \/}
:b (s/alt :i ::my-int :e ::ar))))
:parentheses (s/cat :o #{"("}
:b (s/alt :i ::my-int :e ::ar)
:c #{")"})))
#2016-10-0621:03Yehonathan SharvitIt works for (s/explain-str ::ar (seq "x+(y*x)ā))
#2016-10-0621:03Yehonathan SharvitBut with (s/explain-str ::ar (seq "(y*x)+xā))
I get:
In: [5] val: ("+" "x") fails spec: :my.spec/ar predicate: (alt :operation (cat :a :my.spec/my-int :r (? (cat :op #{"+" "*" "-" "/"} :b (alt :i :my.spec/my-int :e :my.spec/ar)))) :parentheses (cat :o #{"("} :b (alt :i :my.spec/my-int :e :my.spec/ar) :c #{")"})), Extra input
#2016-10-0621:04Yehonathan Sharvit@alexmiller please help (again) š°#2016-10-0621:17Alex Miller (Clojure team)Sorry I'm off duty for the night :)#2016-10-0621:22Yehonathan SharvitNo problem: Actually Iām in Israel and itās 12:30 AM - Iām too tired to continue with this tough topic. Weāll catch up later. If you find something please mention me on slack.#2016-10-0621:45joshgIs there an idiomatic way to define a spec for a constant value? Something like (s/def ::some-constant #(identical? % "foo"))
?#2016-10-0621:45Yehonathan SharvitYou can use set like this: #{āfooā}
#2016-10-0621:46Yehonathan Sharvitbecause sets behave like functions#2016-10-0621:46joshgThat works, thanks#2016-10-0621:46bfabryyes, sets are the idiomatic way to do that, sets also work as generators (while #(identical? % "foo") does not)#2016-10-0702:26richiardiandreaHello folks! I know how to spec a map with keys which I don't care the values of: (s/map-of #{:mid :seq} any?)
thanks to Alex, but is there a way standard pred to say "any non-nil value" or I must create my own predicate?#2016-10-0702:28richiardiandreaI tried (complement nil?)
but then the generator fails#2016-10-0702:29bfabry@richiardiandrea probably some?
#2016-10-0702:29richiardiandreaoh right, let me try#2016-10-0702:30bfabryboot.user=> (s/def ::some some?)
:boot.user/some
boot.user=> (s/exercise ::some)
([{} {}] [{} {}] [(()) (())] [{([[]]) ([])} {([[]]) ([])}] [(-1/2 false) (-1/2 false)] [{{{{{} (-1.5)} ([\=])} []} [{}]} {{{{{} (-1.5)} ([\=])} []} [{}]}] [([:N4oa:*I+-:0:+c8_68:!!-6L]) ([:N4oa:*I+-:0:+c8_68:!!-6L])] [[[([true])]] [[([true])]]] [{} {}] [[()] [()]])
boot.user=>
#2016-10-0702:30richiardiandreayes it looks like that's what I was looking for thanks a lot @bfabry !#2016-10-0702:30bfabrynp#2016-10-0703:12seancorfieldLooks like some?
always generates a collection thoā?#2016-10-0703:14bfabryHUH, that's weird#2016-10-0703:14seancorfield@richiardiandrea what about (s/and any? (complement nil?))
? That generates regular values that arenāt nilā¦ boot.user=> (s/exercise (s/and any? (complement nil?)))
([{} {}] [[] []] [[] []] [([false]) ([false])] [[()] [()]] [[] []] [{[[:Q5B+B:-vS7!+_2a:14s+?:Wd:?:E3+cY7:7q9n_V:+T*_] [L?lQDc.poz2z._rRq+9*Le._?.o.Pq!San9?!.uP36/y?+-i_]] {[] {}}} {[[:Q5B+B:-vS7!+_2a:14s+?:Wd:?:E3+cY7:7q9n_V:+T*_] [L?lQDc.poz2z._rRq+9*Le._?.o.Pq!San9?!.uP36/y?+-i_]] {[] {}}}] [{[] ()} {[] ()}] [[:AE+4CW!863.Rt_??z4!.ob*6.*M.J*+7!.?Wpz.RH?.k+L-L?-2_8.Dz_5E3zaY.!2GNEsM/N51M0Sn7 6 5] [:AE+4CW!863.Rt_??z4!.ob*6.*M.J*+7!.?Wpz.RH?.k+L-L?-2_8.Dz_5E3zaY.!2GNEsM/N51M0Sn7 6 5]] [[{[3 \s] [true], (s) (1/6)} {() (:?tLe9B2-66t.--s6+?8E*.*GVek-7!Y.++5BT3TQ6.x0Al3z.*7sAkrq8.*h9.a+P+.qz6?6?/KT-4!m+_4D -8), {\t v-Y+u+D+vk-.K?XwW-8?1?.+!529+.DaEb.erp7.hl?+??ma+t.-ZiN+y+.VC63aicOOe7.!226W+RFO.e1*??7/-q*.} {-5/3 \?}}] [{[3 \s] [true], (s) (1/6)} {() (:?tLe9B2-66t.--s6+?8E*.*GVek-7!Y.++5BT3TQ6.x0Al3z.*7sAkrq8.*h9.a+P+.qz6?6?/KT-4!m+_4D -8), {\t v-Y+u+D+vk-.K?XwW-8?1?.+!529+.DaEb.erp7.hl?+??ma+t.-ZiN+y+.VC63aicOOe7.!226W+RFO.e1*??7/-q*.} {-5/3 \?}}]])
#2016-10-0703:15seancorfieldHmm, looking at those, most seem to be collections tooā¦?#2016-10-0703:15bfabrysomething weird going on there https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/gen.clj#L133#2016-10-0703:16seancorfieldAh, I think itās down to how any?
generates stuff...#2016-10-0703:16bfabrythere's no reason some? should be only collections, and I think that behaviour might be common#2016-10-0703:17seancorfieldLooks like any?
generates nil
or a collectionā¦ boot.user=> (->> (s/exercise any? 100) (map second) (remove coll?))
(nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil)
#2016-10-0703:18bfabryyeah seems like whatever any-printable is always generates a coll#2016-10-0703:23bfabryI can't be bothered digging but that seems like not-incorrect-but-pretty-undesirable#2016-10-0703:26seancorfield10,000 iterations donāt produce anything that isnāt nil
or a collection: boot.user=> (->> (s/exercise any? 10000) (map second) (remove coll?) (remove nil?))
()
#2016-10-0703:26seancorfield(dang, I love that you can do stuff like that so easily in the REPL!)#2016-10-0703:29bfabryseems like the behaviour is inherited from clojure.test.check.generators/any-printable#2016-10-0703:34bfabryyuuup
boot.user=> (remove coll? (remove nil? (repeatedly 1000 #(clojure.test.check.generators/generate clojure.test.check.generators/any-printable))))
()
#2016-10-0703:41seancorfieldThat probably needs ā¦ enhancing ...#2016-10-0704:10richiardiandreaFor generating maps only I was thinking of using an s/and
but for a first approximation of this spec some?
is perfect#2016-10-0704:10richiardiandreaThat is good investigation though#2016-10-0707:26preporHello. ring-spec had just released. Imagine that library was named not "ring", but "com.stuartsierra.ring". For example, https://gist.github.com/prepor/c9a3ac1c5746146671b55631a543a6e4#file-spec-clj-L107. There are a lot of noise, aren't it?#2016-10-0707:36robert-stuttafordiāve never understood reverse domain name packages#2016-10-0707:36robert-stuttafordi get the rationale, but it almost never pans out in practice. company names suffice, i find#2016-10-0707:44misha@prepor
(require '[clojure.spec :as s])
(create-ns 'com.this.looks.really.inconvenient)
(alias 'inco 'com.this.looks.really.inconvenient)
(s/def ::inco/bar int?)
=> :com.this.looks.really.inconvenient/bar
(s/valid? ::inco/bar 1)
=> true
(s/valid? :com.this.looks.really.inconvenient/bar 1)
=> true
#2016-10-0707:47prepor> iāve never understood reverse domain name packages
this is the only way to guarantee name clashes absence#2016-10-0707:47prepor@misha nice. will it work with aot?#2016-10-0707:48mishakappa no idea#2016-10-0707:50preporyep. i think dynamic ns juggling is controversial way...#2016-10-0710:13borkdudeIām amazed spec can parse the last case:
(s/def ::rhs (s/cat :lo (s/? symbol?)
:r (s/? (s/alt :rsh (s/cat :op #(= % 'RSHIFT)
:ro int?)
:or (s/cat :op #(= % 'OR)
:ro symbol?)
:not (s/cat :op #(= % 'NOT)
:ro symbol?)
))))
(s/conform ::rhs '[bn RSHIFT 2])
(s/conform ::rhs '[cj OR cp])
(s/conform ::rhs '[lx])
(s/conform ::rhs '[NOT ax])
I expected this not to work because NOT is also a symbol and that none other alt matches the rest#2016-10-0711:07borkdudeIs there a conform variant that throws like s/assert but returns the conformed value?#2016-10-0712:23Alex Miller (Clojure team)No but it's easy to make one#2016-10-0712:23Alex Miller (Clojure team)@prepor should work fine with aot#2016-10-0712:24Alex Miller (Clojure team)@seancorfield: if you filed a ticket on that any? some? generator business I would take a look at it. I keep forgetting to do so#2016-10-0712:57prepor@alexmiller ok, thank you#2016-10-0714:26lvh@borkdude btw, consider #{āRSHIFT}
#2016-10-0714:26lvh@borkdude Itās shorter, generally considered more idiomatic, and spec can inspect it#2016-10-0714:27borkdude@lvh my current spec:
(s/def ::val (s/alt :name symbol? :value int?))
(s/def ::binary-operator #(#{'LSHIFT 'RSHIFT 'AND 'OR} %))
(s/def ::binary-expression (s/cat
:left-operand ::val
:operator ::binary-operator
:right-operand ::val))
(s/def ::not (s/cat :not #(= % 'NOT) :operand ::val))
(s/def ::lhs (s/alt :simple-value ::val
:binary-expression
::binary-expression
:not ::not))
(s/def ::rhs symbol?)
(s/def ::expr (s/cat :lhs ::lhs :arrow #(= % '->) :rhs ::rhs))
#2016-10-0714:27lvhthat works too#2016-10-0714:27lvhstill gotta fix the #{āNOT}
if you want generation to work though#2016-10-0714:27borkdudeah#2016-10-0714:27lvhalso: arrow#2016-10-0714:27borkdudeoh wow, I didnāt even think of generation#2016-10-0714:28borkdudemust. try.#2016-10-0714:28lvhborkdude Itās pretty cool. Iām currently generating specs for swagger from the swagger json-schema definition, which I then use the generate swagger instances, from which I then generate instances of that swagger schema, which I then validate against the original swagger#2016-10-0714:29borkdudeš#2016-10-0714:29borkdudewhat reads better, '#{LSHIFT RSHIFT AND OR}
or quotes before all symbols?#2016-10-0714:30lvherrr#2016-10-0714:30lvhI dunno; Iād go with quotes before syms#2016-10-0714:35borkdudeIt works. How do I get s/cat to generate vectors instead of whatever it generates now?#2016-10-0714:44borkdudeHow do I get a generator that only generates lower case alphabetic strings?#2016-10-0714:55nwjsmith(gen/not-empty
(gen/fmap
(fn [s] (str/replace s #"[A-Z0-9]" ""))
gen/string-alphanumeric))
#2016-10-0714:56nwjsmiththat's a really naive generator, but it'll get you there#2016-10-0715:07borkdudeThanks!
I needed symbols of one or two character, so I got this now:
(s/with-gen symbol?
#(gen/fmap (fn [[i j]]
(symbol (string/lower-case (str i j))))
(gen/tuple (gen/char-alpha) (gen/char-alpha))))
#2016-10-0715:09borkdudehow do I get s/exercise to generate vectors instead of seqs for s/cat?#2016-10-0715:41nwjsmithShould work with :kind
, no?
(s/exercise (s/coll-of (s/cat :a nat-int? :b string?) :kind vector?))
#2016-10-0715:53nwjsmithOtherwise, you might want to check out s/tuple
#2016-10-0716:54seancorfield@alexmiller https://clojurians.slack.com/archives/clojure-spec/p1475843093004536 http://dev.clojure.org/jira/browse/CLJ-2036#2016-10-0716:55bfabry@seancorfield @alexmiller it actually seems like there's already a ticket on test.check http://dev.clojure.org/jira/browse/TCHECK-111#2016-10-0716:56seancorfieldThanks @bfabry ā cross-linked#2016-10-0721:55jrheardha, hadnāt seen ffirst before, thx for the tip#2016-10-0721:57jrheardhadnāt seen advent of code either, will have to take a look#2016-10-0800:31jrheard(update: itās pretty fun! :D)#2016-10-0803:30richiardiandreaA question, I can open a JIRA bug as well, shouldn't exercise-fn
accept custom generators?#2016-10-0809:33Alex Miller (Clojure team)Yes#2016-10-0815:22jrheard@borkdude just caught up - https://github.com/jrheard/advent-of-code/blob/master/src/advent_of_code/day_7.clj#2016-10-0815:23jrheardthis is fun, donāt know why i didnāt do it when it came out! looking forward to the 2016 edition#2016-10-0815:23jrheard[assuming i can solve the next 18ā¦]#2016-10-0815:23borkdudegreat!#2016-10-0815:24jrheardi like your use of s/conform, it felt like that was super relevant re: parsing but i guess i just didnāt reach for it; gonna take another look and see how you accomplished it#2016-10-0815:26jrhearddelayās neat too, havenāt used that before - thanks for sharing your solution, v educational! š#2016-10-0822:01jrheards/conform came super in handy for day 8 š https://github.com/jrheard/advent-of-code/blob/master/src/advent_of_code/day_8.clj#2016-10-0822:02jrheardspec rules#2016-10-0907:01bensuI defined an fdef
spec for a macro and Iām testing it with (is (thrown? Exception (macroexpand ā(my-macro bad-args))))
#2016-10-0907:02bensuwhen I run it in the repl lein with-profile +test repl
the test passes#2016-10-0907:02bensubut when I run it with lein test
the arguments are not checked and the test fails#2016-10-0907:03bensuHas anybody hit a similar problem?#2016-10-0907:17hiredmanis your macro code actually being loaded? macroexpand doesn't throw an error if my-macro doesn't exist and isn't defined as a macro#2016-10-0907:29bensuit was getting loaded#2016-10-0907:29bensuthe problem turned out to be related: it was loaded but not recognized by macroexpand
because I was not using a syntax-quote#2016-10-0907:29bensu@hiredman thank you for your help#2016-10-0909:17Yehonathan SharvitSometimes s/conform
and s/unform
are not inlined. For instance:
(s/def ::my-spec (s/cat :op (s/? (s/cat :a integer?))))
(->> (s/conform ::my-spec '(1))
(s/unform ::my-spec))
It returns ((1))
#2016-10-0909:17Yehonathan SharvitThere is an extra nesting#2016-10-0909:18Yehonathan SharvitLive demo here: https://bit.ly/2efyHkI#2016-10-0909:18Yehonathan SharvitIs it a bug in clojure.spec
@alexmiller ?#2016-10-0912:17sanderis there a way to get a spec that (s/conform ::my-spec v)
will always conform to? I want to fspec
a function saying that it takes a spec-destructured value as an argument, and returns another spec-destructured value#2016-10-0912:22Yehonathan Sharvitdid you try s/any?
?#2016-10-0912:22Yehonathan Sharvit@sander#2016-10-0912:23sanderI should rephrase, I want a specific spec that only accepts values in the shape that (s/conform ::my-spec v)
returns#2016-10-0912:30Yehonathan SharvitInteresting question...#2016-10-0912:44Alex Miller (Clojure team)@viebel yes, generally anytime conform unform returns a different result, that's a bug. This particular one is already logged.#2016-10-0912:44Yehonathan Sharvitjira?#2016-10-0912:47Alex Miller (Clojure team)@sander no, other than saying what you just said as a custom predicate - call s/valid? on the result with another spec#2016-10-0912:47sanderok, should be easy enough, thanks!#2016-10-0912:51Alex Miller (Clojure team)@viebel http://dev.clojure.org/jira/browse/CLJ-2003#2016-10-0912:54Yehonathan Sharvitthx @alexmiller#2016-10-0912:54Alex Miller (Clojure team)Rich and I talked about this one on Friday in particular - it's tricky#2016-10-0912:55Yehonathan SharvitWhy is it tricky?#2016-10-0912:56Alex Miller (Clojure team)Conceptually tricky about what an optional regex should return in different contexts, how it nests, etc#2016-10-0912:58Yehonathan SharvitIn my specific case, I was able to workaround this bug using s/conformer
. But I still donāt understand fully how s/conformer
works. Could you explain in simple words how does s/conformer
work?#2016-10-0912:58Yehonathan SharvitThe docstring is a bit obscure...#2016-10-0912:59Alex Miller (Clojure team)It acts as a predicate but applies an arbitrary transformation in the conformed value #2016-10-0913:00Yehonathan SharvitWhat is the simplest use case for conformer
?#2016-10-0913:00Alex Miller (Clojure team)Applying an arbitrary transformation to the conformed value #2016-10-0913:01Alex Miller (Clojure team)It is primarily a tool for building more complex spec variants#2016-10-0913:03Alex Miller (Clojure team)It's generally recommended to use sparingly as embedding it in a spec means that you are overriding how spec conforms for all future consumers of your spec#2016-10-0913:03Yehonathan Sharvitwith s/and
?#2016-10-0913:03Alex Miller (Clojure team)Yes#2016-10-0913:04Alex Miller (Clojure team)Like if you used s/or but didn't want a tagged result#2016-10-0913:05Alex Miller (Clojure team)When you do that you throw away info about how something was conformed#2016-10-0913:05Alex Miller (Clojure team)One use might not need it but other possible future uses may#2016-10-0913:06Alex Miller (Clojure team)So baking it into your public spec takes away that option#2016-10-0913:11Yehonathan SharvitThx#2016-10-0913:11Yehonathan SharvitIāll play with that...#2016-10-0913:45Yehonathan Sharvit@alexmiller another use case would be when one wants to match regex with vectors#2016-10-0913:45Yehonathan SharvitThe naive code would be:
(s/def ::vec-of-int-and-str (s/cat :0 integer?
:1 string?))
(->> (s/conform ::vec-of-int-and-str [1 "a"])
(s/unform ::vec-of-int-and-str))
#2016-10-0913:46Alex Miller (Clojure team)There will be a different solution for that#2016-10-0913:46Alex Miller (Clojure team)But yes#2016-10-0913:46Yehonathan Sharvitbut it returns: (1 "aā)
instead of [1 āaā]
#2016-10-0913:47Yehonathan SharvitSo the fix is:
(s/def ::vec-of-int-and-str (s/and
vector?
(s/conformer identity vec)
(s/cat :0 integer?
:1 string?)))
#2016-10-0913:47Yehonathan SharvitIs there a better solution for that?#2016-10-0913:54Alex Miller (Clojure team)Not yet but it's coming#2016-10-0913:55Yehonathan SharvitIām curiousā¦ :relaxed:#2016-10-0915:31jrheardnoob question, iām sure, re: conformer - i reached for conformer when i was operating on data that looked like ā123ā, but i wanted (s/conform ::my-spec) to parse the integer such that it ended up being 123. what should i use instead of s/conformer in situations like that, for coercion of this sort?#2016-10-0915:31jrheardi felt like i read about something that would solve this problem idiomatically in the rationale or guide, but couldnāt find it when i went back to look for it#2016-10-0916:30seancorfield@jrheard: yes I have a lot of cases where I have strings as input but I want them conformed to integers (for API input specs). But @alexmiller has told me a couple of times that is "not idiomatic" and to avoid conformers like that :(#2016-10-0916:45jrheardhm, i donāt understand - the strings have to be turned into integers at some point, right?#2016-10-0916:46jrheardis it more idiomatic to have a secondary, non-spec transformation layer that happens after s/conform?#2016-10-0916:47jrheardiām sure @alexmiller has good reasons for saying that, but i donāt understand what spec users are actually supposed to do here#2016-10-0917:44Alex Miller (Clojure team)The downside is that once you enshrine a conversion into your spec, you have made a conformance decision for all future consumers that has removed information. If you understand this ramification and are willing to live with the consequences, then that's up to you.#2016-10-0917:46Alex Miller (Clojure team)There are many ways to apply conversions - you should be aware of when you are conflating conformance and conversion though#2016-10-0918:01jrheardi think i follow those statements, but a āin situations like that, try doing this insteadā example would be supremely helpful š#2016-10-0919:40Alex Miller (Clojure team)There's not one answer#2016-10-0919:41Alex Miller (Clojure team)You can apply post transformations#2016-10-0919:41Alex Miller (Clojure team)You can apply a second spec#2016-10-0919:41Alex Miller (Clojure team)You can apply a different spec in one case /cdn-cgi/l/email-protection#2016-10-0919:41Alex Miller (Clojure team)Etc#2016-10-0919:45Alex Miller (Clojure team)And in some cases conformers might be the right answer#2016-10-1000:21jrheardgotcha, makes sense. thanks!#2016-10-1016:32Yehonathan Sharvithttps://twitter.com/viebel/status/785518288012992512#2016-10-1017:19juliobarrosI'd like to implement a multi step data pipeline where each step's "spec" is slightly modified and based on the previous step's spec.
For example at step 1 an address has some number of fields and at step 2 it has the same fields plus geo coordinates. Or in step 1 it can have optional string geo coordinates and after step 2 it has to have numeric ones.
It seems like spec requires/encourages you to duplicate base address fields in two separate specs (rather than build on a base spec) and possibly have different names for the string geo coords and double geo coords (since they are registered under the same name). This appears to be more straight forward in schema where I can modify the schema map as I go through the pipeline.
Am I reading this right? Am I missing something? What's the best spec based approach to this? How would you approach this? Thanks in advance.#2016-10-1017:31hiredmanif your changes are purely additive between pipeline steps, you can compose specs using and#2016-10-1017:35hiredmanmy perspective on specs is you should treat them like defining columns in a sql database. if I had a column named date
in a sql database, would I want the scheme to allow strings or dates? most likely not, I would have different columns for strings and dates.#2016-10-1017:41juliobarrosThanks @hiredman the sql table analogy is a good way to think about it. #2016-10-1017:57mattlyI think specs that coerce in a conformer are useful, but I've taken to naming them with a !
suffix#2016-10-1017:59mattlyor well, naming the function that the conformer calls with a !
suffix and typically only using them where absolutely necessary#2016-10-1018:19jrheardseems like a reasonable pattern, thanks for sharing#2016-10-1018:23Yehonathan Sharvit@mattly donāt forget to pass the 2nd param to s/conformer
- otherwise s/unform
will fail#2016-10-1018:23mattlyheh#2016-10-1018:23mattlyin this case I'm basically using it to validate environment variables#2016-10-1023:23bbloom@hiredman indeed the table/column analogy is interesting, but itās not the whole story. iāve been thinking about ārefinementā (for lack of a better word) a bunch lately#2016-10-1023:24bbloomspecs (and for that matter: types) have an inherit problem when modeling time#2016-10-1023:24bbloomfor a simple example, consider before/after validation of some property#2016-10-1023:25bbloomyou can have :unvalidated/foo and :validated/foo, but it seems like a weird way to go about that if they are going to be identical?
to each other in all cases#2016-10-1023:25bbloomof course, you can always model the validation as coercion, such that you get the validated value or nil, so having two keys makes sense - and indeed thatās what iād probably do with spec#2016-10-1023:26bbloomthe real problem comes in when you talk about nested structure#2016-10-1023:27bbloomyou donāt want to do O(N) allocations to validate a structure#2016-10-1023:27bbloomrenaming all the keys, losing generality of functions#2016-10-1023:31hiredmanI don't think I follow#2016-10-1023:32hiredmanwhen I said sql column above, that is because I am more familiar with using sql, but obviously the best analogy is a datomic attribute#2016-10-1023:33hiredmanso stick all your maps in datomic so you can scrub forward and backwards in time on them š#2016-10-1023:33bbloomheh, sorry, let me try to explain again#2016-10-1023:34bbloomso iām talking about a more general problem, but i think this neatly specifies one specific instance of it: http://blog.ezyang.com/2013/05/the-ast-typing-problem/#2016-10-1023:35bbloomthe ast typing problem is double bad b/c they arenāt modeling things with extensible records#2016-10-1023:35bbloombut the notion of having two recursive structures that are subtle distinct at different stages in a pipeline is a common theme#2016-10-1023:36bbloomit sucks to have to do O(N) work (eg copy a whole AST) in order to make some small changes to a subset of nodes#2016-10-1023:36hiredmanisn't that the same argument against immutable datastructures?#2016-10-1023:37bbloomnot quite, itās more related to the argument in favor of covariant arrays in java/c# š#2016-10-1023:38bbloomeg you have an array of integers and need an array of objects, so why should you have to copy every integer in order to satisfy the type checker? obviously you donāt have to do that with spec, but itās still a problem if you have a recursive structure and use different keys for different specs#2016-10-1023:38hiredmanI mean, your options are to write novelty in place, or accrete it#2016-10-1023:40bbloomsure, youāre totally right - this problem is usually easily avoidable in spec thanks to the dynamic instrumentation and extensible records#2016-10-1023:42bbloomi just think its interesting to think about refinement of specifications without having to introduce new names#2016-10-1023:42bbloomthereās definitely use cases to have āchangedā structures vs āextendedā structures, but yeah - go with the latter whenever possible#2016-10-1108:02danielstocktonI'm calling explain on a spec and it's returning nil
, why would that be?#2016-10-1108:06danielstocktonanswer: i need to use explain-str
, explain
just prints to out#2016-10-1108:59borkdudeInteresting clojure.spec + datomic question: https://stackoverflow.com/questions/39971727/using-clojure-spec-with-datomic-entities#2016-10-1109:05dm3how would you create a spec for the following:
1) ns store
- defines ::state
as (s/keys :req [::data ::metadata])
where ::data
is a map?
2) ns x
produces a ::state
with concrete ::data
How would the ::x-state
spec be defined? (s/and ::store/state #(s/assert ::x-data (::store/data %)))
doesn't look right...#2016-10-1109:48danielstocktonOn it's own, [1 1]
conforms to the ::pattern
spec#2016-10-1114:52samedhi(ns example.paths
(:require
[cljs.spec :as s]
[cljs.spec.impl.gen :as gen]
[cljs.spec.test :as test]))
(s/def ::keyword-or-string
(s/with-gen
(s/or :keyword keyword?
:string string?)
#(s/gen #{"a" "b" :c :d :e/z})))
(def path (s/coll-of ::keyword-or-string
:min-count 1
:kind vector?))
(s/def ::path
(s/with-gen
path
(fn []
(s/gen
(gen/such-that
#(-> % count (< 5))
(s/gen path))))))
#2016-10-1114:52samedhi#object[Error Error: Unable to construct gen at: [] for: [object Object]]
Error: Unable to construct gen at: [] for: [object Object]
#2016-10-1114:53samedhiSo I am trying to reduce the size of generated ::paths
to at most be of length 5. It isnāt part of the actual spec for a ::path
, but it is something I would like when generating, any idea what I am doing wrong?#2016-10-1115:00jrheard@samedhi there is a 20% chance that what iām about to say is relevant, but iāll say it just in case#2016-10-1115:00jrheardiāve run into trouble with s/coll-of specs that define a :kind but not an :into#2016-10-1115:00jrheardtry adding :into [] and see if it does anything#2016-10-1115:01jrhearditās probably irrelevant, but who knows#2016-10-1115:01jrheardthis is definitely a thing if your :kind is set?
, you need to define :into #{}
for generators to work afaict#2016-10-1115:05samedhi(ns example.paths (:require [cljs.spec :as s] [cljs.spec.impl.gen :as gen] [cljs.spec.test :as test]))
(s/def ::keyword-or-string
(s/with-gen
(s/or :keyword keyword?
:string string?)
#(s/gen #{"a" "b" :c :d :e/z})))
(gen/generate (s/gen ::keyword-or-string)) => :d
(def path (s/coll-of ::keyword-or-string
:min-count 1
:kind vector?
:into []))
(gen/generate (s/gen path)) => [āaā :c ābā ābā ābā ābā āaā]
(s/gen
(gen/such-that
#(-> % count (< 5))
(s/gen path)))
=> (errors with)
#object[Error Error: Unable to construct gen at: [] for: [object Object]]
Error: Unable to construct gen at: [] for: [object Object]
...
#2016-10-1115:10samedhi@jrheard is this what you mean? In this case I am trying to return a vector of keywords or strings, so I set :kind
to vector?
and :into
to []
#2016-10-1115:14jrheardyeah, thatās what i mean; looks like it wasnāt relevant, sorry š#2016-10-1115:22samedhiHey, no problem, I appreciate it.#2016-10-1115:40thegeez@samedhi should that be gen/generate instead of s/gen around the gen/such-that?#2016-10-1115:42samedhi@thegeez, actually, that does workā¦ Let me play a bit with it.#2016-10-1115:43samedhiOn my first post, 2 paste up, I had.#2016-10-1115:43samedhi(s/def ::path
(s/with-gen
path
(fn []
(s/gen
(gen/such-that
#(-> % count (< 5))
(s/gen path))))))
#2016-10-1115:44samedhiI think s/with-gen
requires a generator as the second argument...#2016-10-1115:45samedhiI think I am basing it off of this from http://clojure.org/guides/spec#_explain#2016-10-1115:45samedhi(s/def ::kws (s/with-gen (s/and keyword? #(= (namespace %) "my.domain"))
#(s/gen #{:my.domain/name :my.domain/occupation :my.domain/id})))
#2016-10-1115:46samedhi> Note that with-gen (and other places that take a custom generator) take a no-arg function that returns the generator, allowing it to be lazily realized.#2016-10-1115:47samedhiSo it sounds like I want a zero-args function that returns a generator.#2016-10-1115:47samedhiKind of seems like that is what I have aboveā¦ right?#2016-10-1115:48samedhiMaybe gen/such-that
is already a generator or maybe it is a spec?#2016-10-1115:48thegeezremove the s/gen around the gen/such-that#2016-10-1115:49thegeezgen/such-that is already a generator indeed#2016-10-1115:51samedhi@thegeez You got it!#2016-10-1115:51samedhi(s/def ::path
(s/with-gen
path
(fn [] (gen/such-that #(-> % count (< 5)) (s/gen path)))))
#2016-10-1115:55samedhiThanks, that helped me out a lot, data seemed to just take forever on generating without this. (though that may actually be my formater and not the actual generation). Fixed loading issues.#2016-10-1117:09samedhi(s/def ::path
(s/with-gen
path
(fn [] (gen/fmap #(vec (take 5 %)) (s/gen path)))))
#2016-10-1117:10samedhiIs actually slightly better, as it never fails to generate under the 100 try limit.#2016-10-1117:18lvh@alexmiller Suggestion: a special s/cat, primarily intended for specifying args in fdef, that automatically calls s/spec
on each of its arguments#2016-10-1117:18lvh@alexmiller I feel like a lot of people have bumped their toe on the nested regex gotcha#2016-10-1117:20Alex Miller (Clojure team)I think thatās unlikely something we would add#2016-10-1117:21Alex Miller (Clojure team)and in general, I donāt find that that is (usually) what you want#2016-10-1117:24Alex Miller (Clojure team)it seems more common from specs Iāve written to have s/cat of s/coll-of for functions. For macros, you sometimes have this but I find itās really important in that case to develop the awareness of when you are really needing to drop nested levels - thatās critical to build reusable parts, and often for making workable recursive specs#2016-10-1118:56danielstocktonI think just making this clearer in the guides, in the specing functions section, would help#2016-10-1118:57danielstocktonI found where it's mentioned now, but not when I was trying to work out what was wrong#2016-10-1118:58danielstocktonYou could argue I should have read the guide more thoroughly#2016-10-1121:15bhaganyfor me, it was just forgetting that cat
is a regex operator. I didnāt feel like the guide could have been much clearer about it.#2016-10-1208:04jmglovIs it possible to spec functions defined in a protocol?#2016-10-1208:04jmglovOr the implementations thereof in a defrecord
or reify
?#2016-10-1208:16hiredmanyou can fdef a protocol function, and you'll be able to validate it, but instrument may behave oddly#2016-10-1208:18danielstocktonI found instrument doesn't work, I make my protocol functions a simple proxy to another function that I can spec#2016-10-1210:31danstoneHas anyone had any success spec'ing high level components, such as ring handlers? I would like to spec boundaries rather than implementation details in an app I'm working on and it seems like this will be a ton of work.#2016-10-1211:38jmglov@danielstockton That's what I'm doing for now. It would be really nice if instrument worked, though. š#2016-10-1211:38jmglov@alexmiller Is that something worth submitting as a feature request? Or is unlikely to be considered?#2016-10-1211:39jmglov@hiredman So I can do something like this?
(defprotocol Foo
(bar [this]))
(s/fdef bar
:args (s/cat :x int?
:y int?)
:ret string?)
(defrecord MyFoo [_]
Foo
(bar [_] (str a "+" b))
#2016-10-1211:40danielstocktonIt was asked before on the google group: https://groups.google.com/forum/#!topic/clojure/f068WTgakpk#2016-10-1213:11Alex Miller (Clojure team)I'm not sure it's possible to do#2016-10-1213:12Alex Miller (Clojure team)I'm certainly not opposed to it#2016-10-1214:52jfntnIs there a working equivalent of (s/map-of ::k ::v :kind sorted?)
for sorted maps?#2016-10-1215:13Alex Miller (Clojure team)There is sorted-map? predicate#2016-10-1215:13Alex Miller (Clojure team)I think? On phone so going from memory#2016-10-1215:22jfntnHmm I donāt think so, at least in alpha12?#2016-10-1216:55clojuregeekTrying to use spec, and have a problem with setting up my spec ... basically i want style? to be one of those two values (for now :one :two) ... but it is not working, i've been studying the docs and i can't figure out where i am doing something wrong .#2016-10-1216:56clojuregeekhttps://gist.github.com/rubygeek/065c2f32147cf09f5148d9fd4f7668c3#2016-10-1216:57jrheard@clojuregeek - i think you want to either do (def style? #{:one :two})
and (s/def ::style style?)
, or just (s/def ::style #{:one :two})
#2016-10-1216:57jrheardi donāt think (s/def) expects you to give it a symbol like style?
as the first argument, i think it expects a namespaced keyword like ::style
#2016-10-1216:58jrheardalso you have ::style?? in there at one point, which is probably confusing things as well#2016-10-1216:58bfabryyes, you've registered the spec under the symbol style?
instead of under the keyword ::style?
#2016-10-1216:59clojuregeekok let me try some more ..#2016-10-1217:00clojuregeekok i think maybe it was extra ? on style in job not a keyword when i defined style? spec#2016-10-1217:00clojuregeekthanks guys :+1:#2016-10-1217:03clojuregeeknow if i can only stop typing confirm instead of conform š#2016-10-1306:52tianshuIt seems clojure.spec/assert always return original data in clojurescript.#2016-10-1306:53tianshu(s/assert (s/keys :req-un [::a]) {:b 10})
return {:b 10}#2016-10-1309:05Oliver George@doglooksgood asserts aren't enabled by default. Try (s/check-asserts true)
.#2016-10-1309:05Oliver Georgehttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/check-asserts#2016-10-1309:17yendais there a recommended place to put (s/check-asserts true)
?#2016-10-1309:46jeroenvandijkIs this a bug? (def my-keys [:my.ns/some-key])
(s/def :my.ns/ok (s/keys :req-un ~my-keys))
;=> :my.ns/ok
(s/def :my.ns/not-ok (s/keys :opt-un ~my-keys))
; CompilerException java.lang.AssertionError: Assert failed: all keys must be namespace-qualified keywords
#2016-10-1309:47jeroenvandijksome specific behaviour for :opt-un
, Iāll try to dig further#2016-10-1311:50jeroenvandijkAfter looking a bit further I fell into a macro trap. I guess at most it is inconsistent behaviour#2016-10-1312:30jeroenvandijkThis terrible hack does what I want (defmacro dynamic-keys [& opts]
(let [args (mapcat (fn [[k v]]
[k v])
(partition 2 opts))]
`(eval (list `s/keys
#2016-10-1312:30jeroenvandijkIs there a better way?#2016-10-1312:43odinodinwhat is the preferred convention for fdef specs, should they come before or after the function they spec?#2016-10-1312:45darwin@jeroenvandijk you could write a macro which emits my-keys#2016-10-1312:46darwinbut your solution is fine, under given circumstances, I think it leads to more-readable code#2016-10-1312:47darwininstead of eval
you could use ns-resolve
and var-get
, if you expected just a symbol as arg, I think#2016-10-1312:48darwineval
could be a weapon of mass destruction š#2016-10-1312:49immoh@jeroenvandijk You donāt necessarily need macros. Iāve been using these to create specs dynamically:#2016-10-1312:50jeroenvandijkah yeah could just use eval, slightly better#2016-10-1312:51jeroenvandijkThanks @darwin and @immoh#2016-10-1320:17mlimotteAre there any examples of how to instrument with a :spec override? For example:
(defn foo [x] (inc x))
(s/def ::x clojure.future/pos-int?)
(s/fdef foo :args (s/cat :x ::x))
; Attempt #1
(stest/instrument `foo {:spec {`foo {:args (s/cat :x zero?)}}})
(foo 1)
=> 2
; Attempt #2
(stest/instrument `foo {:spec {::x zero?}})
(foo 1)
=> 2
#2016-10-1320:47clojuregeekI want to say that my function returns a vector of 1 or more jobs (a map) is this correct?
30 ā (s/def ::job (s/keys :req [::type ::meta]
31 ā :opt [::payload]))
32 ā
33 ā (s/fdef sample-jobs
34 ā :args int?
35 ā :ret (s/+ ::job))
#2016-10-1321:25mlimotte@clojuregeek I think you need to use (s/coll-of ...)
for :ret
#2016-10-1321:27clojuregeekhmm .. i would expect a better doc string?
kraken-consumer.job/sample-jobs
([] [x])
Spec
args: int?
ret: (every :kraken-consumer.job/job :clojure.spec/cpred #function[kraken-consumer.job/fn--12178] \
:clojure.spec/kind-form nil :clojure.spec/conform-all true)
#2016-10-1321:35mlimotteFound the answer to my instrument with :spec
question above. Should be done like this:
(defn foo [x] (inc x))
(s/def ::x clojure.future/pos-int?)
(s/fdef foo :args (s/cat :x ::x))
(stest/instrument `foo {:spec {`foo (s/fspec :args (s/cat :x zero?))}})
(foo 1)
=>
ExceptionInfo Call to #'user/foo did not conform to spec:
In: [0] val: 1 fails at: [:args :x] predicate: zero?
:clojure.spec/args (1)
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file "form-init5306699344209214826.clj", :line 5, :var-scope user/eval24489}
clojure.core/ex-info (core.clj:4617)
#2016-10-1321:37clojuregeekusing + has a decent doc string
175 ā kraken-consumer.job/sample-jobs
176 ā ([] [x])
177 ā Spec
178 ā args: int?
179 ā ret: (+ :kraken-consumer.job/job)
#2016-10-1322:25danielcomptonIs there a way to spec a sequential collection? s/coll-of
throws a massive error if you pass a map to it, as it tries to check each mapentry against a spec and displays a failure for each mapentry#2016-10-1322:26jrheards/map-of
might be useful for you#2016-10-1322:26danielcomptonI want my spec to be a sequence of values, each conforming to a spec#2016-10-1322:27danielcomptonIf I pass a map, then each mapentry gets evaluated against that spec instead#2016-10-1322:27jrheardhuh!#2016-10-1322:27jrheardwould you mind pasting an example?#2016-10-1322:29jrheards/cat
is a good way of speccing sequences, but youāre probably already familiar with it so i hesitate to mention it#2016-10-1322:29jrheardi havenāt seen the behavior you describe re: coll-of and maps#2016-10-1322:29jrheardam very curious š#2016-10-1322:30danielcomptonI thought about s/cat
but didn't think it was for this use case? I could be wrong though#2016-10-1322:30jrheardif iām reading this right, youāre passing in {a-map}, and not [{a-map}] - could that be the issue?#2016-10-1322:31jrheardrereading, iāve probably missed something š#2016-10-1322:31danielcomptonI know what the issue is, I'm saying that the spec error message telling me that wasn't very useful when I used s/coll-of
, so I was wondering if there was something else?#2016-10-1322:43Alex Miller (Clojure team)coll-of seems like what you want although I'm not entirely positive I understand what you're doing#2016-10-1322:53danielcomptonI want my spec to be a sequential?
collection of ::request-maps
. I define ::request-map
, then create (s/def ::request-maps (s/coll-of ::request-map))
. If I pass "abc"
and check it against my spec, I get
{:cljs.spec/problems
[{:path [],
:pred coll?,
:val "abc",
:via [:day8.re-frame.http-fx/request-maps],
:in []}]}
as it fails on the coll?
predicate. This is a pretty easy to understand problem.
However if I check a single map against this spec, I get a very large error, because the map is treated as a sequence, and each MapEntry in the map is checked against (and fails) my spec ::request-map
.#2016-10-1322:55danielcomptonI can change my spec to (s/and sequential? (s/coll-of ::request-map))
but this seems like a fairly common use case, and wondered if there was another way to spec this more precisely?#2016-10-1323:19hiredmanuse the regex ops, + or *#2016-10-1323:19hiredman(s/* ::request-map)
#2016-10-1323:20hiredman(I mean, I don't know, that is just a suggestion, seems nicer than coll-of)#2016-10-1323:27danielcomptonThat's a bit better, thanks. It short circuits earlier, but the error message is still not very illuminating:
{:cljs.spec/problems
[{:path [],
:pred map?,
:val [:method :post],
:via
[:day8.re-frame.http-fx/request-maps
:day8.re-frame.http-fx/request-maps
:day8.re-frame.http-fx/request-map
:day8.re-frame.http-fx/request-map],
:in [0]}]}
#2016-10-1323:28danielcomptonIt still treats a map as a sequence of map entries#2016-10-1323:38Alex Miller (Clojure team)I think coll-of is the best match for what you are trying to say#2016-10-1323:40Alex Miller (Clojure team)Using s/* by itself without other regex ops is less good#2016-10-1323:41Alex Miller (Clojure team)As it conveys a sequential with more interesting internal structure#2016-10-1323:41danielcomptonDoes it make sense to include maps as valid coll-of
's?#2016-10-1323:41Alex Miller (Clojure team)Yes#2016-10-1323:41Alex Miller (Clojure team)They are a collection of tuples #2016-10-1323:41Alex Miller (Clojure team)And that's very useful#2016-10-1323:42danielcomptonIs there scope to add a sequential-of?
or something similar?#2016-10-1323:42Alex Miller (Clojure team)No, you could use :kind though#2016-10-1323:43Alex Miller (Clojure team):kind sequential?#2016-10-1323:43danielcomptonNice!#2016-10-1323:43danielcomptonThat's wha I was after#2016-10-1323:43Alex Miller (Clojure team)You may also want :into []#2016-10-1323:44Alex Miller (Clojure team)Can't remember how it will conform without that#2016-10-1323:44Alex Miller (Clojure team)That might already be the default#2016-10-1323:44danielcomptonI think so#2016-10-1323:45Alex Miller (Clojure team)That's used by gen too if you care about that so might double check that too#2016-10-1323:46danielcomptondocs seem to suggest an :into is needed as well here, as I don't know if sequential?
can generate?#2016-10-1401:39Alex Miller (Clojure team)I don't know - try it? You can specify a generator too#2016-10-1402:02danielcomptonyep, generating works fine with :kind sequential?
#2016-10-1415:04robertofyi, snippets
slow down slack a lot. Maybe we should default to creating posts
instead of snippets
.#2016-10-1415:29samedhiI keep having to with-gen
or overspecify my generator in clojurescript as generation just takes too long and eventually times out.#2016-10-1415:29samedhiSpecifically coll?
as a :args
always causes me to eventually time out#2016-10-1415:29samedhiAnything clever to fix this?#2016-10-1415:30samedhiOh, I mean when using clojure.spec.test/check
.#2016-10-1415:37samedhi> (defn fx [coll] coll)
> (s/fdef fx :args (s/cat :coll coll?))
> (stest/summarize-results (stest/check `fx))
#2016-10-1415:37samedhiAs an example of timing out in cljs.#2016-10-1415:39Alex Miller (Clojure team)Have you tried using the :gen-max option?#2016-10-1415:39samedhinope#2016-10-1415:39Alex Miller (Clojure team)For spec'ing collections that is#2016-10-1415:40samedhiyeah, I saw one example of that in the spec getting started guide, forgot about it.#2016-10-1415:40Alex Miller (Clojure team)You could use (s/coll-of any? :gen-max 3)#2016-10-1417:00spiedeniām looking at doing form validation using spec. any thoughts on mapping from an explain to a slightly more human readable messages to put under fields? canāt find anything to put metadata on#2016-10-1417:01spiedenmaybe just a map from spec keys to messages would do#2016-10-1417:06spiedenhmm, looks like iād need to key off the pred symbol actually user=> (s/explain-data (s/keys :req [::foo]) {::foo ""})
#:clojure.spec{:problems ({:path [:user/foo], :pred not-empty, :val "", :via [:user/foo], :in [:user/foo]})}
#2016-10-1417:25samedhi@alexmiller Thank you for your help. I played around with it since then. Actually, the real issue was that I was including a spec in the :ret
part of my s/fdef
. This causes the value in :ret
in the :fn
part of the same s/fdef
to be the spec-conformed-return version of the :ret
, not the function-value-return version of the return. Kept causing failures that I just couldnāt understand. š#2016-10-1417:27samedhitldr; If your spec does not conform to the identity of the value passed to it, you should think about what you are doing when using it in s/fdef
ās :ret
#2016-10-1417:31spiedenhrm, wrapping it in s/and allows you to retrieve the āgroundā spec: cljs.user=> (cljs.spec/explain-data (cljs.spec/keys :req [::foo]) {::foo ""})
{:cljs.spec/problems {[:cljs.user/foo] {:pred not-empty?, :val "", :via [:cljs.user/foo], :in [:cljs.user/foo]}}}
cljs.user=> (cljs.spec/def ::foo (cljs.spec/and ::not-empty))
:cljs.user/foo
cljs.user=> (cljs.spec/explain-data (cljs.spec/keys :req [::foo]) {::foo ""})
{:cljs.spec/problems {[:cljs.user/foo] {:pred not-empty?, :val "", :via [:cljs.user/foo :cljs.user/not-empty], :in [:cljs.user/foo]}}}
#2016-10-1417:32spieden(difference is in :via values)#2016-10-1419:21samedhi@spieden Thanks you for letting me know, I did reach that solution as well. It all makes pretty good sense, it just caught me by surprise that :ret
also effected the :ret
in :fn
.#2016-10-1520:53samedhihttps://gist.github.com/samedhi/3ec2d55f2a9b554c74737b347a911210#2016-10-1520:53samedhi(above) Having a bit of confusion about when specs can be used in :ret
and when they can be used in spec/and
.#2016-10-1520:55gfredericks@samedhi the :fn
on line 24 is not supposed to be a spec#2016-10-1520:56gfredericksit's supposed to be a function that compares the args and the return value#2016-10-1520:56gfredericksand you can leave it off if you don't have anything obvious to check#2016-10-1520:56gfredericksthat might not be your problem, I'm just guessing#2016-10-1520:59samedhi@gfredericks Looking at https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/fdef, seems like a spec should be ok.#2016-10-1520:59samedhiOh, you mean within it#2016-10-1521:00samedhiYes, it does state it takes a spec, but it should ācontain predicates"#2016-10-1521:00samedhiinnnteresting#2016-10-1521:01gfredericksI would wager "A spec of the relationship..." is a typo, unless things have changed recently#2016-10-1521:01gfredericksI'll check on that actually#2016-10-1521:01gfrederickswhat's the latest clojure version#2016-10-1521:01samedhi1.9.0-alpha13 I think?#2016-10-1521:02gfrederickslooks like there's 13#2016-10-1521:02samedhiYeah, I just checked and saw that. š#2016-10-1521:04samedhi> Note that :fn specs require the presence of :args and :ret specs to conform values, and so :fn specs will be ignored if :args or :ret are missing.#2016-10-1521:05samedhiUgh, that line alone would have saved me hooours. I donāt know what you can do for people who donāt read the api docs (sorry), but figuring this out on my own just hurt.#2016-10-1521:06samedhiMaybe a warning if you include a :fn
but not :args
and :ret
#2016-10-1521:13gfrederickslooks like the s/and
thing does work, and I don't know why#2016-10-1521:13gfredericksso TIL I guess#2016-10-1521:15samedhi@gfredericks Sounds good, yeah, no idea. Thanks for you help. TIL I should always read the docs right after going through the getting started section š¬#2016-10-1605:21samedhi> (->> (with-meta #{} {:I :am-lost}) (s/conform (s/coll-of strings?)) meta)
{:I :am-lost}
> (->> (with-meta #{} {:I :am-lost}) (s/conform (s/coll-of string?)) meta)
nil
#2016-10-1605:22samedhiI get the behavior, but I wanted to fdef
check the metadata after a function call in the :fn
. However, the :fn
only has the conformed return values, so I donāt have the metadata. Any ideas how to do this. Or would you just say I shouldnāt include it in metadata?#2016-10-1613:53ggaillardHello everyone! I am going crazy on a cljs.spec strange behavior.
I use (s/fdef)
and (cljs.spec.test/instrument)
in my dev mode to catch non-conform arguments. For a strange reason my spec-ed functions keep beeing called as if a generator was forever running. I have cljs.spec.test
in my dependencies but I don't call it anywhere at the momentā¦ any idea why ?#2016-10-1615:37jetzajacHi! How do I spec named arguments of a function/marco?#2016-10-1616:10Alex Miller (Clojure team)cat#2016-10-1616:35gfredericksjetzajac: s/keys
?#2016-10-1616:35gfredericksdepends on what you meant by "named arguments" I guess#2016-10-1617:16jrheard@ggaillard i have the same problem using cljs.spec.test/instrument! iām glad itās not just me#2016-10-1617:16jrheardi canāt reproduce it reliably though#2016-10-1617:16jrheardi really hope youāre able to figure out whatās going on and open a jira issue#2016-10-1617:16jrhearditās been driving me nuts#2016-10-1617:20ggaillardI'm not crazy! Thank you! ^^
I can't manage to reproduce it too. I'm building a re-frame app and this situation always appear after a page refresh, when React tries to render some spec-ed Reagent components. I searched deeply into tens of stacktraces but found nothing particular š#2016-10-1617:22jrheardfwiw this happens for me in https://github.com/jrheard/voke, which has very few dependencies#2016-10-1617:22jrheardand does not include re-frame#2016-10-1617:23jrheardbut again i canāt figure out whatās going on, mainly because the behavior is inconsistent so coming up with a repro is maddening/difficult#2016-10-1617:39Alex Miller (Clojure team)@jetzajac: oh if you mean kwargs then cat and a nested keys* for the options#2016-10-1617:50jetzajac@alexmiller: thanx! this is what i was looking for#2016-10-1619:15jrheard@ggaillard btw while youāre using cljs.spec.test, be wary of http://dev.clojure.org/jira/browse/CLJS-1808 and http://dev.clojure.org/jira/browse/CLJS-1812 ; fixes for both are awaiting review by @dnolen#2016-10-1619:15jrheard(the first one's just a documentation issue)#2016-10-1619:18ggaillardI struggled with the first one (ended up looking the source). Thanks a lot for the second one !#2016-10-1710:53akielHi. Is there something like s/assert
which is executed every time? I like to reserve s/assert
for checks I want potentially disable in production. But I have some checks I like to execute in all cases.#2016-10-1711:04NiclasHow would one go about specing a function that takes a callback that may take any kind of argument? For ex: (defn myfunc [k cb]
(do-something-async k (fn [res err]
(check-err err)
(cb res err))))
#2016-10-1711:05akielWhy takes the callback any kind of argument?#2016-10-1711:07NiclasIf do-something-async
in this example is an un-speced library function where I only care about specing functions in my project and not external ones#2016-10-1711:09akielBut if you know what cb should look like, it would be beneficial to fail fast at your function.#2016-10-1712:31Alex Miller (Clojure team)You can use fspec to spec a function arg and any? to spec an arg that takes any value#2016-10-1712:32Alex Miller (Clojure team)So something like #2016-10-1712:33Alex Miller (Clojure team)(s/fdef myfunc :args (s/cat :k keyword? :cb (s/fspec :args (s/cat :res any? :err any?))))#2016-10-1715:34Niclas@alexmiller Thanks, thatās what I was looking for!#2016-10-1715:45vikeriShould I be able to spec an infinite sequence as an argument to a function? I tried s/every
but it got stuck in an infinite loop.#2016-10-1716:02Alex Miller (Clojure team)s/every samples so that should be able to work. Would be interested in seeing more.#2016-10-1716:35Alex Miller (Clojure team)(s/valid? (s/every int?) (range)) ;; => true
#2016-10-1720:52akhudekIs there a recommended way to handling function arguments like [a b & [foo]] ?#2016-10-1720:52akhudekI suppose you could do (s/or :two-args ā¦ :three-args ..)#2016-10-1721:09arohneralso (s/cat :a a :b b :foo (s/? foo?))
#2016-10-1721:09arohnerthe choice partly depends on intent#2016-10-1723:52akhudekah, interesting thanks#2016-10-1811:48ewenHi, some spec descriptions return fully qualified symbols while others don't#2016-10-1811:49ewenFor example
(s/form (s/every string?))
=> (clojure.spec/every string? ...)
(s/form (s/tuple string?))
=> (clojure.spec/tuple clojure.core/string?)#2016-10-1811:49ewenis this the expected behavior ?#2016-10-1812:40Alex Miller (Clojure team)No and I am working on fixing all of those right now#2016-10-1812:42Alex Miller (Clojure team)This particular one is http://dev.clojure.org/jira/browse/CLJ-2035#2016-10-1812:43ewenok thank you !#2016-10-1819:55decoursinIs there a way to stub clojure.spec.test/check
?#2016-10-1819:56decoursin(clojure.spec.test/check `foo {:stub #{`bar}})
doesn't work#2016-10-1820:06donaldballHas anyone used clojure.spec at runtime, e.g. to validate the arguments of fns at system boundaries?#2016-10-1820:20seancorfieldYes, thatās how we use it @donaldball#2016-10-1820:21donaldballWhatās your technique? Do you instrument the namespaces or fns about which you care, or manually check the args using a precondition or e.g. a custom defn
macro?#2016-10-1820:22Alex Miller (Clojure team)@decoursin do that with instrument
prior to the call to check
#2016-10-1820:24seancorfield@donaldball We explicitly call conform
and see whether we get invalid?
data back, since we need to map to a set of known error codes that our API returns.#2016-10-1820:25seancorfieldIn a couple of places we use explain-data
to help figure out which error code to return (but most are simply "parameter X was not valid").#2016-10-1820:26seancorfieldMost of our specs are data structure specs. We have some function specs. We call instrument
when weāre testing, and we have a few places where we run check
explicitly as well. Mostly weāve been doing the latter manually (in the REPL) so far.#2016-10-1909:16NiclasHow does one set the default number of tests to run with cljs.spec.test/check
?#2016-10-1912:24bhagany@looveh based on my reading of the source (specifically, https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L304), I donāt believe the default number is configurable.#2016-10-1912:31bhaganyhopefully I didnāt misunderstand you there - if youāre looking for how to specify the number of tests to run when you invoke check
, youād pass an options map that looks like {:clojure.spec.test.check/opts {:num-tests 500}}
#2016-10-1912:44decoursin@alexmiller Thanks Alex! I'll try next opportunity#2016-10-1912:51NiclasThanks @bhagany! Yes, that was my understanding too from the docs, but the call doesnāt seem to take the option into account when I test it locally: (cljs.spec.test/check
`my-func
{:clojure.spec.test.check/opts {:num-tests 200}}) =>
[{:spec #object[cljs.spec.t_cljs$spec16362],
:clojure.test.check/ret {:result true,
:num-tests 1000,
:seed 1476881391908},
:sym my-ns/my-func}]
#2016-10-1912:53minimal(st/check `my-func {:clojure.test.check/opts {:num-tests 15}})#2016-10-1912:55Niclas@minimal Yes! That did it#2016-10-1914:38bhaganyAh yes, it is different in clojurescript. I believe it's a difference in test.check iirc#2016-10-1914:42minimalYeah took me while to work it out in cljs#2016-10-1915:51jrheardhttp://dev.clojure.org/jira/browse/CLJS-1808 š#2016-10-1915:52jrheard@dnolen - this is what patch 1 on that issue fixes#2016-10-1915:52jrhearddeleting patch 2, pretend you never saw it#2016-10-1915:57Alex Miller (Clojure team)would be nice to get that fixed#2016-10-1916:08dnolen@jrheard applied to master#2016-10-1916:08jrheardš® š® š®#2016-10-1916:08jrheardthanks! š my first cljs commit!#2016-10-1916:16dnolen@jrheard nice š#2016-10-1920:34Timanyone using clojure-future-spec ?#2016-10-1921:23darwin@tmtwd yep, me#2016-10-1921:26liamdto use a spec in another namespace do you have to require anything?#2016-10-1921:28borkdudeyes, we are using it too at work#2016-10-1921:49jrheard@liamd - my understanding of the world is that you need to either require the namespace that the spec was defined in, or require another namespace that requires that namespace#2016-10-1921:50jrheardlike, the (s/def ::foo) line is a side effect, and so in order for you to use ::foo
somewhere else, you need to ensure that that side effect was run#2016-10-1921:50jrheardthis could be a poor mental model to use, but itās the one iām currently using š#2016-10-2002:16Alex Miller (Clojure team)That's right#2016-10-2002:17Alex Miller (Clojure team)s/def updates the registry so that needs to occur before you can use the spec#2016-10-2009:12mpenethow do you wrap a function that returns values to be used as gen that don't depend and randomisation at test.check level (it's an external lib that already does this (sentence generation))#2016-10-2009:13mpenetI know I can (gen/fmap (fn [_] (some-lib/foo)) (gen/any))
but that seems a bit too convoluted#2016-10-2011:34Alex Miller (Clojure team)gen/return ?#2016-10-2011:35mpenetI don't think so gen/return will always return the same value#2016-10-2011:36Alex Miller (Clojure team)The fact that foo doesn't rely on test.checks notions of randomness is actually a problem (as it can't play in growing, shrinking, or creating reproducible results)#2016-10-2011:37mpenetI know it's a bit "dirty"#2016-10-2011:37Alex Miller (Clojure team)But I think you could also use gen/bind to do this#2016-10-2011:37mpenetbind is from generator to generator, I am not sure this works too#2016-10-2011:38Alex Miller (Clojure team)Yeah, I'm thinking of it as a gen#2016-10-2011:38mpenetI am not sure I understand, you have an example?#2016-10-2011:39Alex Miller (Clojure team)I guess its not any better than what you had with fmap#2016-10-2011:40Alex Miller (Clojure team)I mean you could directly satisfy the test.check requirements to be a generator#2016-10-2011:40Alex Miller (Clojure team)I think it's just a record with a function field#2016-10-2011:41Alex Miller (Clojure team)Something like that#2016-10-2011:41mpenetyep I saw that too, but it's almost the same as the fmap solution in the end#2016-10-2011:41mpenetit's a 2 arg func#2016-10-2011:41mpenetdoesn't feel right to use it either, the "make-gen" function is private for instance#2016-10-2011:42Alex Miller (Clojure team)Well you're starting from the position of doing something you shouldn't :)#2016-10-2011:42mpenettrue#2016-10-2011:42Alex Miller (Clojure team)So it's going to feel wrong whatever you do#2016-10-2011:42mpenetit's probably not an uncommon need tho#2016-10-2011:43Alex Miller (Clojure team)First time I've seen it asked#2016-10-2011:43mpenetbut I can live with the "hack" for now#2016-10-2011:43Alex Miller (Clojure team)Since you're not going to use the value, I'd pick a simpler generator than any?#2016-10-2011:44Alex Miller (Clojure team)If you're doing the fmap one#2016-10-2011:44mpenetright#2016-10-2011:44Alex Miller (Clojure team)boolean?#2016-10-2011:45mpenetgood idea#2016-10-2011:45mpenetor even (gen/return nil)#2016-10-2011:45mpenetbut it doesn't make much diff#2016-10-2011:45Alex Miller (Clojure team)Yeah#2016-10-2011:45mpenetš#2016-10-2012:23mpenetis there a way to increase "randomness" of num generators in test.gen?#2016-10-2012:23mpenetit's often just +-1#2016-10-2012:24mpenetwell, I guess it's just a matter of gen size actually#2016-10-2012:24mpenetnevermind#2016-10-2012:25odinodinDoes anyone have any tips on how to use clojure.spec/keys on Datomic entities? Looking for a workaround for the following http://dev.clojure.org/jira/browse/CLJ-2041#2016-10-2012:37Alex Miller (Clojure team)I don't know that there is a solution to 2041 btw#2016-10-2012:38odinodincurrently hurting us bad, since we work with entities all over, treating them as maps#2016-10-2012:38Alex Miller (Clojure team)The lighter weight lookup interfaces do not have a way to iterate over entries which keys needs to do#2016-10-2012:39Alex Miller (Clojure team)I know this has been discussed re Datomic but I'm not sure where things will end up#2016-10-2012:39odinodinright, also guess conform would call assoc as well#2016-10-2021:10liamdis there a spec equivalent of annotating types using the s/defn
, etc. macros in schema with :-
and all that?#2016-10-2021:23liamdor do i just use fdef
#2016-10-2021:31liamdalso, so the name of a spec is a key that corresponds to what it must be called in a map, but also is just the name of a thing if youāre using it not in a map?#2016-10-2021:45liamdso how do i spec a map where the keys are strings?#2016-10-2021:59jasonjcknuse conformer#2016-10-2022:55bfabry@liamd s/keys requires keyword keys for validating maps. if you don't have keyword keys, you'll need to use map-of, coll-of, custom predicates etc#2016-10-2022:55bfabryand yes, use fdef#2016-10-2023:32jasonjcknone of my dependencies is broken ""call to clojure.core/defn did not conform to spec""#2016-10-2023:32jasonjcknis there a way to disable spec checks?#2016-10-2023:33seancorfieldMost of the libraries "broken" by recent spec versions have already released new versions. What is breaking?#2016-10-2023:34jasonjckna few different deps, i'm trying to upgrade them now, i'll let you know if it's still broken#2016-10-2023:35jasonjcknalso fyi the error mesasge makes it very hard to know who the culprit is#2016-10-2023:35seancorfieldThis page lists the known ones: http://dev.clojure.org/display/design/Errors+found+with+core+specs#2016-10-2023:35seancorfield(and what releases have fixes)#2016-10-2023:36seancorfieldI would have expected the error to fairly clearly indicate the source file containing the illegal defn
?#2016-10-2023:39seancorfieldMonger...#2016-10-2023:40seancorfieldCurrent version is 3.1.0#2016-10-2023:45seancorfieldThe problem is the format of :or [auto-connect-retry true]
which should be :or {auto-connect-retry true}
#2016-10-2023:46seancorfieldLooks like it was reported in early June and fixed in mid-August https://github.com/michaelklishin/monger/issues/142#2016-10-2023:48seancorfieldSo, yeah, youāll need a newer version of Monger than 3.0.2 @jasonjckn#2016-10-2023:48jasonjcknkk, thanks sean#2016-10-2023:50seancorfieldI updated http://dev.clojure.org/display/design/Errors+found+with+core+specs to show 3.1.0 contains the fix for Monger 3.0.2ās breakage.#2016-10-2107:25tengIf I want to make sure that a map does not contain a specific attribute, for example :user/password, how do I do that?#2016-10-2108:12pyrhola specers!#2016-10-2108:12pyrI finally bit the bullet and am moving some of my work to spec.#2016-10-2108:12pyrI have a few questions#2016-10-2108:13pyrMy first one is how to "compose" or programmatically generate specs, I have a multi spec with a lot of repetitive code#2016-10-2108:13pyrwhere each spec in the multi-spec has common members and a few things that vary#2016-10-2108:14pyrso i'd like to be able to do something along the lines of (defn base-op [& ks] (spec/keys :req (concat [::base1 ::base2] ks)))
#2016-10-2108:15pyrThis doesn't work because spec/keys
is a macro#2016-10-2108:15pyrAny hint as to how to approach this?#2016-10-2108:20pyrRight now my method is to go with macros:#2016-10-2108:21pyr(defmacro base-op [& ks] `(spec/keys :req ~(concat [::base1 ::base2] ks)))
#2016-10-2108:22pyrThis works but doesn't feel right, if there's a better approach, happy to hear about it š#2016-10-2108:25mpenetI guess you can use s/and
#2016-10-2108:26mpenetthere's also s/merge
for this purpose, but I can't say really, haven't used it much so far#2016-10-2108:29mpenet(s/def ::foo string?)
(s/def ::bar string?)
(s/def ::base (s/keys :req [::foo]))
(s/def ::stuff (s/keys :req [::bar]))
(s/def ::n (s/and ::base ::stuff))
(s/valid? ::n {::foo "0" ::bar "1"}) => true
#2016-10-2109:01mpenetI did it wrong again: (gen/large-integer* {:min 1e6 ...})
-> Doubles (my fault I know), but maybe there could be an assert (or casting) there#2016-10-2109:37pyr@mpenet, thanks!#2016-10-2109:40mpenetIs it possible to "share" a generator value for multiple keys in a map (or values in a spec) -> ex: a user profile with N fields?#2016-10-2109:43mpenettime to look up gen overrides I guess#2016-10-2111:17jetzajacHi! How do I override generators for test/check? doc says we can pass :gen parameter in there, but canāt find any example. passing something like (stest/check split-operation {:gen {:onair.ot/count (s/gen :onair.ot/bounded-count)}})
shows that it is not generator which is expected as a value, but what then?#2016-10-2111:18jetzajacgetting error like clojure.test.check.generators.Generator cannot be cast to
clojure.lang.IFn#2016-10-2111:20jetzajacthe problem Iām trying to address is actually testing for interger overflow, I donāt care about it, but generator for int passes huge numbers to a function, which causes it to fail with overflow exception. bounding value with predicate in spec causes a fail in conforming results =(#2016-10-2111:27jetzajaclooks like (stest/check split-operation {:gen {:onair.ot/count #(s/gen :onair.ot/bounded-count)}})
helps š#2016-10-2118:01hiredmanclojure.spec.gen wraps clojure.test.check.generators, and the way it wraps it results in generators becoming no argument functions that return generators (this is to make the dependency on test.check optional). It can make it kind of confusing about when to use a test.check generator and when to use a function returning a generator, most notably when using the generator combinators in clojure.spec.gen#2016-10-2118:33Alex Miller (Clojure team)the answer is to always use a function returning a generator#2016-10-2118:33Alex Miller (Clojure team)I only find it to be confusing when using both spec.gen and test.check.generators at the same time#2016-10-2118:33Alex Miller (Clojure team)so I mostly try not to do that :)#2016-10-2119:37liamdiām trying to s/exercise
a function i specād and iām getting:
ExceptionInfo Unable to construct gen at: [] for:
what does it mean ā`at: []`"#2016-10-2120:15hiredmanuser=> (require '[clojure.spec :as s])
nil
user=> (s/def ::foo (constantly false))
:user/foo
user=> (s/exercise ::foo)
ExceptionInfo Unable to construct gen at: [] for: :user/foo clojure.core/ex-info (core.clj:4725)
user=>
#2016-10-2120:15hiredmanyou have a predicate that spec doesn't know how to find the generator for#2016-10-2120:17kennyWhere are you guys putting your clojure.spec.test/check
calls? Are they in an is
where you check if :result
is true?#2016-10-2120:19hiredman'[]' is the path to the spec, so an empty path means the top level#2016-10-2120:24kennyWhen using test.check I would use defspec
. Is there some similar integration point with clojure.spec.test?#2016-10-2120:48kennyMaybe something like this?
(defmacro defspec-test
[name & test-check-args]
(when t/*load-tests*
`(def ~(vary-meta name assoc :test `(fn [] (clojure.spec.test/check
#2016-10-2121:10liamdhm so it canāt find a generator for my function#2016-10-2121:11liamdthis is right beneath it:
(s/fdef option-settings->environment-variables
:args :elb/option-settings
:ret :elb/option-setting
:fn #(= "EnvironmentVariables" (-> % :ret :option-name)))
#2016-10-2121:11liamdis that enough to give it a generator or am i missing something?#2016-10-2121:19liamdoh i should probably use exercise-fn
#2016-10-2121:55liamdi followed this exactly https://asciinema.org/a/87157 to use exercise-fn
and get:
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:583)
=/#2016-10-2123:19jrheard@kenny i havenāt seen official guidance on this. have you seen the summarize-results function, though?#2016-10-2123:20kennyYes#2016-10-2123:20jrheardiāve seen people assert on its return value in regular deftests#2016-10-2123:21kennyI wrote a version of defspec
that uses abbrev-result
. So you could do something like (defspec-test qc-myfn? `myfn)
#2016-10-2123:21jrheardnice#2016-10-2123:21jrheardhavenāt seen abbrev-result#2016-10-2123:21kennyIt works for now. I'm curious what the official solution will be#2016-10-2123:22jrheardme too! š#2016-10-2123:22kennyIt's clojure.spec.test/abbrev-result
#2016-10-2123:22kennyAlso combined it with clojure.spec/explain-out
#2016-10-2123:23kennyclojure.test + generative tests don't fit perfectly -- it's not clear what the :expected
and :actual
values in the clojure.test report should be.#2016-10-2123:24kennyIn my case I used the :ret
for :expected
and :clojure.spec.test/val
for :actual
#2016-10-2216:12cddrHas anyone thought much about using spec definitions to generate avro/protocol buffer/thrift schemas (and corresponding coercion of data in those formats into clojure and back) or is that a fools errand?#2016-10-2217:04cddrI think what I really want to do is associate specs (or perhaps conformers) with "hints" that would document the low-level type implied by some conformer.#2016-10-2219:32donaldballI have had modest success using them as parts of fixed-width file parsers#2016-10-2300:52cddrDoes your approach require the generation of some sort of schema artifact representing the on-disk representation of your data?#2016-10-2322:38bfabry@cddr I fully intend to be using something that does spec->avro schema within the next 6 months. either someone else will write it and I'll use it or I'll write it#2016-10-2322:50cddrWould you want it to infer the avro type from the spec? Or would you be happy to annotate the avro part.#2016-10-2323:10bfabryprobably shoot for from spec. I'm thinking if spec can do this with generators https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/gen.clj#L128 I can do something similar with avro types#2016-10-2323:14cddr:+1:#2016-10-2407:37mpenet@bfabry: In my small experience with this stuff specs is too "open" to allow conversions like this without hitting tons of problems down the road. Prolly better to limit yourself to a subset of specs, and maybe front all this will a small DSL that would avoid people to go wild with theirs spec#2016-10-2422:35bfabry@mpenet yeah, I definitely wasn't planning on trying to create a "complete" spec->avroschema thing. just register arbitrary predicates against their avro conversion#2016-10-2510:48pviniswhat is the idiomatic way to write specs? in a spec
directory? how do you run them? is there a lein spec
like lein test
? how do you guys do that? i guess specs are not just to be run in a repl while developing..#2016-10-2512:39Alex Miller (Clojure team)Specs are code. You can either put them inline with your other code or in a separate namespace.#2016-10-2512:40Alex Miller (Clojure team)You can use them in a variety of ways#2016-10-2512:41Alex Miller (Clojure team)If you want to use them in your production code you can call s/conform or s/valid?#2016-10-2512:41Alex Miller (Clojure team)If you want to be able to toggle them on or off you can use s/assert#2016-10-2512:43Alex Miller (Clojure team)If you want to use them during testing you can use stest/instrument (to check calls to a function) and stest/check to generatively test a spec'ed function#2016-10-2518:21donaldballI think I may have run into a bug in conform
:#2016-10-2518:21donaldball(s/def ::mapping-offset
(s/cat :offset nat-int? :length nat-int?))
(s/def ::mapping-offsets
(s/map-of ::mapping-offset ::mapping-offset))
#2016-10-2518:21donaldball(s/conform ::mapping-offsets {[0 3] [4 3]}) => {[0 3] {:offset 4, :length 3}}
#2016-10-2518:22donaldballEasy enough to work around using s/tuple
instead of s/cat
, but figured Iād mention it here. Should I file a bug report?#2016-10-2518:48Alex Miller (Clojure team)no, thatās expected behavior#2016-10-2518:49Alex Miller (Clojure team)you can pass :conform-keys true
as additional map-of options to get both conformed#2016-10-2518:49Alex Miller (Clojure team)default is false (as often you donāt need it, and thereās a perf cost, and thereās the possibility of conformed keys overlapping and losing data in the conformed map)#2016-10-2518:50Alex Miller (Clojure team)this is in the docstring for map-of
too#2016-10-2518:51Alex Miller (Clojure team)Iām assuming that you were objecting to the vals being conformed but not the keys#2016-10-2518:51Alex Miller (Clojure team)if you actually want neither conformed, you can do that with a conformer on the val spec#2016-10-2518:52Alex Miller (Clojure team)or change it to use tuple as you suggested#2016-10-2522:10Oliver GeorgeHey @alexmiller, I think there's a bug in (s/form (s/every pred ...)). (s/form (s/coll-of (s/keys :req-un [:PROJ_CONTACT/NAME])))
returns a pred of s/keys
instead of clojure.spec/keys
.#2016-10-2522:11Alex Miller (Clojure team)yes, thereās a ticket and a patch waiting for Rich already on that#2016-10-2522:12Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2035#2016-10-2522:13Oliver GeorgeThanks Alex.#2016-10-2522:19Alex Miller (Clojure team)that patch was super annoying with all the macrology - itās hard to tell but I put a lot of work into making that as small a change as it is#2016-10-2522:19Oliver GeorgeThat code is quite tricky.#2016-10-2522:20Oliver GeorgeWhile you're about. I'm interested in finding a good approach for walking through specs to transform. In my case I'd like to use specs to produce a simple version of a datomic pull query.#2016-10-2522:20Oliver GeorgeThis is what I have https://gist.github.com/olivergeorge/8368ab3637d0ade128166de78bac7903#2016-10-2522:21Oliver GeorgeI suspect there's a slightly more artful approach. Any chance you can point me at an example.#2016-10-2522:24Alex Miller (Clojure team)so one thing that is coming (and is behind all my spec form fixes) is specs for s/formās return, basically specs for spec forms, which is the path youāre on there#2016-10-2522:24ericfodeHas anyone tried using test.check cljs.spec and core.async together?#2016-10-2522:25Alex Miller (Clojure team)@olivergeorge so I think using spec to conform spec forms is imo a good approach#2016-10-2522:26ericfodeI am having some trouble getting a test that uses promises wired up to be the body of a property. (the example i am extending is here https://github.com/clojure/clojurescript/wiki/Testing#async-testing and here https://github.com/clojure/test.check#clojurescript)#2016-10-2522:27Oliver George@alexmiller Thanks. The spec form specs (!) sound useful. Perhaps those could be the basis for post-walk wrapper or zipper. I'm thinking that might make walking/transforming the spec tree less quirky.#2016-10-2522:28Oliver GeorgeNice to know i'm on the right track. Much appreciated.#2016-10-2522:40donaldballThanks, alexmiller, I should have checked the docstring first.#2016-10-2522:46gfredericksis (s/keys)
the best way to express s/map-of
-style varargs?#2016-10-2522:47gfredericksassuming my key-spec is keyword?
of course#2016-10-2522:50bfabry@gfredericks s/keys* is for varargs#2016-10-2522:51bfabryassuming you meant what I thought you did by that#2016-10-2523:02gfredericksyeah I did#2016-10-2523:02gfredericksin particular I mean (s/keys*)
with no args#2016-10-2523:14Alex Miller (Clojure team)Why no args?#2016-10-2523:14Alex Miller (Clojure team)I mean, that's fine but opt-un is useful on the gen side#2016-10-2600:21gfredericks@alexmiller it would be better as a s/map-of
but afaik I can't use that in a regex#2016-10-2600:21gfredericksI don't have any explicit keys#2016-10-2600:23Alex Miller (Clojure team)Well then sounds good#2016-10-2600:23Alex Miller (Clojure team)Any registered keys will get checked#2016-10-2601:20gfredericksthat would be bad actually#2016-10-2601:21gfredericksI'm trying to spec gen/hash-map
#2016-10-2601:21gfrederickscome to think of it I can just do (s/* (s/cat :k keyword? :gen gen?))
#2016-10-2601:21gfredericksand that's perfect#2016-10-2613:34gfredericksSo I'm specing some macros#2016-10-2613:35gfredericksAnd it seems common for there to be macro args where there's a straightforward spec for the runtime value, but the macrotime form could be almost anything#2016-10-2613:36gfredericksSo my first thought is to use any?
, but that's unsatisfying since it doesn't tell users anything and doesn't even try to catch problems evident at macroexpansion time#2016-10-2613:37gfredericksBut anything fancier would be somewhat complex#2016-10-2613:37gfredericksSo I'm wondering if it's worth it#2016-10-2613:37gfredericksOr if we should just expect macro specs to have a lot of any?s in them#2016-10-2613:41Alex Miller (Clojure team)yes#2016-10-2613:42moxaj@gfredericks I think your macro specs shouldn't be (can't be?) concerned about runtime semantics#2016-10-2613:42Alex Miller (Clojure team)Iāve spent a fair amount of time looking at this and in general, itās quite common for :ret to not be worth spec'ing#2016-10-2613:42Alex Miller (Clojure team)and in that you should just omit it#2016-10-2613:43Alex Miller (Clojure team)currently macros are skipped during check and instrument only checks :args, so there is not much value in specāing :ret anyways beyond docs#2016-10-2613:44Alex Miller (Clojure team)for :args where some can be scoped but others canāt, I think use of any? is fine (you can see some of this in the specs Iāve done in clojure.core.specs)#2016-10-2613:45Alex Miller (Clojure team)and then consider what you are expanding to - sometimes itās better to spec that instead (particularly if it bottoms out at a function)#2016-10-2614:23gfredericksThat makes sense; thanks#2016-10-2615:11quollIs there a suggested approach to building specs in Clojure-1.8 code in anticipation of the 1.9 release? The āalphaā label puts managers off systems that are rapidly going into production, but I would rather be writing specs as I write code, rather than trying to go back to older code and updating it after 1.9 comes out.#2016-10-2615:15trptcolini haven't used it, but https://github.com/tonsky/clojure-future-spec is the effort i've heard about#2016-10-2616:56seancorfield@quoll Any chance that you can persuade them that Clojure 1.9 Alpha = Clojure 1.8 + spec?#2016-10-2616:58Alex Miller (Clojure team)not to shoot down a direction towards more spec usage, but that is not exactly true and will become less so#2016-10-2616:59Alex Miller (Clojure team)1.9 is still alpha for good reasons and will not go beta until there are a few more things that have been stabilized. It is still reasonably likely that there will be significant changes in the spec internals.#2016-10-2616:59Alex Miller (Clojure team)most of that would probably not affect the api but it may affect the idea of forms as the information model for specs#2016-10-2617:00Alex Miller (Clojure team)there is a lot of concern that we get certain parts of spec ārightā as they will be very difficult to change later#2016-10-2617:01Alex Miller (Clojure team)so if youāre worried about investing in something that might change, then your managers are right and you should hold off :)#2016-10-2617:02Alex Miller (Clojure team)if youāre worried about using alpha software because it may have known and significant bugs, you should also hold off, because there are some :)#2016-10-2617:04Alex Miller (Clojure team)1.9 beta 1 to us signals that the first part (deciding what features to include and provide) is over#2016-10-2617:04Alex Miller (Clojure team)1.9 rc1 signals that we believe the major bugs have been fixed#2016-10-2617:08Alex Miller (Clojure team)That said, I think the two approaches are to use clojure-future-spec with 1.8 or to separate your specs from your core so that you can build and deploy with 1.8 but test with 1.9 if you like#2016-10-2617:30seancorfieldUnderstood. My point was more that clojure.spec is (almost) the only new thing in 1.9 so the risk of using the current alphas is actually very low, as far as existing 1.8 behavior is concerned.#2016-10-2617:30seancorfieldI like the suggestion to have a 1.8-buildable system and an option to test against 1.9 with spec on top of that.#2016-10-2617:32seancorfieldFor us, weāve used Clojure Alpha builds in production dating back to 1.3 Alpha 7 or 8 in 2011 and weāve found it amazingly stable. We find the benefits of being able to leverage new features far outweighs the risks and I think weāve only had to roll back one or maybe two prerelease builds in those five years.#2016-10-2617:32Alex Miller (Clojure team)@seancorfield weāve had several regressions related to the changes in destructuring and namespaced map syntax#2016-10-2617:33seancorfieldYup, and there are often small regressions or breaking changes in each release but those have either not affected us or been very minor impact issues.#2016-10-2617:35seancorfieldI can certainly understand some companies being alpha-averse thoā. With some of the projects I maintain, Iāve encountered companies who wonāt even consider an RC build. I find the most risk-averse companies tend to be the ones with the worst test coverage and therefore more likely to be unable to detect a regression in their own codeā¦ š#2016-10-2617:35seancorfieldI think the Clojure/core team has done an incredible job of stability with Clojure releases over the years.#2016-10-2617:37seancorfieldWe always run our test suite against both our regular selected Clojure release and against Clojure master snapshots to catch any new breaking changes early.#2016-10-2702:46cddrI sense that there is some profound wisdom in the "informational vs implementation" section of the spec overview but I fear it goes over my head. Is it a hint to users to not do something with spec that might seem tempting? Or is it a note to implementors to stay away from problems that cannot be solved in the general case.
Could anyone provide an example of an implementation decision that should not go into a spec.#2016-10-2702:51cddrAlso it seems like most people talking about it here are using it to spec inputs/outputs of functions and macros. Is anyone using (or planning on using it) to define system boundaries? The note in the same doc about starting a dialog about semantic change seems particularly relevant to this type of usage.#2016-10-2703:07seancorfield@cddr We're using it to spec system boundaries more than function inputs/outputs.#2016-10-2703:08cddrHow do you share the specs between the two systems?#2016-10-2703:08seancorfieldWe're spec'ing our REST API at the user boundary, the domain model boundary, and the persistence boundary.#2016-10-2703:08seancorfieldRight now, those are still in a single process but that is changing.#2016-10-2703:09seancorfieldWe keep the specs mostly separate from the code. We haven't completely decided how to package that up in libraries for sharing across systems, but we expect to have the spec libraries as dependencies to both systems (everything's Clojure).#2016-10-2703:10cddrThose categories are interesting too. Is there any overlap between domain model and persistence for example?#2016-10-2703:10seancorfieldWe're mostly focusing on data -- and generators -- rather than functions.#2016-10-2703:11seancorfield@cddr We're still looking at whether we can (mostly) derive one from the other... we've had some success with that but there are... special cases... shall we say.#2016-10-2703:13seancorfieldSome parts of our API boundary match our domain model better than others š#2016-10-2703:14cddrYeah, it seems like it would be easy to generate a spec from some sort of schema like avro, but a bit harder to go the other way.#2016-10-2703:15cddrThanks for your thoughts. Are you going to be at the conj? I'd love to chat some more about this.#2016-10-2703:15seancorfieldWe are started with a legacy API and evolving a new one, piecemeal, so we don't have the luxury of anything like Swagger at the moment etc.#2016-10-2703:16seancorfieldYeah, I'll be there. I submitted a talk about this topic -- and it got accepted -- but partly for personal reasons I've had to withdraw it.#2016-10-2703:16cddrAh well look forward to seeing you then. š#2016-10-2703:16seancorfieldI love Austin š#2016-10-2712:19ikitommi@seancorfield @ccdr we have been writing the spec->swagger/json-schema conversion, should help in the docs part. Also, the JSON/String conforming of the specs. Trying to make those transparent. #2016-10-2712:22mpenet@ikitommi anything public ?#2016-10-2712:24mpenetalso @alexmiller has been hinting about changes in spec forms (https://clojurians.slack.com/archives/clojure-spec/p1477501089007213), I guess this might (or not) change the situation for these kind of things#2016-10-2712:25mpenetpersonally I am holding my horses until this stabilizes (I am very much interested in having something solid for swagger, json-schema & all)#2016-10-2713:24ikitommi@mpenet nothing really usable as public. There is stuff under metosin/spec-tools (second take on the ->json-schema & original spike on the dynamic conformation). But just planned to put effort on the next two weeks with these. I'm working on the spec->json-spec & spec->string-spec transformation - e.g. generating differently conforming copies of the specs for different formats/use cases. If that works, will throw the dynamic conformations into trash bin. Coercion rules in both from Schema/Ring-swagger. Were you doing the json schema -> spec side? or the same?#2016-10-2713:25ikitommiand thanks for pointing out Alexās message. Would really like to hear how thing will evolve.#2016-10-2713:29mpenetSomething similar (or not), not sure. We have a thin facade that generate both specs with conformers and json-schemas, this way this shields us from changes in the underlying libs for now. Moderately happy about the current situation, hopefully the changes alex mentioned will make this stuff easier.#2016-10-2713:29mpenetgoing from spec to json-schema directly is something we tried but this was brittle, tons of possible breakage unless you set some strict rules of what you can / cannot use in spec#2016-10-2713:31ikitommisounds cool. So, you are describing something in your own way and generate specs out of those? of do you start from plain specs too?#2016-10-2713:32mpenetyes, at the edge we have our own "spec" language#2016-10-2713:32mpenetI think also onyx does something similar#2016-10-2713:32ikitommido you have something of that in public?#2016-10-2713:32mpenetbut hopefully this is temporary#2016-10-2713:33mpenetno not yet, we might release this but no promises.#2016-10-2713:34ikitommithe old (plumatic) schema -> json schema was partly brittle too, and as the json schema (and expecially the swagger version of it) is less powerfull, only parts of the Schemas could be presented in it.#2016-10-2713:35ikitommiwhat did you find brittle in the direct spec->json-schema conversion?#2016-10-2713:35mpenetindeed. same problem#2016-10-2713:36mpenetthe fact that a spec is a predicate makes it too open and hard to "understand", unless you are 100% aware of the limitations you can cause the generation of json-schema to go "wtf is that" to some validators/converters#2016-10-2713:38mpenetdunno what's best between limiting ourselves to a subset our the facade solution. the facade has the advantage of shielding us from potential dependency changes (Schema -> Spec), and allows the devs not to be afraid to break stuff in a dependency they never inspected#2016-10-2713:39mpenetseems like that could be a good solution for compojure-api & related libs imho#2016-10-2713:43ikitommiyes, at least the current spec maps are quite hard to use, was thinking of adding schema-like map -support ~ {::name string?, ::id integer?}
#2016-10-2713:44ikitommiin web apis, currently we have lotās of anonymous maps to describe the different paramerer sets for endpoints.#2016-10-2713:48mpenetI think the validators should be more static (for compojure-api & co), using string? integer? is not so good in that context, just keywords (:string :integer etc) that dispatch to a multimethod that knows what to make out of it and act as some sort of registry that's limited to that dsl. Allowing the user to create schemas and register/use them too in that context since they're known to be understandable by the coercion/validation chain later.#2016-10-2713:49mpenetanyway, I am curious to see what will come next#2016-10-2715:27zaneAre conformers intended to be limited in scope? As in, should I use them exclusively for minor transformations, or would it still be idiomatic to use them for larger-scale manipulation?#2016-10-2715:27zaneHope that question make sense. Sorry for being vague.#2016-10-2715:42Alex Miller (Clojure team)they should be used cautiously, particularly when using them with registered specs as you are making decisions for all consumers of your specs (for now and the future, which is long :)#2016-10-2715:42Alex Miller (Clojure team)in general, I think generic data transformation is better done explicitly with functions in normal Clojure ways#2016-10-2716:24seancorfieldThe problem we ran into is that we had a lot of specs for our API that needed to accept a "string that could be converted to something that conforms to another spec" ā and so your choice is either to make the spec produce the converted value (using conformer
) or run that conversion twice: once to check you have a compliant string, and then again as Clojure code once you have "validated" your string input.#2016-10-2716:26seancorfieldYour other choice is to run the conversion as a "pre-spec" layer and deal with conversion failures outside specā¦ which is a lot of boilerplate.#2016-10-2716:28seancorfieldGiven Alexās dark warnings about using conformer
"extensively" (which we were), we have sort of called a halt to our spec usage in order to see what falls out as the recommended approach for API specs (i.e., specs that must accept strings that essentially conform to other specs after "conversion").#2016-10-2716:29seancorfieldIf that recommended approach is "run the conversion twice", well, fair enough, but it seems a bit of a waste of effort.#2016-10-2716:56Alex Miller (Clojure team)I think perhaps you take my words more apocalyptically than they were intended :)#2016-10-2716:57zaneThanks for the exposition, @seancorfield. That's precisely the situation I'm in.#2016-10-2716:59seancorfield@alexmiller Well, you have repeatedly cautioned against (overuse of) conformer
whenever this sort of pattern comes up ā and Iām not seeing a recommendation for how to handle it otherwise.#2016-10-2717:00seancorfieldFor us, having the API spec also handle the conversion (via conformer
) is the easiest way to do things. But easy != simple and I understand that it complects validation with transformation...#2016-10-2717:04Alex Miller (Clojure team)My point is and has always been: you are making a choice for all future consumers of your registered specs and you should think about the ramifications of this before you use conformers on everything#2016-10-2717:05Alex Miller (Clojure team)thatās it#2016-10-2817:46drewrI'm having a time with clojure.test.spec/check
#2016-10-2817:46drewr{:spec
(fspec :args (cat :m :runbld.store/es-opts) :ret string? :fn nil),
:sym runbld.store/make-connection,
:failure #error {
:cause "clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn"
:via
[{:type java.lang.ClassCastException
:message "clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn"
:at [clojure.spec$map_spec_impl$reify__13764 gen_STAR_ "spec.clj" 821]}]
:trace
[[clojure.spec$map_spec_impl$reify__13764 gen_STAR_ "spec.clj" 821]
[clojure.spec$gensub invokeStatic "spec.clj" 269]
[clojure.spec$re_gen invokeStatic "spec.clj" 1565]
[clojure.spec$re_gen$ggens__14211$gen__14212 invoke "spec.clj" 1554]
[clojure.core$map$fn__6882 invoke "core.clj" 2739]
[clojure.lang.LazySeq sval "LazySeq.java" 40]
[clojure.lang.LazySeq seq "LazySeq.java" 49]
[clojure.lang.RT seq "RT.java" 525]
[clojure.core$seq__6416 invokeStatic "core.clj" 137]
#2016-10-2817:47drewrwhich comes from this fdef
(s/fdef make-connection
:args (s/cat :m ::es-opts)
:ret string?)
(defn make-connection
[args]
(http/make args))
#2016-10-2817:48hiredmanyou are using a test.check generator directly somewhere, maybe for ::es-opts#2016-10-2817:48hiredmanyou need to wrap them in a thunk#2016-10-2817:48drewryeah, I suspected that, but I don't :require
test.check anywhere#2016-10-2817:49drewrah wait, I see it now#2016-10-2817:49drewryeah ::es-opts
was it#2016-10-2817:50drewrI was getting that exception for a with-gen
in a separate place and kept thinking it was the same place#2016-10-2817:50drewr@hiredman thanks kev!#2016-10-2817:50hiredmanš#2016-10-2818:04cap10morganWhat are folks' thoughts on generative testing w/ spec on stateful fns (in this case modifying an atom)? How do you setup enough variations on the state for meaningful tests? Or is the answer that I should go to test.check
directly instead of relying on what clojure.spec
generates?#2016-10-2818:07bhagany@cap10morgan: I havenāt done anything stateful yet, but if I had special generative needs, Iād reach for passing a custom generator in with-gen
first. If I understand you correctly, this is what youāre reaching for with the idea to use test.check
directly.#2016-10-2818:08cap10morgan@bhagany OK, thanks. You answered my next question too. š#2016-10-2818:11seancorfield@cap10morgan Iād try to separate the logic from the state and test the logic separately if possible.#2016-10-2818:12cap10morgan@seancorfield Hmm, that's a good idea. I need to get into the habit more often. Thanks!#2016-10-2818:13seancorfieldI was just having a similar conversation in #component š#2016-10-2818:15drewrhow do you figure out which spec or gen is problematic when you get Couldn't satisfy such-that predicate after 100 tries.
#2016-10-2818:16drewrthe stacktrace doesn't provide any clues that I can see#2016-10-2818:19hiredmanI think alexmiller has mentioned that as a pain point, that error comes out of test.check and plumbing the spec information through from spec to test.check and back again hasn't been done#2016-10-2818:19hiredmanI would start looking at your s/and
s I guess#2016-10-2818:21hiredmanif I recall, the way generators for s/and specs are constructed is you take the generator for the first spec in the s/and, and then such-that the rest of the specs (as predicates)#2016-10-2818:21hiredmanso if you have a really generic first thing, and more specific predicates afterwards, you can easily end up with a such-that that doesn't work#2016-10-2818:24drewryep, that's exactly what it was#2016-10-2818:24drewr(s/and string?
#(.startsWith % "http"))
#2016-10-2818:25drewrI had a string, but it was "foo"
#2016-10-2818:41Alex Miller (Clojure team)there is an exactly almost the same as this in the spec guide http://clojure.org/guides/spec#2016-10-2818:42Alex Miller (Clojure team)that shows how to make a custom generator for it#2016-10-2818:45drewr@alexmiller it wasn't that I misunderstood the spec, I just couldn't find the problematic one#2016-10-2818:46drewrthanks for that doc!#2016-10-2902:03steveb8nI have zipper data structure using the techniques described in https://puredanger.github.io/tech.puredanger.com/2010/10/22/zippers-with-records-in-clojure but Iām finding it hard to write a spec that validates child nodes using the protocol. I canāt use a map key because different impls of TreeNode (Zippable in my case) may use different keys. Iāll paste an e.g. next....#2016-10-2902:04steveb8nAny suggestions would be much appreciated#2016-10-2904:04grantIs there an easy way to figure out which spec in a large nested spec that is causing "ExceptionInfo Couldn't satisfy such-that predicate after 100 tries." when using (s/exercise ...)
?#2016-10-2904:25jrheardnot afaik but iām not an expert#2016-10-2904:27jrheardusually itās down to a string or something that has lots of constraints put on it and is difficult to generate reliably, but you probably already know about that; one gotcha that i donāt think is as well-known is that if you have a coll-of thatās got a :kind set?
but doesnāt have an into #{}
, that can cause the same couldnāt-satisfy-after-100-tries behavior#2016-10-2904:28jrheardmight not be your issue, but i always feel compelled to mention it when people have this problem because it happened to me and took me a while to figure out#2016-10-2904:30grantAh, good tip on the set thing, I haven't run into that yet, but I could see making that mistake.#2016-10-2904:37seancorfieldIt doesn't really help you but I highly recommend trying to s/exercise
each spec as you write it and build up complex specs in terms of other specs (not just predicates).#2016-10-2904:38seancorfieldPredicates tend to be opaque when you combine them into specs. If you define specs for those predicates, when you combine the specs, you get better error reporting.#2016-10-3021:34drewrI'm having trouble tracking down a :clojure.spec/unknown
#2016-10-3021:35drewrthe :clojure.spec.test/caller
, :sym
, and :via
all point to different things, it doesn't seem to make any sense#2016-10-3021:59cddrFolks interested in clojure-spec might be interested to know about some prior art in the pharma industry. They have an organization called "CDISC" that defines a collection of XML standards for defining clinical databases for pharma trials. Their "operational data model" has an info model quite similar to spec. It used to be private only to members but looks like they've opened it up recently: https://www.cdisc.org/standards/foundational/odm#2016-10-3022:25mac@cddr How is this different from any other XML Schema ?#2016-10-3022:27cddrThe bit that spec borrows is the way it defines "ItemDef"s independently of their containers#2016-10-3022:30mac@cddr Ok...#2016-10-3112:03tcouplandI feel like there's a missing flag on s/keys, something like 'complete' or {:allow-unspecified false} as an option. A couple of times while using s/check i've spelled something wrong, in the spec, or set a key badly in the transform code, and spec isn't yelling at me about it, despite everything being spec'd up, it comes down to this for me:
(s/def ::key integer?)
(s/def ::other integer?)
(s/def ::map
(s/keys :req [::key]
:opt [::other]))
(s/valid? ::map {::key 1 ::others "2"}) => true
I've clearly got this wrong (::others instead of ::other), but it's really easy to just not notice something like this, but if I could do this:
(s/def ::map
(s/keys :req [::key]
:opt [::other]
:complete true))
and have spec shout if it see's any key it's not ready for, then I'd be getting more out of my spec's! Just something that's bothered me this morning as i realised I'd mistakenly pluralised a key word š#2016-10-3112:04tcouplandhopefully someones going to tell me it already exists somewhere now!#2016-10-3114:00donaldballThe usual workaround is to combine the s/keys
spec with an s/map-of
spec#2016-10-3114:02donaldball(I agree that a :complete
flag would be a welcome addition to the s/keys
macro, but Rich seems committed to open maps as the preferred general data structure)#2016-10-3114:06Alex Miller (Clojure team)specs always make positive statements about data, not negative ones and we do not plan to add anything to s/keys along these lines#2016-10-3114:06Alex Miller (Clojure team)however there is something weāre considering that would allow making these additional assertions easier#2016-10-3114:27tcoupland@donaldball that does work, but is pretty ugly isn't it. Sound like a 'wait and see' situation#2016-10-3114:46donaldballItās not a hard macro to write, a keys
variant that allows a complete?
flag#2016-10-3114:48tcouplandyeah i just started bashing it out š#2016-10-3114:48drewr{:clojure.spec/problems
[{:path [:args :idx],
:pred :clojure.spec/unknown,
#2016-10-3114:49drewrcan anyone point me in the direction of tracking down why that's unknown?#2016-10-3114:53drewrI get a caller, but I go to that line, and it's downstream from the function that :sym
references š¤#2016-10-3114:53drewrso I must be misunderstanding something fundamental#2016-10-3115:23Alex Miller (Clojure team)the fact that itās printing unknown there is a known issue with describing preds#2016-10-3115:23Alex Miller (Clojure team)so that part is a known bug#2016-10-3115:25Alex Miller (Clojure team)but you have the path so it should be pretty narrowly scoped to where the pred is failing?#2016-10-3115:25drewrok, that helps, so go through and perhaps exercise
my specs to see if anything's funny?#2016-10-3115:25drewrI'm probably doing this the worst way, by trying to migrate from schema instead of using on a fresh codebase#2016-10-3115:26Alex Miller (Clojure team)as with most programming things, itās best to start small and build up#2016-10-3117:22drewras a general comment, the jump to mandated generative testing is a bit of a leap#2016-10-3117:22drewrmaybe I'm doing something wrong, but I don't always need this#2016-10-3117:23drewrwith a lot of side-effects, traditional unit-testing can work just fine, and I just want to make sure functions deep in the call stack are getting the right arguments and returning something valid#2016-10-3117:24drewrmy impression after a couple weeks is that spec makes this some of this more difficult over, say, schema#2016-10-3117:24drewrand maybe that's the intent: lower-level, more correct, more powerful, etc., it just also means there aren't yet companion libraries to help with the porcelain#2016-10-3117:25drewrmaybe I'm just spoiled by clojure's quality... alpha is usually good enough š#2016-10-3117:30donaldballYou donāt need to buy into generative testing to use clojure.spec. instrument
with traditional example-based tests is pretty useful.#2016-10-3117:31drewr@donaldball that was my early reaction as well, but I lost the ability to check return values, which I get, that's for check
, but then that forces me into the generative style#2016-10-3117:32drewrhappen to have a piece of code you transitioned that you're happy with?#2016-10-3117:33donaldballAlas, nothing public, sorry#2016-10-3117:34donaldballIIRC clojure.java.jdbc uses this to good effect tho#2016-10-3117:36jrheardi also kinda wish instrument
checked return values#2016-10-3119:20drewr@jrheard this thread was helpful in describing the philosophy https://groups.google.com/d/msg/clojure/JU6EmjtbRiQ/WSrueFvsBQAJ#2016-10-3119:21jrheardthanks for the link, havenāt seen this#2016-10-3119:21drewrand I think I agree, I just have the same desire as this guy https://groups.google.com/forum/#!msg/clojure/RLQBFJ0vGG4/E0tHqVyQBgAJ#2016-10-3119:54mattlyis there any way to view those google groups links without having a google account?#2016-10-3119:55shaun-mahood@mattly: The link works for me with no google account signed in#2016-10-3119:55drewr@mattly I brought one up in incognito#2016-10-3119:55mattlybah, ok, so it wants me to sign in with one of the google accounts I'm signed in as#2016-10-3119:56mattlywhich doesn't work because I signed into my google accounts in the wrong order#2016-10-3119:56mattly...sorry, wrong place for this rant#2016-10-3119:56drewrI use chrome profiles, separate windows, yada yada#2016-10-3121:02zane(s/def ::select
(s/coll-of (s/or :attribute string?
:subselect (s/every-kv string? ::select :count 1))))
(clojure.pprint/pprint
(s/conform ::select
["1"
{"2" ["2.1"
{"2.2" ["2.2.1"]}]}]))
;; => [[:attribute "1"] [:subselect {"2" ["2.1" {"2.2" ["2.2.1"]}]}]]
Why does that inner ::select
clause not tagged the way the outer one is?#2016-10-3121:29seancorfieldYou need conform-keys#2016-10-3121:31seancorfieldHmm, :conform-keys
is only listed as an option for map-of
but I would expect it to work here too...#2016-10-3121:33zaneWhy would I need conform keys? "2.1"
is being conformed by the s/or
.#2016-10-3121:34zaneIt's like conforming stops at the recursive step.#2016-10-3121:34seancorfieldNope, only works with map-of
ā but if you change :subselect
to be (s/map-of string? ::select :count 1 :conform-keys true)
then you get this [[:attribute "1"]
[:subselect
{"2"
[[:attribute "2.1"] [:subselect {"2.2" [[:attribute ā2.2.1ā]]}]]}]]
#2016-10-3121:35zaneI still don't understand why :conform-keys
has anything to do with this.#2016-10-3121:35zaneIt's the values that need conforming.#2016-10-3121:35seancorfieldYeah, I think I misunderstood what you were saying didnāt work..#2016-10-3121:36seancorfieldIs the output I got with map-of
what you want?#2016-10-3121:36zaneI would just expect that in my original example "2.1"
would be [:attribute "2.1"]
.#2016-10-3121:37zaneWhy is the recursive s/or
not being conformed?#2016-10-3121:37seancorfield(Youāre right, you donāt need :conform-keys
but you do need map-of
, not every-kv
)#2016-10-3121:37zaneIt seems like it happens, but only sometimes.#2016-10-3121:37seancorfieldPer the docs, every-kv
does not conform all values ā map-of
does.#2016-10-3121:37seancorfield"Unlike 'every-kv', map-of will exhaustively conform every value."#2016-10-3121:38zaneAhhh.#2016-10-3121:38zaneI missed that in the docs.#2016-10-3121:38seancorfield"Note that 'every' does not do exhaustive checking, rather it samples
coll-check-limit elements. Nor (as a result) does it do any
conforming of elements.ā ā and every-kv
is like every
#2016-10-3121:39zaneGot it.#2016-10-3121:44zanePhew. Thanks, Sean.#2016-10-3121:58seancorfield(I must admit, I find it counter-intuitive that every / every-kv do not in fact check every element but coll-of / map-of do)#2016-10-3121:58seancorfieldAfter all, they have every in their name so youād sort of expect them to check every item...#2016-10-3121:59jrheardheh yeah#2016-10-3122:19bhaganyit is the least-intuitive thing. I wish I understood the rationale.#2016-10-3122:27zaneIt's heartening that I'm not the only one. š
#2016-10-3122:59Alex Miller (Clojure team)"every" is an assertion of belief#2016-10-3122:59Alex Miller (Clojure team)Not a statement of what they do#2016-10-3123:24seancorfield^ rationale @jrheard @bhagany @zane#2016-11-0103:33settingheadI'd like to conform this vector [["John" 20] ["Jane" 30] ["Bob" 40]]
into [{:name "John" :age 20} {:name "Jane" :age 30} {: name "Bob" :age 40}]
. how should I write my spec?#2016-11-0106:23curlyfry@settinghead
(def john-jane-bob [["John" 20] ["Jane" 30] ["Bob" 40]])
(s/def ::person (s/cat :name string? :age integer?))
(s/def ::persons (s/coll-of ::person :kind vector?))
(s/conform ::persons john-jane-bob) => [{:name "John", :age 20} {:name "Jane", :age 30} {:name "Bob", :age 40}]
#2016-11-0115:11dominicmHow does clojure-spec work with the reloaded workflow / tools.namespace?
I notice both that:
1. https://clojurians-log.clojureverse.org/clojure-spec/2016-09-09.html#inst-2016-09-09T18:34:11.000362Z
2. specs aren't/can't be, deleted or unloaded in any way.
Any success with getting it to work in some way?#2016-11-0115:40zaneMy experience has been that needing to unload a spec is a rare occurrence.#2016-11-0116:46hiredmanis there something like s/keys, but instead of looking up specs via global names, I can pass it a spec for the value of each key?#2016-11-0116:51hiredmanI have these tests that result in a data structure that is sort of a log of activity, and the result of the test is determined by checking that log#2016-11-0116:53hiredmanthe log is a sequence of maps, and it has a :messages
key, and at each entry in the log, I know that key should have a specific value#2016-11-0116:54hiredmanif I have to define the value globally (use s/def and s/keys) then all I can say is that it could be any of a number of different values#2016-11-0116:55hiredmanah, maybe s/map-of and s/merge#2016-11-0117:03hiredmannope#2016-11-0117:06bfabry@hiredman what do you mean "at each entry in the log", like you dynamically know what it should be or you have a list of things it should be in certain circumstances? because if the latter you could always use :req-un#2016-11-0117:07hiredman:req-un
still looks up specs via a global name in the spec registry#2016-11-0117:07hiredmanI have more refined information locally than globally#2016-11-0117:08hiredmanglobally all I can say is the value for the key should be either x y or z, but locally I know exactly what the value for the key should be#2016-11-0117:11settinghead@curlyfry thanks#2016-11-0117:11bfabryright, but I meant you can define multiple versions of :messages using :req-un#2016-11-0117:11hiredmanyeah#2016-11-0117:12bfabry(s/def ::messages-1 (s/keys :req-un [:messages-1/message])) and then switch in the spec you need later on#2016-11-0117:14hiredmanI was hoping to use spec as a sort of a data regex to validate these test results, but having to globally define map validators is a pain#2016-11-0117:15hiredmanlike if you had to globally define your capture groups before you could use them in a regex engine#2016-11-0117:18bfabryI see your point#2016-11-0117:21hiredmanand if I had followed "best practices" and namespaced the :messages
key, that work around wouldn't work at all#2016-11-0117:25hiredmanalso clojure.spec has a declare of map-spec, which is never defined https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L348 maybe a left over from a previous version#2016-11-0117:28hiredmanI guess I can build something using s/alt#2016-11-0118:26madstap(s/fdef update*
:args (s/cat :m map?
:k keyword?
;; I want to say:
;; The function needs to take at least one arg,
;; the current value in the map,
;; and can take zero or more extra arguments.
:f (s/fspec :args (s/cat :value-in-map any?
:args (s/* any?)))
:args (s/* any?)))
(defn update* [m k f & args]
(apply update m k f args))
(test/instrument)
(update* {:a 10} :a inc)
#2016-11-0118:26madstapHow do I do this in spec?#2016-11-0118:28madstap;;=> Call to #'foo.core/update* did not conform to spec: In: [2]
val: (nil) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args ({:a 10} :a #function[clojure.core/inc])
:clojure.spec/failure :instrument :clojure.spec.test/caller
{:file "form-init8332152340913865653.clj", :line 342, :var-scope
foo.core/eval12509}
#2016-11-0118:31zane@madstap You're probably better off just using a function.#2016-11-0118:31madstapLike just :f
ifn?
?#2016-11-0118:36hiredmanmaybe something with http://clojure.org/guides/spec#_multi_spec#2016-11-0118:46madstapFrom that example I'm having truble seeing how I could apply multi-spec
to my problem#2016-11-0118:47madstap@hiredman could you explain?#2016-11-0118:50hiredmanmaybe not, multi-spec would only be useful if you knew the set of keys you would be pass to update*#2016-11-0118:52hiredmanI guess that is not entirely true, if you knew the set of keys entirely before hand, you could use an s/alt or s/or, the advantage using multi-spec would be that choice would be open ended#2016-11-0118:52hiredmanso easier to extend#2016-11-0118:54hiredman(defmulti update-multi-spec (fn [m k fun & args] k))
(s/fdef update*
:args (s/multi-spec update-multi-spec)
:ret any?)
(defmethod update-multi-spec ::a [m k fun & args]
(s/cat
:map (s/keys :req [::a])
:key #{k}
:fun (s/fspec :args ...)
:args ...
))
#2016-11-0119:09madstapI think I explained myself badly, what I want to do is say that the function passed to update needs to take at least one argument, and may take more.#2016-11-0119:17madstapYour multi-spec example helped me grok why multi-spec is useful, though, thanks#2016-11-0119:36hiredman(s/+ any?)#2016-11-0119:41madstap@hiredman Doesn't work..
(s/fdef update*
:args (s/cat :m map?
:k keyword?
:f (s/fspec :args (s/+ any?))
:args (s/* any?)))
(defn update* [m k f & args]
(apply update m k f args))
(test/instrument)
(update* {:a 10} :a inc)
#2016-11-0119:42madstapCall to #'foo.core/update* did not conform to spec: In: [2]
val: (nil) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args ({:a 10} :a #function[clojure.core/inc])
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file
"form-init8954135211496312533.clj", :line 478, :var-scope
foo.core/eval13250}
#2016-11-0119:43hiredmanyeah, because any? is generating bad inputs for the function#2016-11-0119:43hiredmanthe way test check validates the function is it generates inputs for it#2016-11-0119:44hiredmanyour function is inc
, in the spec you say it can take an any?, so spec is checking that by feeding it all kinds of values#2016-11-0119:51madstapRight, what I wan't to say that this function needs to take at least one arg, but doesn't necessarily take any more args.#2016-11-0119:52madstapIn this case that passing (fn [] (rand-int 10))
to update will fail, but inc
will work, as will (constantly true)
.#2016-11-0119:57hiredmanthe problem is when you turn instrumentation on, spec is validating inc
against your fspec#2016-11-0120:00hiredmana spec that is general enough isn't going to be valid against a specific function, you will generate very general inputs that the specific function won't handle#2016-11-0120:01hiredmanI think the best you could do would be a multi-spec, with a defmethod for every function/key combination you pass to update*#2016-11-0120:26madstapGot it, thanks#2016-11-0209:55vandr0iyHi, #clojure-spec! came to ask: what's the difference between clojure.spec/map-of and clojure.spec/every-kv?#2016-11-0212:46bhagany@vandr0iy map-of
checks all the entries of the map, while every-kv
samples a subset of the entries#2016-11-0212:47bhaganyevery-kv
is designed to be used with maps that may be large#2016-11-0212:47vandr0iynice explanation, thanks @bhagany !#2016-11-0213:06lopalghostAny thoughts on writing separate specs for input data that needs to be coerced, vs using spec/conformer?#2016-11-0213:06mpenetcoerce separately from your main specs yes, with or without spec#2016-11-0213:07mpenetit gets very messy otherwise imho#2016-11-0213:07lopalghostThat's what I'm finding to be the case#2016-11-0213:09lopalghostThe only problem is I'm duplicating my validation logic, because I want to be able to detect invalid data at the point of entry#2016-11-0213:09lopalghostI'm using spec in both cases because the generative testing is super-helpful#2016-11-0213:12mpenetyou can create predicates separately that you re-use on both specs#2016-11-0213:12mpenetbut yes, there will be duplication here and there#2016-11-0213:36drewrwhat are you all doing to cleanup after check
on a fn with side effects?#2016-11-0213:36drewrit doesn't look immediately possible with :fn
unless I pass context back through the return value#2016-11-0213:37drewrI suppose I could with-redefs
in there, but that doesn't feel right#2016-11-0214:01Alex Miller (Clojure team)@vandr0iy additionally, every-kv will not conform its keys or values (because it doesnāt check all of them) whereas map-of will conform all values (and optionally all keys)#2016-11-0214:02Alex Miller (Clojure team)@drewr donāt write fns with side effects? :)#2016-11-0214:02Alex Miller (Clojure team)seriously though, you might look at the stub and replace options on stest/instrument
#2016-11-0214:03drewryeah, would love to avoid them, but have to do things other than burn cpu š„#2016-11-0214:03drewrcool, didn't notice those opts on instrument
#2016-11-0214:03Alex Miller (Clojure team)particularly for the case where you are checking a function that calls a side-effecting function#2016-11-0214:03Alex Miller (Clojure team)you can stub or replace it instead#2016-11-0214:06drewrI didn't think to look at instrument
, though, because this goes farther than just args checking#2016-11-0214:07drewrseems like something that would happen with test.check... ie, a function that somewhere ends up writing a value to a db, and I need to make sure that value is deleted (or the db is removed)#2016-11-0214:07Alex Miller (Clojure team)you should instrument, then check#2016-11-0214:07Alex Miller (Clojure team)then unstrument#2016-11-0214:08drewrright now I'm doing the whole enumerate-namespace
-> check
-> summarize-results
dance, sounds like I need to break out of that pattern#2016-11-0214:09Alex Miller (Clojure team)yes#2016-11-0214:27gfredericks@lopalghost I played around with those issues in https://github.com/gfredericks/schema-bijections; I'd love to see that sort of thing implemented for spec#2016-11-0215:01jrheardwhatās the alternative to the pattern @drewr gives above?#2016-11-0215:03jrheardi havenāt used the stub-and-replace options alex mentions, so maybe you guys just mean that using them necessitates breaking out of the pattern he gives#2016-11-0215:03jrheardjust curious š#2016-11-0215:04drewr@jrheard I'll give you an example when I have one š#2016-11-0215:15lopalghost@gfredericks I think spec/conformer is a pretty good solution to the problem of coercing back and forth, at least #2016-11-0215:16lopalghostThe problem is you lose information about the input, making explain and check less useful#2016-11-0215:17lopalghostI've tried writing some DSLs to generate specs, but I was never really happy with them#2016-11-0215:19gfredericksYeah, I remember an earlier conversation that ended up recommending mapping to new specs rather than just conforming#2016-11-0215:20gfredericksBut spec isn't good at, e.g., describing maps with string keys, though :/#2016-11-0215:25mpenetanything other than keyword keys (sadly)#2016-11-0215:27mpenetyou end up having to resort to weird, half working, set + map-of hacks#2016-11-0215:27zaneIt feels like passing around conformed values is discouraged. Is that correct?#2016-11-0215:29mpenetI often wonder why namespaced keys couldn't just be a collection of keywords, since that's just values#2016-11-0215:30mpenetI guess there's a good reason behind this (interning magic leading to better perf ?)#2016-11-0215:31lopalghostI don't like set + map-of because it makes explanations less clear#2016-11-0215:31mpenet[:some-ns :some-key]#2016-11-0215:31lopalghostIt might be better to coerce the keys in a separate step#2016-11-0215:31mpenet@lopalghost more importantly it breaks link between key and value#2016-11-0215:31lopalghostRight#2016-11-0215:32mpenetit just guarantees the key is here#2016-11-0215:37mpenetso yeah either coercion, or it might be better to just have a custom predicate to validate such maps#2016-11-0216:30Alex Miller (Clojure team)you can use every
on maps to describe the map entries as k-v tuples#2016-11-0216:31mpenetneat, I didn't think of that one#2016-11-0216:32Alex Miller (Clojure team)like (s/every (s/or :name-entry (s/tuple #{ānameā} string?) :id-entry (s/tuple #{āidā} int?)) :kind map? :into {})
#2016-11-0216:32Alex Miller (Clojure team)but whatever orāed entry types you need#2016-11-0216:33Alex Miller (Clojure team)for non-kw key maps, this can be a good approach#2016-11-0216:33Alex Miller (Clojure team)(thatās how every-kv
and ultimately map-of
are actually implemented)#2016-11-0219:45nwjsmithWhere can I vote for adding docstrings to specs?#2016-11-0219:48nwjsmithI think I saw it being discussed here a few weeks ago, and I though āwho would ever want to do that?ā Turns out I am someone who would want to do that.#2016-11-0219:50jrhttp://dev.clojure.org/jira/browse/CLJ-1965#2016-11-0220:14potetmIs it appropriate to give more rationale for that ^ ticket in a comment on JIRA?#2016-11-0220:15potetmI've thought about it a little bit, and I think there's a really good justification for adding it.#2016-11-0222:21jasonjcknis there a general recommendation for versioned schema?#2016-11-0222:21jasonjcknmulti spec dispatching on version tag?#2016-11-0222:21jasonjcknwhat if i'm evolving my spec over time, deprecating fields, introducing new ones, changing existing ones.#2016-11-0305:29Alex Miller (Clojure team)@potetm feel free, although I donāt know that there is much need :)#2016-11-0305:30Alex Miller (Clojure team)@jasonjckn backwards compatible changes do not necessarily require a new spec (for example, s/keys will always validate all registered keys, so adding new ones does not require a spec change)#2016-11-0305:31Alex Miller (Clojure team)@jasonjckn breaking changes should mean a new spec#2016-11-0305:32Alex Miller (Clojure team)I expect this area will receive more attention as its something Rich has thought about a lot#2016-11-0305:34Alex Miller (Clojure team)and heās really serious about not just āchangingā specs but actually leaving the existing one and creating a new one (::person2, ::person3, etc)#2016-11-0305:34jrheardthe evils of āusing the same name to refer to different things"#2016-11-0305:34Alex Miller (Clojure team)hello immutable data#2016-11-0305:34jrheardš#2016-11-0305:56ikitommihas anyone managed to do the ātake a spec, walk it and transform it to another (differently conforming) specā? I tried, but for s/keys
it seems hard/impossible. Have already copied much of the clojure.spec internals into my project for this. With un
-keys, itās easy, with the non-un keys, itās not: the key names should remain same, with a different implementation.#2016-11-0305:59ikitommiwould be nice, if there was a map-kind of Spec in clojure.spec that would allow separate keys and values - I could just swap the values in this case. Could there be such a thing under the hoods of s/keys
@alexmiller?#2016-11-0318:30gfredericks@ikitommi having a consistent meaning for a namespaced keyword is one of the core ideas of clojure.spec#2016-11-0318:56zaneikitommi's question relates to my earlier one about whether conformed values are more or less intended to remain internal to the function in which they were conformed.#2016-11-0318:56zaneWhen used idiomatically, anyway.#2016-11-0319:05ikitommi@gfredericks true that, but for external input, there is a need to conform/coerce values differently, based on capabilities of the wire format. Stuarts thought was to transform raw specs into new specs, with different conformers.#2016-11-0319:06zaneHow would that preclude you from using different keys for the new specs, ikitommi?#2016-11-0319:07ikitommiwhat if I want to expose a fully-qualified key?#2016-11-0319:09ikitommifor unqualified keys, I can just create a new spec key, e.g. :mydomain/age
=> :JSON.mydomain/age
, as both can be mapped to the use the :age
key.#2016-11-0319:41kuzmin_mHi!
I play with spec and found one dangerous point.
I use s/fspec to specify high-order function foo
with bar
argument.
In several cases I may call foo
with bar
that have some side effects.
When instrument is enabled then bar
will be called more than onŃŠµ
because spec will check bar
.
For instance I may write smth like that: (foo delete-user-by-id)
and debug it many hours.
I can check only bar
arity but stest/check will not work.
(ns spec.func.example
(:require [clojure.spec :as s]
[clojure.spec.test :as stest]))
(defn foo [bar]
(bar 42))
(s/def ::bar (s/fspec :args (s/cat :x integer?)
:ret integer?))
(s/fdef foo
:args (s/cat :bar ::bar)
:ret integer?)
(stest/instrument)
(foo #(do
(println %) ;; print more than once !!!
(inc %)))
#2016-11-0319:53drewr@kuzmin_m I've been dealing with exactly this#2016-11-0319:54drewryou might not want to use check
at all, but instrument
with its :replace
option, and then call the function normally#2016-11-0319:58kuzmin_mYes. But people are careless. If something is breakable itāll be broken.#2016-11-0320:00kuzmin_mI may forget specify replace option#2016-11-0320:00drewrnot sure what you mean#2016-11-0320:02kuzmin_mI need an extraordinary action. I need remember about s/fspec behaviour and specify instrument
option.#2016-11-0320:13kuzmin_mMay we write a function that replace s/fspec
by fn?
when we use instrument
?
We donāt call fn and we only check an argument type in this case.#2016-11-0320:14bfabry@kuzmin_m that would make instrument
significantly less useful#2016-11-0320:16bfabryfwiw, I think there should probably be more explicit handling of functions with side-effects for both fspec and fdef, and I'm hopeful that something is coming#2016-11-0320:18kuzmin_mSure! But it only about fspec, not about fdef.#2016-11-0320:20bfabrythey're kinda the same thing though, fdef as far as I understand it is just (s/def name (s/fspec blah))#2016-11-0320:23kuzmin_mYes. I just write about one case: s/fspec for function argument.#2016-11-0320:24bfabryright, I just mentioned fdef because you can use the name of something fdef'd somewhere in place of an fspec and you'll get the same behaviour. but anyway you're right that it would be on fspec where things need to be handled#2016-11-0320:25kuzmin_mright :+1:
I forget about that behavior š#2016-11-0320:27kuzmin_mItās just alpha version.
I hope that we can fix this issue.#2016-11-0321:41mrcncis there a ābest practicesā way to run spec tests from lein?#2016-11-0321:41mrcnci found this https://gist.github.com/tgk/be0325e7b78bc692ad6c85ef6aca818d#2016-11-0323:09jrheardhave you seen summarize-results
? https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L448#2016-11-0323:11jrheardi havenāt seen official guidance on the one true specifically correct invocation of clojure.spec.test/check, but when i dig around in github search i usually see people write clojure.test assertions like (is (= 0 ((summarize-results foo) :failures)))
or whatever#2016-11-0323:11jrheardhandwaving here, but you get the idea#2016-11-0323:53kennyIt works well with Cursiveās run tests REPL action#2016-11-0323:54kennyExample usage:
(t/defspec-test st-test-name `you-function-to-test)
#2016-11-0411:54mpenetjust wondering: why in multispec we have to repeat the dispatch fn/key in s/multi-spec call itself (since it's already in the defmulti definition)#2016-11-0411:54mpenetas seen here: http://clojure.org/guides/spec#_multi_spec#2016-11-0411:59mpenet-> retagging#2016-11-0411:59mpenetok#2016-11-0412:44Alex Miller (Clojure team)You can use multispec on cases other than keyword maps too - in that case you need a fn for retagging instead.#2016-11-0413:50mpenetyup, makes sense#2016-11-0413:51mpenetit's not obvious from the guide tho#2016-11-0413:51mpenetbut the docstring is clear about it#2016-11-0417:59sveriLets say I have two specs A and B and now I have a function that has a paramater that is the union of the two maps so like (keys ::A ::B). How would I define that spec?#2016-11-0417:59sveriI tried a (merge ::A ::B) but that does not work#2016-11-0418:01bfabry@sveri did you use core/merge or spec/merge?#2016-11-0418:02sveri@bfabry Ah, I did not know there was a spec/merge, thank you, I used the core one. With spec/merge it works š#2016-11-0423:12d._.bI'm feeling dumb. I have a map like
{:abc "abc" ; required key, and must have a value of "abc"
:x {:y nil ; required key, but value can be anything
:z 123 ; same as above}
}
#2016-11-0423:12d._.bHow do I spec this?#2016-11-0423:16bfabry(s/def :your/abc #{"abc"})
(s/def :your/x (s/keys :req-un [:your/y :your/z]))
(s/def :your/y any?)
(s/def :your/z any?)
(s/keys :req-un [:your/abc :your/x])#2016-11-0423:16d._.b(s/def ::abc "abc")
(s/def ::x (s/map-of :req-un [keyword? identity])
(s/def ::my-map (s/keys :req-un [::abc ::x]))
#2016-11-0423:17d._.b@bfabry thank you, i knew i was missing something#2016-11-0423:17d._.bis it possible to also use every-kv
or map-of
?#2016-11-0423:18bfabryanything is possible#2016-11-0423:18d._.bhaha, im just trying to understand when to reach for map-of or every-kv#2016-11-0423:18d._.band when not to#2016-11-0423:18bfabryreach for s/keys if you know what your keys will be#2016-11-0423:18d._.bin my example, i know that all of the k/v pairs will be :key any?#2016-11-0423:18d._.bunder :x#2016-11-0423:19d._.b@bfabry why do you not s/def the last line in your example?#2016-11-0423:20bfabryshrug I didn't need to refer to it by a name#2016-11-0423:20bfabryif I did I would#2016-11-0423:20bfabrygotta go#2016-11-0423:40d._.bIs there a way to def multiple things at once?#2016-11-0423:41Alex Miller (Clojure team)No#2016-11-0423:41d._.bUse for
or doseq
to generate the defs? What's idiomatic?#2016-11-0423:41Alex Miller (Clojure team)You can write a macro#2016-11-0423:42d._.bSomething like (doseq [k [::abc :def]]
(s/def ~k any?))` ?#2016-11-0423:44d._.bI think I must just be missing something obvious#2016-11-0423:45d._.bIf I have like 20 keys which are req-un in a map, all of which should have values checked of any?
, that seems like it should be more straightforward than resorting to doseq'ing to generate s/def
s#2016-11-0423:48d._.bSorry for the repost but, just to reiterate:
{
:abc "abc" ; required key, and must have a value of "abc"
:x {:y nil ; required key, but value can be anything
:z 123 ; same as above
:w :wow ; same as above
:o :out ; same as above
}
}
#2016-11-0423:51d._.b(s/def ::abc #{"abc"})
(s/def ::x (s/keys :req-un [::y ::z ::w ::o]))
(s/def ::y any?)
(s/def ::z any?)
(s/def ::w any?)
(s/def ::o any?)
(s/def ::entire-thing (s/keys :req-un [::abc ::x]))
#2016-11-0423:51d._.bDo I really need to write all 4 of those defs?#2016-11-0423:53d._.bI was thinking I could (s/every-kv #{::y ::z ::w ::o} any?)
or something... No?#2016-11-0500:50bbloomfinally getting around to trying spec in anger#2016-11-0500:50bbloomis there a standard conform-or-throw function?#2016-11-0501:27hiredmanthere is a/assert, but you need to set a property to turn it on#2016-11-0501:35richiardiandrea@bbloom I so wanted that too, unconditionally, I could not find it and copied over s/assert
and named it valid-or-throw
š#2016-11-0501:38kenny@richiardiandrea Done š https://gist.github.com/kennyjwilli/8bf30478b8a2762d2d09baabc17e2f10#2016-11-0501:39richiardiandreagreat I'll tweet you š#2016-11-0512:44dominicmIs there yet a way to "scope" an error done against a whole map?
Given that I have:
{:x 1
:y [2 3 4 5]}
And want to validate that :x
must be contained in :y
, but I want my error message to state that it is :x
that is wrong, not the whole map.
How would I do it?#2016-11-0513:02Alex Miller (Clojure team)s/and a function that checks that x is in y?#2016-11-0513:03Alex Miller (Clojure team)The error should list that function#2016-11-0513:03Alex Miller (Clojure team)As the failing predicate#2016-11-0514:17dominicm@alexmiller I'm thinking more of the :path
of the error (I think that's the key)#2016-11-0515:56tianshuhow can i turn on the :ret
check for clojure.spec/fdef
.#2016-11-0519:47mpenetI had deal with that pb recently, If you check on test.check wiki there's a page explaining how to deal with recursive gen and how to limit gen depth/size#2016-11-0519:51mpenet(I am on my phone, not easy to find)#2016-11-0522:44bbloomis there a spec somewhere for the ns form or ālibspecā data?#2016-11-0523:43gfredericks@bbloom I think you can poke around to find how the spec for ns is implemented#2016-11-0523:43gfredericks(doc ns) might even help with that#2016-11-0523:43bbloomduuuuh i forget that core had all the specs in there now#2016-11-0523:43bbloomthanks#2016-11-0523:43gfredericks(clojure.spec/form :clojure.core.specs/ns-clauses)
#2016-11-0523:44gfredericksand clojure.spec/form
#2016-11-0523:44bbloomanyway - i decided to do my own spec for a tiny subset of the complexity of the full ns form#2016-11-0523:44bbloompretty narrow use case#2016-11-0600:50hiredmanso, anyone write a spec -> gloss codec compiler yet?#2016-11-0601:42keithsparkjoyWorks fine if I define it all on one line, but include the newline and the repl chokes.#2016-11-0601:42keithsparkjoyCurious if anyone has experienced this or if itās just me?#2016-11-0601:42keithsparkjoyTrying to use the new map literal syntax with clojure.spec, which is why Iām asking here...#2016-11-0601:43keithsparkjoyThe error I get is java.lang.RuntimeException: Unmatched delimiter: )#2016-11-0614:55madstap@keithsparkjoy Works for me on alpha14#2016-11-0616:11keithsparkjoyOkay must be something about my environment. Thanks for trying.#2016-11-0707:36nortonHas someone an example where spec instrumentation using the :replace option works? I tried a simple example to override the ranged-rand function with one that always returns the start value ā¦ but so far no luck.#2016-11-0718:11uwois there anyway to say that a map should contain one or the other key, besides this?
(s/def ::one-or-the-other
#(let [ks (set (keys %))]
(xor (ks :key1) (ks :key2))))
#2016-11-0718:13minimal@uwo you can use or
in s/keys
, see the docstring#2016-11-0718:13minimalThe :req key vector supports 'and' and 'or' for key groups:
(s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
#2016-11-0718:14uwo@minimal wicked, thanks!#2016-11-0718:15uwortfmftw!#2016-11-0816:09drewr@norton it's tricky... you have to make sure there's an fdef for the thing you're :replace
-ing#2016-11-0816:43norton@drewr Thank you for the feedback. I posted this question (with more details) to the Google clojure mailing list with an example that is using :replace. Alex Miller responded and corrected my example. It works as expected!#2016-11-0819:38mpenets/keys question, I need to express key ::foo is optional, but if it's there ::bar key should be too, since there's no or/and in opt it's a bit tricky I think.#2016-11-0819:41mpenetI guess i ll split that in multiple s/keys#2016-11-0819:41zaneI don't think you can do all ā yes.#2016-11-0819:41zaneThat's what I'd do. :relaxed:#2016-11-0819:41mpenetnot sure why and/or is not supported in s/keys :opt, it would be useful#2016-11-0819:43mpenet@alexmiller might know š#2016-11-0819:45Alex Miller (Clojure team)Why not req (or foo (and foo bar))#2016-11-0819:45zaneI'd probably replace the #(or ā¦
with another (s/keys ā¦
so you get enforcement of the values for :foo
and :bar
.#2016-11-0819:46mpenet@alexmiller because foo is required like this#2016-11-0819:46mpenetit's should be optional, but if present there should be :bar too#2016-11-2100:40bbloomso that i can refer to things w/o a cyclic dep#2016-11-2100:41Alex Miller (Clojure team)There is a patch I wrote to auto-create #2016-11-2100:41Alex Miller (Clojure team)So just alias would be enough#2016-11-2100:41bbloomah, nice#2016-11-2100:41bbloomthat would be welcome#2016-11-2100:41Alex Miller (Clojure team)Rich hasn't looked at it yet#2016-11-2100:41bbloomĀÆ\(ć)/ĀÆ#2016-11-2100:43Alex Miller (Clojure team)CLJ-2030#2016-11-2100:44bbloomvoted.#2016-11-2100:44bbloomthx#2016-11-2100:45Alex Miller (Clojure team)Well, it's already screened, just waiting for a slice of rich time#2016-11-2100:45bbloomnot complaining - you know iām in your camp on the processes š#2016-11-2101:43twashing@alexmiller defspec-test
worked like a charm. I answered my own SO question for future reference. https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite/40711634#40711634#2016-11-2101:44twashingand Iām hoping that such a useful function makes it into the core library.#2016-11-2101:48Alex Miller (Clojure team)Not planning on it right now#2016-11-2104:20bbloomiām using spec in anger to solve a bug in an otherwise un-specād namespace right now#2016-11-2104:20bbloomi havenāt found the bug i am looking for, but i did find three other lurking bugs#2016-11-2104:21bbloomitās kinda awesome.#2016-11-2104:34twashing@alexmiller Curious as to why not. defspec-test
, or something similar, seems to be low hanging fruit for adopting specād function tests, into a projects overall test suite.#2016-11-2105:56kenny@twashing the one issue with the macro is it only counts generative tests as one test in the test summary (e.g. You run 1000 generative tests but the summary results only count it as one). I've been meaning to add it but haven't had time. #2016-11-2105:59kennyIf something like it isn't included in Clojure core when 1.9 is released then we can move it into a micro library instead of a gist#2016-11-2106:43twashing@kenny Right, I was thinking just that. Kewl :+1:#2016-11-2114:33alqvist@kenny Would make may day if it found its way into 1.9#2016-11-2114:40Alex Miller (Clojure team)in general, spec tests have a different shape than example based tests. I think itās good to step back and think about why or if itās valuable to force everything into the shape of something that clojure.test can run. This feels to me like the limits of our tool (particularly reliance on lein test to run and report everything) are affecting our ability to think about the problem. Why does there need to be only one kind of test runner? Why does there need to be only one kind of test? Generative tests are (by their nature) longer running and often donāt mix well with example based tests in a single suite. There is more to say on this.#2016-11-2115:00bplatzIs there a way to use conform
without destructuring the values? (i.e. Iād like have an s/or
within the spec but not have change the original value).#2016-11-2115:42Alex Miller (Clojure team)no, but you can use a conformer of val
to undo conforming an or
#2016-11-2115:43Alex Miller (Clojure team)(s/and ::my-or-spec (s/conformer val))
#2016-11-2115:44Alex Miller (Clojure team)Rich toyed with s/nonconforming (still there but not in docs) but I suspect it will probably get removed before final release#2016-11-2115:54dergutemoritz@bplatz You can fudge it by wrapping your or
spec like this: (s/and (s/or ...) (s/conformer val))
#2016-11-2115:54dergutemoritzThough this is probably dangerously close to meat grinder territory again š#2016-11-2115:56dergutemoritzOops, sorry, I overlooked the first reply of yours @alexmiller#2016-11-2116:12bplatzThe use case here, which Iād think is a decent one, is validating and coercing JSON input - but downstream code expects data in a specific way.#2016-11-2116:13bplatzThanks for the s/and
tip, Iāll use that for the time being until either it gets officially supported or we just use a different mechanism to coerce the data.#2016-11-2116:51vikeriHow does one go about speccing a multimethod? Can the different defmethods have different :args
?#2016-11-2116:59Alex Miller (Clojure team)I donāt think s/fdef
works on multimethods right now#2016-11-2117:10vikeri@alexmiller Ok, but if I donāt use multimethods but dynamically dispatches inside the function, can I somehow define a relation between the function arguments i.e.:
> If the first argument is a number, then the second should conform to :c/myspec1
, but if the first argument is a keyword then the second argument should conform to :c/myspec2
?#2016-11-2117:10Alex Miller (Clojure team)there are a couple options#2016-11-2117:11Alex Miller (Clojure team)one is to write an fdef
with a :fn
spec which can describe a relationship between args and ret#2016-11-2117:11Alex Miller (Clojure team)another is to use s/multi-spec
which can yield a different spec based on a separate multimethod#2016-11-2117:11Alex Miller (Clojure team)the latter is really ideal for functions where the spec is open for extension after the fact#2016-11-2117:12vikeriI checked into multi-spec, but then the input has to be a map right?#2016-11-2117:12Alex Miller (Clojure team)I think there are examples of both in the guide http://clojure.org/guides/spec#2016-11-2117:12Alex Miller (Clojure team)no, it can be anything#2016-11-2117:12Alex Miller (Clojure team)as long as you give it a multimethod that chooses the proper spec based on the input#2016-11-2117:13vikeri@alexmiller Ah, just like multimethods the second argument is not a keyword but a function, but in clojure a keyword can be a functionā¦ Then thatās exactly what I need.)#2016-11-2117:13Alex Miller (Clojure team)yes#2016-11-2117:14Alex Miller (Clojure team)you will likely need a custom retag function too, but thatās not a big deal#2016-11-2117:14vikeriretag?#2016-11-2117:15Alex Miller (Clojure team)used during generation#2016-11-2117:15Alex Miller (Clojure team)check the docstring for the details#2016-11-2123:16paytonrulesI have a clojure-spec/clojurescript question if anybody can help. I suspect itās obvious.#2016-11-2123:17paytonrulesI have the following -
(s/def ::box (s/keys :req [::c/height ::c/width ::p/x ::p/y]))
(defn- right [box]
(+ (:width box) (:x box)))
(defn- bottom [box]
(+ (:height box) (:y box)))
(defn intersect? [box-one box-two]
(not
(or
(> (::p/x box-two) (right box-one))
(< (right box-two) (:x box-one))
(> (:y box-two) (bottom box-one))
(< (bottom box-two) (:y box-one)))))
(s/fdef intersect?
:args (s/cat :box-one ::box :box-two ::box)
:ret boolean?)
#2016-11-2123:17paytonrulesWhat I canāt do is get the intersect?
to actually assert when I call it incorrectly#2016-11-2123:18paytonrulesSo for instance:
cljs.user=> (require '[cljs.spec :as s])
nil
cljs.user=> (s/check-asserts?)
true
cljs.user=> (box/intersect? {} {::c/width 100})
true
#2016-11-2123:18paytonrulesIām sure the problem is between brain and keyboard, but Iām pretty stumped. Shouldnāt this go kerblooie?#2016-11-2123:20hiredmancheck-asserts? governs any s/asserts you have in the code, but you don't have any#2016-11-2123:22paytonrulesI thought fdef would instrument the function - thereby adding the asserts automagically#2016-11-2123:23hiredmannope#2016-11-2123:24hiredmanwhen testing you can run instrument, which turns on that sort of thing for testing, but you will also get bits of generative testing happening, so it is not something you generally want to turn on#2016-11-2123:26bfabry@paytonrules you want to run (s/instrument)#2016-11-2123:26paytonrulesI was going by this from fdef docs "Once registered, function specs are included in doc, checked by instrument,
#2016-11-2123:27paytonrulesBut apparently ClojureScript doesnāt have the instrument function#2016-11-2123:27bfabryreally? that surprises me#2016-11-2123:28hiredmaninstrument is in clojure.spec.test (not sure what translation to that you need to do for clojurescript)#2016-11-2123:28bfabryoh yeah, sorry, instrument is in a different ns#2016-11-2123:28hiredmanfrom what I understand, always on checking is a non-goal for spec#2016-11-2123:29paytonrulesAha - it was the clojure.spec.test that I missed bit that I missed.#2016-11-2123:32paytonrulesThanks a lot. It would appear then that specāing functions that you arenāt actively doing property based testing on, or debugging, may not be a useful endeavor.#2016-11-2123:33hiredmanhttp://clojure.org/about/spec#_using_specs#2016-11-2123:36bfabry@paytonrules I'm not sure that's true, I think there's still value in specing things and having instrumentation turned on for development/test#2016-11-2123:39paytonrulesI wonder how much slower things will be turning on instrument in development will be#2016-11-2123:40bfabryunless you're getting into speccing function arguments I don't expect instrument will be super slow in a development context#2016-11-2123:42paytonrulesMaybe to turn it on and turn it off. Iām making an HTML5 game so if I spec everything itāll hurt.#2016-11-2123:52hiredmanif you turn on instrument, invoking functions that are spec'ed will also cause those functions to be genertively tested against those specs#2016-11-2123:56bfabry@hiredman I'm not sure that's right? I think it only generatively tests functions that you pass as arguments, to make sure the argument satisfies the fdef#2016-11-2123:57bfabryuser=> (defn foo []
#_=> (println "foo!"))
#'user/foo
user=> (s/fdef foo :args (s/cat) :ret nil?)
user/foo
user=> (clojure.spec.test/instrument)
[user/foo]
user=> (foo)
foo!
nil
user=>
#2016-11-2123:57hiredmanthe exact behavior is whatever, my point is, don't turn it on in production#2016-11-2123:58hiredmanyou don't want to be running generative tests in production#2016-11-2201:52paytonrules^Thanks for the help. I donāt think I said so properly before as my internet time was up.#2016-11-2208:32yonatanelIs this anything to be worried about?
(s/conform (s/or) :anything)
=> :clojure.spec/invalid
(s/explain (s/or) :anything)
Success!
=> nil
#2016-11-2210:21yonatanelHere's my experiment in returning only the conformed value of an s/or
spec:
(defmacro meat-grinder-or [& preds]
(let [tags (map keyword
(repeatedly (count preds)
#(gensym "tag")))
prepared (interleave tags preds)]
`(s/and (s/or
#2016-11-2212:27Alex Miller (Clojure team)On the first question, that doesn't seem right, agreed. Feel free to log it if you like#2016-11-2212:37yonatanel@alexmiller What do you mean by log it?#2016-11-2212:38Alex Miller (Clojure team)In jira#2016-11-2212:38Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ#2016-11-2212:46yonatanelno https?#2016-11-2213:02Alex Miller (Clojure team)No, sorry (it's on the list)#2016-11-2217:11eggsyntaxIs there an easy way to get the keys of a map spec pulled from the registry? I feel like there must be an obvious way I'm overlooking. I can just generate one & get the keys from that, but that feels inelegant (& might be incomplete if it had optional keys).#2016-11-2217:26Alex Miller (Clojure team)you can use s/form to get the form version of it#2016-11-2217:27Alex Miller (Clojure team)when we release form specs, you could then conform with that to extract the keys#2016-11-2217:27Alex Miller (Clojure team)but you can work around that for the time being#2016-11-2217:27eggsyntaxExcellent. Thanks @alexmiller!#2016-11-2217:30eggsyntaxThat's actually going to be broadly useful for me; I had overlooked s/form until now.#2016-11-2217:49Alex Miller (Clojure team)there are a number of remaining issues with it and I have patches for most of those in the queue#2016-11-2217:50Alex Miller (Clojure team)in particular all of the coll specs and keys* have broken forms atm#2016-11-2217:50Alex Miller (Clojure team)and maybe conformer (canāt remember if thatās been applied yet)#2016-11-2217:50eggsyntaxOK, gotcha, I'll be cautious. Still, it's a great feature!#2016-11-2217:51Alex Miller (Clojure team)those will all be fixed#2016-11-2219:05zaneIs there something I can read to better understand query caching and how to optimize queries?#2016-11-2219:54seancorfield@zane Could you elaborate? That doesnāt sound related to clojure.spec...#2016-11-2220:46zaneThat was indeed for the wrong channel.#2016-11-2220:46zaneMeant for #datomic.#2016-11-2220:47zaneThanks for taking my question seriously anyway, @seancorfield. š#2016-11-2221:03seancorfield@zane Ah, I wondered if it was for JDBC in which case Iād be happy to field itā¦ in #clojure š#2016-11-2222:06rickmoynihanDoes anyone have any tips on using clojure.spec to validate some data conforms to the spec in a clojure.test? Obviously I can do (is (s/valid? ::spec data)
but if its invalid I want to see the explain outputā¦#2016-11-2222:16Alex Miller (Clojure team)we were waiting for you to do it#2016-11-2222:34Oliver GeorgeJust a quick bit of feedback. This is a circular dependency thing. I generated some specs based on my relational database schema
tables became (s/keys ...)
fields became simple (s/def :schem.table/field pred)
one-to-many relations became (s/def :schem.table/rel (s/coll-of :schem/table))
many-to-one relations became (s/def :schem.table/rel :schem/table)#2016-11-2222:36Oliver GeorgeThe slightly fiddly bit was the relations#2016-11-2222:36Oliver GeorgeI couldn't dump these all in one file because the table specs needed to exist before the relation specs could be processed. (more specifically order mattered or it would throw an error)#2016-11-2222:37Oliver GeorgeI was hoping all (s/def ...) statements were essentially lazy but perhaps that's not the case for performance reasons.#2016-11-2222:37Oliver GeorgeI might need to produce a simple example to demonstrate this.#2016-11-2222:41rickmoynihanlol#2016-11-2222:42Oliver George(s/def ::a ::b)
throws
CompilerException java.lang.Exception: Unable to resolve spec: :example.order-matters/b, compiling:(order_matters.clj:11:1)
#2016-11-2222:45Oliver GeorgeMy work around is to do all table defs first since they reflect relations. s/keys
doesn't require the req/opt specs exist before the s/def#2016-11-2222:46rickmoynihan@alexmiller: presumably the right thing to do is to extend assert-expr
to operate on something like 'valid?
#2016-11-2222:47bfabry@olivergeorge I believe you can avoid that by doing (s/spec ::b)#2016-11-2222:48bfabryalthough, that's not what the documentation says, so I'm probably confused#2016-11-2222:59Oliver GeorgeThanks @bfabry I did wonder if there might be something like that. I'll experiment more.#2016-11-2223:00bfabryyeah actually doesn't work at all. my bad#2016-11-2223:00Oliver GeorgeNo worries.#2016-11-2223:02Oliver GeorgeActually this works: (s/def ::a (s/and ::b))
#2016-11-2223:02bfabryI wonder if there's a concise definition of which spec macros require definition and which don't#2016-11-2223:02bfabryha. neat#2016-11-2223:03Oliver GeorgeNot sure I'd say neat exactly but it'll do the trick in my case.#2016-11-2223:04Oliver GeorgeI'd be interested in hearing from @alexmiller if this is technically a bug or if "declare before use" is an implementation requirement/assumption#2016-11-2223:04Oliver George@bfabry thanks for the inspiration š#2016-11-2223:05bfabrylol, glad to "help"#2016-11-2223:05Alex Miller (Clojure team)I'd say bug, file a jira#2016-11-2223:05Alex Miller (Clojure team)Everything should have delays, but there are a few places missing them#2016-11-2223:06Oliver GeorgeThanks Alex. Will do.#2016-11-2223:06bfabryoh well that's concise enough, awesome#2016-11-2223:08Alex Miller (Clojure team)That's either really easy to fix or really hard :)#2016-11-2223:10Oliver Georgeš#2016-11-2223:15Oliver GeorgeLogged. Please tell me if it's not a well formed JIRA ticket. http://dev.clojure.org/jira/browse/CLJ-2067#2016-11-2223:16hiredmaneventually @alexmiller will release a spec for jira tickets so you'll be able to s/valid? them before submitting them#2016-11-2223:20rickmoynihanwhat do people think of this clojure.test extension for spec?
(defmethod assert-expr 's/valid? [msg form]
(let [spec (second form)
input-val (nth form 2)]
`(let [value# ~form]
(if value#
(do-report {:type :pass, :message ~msg,
:expected '~form, :actual value#})
(do-report {:type :fail, :message ~msg,
:expected (:clojure.spec/problems (s/explain-data ~spec ~input-val)), :actual ~input-val}))
value#)))
Output looks like this:
FAIL in (foo-test) (common.clj:18)
expected: [{:path [],
:pred (* :user/foo),
:val "1",
:via [:user/foo],
:in []}]
actual: "1"
#2016-11-2223:31Alex Miller (Clojure team)@olivergeorge good enough, Iāll tweak as needed#2016-11-2223:55Oliver GeorgeThanks Alex#2016-11-2310:24yonatanelIf I define specs such as :event/type, it's something that will likely be defined by more projects, so is the best practice to have a fuller namespace such as :org.my.event/type? This will deviate from the shorter form of my Datomic attributes which I don't want to prefix with the organization name all over the place.
(I assume specs can collide being in the same process, while in Datomic I control what I put there)#2016-11-2312:11yonatanelPerhaps the question is about specs of internal vs external data.#2016-11-2312:52Alex Miller (Clojure team)Yes, you should use a sufficiently unique namespace #2016-11-2312:53Alex Miller (Clojure team)But if you control what namespaces you're using, "sufficiently" can be pretty short#2016-11-2313:03yonatanelBeing paranoid, maybe it could be nice to have separate non-default registry instances, or attaching a spec to a key when calling s/keys somehow.#2016-11-2313:54Alex Miller (Clojure team)Not doing that #2016-11-2315:41gfredericks@yonatanel if you're paranoid then you can just use sufficiently unique namespaces#2016-11-2315:48rickmoynihanOn a similar note, whatās the suggested way to spec a format you didnāt define, that defines the same key with different values/specsā¦ Basically is there an alternative to using s/keys
when I have two different specs that share that prefix? Or should I just move the conflicting definitions into different namespaces?#2016-11-2315:50gfredericksDifferent namespaces#2016-11-2315:50gfredericksIt just occurs to me that spec can't describe something where a namespaced keyword has varying meaning#2016-11-2315:51rickmoynihangfredericks: thatās the problem I have#2016-11-2315:51gfredericksWhich I imagine is rare#2016-11-2315:51gfredericksWho subjected you to this#2016-11-2315:52rickmoynihanwell maybe I donāt have the problem#2016-11-2315:53rickmoynihanI mean I can work around it by defining the specs in different namespaces#2016-11-2315:53Alex Miller (Clojure team)If you have that problem, you may be doing it wrong :)#2016-11-2315:53rickmoynihanentirely possible#2016-11-2315:53rickmoynihanš#2016-11-2315:54Alex Miller (Clojure team)if you have unqualified keys, then you can deal with this using s/keys and :req-un with different namespaced specs#2016-11-2315:54Alex Miller (Clojure team)if you have qualified keys, thenā¦ use the namespaces to differentiate different semantics#2016-11-2315:54rickmoynihanalexmiller: yeah I think I just need to define some new namespaces for those extra keys#2016-11-2315:57rickmoynihanout of interest is it possible to do (s/def :foo/spec ,,,)
and have that work with keys
:req-un
?#2016-11-2315:58gfredericksLooks normal....?#2016-11-2315:59gfredericksEvery registered spec has to have a namespace#2016-11-2316:00rickmoynihangfredericks: What I mean is - is the above possible instead of having to create a new namespace e.g. can you do the above instead of needing (ns blah.foo) (s/def ::spec ,,,)
#2016-11-2316:01gfredericksYes#2016-11-2316:02gfredericksCode namespaces and keyword namespaces are mostly independent#2016-11-2316:04gfrederickshttp://hacklog.gfredericks.com/2013/08/10/clojure-overloads-the-term-namespace.html#2016-11-2316:05rickmoynihanok thatās what I thought, itās just not working - presumably for another reason#2016-11-2316:06rickmoynihanits ok I fixed it#2016-11-2316:06rickmoynihanthanks#2016-11-2316:07rickmoynihancode blindness#2016-11-2316:08gfredericksThe eternal bug#2016-11-2316:10rickmoynihanyup - thatās one thing about software development, you have to be pretty thick skinned as youāre continually told by the computer that āyou suck! Another mistake, are you some kinda noob? Oh and another mistake, you really arenāt good enoughā¦ā ad-infinitumā¦ I have a hypothesis that this is why people donāt stick it out as a career#2016-11-2316:11rickmoynihanyou basically have to be a bit of a masochist to stick it out#2016-11-2316:16yonatanel@rickmoynihan My take on this is that you can't yet have "wonderful" mistakes when programming, unlike art where if you use some new material or do a quick sketch it can still look good and expressive.#2016-11-2316:21rickmoynihanyonatanel: yeahā¦ programming is so precise one bit out and it's a catastrophic failure and how many trillion bits are we handling these days?#2016-11-2316:21rickmoynihanitās a miracle of engineering anything works at all#2016-11-2318:08seancorfield(for me, itās sheer stubbornness: āI will not be beaten by an inanimate object!ā so I must ādefeatā the computer š )#2016-11-2318:39rickmoynihana quixotic quest if ever there was one š#2016-11-2318:47aengelbergThe docstring of s/def
indicates that one can provide a symbol instead of a keyword as the name for a spec, but I cannot find any examples of that. What use case is that for?#2016-11-2318:49bfabry@aengelberg s/fdef
is equivalent to (s/def symbol (s/fspec ...))
#2016-11-2318:50bfabrysame as defn
is (def symbol (fn ...))
š#2016-11-2318:52aengelberg@bfabry: interesting, thanks. So when would one use specs that have been set to a symbol?#2016-11-2318:54bfabrywell you're implicitly using them anytime you use instrument#2016-11-2318:54bfabryexplicitly? I dunno. I can't think of a good one off the top of my head, but that doesn't mean there's not#2016-11-2318:57bfabryyou could use the registered fspec's to spec a function to be passed in as (s/or :this `this :that `that :the-other `the-other)
I guess. seems a little contrived#2016-11-2318:58hiredman@aengelberg https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/specs.clj#L197-L202#2016-11-2319:00hiredmanas of the 1.9 alphas, macro inputs are checked via spec, so because the spec is named with the same symbol as the macro, the ns macro's inputs are checked with that spec#2016-11-2319:03aengelbergthanks, that helps a lot#2016-11-2319:07aengelbergSuppose I have a datatype (or {:foo/name-type :keyword, :foo/name :my-foo} {:foo/name-type :string, :foo/name "my-foo"})
. How can I write that in clojure.spec, if s/keys
forces me to commit to one possible spec for a given key?#2016-11-2319:08bfabryfoo/name will need to be an or spec, then you can enforce the dependency using and with a predicate#2016-11-2319:09aengelbergIs the fact that :a/b/c
gives me a keyword with namespace "a"
and name "b/c"
an implementation detail?#2016-11-2319:10aengelbergBecause I could technically leverage that with :req-un
to have different versions of a namespaced key.#2016-11-2319:13hiredmanI think that is sort of a design decision baked in to spec, a namespaced key should only ever be mapped to the same kind of thing#2016-11-2319:15hiredmanspec is sort of a global schema for data in your program#2016-11-2319:16hiredmanso similar to having a sql table with a schema where you say a column has some type, you don't really say sometimes this column has this type, some times it has this other, depending on where the data comes from#2016-11-2319:16bfabryI would guess that a symbol with / in its name is undefined in terms of what will happen#2016-11-2319:18bfabry"Symbols begin with a non-numeric character and can contain alphanumeric characters and *, +, !, -, _, ', and ? (other characters may be allowed eventually)."#2016-11-2319:18bfabry'/' isn't in that list, so I'd say you're off the edge of the map if you use symbols with it in them#2016-11-2319:19aengelbergGot it, makes sense. In this case I am working with an existing program that has keys not set up with that philosophy, so I'm just trying to figure out if there's any way I can fall back to more verbose but more flexible behavior a la prismatic schema. Sounds like not.#2016-11-2319:20aengelberg@bfabry: I was referring to keywords, not symbols.#2016-11-2319:20bfabrykeywords have the same rules as symbols, but can't contain .#2016-11-2319:20hiredmanthere was a discussion about / in keywords, and it comes up over and over again because clojure allows a larger range of inputs than it should in many places#2016-11-2319:21hiredmananswer is just don't do that#2016-11-2319:21hiredmanmy answer#2016-11-2319:21bfabryyes, you can make all sorts of symbols and keywords that the documentation doesn't say is permitted, which is very unfortunate#2016-11-2319:24hiredmanhttp://dev.clojure.org/jira/browse/CLJ-1530#2016-11-2319:25aengelbergSo what you're saying is that I should use the :a/b/c
keyword NOW while I can before it becomes officially unusable.#2016-11-2319:25hiredmanno#2016-11-2319:25aengelbergJust kidding. I don't in fact want to watch the world burn.#2016-11-2319:25hiredmanš#2016-11-2319:28aengelbergIs this intentional?
user> (s/explain #{1 2 3} 4)
val: 4 fails predicate: :clojure.spec/unknown
nil
Specifically the "unknown" part. Seems like the predicate should be more informative when it fails.#2016-11-2319:30bfabryspecial case. if you def'd that predicate to the registry you'd get better message#2016-11-2319:30bfabryuser=> (s/def ::foo #{1 2 3 4})
:user/foo
user=> (s/explain ::foo 5)
val: 5 fails spec: :user/foo predicate: #{1 4 3 2}
nil
#2016-11-2319:30hiredmanuser=> (s/explain (s/spec #{1 2 3}) 4)
val: 4 fails predicate: #{1 3 2}
nil
user=>
#2016-11-2319:30bfabrybasically, explain is not a macro#2016-11-2319:31hiredmanspec tries to transparently treat things that could be specs (like sets and predicates) as specs#2016-11-2319:32hiredmanso those generally work#2016-11-2319:32hiredmanbut there are places where it doesn't, like in the message, there, I suspect its and easy fix if you open an issue#2016-11-2319:33hiredmans/explain must be calling whatever part of the spec protocol implements s/form on the set, and the spec protocol has a fallback on object which returns the unknown keyword#2016-11-2319:34hiredmanthere may already be an issue, spec is only out in alphas#2016-11-2319:38Alex Miller (Clojure team)@aengelberg re the unknown in explain - thatās a known bug and will be fixed#2016-11-2319:38Alex Miller (Clojure team)Iāve worked on it a bit but waiting for some feedback from Rich#2016-11-2319:38Alex Miller (Clojure team)I should probably log an actual jira for it so I can just point people to it#2016-11-2319:39Alex Miller (Clojure team)need to unmunge the fn#2016-11-2319:41Alex Miller (Clojure team)re your prior question, depending on your goal, you can s/def :foo/name with an s/or or you could define different s/keys specs and s/or those#2016-11-2319:42Alex Miller (Clojure team)but the latter seems like it would be papering over the true range of variability in the spec for that attribute#2016-11-2319:42Alex Miller (Clojure team)if your goal is really coercion, then there may be other options (conformers) to consider#2016-11-2319:46Alex Miller (Clojure team)Filed http://dev.clojure.org/jira/browse/CLJ-2068 for the s/explain problem#2016-11-2322:19weiany best practices for ns qualifying keywords that come out of a db?#2016-11-2322:20weiin particular is there a function for adding a namespace to all bare keys in a map#2016-11-2322:23rickmoynihanOne observation whilst playing with spec is that it seems to be āstaticā by design, in that it doesnāt seem to encourage or allow you to programmatically build/modify and refine specs. For example one thing I used to do with schema (especially in tests) was define a generic schema for something and then in specific tests where I want to test something more specifically e.g. the presence of a specific test value in the output, Iād simply assoc
a new specific value into the schema, and validate against that.
Is this a fair point in understanding the proās and conās of spec vs other schema libraries?#2016-11-2322:23rickmoynihan(I should say it seems totally fair to me that spec trades off dynamism for something more static and comprehensible)#2016-11-2322:26hiredmanhard to say#2016-11-2322:26hiredmanthe combinators are different#2016-11-2322:26hiredmaninstead of associng in, you might s/and a new s/keys spec#2016-11-2322:27hiredmanI think there is a bias in spec towards naming things globally, vs some kind of local names or anonymous things#2016-11-2322:33hiredmanI guess I'd say, I don't think spec is particularly static, but the bias towards global names can make it feel that way some times. And the bias is just a bias, you can do that kind of stuff, you might just have to venture off the beaten path of what is provided in the box#2016-11-2322:36rickmoynihanhiredman: yeah I think youāre right#2016-11-2323:40aengelbergHow can you define mutually recursive specs?#2016-11-2323:45zaneI don't remember having to do anything special.#2016-11-2323:59Alex Miller (Clojure team)Just refer to them by (qualified keyword) name#2016-11-2323:59Alex Miller (Clojure team)They delay evaluation so should just work#2016-11-2406:18ikitommiRuntime defined specs/schemas have also a good use case with web apis & dynamic forms. Iām thinking of creating a MapSpec -kinda Spec Record which would allow non-registered (global) specs to be used and which could manipulated as normal Clojure Map. Adding generated specs to registry at runtime might not be a good idea...#2016-11-2406:25ikitommiRelated: toyed with a simple collection spec macro that allows specs to be created from nested maps/vectors/sets. Should help to define partially anonymous map specs. For things like http query-parameters.#2016-11-2406:29ikitommiemits just recursively s/keys
& s/coll-of
s.#2016-11-2407:11ikitommiHmm... but I think after runtime modification of a MapSpec, one would still need to have the full source-code of it for the s/form
to work. So adding values at runtime would require both the value and itās source code to be added. Removing keys would be easy, same as merging existing MapSpecs. Maybe not such a good idea after all.#2016-11-2410:20rickmoynihanikitommi: Interesting - I was having similar thoughts about the possibility of building more specialised libraries that can consume ::specs - for more dynamic use casesā¦ e.g. you could imagine something like plumatic being able to consume specs in some circumstances- It seems youāre a long way ahead of me š#2016-11-2410:20rickmoynihanstill very much at the getting started stage with spec#2016-11-2414:24ikitommirickmoynihan: we are all still spec newbies, exiting times.#2016-11-2418:24yonatanel@ikitommi Talk to me about that runtime specs you are planning. I've just written down my requirements for this kind of thing yesterday and would love to collaborate.#2016-11-2500:08trisshi all. Iāve been looking at extracting the minimum and maximum values from specs created with s/int-in
and s/double-in
...#2016-11-2500:08trissWhat Iāve done is create a spec that recognises the formās produced those functions...#2016-11-2500:09trissand then created seperate functions to pull the mins and maxs from the conformed maps that I create with these.#2016-11-2500:09trissbut the code is 50 lines long? https://gist.github.com/triss/1128bcfeb1a71c7e09ffd9cf10be0370#2016-11-2500:10trisscan anyone tell me if this is a sane approach?#2016-11-2500:11trissor am I totally bonkers?#2016-11-2502:14Alex Miller (Clojure team)So those will s/form back to their original form in the future (I know they are a mess now). That plus a spec on those forms makes this just a conform away.#2016-11-2507:16ikitommi@yonatanel plan is just to test (and built utilities) out things that we are using with Schema. For Schema we have the schema-tools library (https://github.com/metosin/schema-tools) having modified versions of the core functions for maps. In top there are special walkers, transformers and matchers. Using those both at design & runtime. At runtime, have used schemas for example for frontend form validation, and the forms/schemas can change based on the user input. With Spec, there could be either more functions/macros to manipulate the Specs (like s/merge
) or a new MapSpec record which could be used like a regular map. Maybe someone has tried the map-approach already? What use cases / requirements do you have?#2016-11-2509:35slipsetSorry if this has been asked/answered already, but initial googling showed nothing#2016-11-2509:35slipsetI have a map like#2016-11-2509:35slipset{:timestamp <jodatime instance> :value number?}
#2016-11-2509:36slipsetHow do I write a spec for the :timestamp
that s/exercise
understands?#2016-11-2509:37slipsetMy first attempt is (s/def ::timestamp (partial instance? DateTime))
#2016-11-2509:38slipsetbut s/exercise
doesnāt know how to create instances of this.#2016-11-2509:52slipsetok, so I see there is a inst?
which works on java.util.Date. Might get that to work.#2016-11-2509:56slipsetwhich is part of clojure-1.9 but not of future-spec š#2016-11-2510:16slipsetThis is what I ended up with:#2016-11-2510:16slipset(s/def ::timestamp (s/with-gen (partial instance? DateTime)
(fn [] (gen/fmap #(DateTime. %)
(gen/large-integer*
{:min (c/to-long
(t/minus (t/now)
(t/days 365)))})))))
#2016-11-2513:13Alex Miller (Clojure team)in 1.9, you can extend the (new) Inst protocol to DateTime - if you do so, you should be able to use s/inst-in
#2016-11-2513:13Alex Miller (Clojure team)but what you have should work#2016-11-2513:26slipsetYes, I saw that, but Iām just playing with future-spec by @tonsky.#2016-11-2513:26slipsetItās missing some parts, like eg the much debated any?
#2016-11-2513:27slipsetBut it was a good exercise writing ones own generator.#2016-11-2514:02slipsetIām playing around with higher-ordered functions, so I created this:#2016-11-2514:02slipset(s/fdef foo
:args (s/fspec :args any?
:ret any?)
:ret 3)
(defn foo [f] (f 3))
#2016-11-2514:02slipsetSo I do (stest/instrument `foo)#2016-11-2514:03slipsetand running (foo identity)
gives me something like:#2016-11-2514:04slipsetExceptionInfo Call to #ābar/foo did not conform to spec:
val: (#function[clojure.core/identity]) fails at: [:args] predicate: ifn?
:clojure.spec/args (#function[clojure.core/identity])
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file "form-init100394334971998293.clj", :line 917, :var-scope bar/eval55924}
clojure.core/ex-info (core.clj:4617)
#2016-11-2514:04slipsetwhat am I missing?#2016-11-2514:08slipset(s/fdef foo
:args (s/cat :function ifn?)
:ret 3)
(defn foo [f] (f 3))
#2016-11-2514:08slipsetworks, but leaves stuff to be desired#2016-11-2514:12slipsetok, got it#2016-11-2514:12slipset(s/fdef foo
:args (s/cat :function (s/fspec :args (s/cat :arguments number?)
:ret any?))
:ret 3)
(defn foo [f] (f 3))
#2016-11-2521:03leongrapenthinhttp://dev.clojure.org/jira/browse/CLJ-2013 - Are there plans to fix this for 1.9? It would greatly increase error message quality. Is work on it welcome?#2016-11-2522:05Alex Miller (Clojure team)Sure#2016-11-2603:47lvhIs there a way to specify that I expect the value of a var to be valid according to a particular spec, such that it is automatically tested?#2016-11-2603:48lvhSometimes theyāre just data structures, but they might also be the result of a higher order function#2016-11-2604:59lvhAlso; why isnāt there a map-of that takes specs instead of preds?#2016-11-2605:56seancorfield@lvh map-of
does take specs...?#2016-11-2605:58seancorfield(s/map-of ::foo ::bar)
works...#2016-11-2606:04Alex Miller (Clojure team)@lvh you could just check s/valid?
when you set the var. You can use set-validator!
on vars but Iām assuming youāre not actually changing it after you set it.#2016-11-2613:37kestrel7Is it possible to inspect a spec to get, e.g., the list of keys defined by (s/keys :req [a b c])? Iāve created a spec for a map and want to retrieve the list of keys expected for that map for a purpose other than checking the spec.#2016-11-2613:57dergutemoritz@kestrel7 clojure.spec/form
is what you're looking for#2016-11-2614:02kestrel7Perhaps a noob question but from that how do I extract the actual list of keys from the resulting structure, which in this case looks like the following example but could I guess be more complex? Naively I could just say (last (s/form the-spec)) but thatās not going to be robust if the form contents change.
(clojure.spec/keys
:req
[:k.specs.person/title
:k.specs.person/first-name
:k.specs.person/other-names
:k.specs.person/last-name])#2016-11-2614:03dergutemoritzThat's the tricky bit, hehe#2016-11-2614:03kestrel7LOL#2016-11-2614:04kestrel7I guess I could use core.match somehow. It would be useful to get the spec as data thatās easier to interrogate. Thanks for the help @dergutemoritz#2016-11-2614:05dergutemoritzYou can do something like that for the simple case of a single s/keys
spec. However, when stuff like s/merge
is involved, it gets a lot trickier#2016-11-2614:05dergutemoritzIt would be useful to have specs for those forms so you could s/conform
them.#2016-11-2614:06kestrel7For the moment Iām going to cheat:
(def the-keys [::a ::b ::c]
(s/def ::m (s/keys :req the-keys)#2016-11-2614:06dergutemoritzI'm not sure if anyone has put effort into defining such specs already. I was gonna take a stab at that myself when I find some time#2016-11-2614:06dergutemoritz@kestrel7 That cheat won't work I'm afraid#2016-11-2614:07dergutemoritzThe keys vector needs to be literal IIRC#2016-11-2614:07dergutemoritzBecause it's processed at macro expansion time#2016-11-2614:07dergutemoritzRight, just try it in the REPL and you'll see#2016-11-2614:09kestrel7@dergutemoritz youāre right š#2016-11-2614:10dergutemoritz@kestrel7 (apply hash-map (rest (s/form your-keys-spec)))
should give you a reasonable data structure to work with for bare s/keys
specs#2016-11-2614:11kestrel7@dergutemoritz Iāll try that.#2016-11-2614:19kestrel7Hereās my workaround for now#2016-11-2614:19kestrel7(defn spec-keys
"Return the :req keys from the spec"
[spec]
(let [spec-form (s/form spec)]
(assert (= 'clojure.spec/keys (first spec-form)))
(:req (apply hash-map (rest spec-form)))))#2016-11-2614:32dergutemoritzShould do the trick#2016-11-2614:32dergutemoritzYou could destructure spec-form
right there in the let
but I guess that's just cosmetics š#2016-11-2615:13lvh@seancorfield huh! I knew that compiled but I assumed it was treating the keyword as a predicate#2016-11-2616:14dergutemoritzIs there a way to unwrap a regex op context that was previously created with s/spec
?#2016-11-2616:16dergutemoritzHmm regex-spec-impl
doesn't look like it offers something like that#2016-11-2618:21Alex Miller (Clojure team)I have specs for all the forms and those will be published, but I'm still working through fixing all the bugs in form first#2016-11-2618:21Alex Miller (Clojure team)With those, you will be able to conform any spec form to get at its parts#2016-11-2619:46dergutemoritzAh heck š throws his Saturday afternoon hack into the trash bin#2016-11-2619:52dergutemoritzWell, I'll keep it around, perhaps I came up with some things that could be added to your implementation#2016-11-2715:17ikitommithe new specize
seems to turn any function into a spec. As collections are also functions, this gives bit unintuitive results:
(s/valid? {:a 1 :b 2} :b) ; true
(s/valid? #{1 2 3} 3) ; true
(s/valid? [1 2 3] 1) ; true
(s/valid? [1 2 3] 4) ; java.lang.IndexOutOfBoundsException
(s/conform > 10) ; true
(s/conform > >) ; #object[clojure.core$_GT_
#2016-11-2715:24ikitommishould the collections have special behavior with specize? would not turn into specs? That would help my case, I would like to identify and convert nested vectors and sets into collection specs via my coll-spec
macro, but now they can be used as (strangely behaving) specs. Set seems to be the only one of those three that could act as a spec itself. But there could be s/set
for it?#2016-11-2715:33Alex Miller (Clojure team)What's unintuitive about the above?#2016-11-2715:34Alex Miller (Clojure team)Any function that takes 1 arg can be used as a spec#2016-11-2715:35Alex Miller (Clojure team)Sets, maps, and vectors satisfy that#2016-11-2715:35Alex Miller (Clojure team)It's very intentional that this works#2016-11-2715:37ikitommiset as a spec means āone of the valuesā and it has a generator. what does a vector as a spec mean?#2016-11-2715:46Alex Miller (Clojure team)Nothing very useful but it would mean "has a valid index"#2016-11-2715:47ikitommiok, thanks for the answers.#2016-11-2717:46yonatanelIs there a way to directly reuse a spec defined as multimethod implementation? I mean just one of them.#2016-11-2719:43dergutemoritz@yonatanel How do you mean "directly"?#2016-11-2719:44dergutemoritzDoes calling the multimethod with an appropriate value qualify? š#2016-11-2721:44yonatanelNo, I want to define a similar spec by using the one I already defined with one of the multimethod instances, without calling it with a fake map just for the dispatch value. In the end I just defined the common spec as a keyword but I thought there might be a trick to do that.#2016-11-2721:49Alex Miller (Clojure team)Why don't you do the reverse? Define the spec and then have the multimethod return it.#2016-11-2721:52yonatanel@alexmiller That's what I ended up doing.#2016-11-2814:30vikeriHmm, I have an issue with generating data for a multi-spec. It works fine for testing with s/valid?
but not when using (gen/generate (s/gen ::myspec))
. I have the following setup:
(defmulti myspec first)
(defmethod myspec :option/a [_] (s/cat :opt #{:option/a} :val boolean?))
(defmethod myspec :option/b [_] (s/cat :opt #{:option/b} :val string?))
(s/def ::myspec (s/multi-spec myspec first))
Then the following crashes:
(gen/generate (s/gen ::myspec))
;; #object[Error Error: :option/b is not ISeqable]
Iām on cljs FYI.#2016-11-2815:48Alex Miller (Clojure team)the problem is your retag function (first)#2016-11-2815:49Alex Miller (Clojure team)re the docs "retag is used during generation to retag generated values with
matching tags. retag can either be a keyword, at which key the
dispatch-tag will be assoc'ed, or a fn of generated value and
dispatch-tag that should return an appropriately retagged value."#2016-11-2815:50Alex Miller (Clojure team)in this case, I think the value generated by each method spec is probably fine without modification, so you can just use (fn [val tag] val)
#2016-11-2815:50Alex Miller (Clojure team)note that you canāt easily use the anon function syntax here as it must be a function with arity 2#2016-11-2815:51Alex Miller (Clojure team)you could sneakily use it via something like #(first %&)
but I find the fn
version to communicate much better#2016-11-2816:07vikeri@alexmiller Awesome, honestly I read the docs but did not quite understand the retagging partā¦ Now it works as expected.#2016-11-2819:03yonatanelWill specs be re-resolved on Component system restart/refresh?#2016-11-2819:06zaneSpecs are re-added to the registry when you use c.t.n.r/refresh
and the like, but any deleted specs will not be removed from the registry.#2016-11-2819:06zaneIf that helps.#2016-11-2819:14yonatanelThanks#2016-11-2819:39yonatanelIs this a bug? s/merge
specs are not conforming in this case:
(s/def ::status (s/conformer (fn [x] (or (keyword x) ::s/invalid))))
=> :dev/status
(s/def ::a (s/keys :req-un [::status]))
=> :dev/a
(s/def ::b (s/merge ::a (s/keys :req-un [::id])))
=> :dev/b
(s/conform ::b {:id 1 :status "hi"})
=> {:id 1, :status "hi"}
(s/conform ::a {:id 1 :status "hi"})
=> {:id 1, :status :hi}
#2016-11-2819:39yonatanel"hi" should become a keyword#2016-11-2819:55Alex Miller (Clojure team)Merge does not flow conformed values like and#2016-11-2819:55Alex Miller (Clojure team)You'll get the conformed value of the last spec#2016-11-2819:58Alex Miller (Clojure team)Each spec basically has to independently conform for the merge to succeed#2016-11-2819:59Alex Miller (Clojure team)If you swap the order in your merge you should get what you want#2016-11-2820:02yonatanelDon't you want the behavior to be more like map merge? First determine the keys and then conform them.#2016-11-2820:02Alex Miller (Clojure team)No#2016-11-2820:03Alex Miller (Clojure team)That's not what merge is#2016-11-2820:06yonatanelOK. I could read the docs that way.#2016-11-2820:11yonatanelI see bare (s/keys)
doesn't conform either.#2016-11-2820:12yonatanelI wonder how clear these things should be in the docs or if I'm pushing it too far.#2016-11-2821:01potetmIs there a way to spec a multimethod where you can add to the spec in an open-ended way?#2016-11-2821:01potetmJust like you can add to the multimethod in an open-ended way.#2016-11-2822:51mathpunkI had an idea to define a spec, ::item, that specifies that a thing either has an :id key or that it can have a function called id
called on it. I'm not sure, though, if that's a job for protocols instead of specs.#2016-11-2822:56mathpunkThe other thing I'm struggling with is, I can use specs when I define them in the namespace in which I want to use them, but I don't understand how to require them from another namespace. I gathered that spec/def
registered them globally somehow, but other-name-space/foo
does not seem available.#2016-11-2823:28hiredmanit does register them globally, but if you don't load the code it doesn't happen#2016-11-2823:31Alex Miller (Clojure team)@mathpunk re the first question - you could do that but you would need some way to tell that a function can do that (and protocols is maybe the best way) - something like (s/def ::item (s/or :has-key (s/keys :req-un [:id]) :has-fn #(satisfies? HasId %)))
#2016-11-2823:32Alex Miller (Clojure team)where HasId is (defprotocol HasId (id [x]))
#2016-11-2823:32Alex Miller (Clojure team)re the second, as hiredman said, they are registered globally and can be referred to by (qualified-keyword) name, assuming you loaded the code that registered them#2016-11-2908:13mathpunk@alexmiller That's a very helpful example, thank you.#2016-11-2908:22mathpunk@alexmiller @hiredman: As for loading the code that registered them -- I was missing that using ::foo
expands to the whole namespace name i.e. including the name of the app. The 'fully' in fully qualified š#2016-11-2914:32nwjsmithAre there plans to dynaload the shuffle
combinator from test.check
?#2016-11-2914:33nwjsmithWould a patch be welcome for such a change?#2016-11-2914:34Alex Miller (Clojure team)I think thereās actually a pending patch that has that included in it#2016-11-2914:35Alex Miller (Clojure team)but yes, would be happy to add it#2016-11-2914:35nwjsmithstarts digging through Jira to upvote#2016-11-2914:36Alex Miller (Clojure team)yeah, itās in http://dev.clojure.org/jira/browse/CLJ-2046#2016-11-2914:36Alex Miller (Clojure team)which is ready for Rich to look at so I expect that will happen whenever heās looking at tickets next#2016-11-2914:36nwjsmithExcellent, thanks Alex#2016-11-2920:09bbloomcreate-ns and alias seem relatively useful for spec, but clojurescript doesnāt really support these - any good alternative other than verbose keywords?#2016-11-2920:11Alex Miller (Clojure team)prob best to ask this in #clojurescript or #cljs-dev#2016-11-2920:11bbloomok#2016-11-2920:31dnolen@alexmiller I think this is good argument for :alias
at the ns form#2016-11-2920:31dnolenitās already come up before#2016-11-2920:31dnolenand I believe continued usage of clojure.spec will drive the desire for such a feature#2016-11-2920:32Alex Miller (Clojure team)well youāre talking to the wrong person :)#2016-11-2923:26jfntnIs there a way to add meta data to spec definitions without re-implementing s/def
?#2016-11-2923:41hiredmanmetadata?#2016-11-2923:43hiredmanspecs are "named" by symbols (which can have metadata attached) and keywords (which cannot)#2016-11-2923:44hiredmanbut symbols are not interned, so given a symbol 'a if you associated some metadata, another instance of that symbol will not have that metadata#2016-11-2923:45hiredmanby that I mean, even if you did re-implement s/def you would have a hard time putting metadata somewhere#2016-11-2923:48Alex Miller (Clojure team)The answer is no (for now) but it would be useful to have at least a doc string and I'd say that's still a possibility#2016-11-2923:48Alex Miller (Clojure team)As hiredman mentioned, the tricky part is where to put it#2016-11-2923:49Alex Miller (Clojure team)There are some options but none that are obviously good#2016-11-3000:29Alex Miller (Clojure team)https://github.com/prayerslayer/js.spec#2016-11-3020:02nwjsmithThereās also https://github.com/settinghead/specky#2016-12-0100:15stathissiderishello, do function specs and other specs end up in different parts of the registry?#2016-12-0100:15stathissiderisfor example, I have a call like that: (foo {::foo 1})
#2016-12-0100:15stathissideriscan I safely do (s/def ::foo integer?)
and (s/fdef ::foo ā¦)
#2016-12-0100:17Alex Miller (Clojure team)No, those will collide (and the fdef will not be used)#2016-12-0100:17Alex Miller (Clojure team)Def puts keyword keys in the registry and fdef puts symbol keys in the registry#2016-12-0100:22stathissiderisoh so my example should really be (s/fdef foo ā¦)
instead, in which case no collision#2016-12-0100:22stathissideriscorrect?#2016-12-0100:32stathissiderisI have another one: is there a shorter form for (let [{:keys [::stats/foo ::stats/bar]} {::stats/foo 2}] foo)
#2016-12-0100:33stathissideris@alexmiller btw, I enjoyed your interview for the defn podcast, quite intriguing (all the stuff about secret future plans for clojure)#2016-12-0100:33stathissideristhanks#2016-12-0100:47Alex Miller (Clojure team)@re collision - right#2016-12-0100:48Alex Miller (Clojure team)Re destructuring, yes#2016-12-0100:49Alex Miller (Clojure team)Do ::stats/keys [foo bar]#2016-12-0101:28clojuregeekLive stream from AustinClojure meeting by Stu on spec https://m.youtube.com/watch?v=dQcNAscSTSw#2016-12-0103:28bbloomiām totally blown away by how i have a namespace with > 100 defns, only 4 fundamental ones of which have specs, and they help me catch tons of errors - super nice#2016-12-0103:29bbloomthe return value of instrument reminds me of just how little spec you actually need to get some value from it#2016-12-0103:29bbloomvs mandatory type checking everywhere#2016-12-0105:57naomarikthe official spec guide is very well written, had no issues consuming it to once for deep comprehension and also been able to quickly scan it for answers#2016-12-0105:57naomarikjust want to say thanks for that effort š#2016-12-0109:51mishadoffHey, I experienced an issue in clojure.spec
where :fn
key in fdef
is ignored (clojure 1.9-alpha14)
Here is MVP demonstrating the problem.
(defn add [x y]
#_(+ x y)
2)
(s/fdef add
:args (s/cat :x number? :y number?)
:ret number?
:fn #(= (+ (-> % :args :x)
(-> % :args :y)) (:ret %)))
(clojure.spec.test/instrument `add)
If I call (add 1 nil)
I got spec error, but when I call (add 1 2)
spec says everything is ok, despite the fact add
always returns 2
Can anybody help me?#2016-12-0110:06joost-diepenmaat@mishadoff AFAICT instrument
only checks :args (and apparently not :ret or :fn)#2016-12-0110:06joost-diepenmaatthis is surprising to me too but thatās how itās documented:#2016-12-0110:07joost-diepenmaatfrom the instrument
docstring#2016-12-0110:13pithyless@mishadoff @joost-diepenmaat - this has got to be FAQ #1 on clojure-spec š clojure.spec.test/instrument
will only check :args
(itās meant to throw errors if you are using the function incorrectly); clojure.spec.test/check
will run generative tests and check that the function is implemented correctly.#2016-12-0110:22mishadoffGot it#2016-12-0110:22mishadoffStrange, but.. Thanks š#2016-12-0113:56ikitommiBlogged about Schema & Clojure Spec for Web Developers. http://www.metosin.fi/blog/schema-spec-web-devs/ hopefully everyone's spec lib got mentioned.#2016-12-0114:41nha@ikitommi the link to yada is wrong. Nice article š#2016-12-0115:05ikitommi@nha thanks! fixed the link.#2016-12-0115:49camdezWell, I decided to finally give spec a whirl, moved to 1.9.0-alpha14, and promptly discovered that I canāt get a REPL running because one of my dependencies has a non-conforming defn
. š Anything I can do here short of replacing / removing the (slightly busted) library?#2016-12-0115:51Alex Miller (Clojure team)Most offending libs have newer versions that fix these problems#2016-12-0115:52camdez@alexmiller: Iām awareāyou actually patched the library in question š (Aleph). But a new release hasnāt been cut yet.#2016-12-0115:52camdezBut Iām curious conceptually what the options are here.#2016-12-0115:52camdezCertainly I can cut my own release.#2016-12-0115:52Alex Miller (Clojure team)You could s/fdef a dummy spec over the defn spec#2016-12-0115:53Alex Miller (Clojure team)But might be tricky to get that loaded at the right time#2016-12-0115:53camdez@alexmiller: Gotcha. But Iād basically be overriding it globally, right?#2016-12-0115:53camdezFor all defn
s#2016-12-0115:53Alex Miller (Clojure team)Yes#2016-12-0115:53camdezOr no?#2016-12-0115:53camdezk. Thanks.#2016-12-0115:59camdezAh, my mistakeāthere is an alpha release of Aleph with a conforming defn
. (Didnāt release lein ancient
wouldnāt show that.)#2016-12-0123:08bbloomi saw some discussion yesterday about docstrings#2016-12-0123:08bbloomwould be super nice to have some mechanism for that#2016-12-0123:08bbloomiāve got some metadata on some objects that iād like to document better than with a comment#2016-12-0123:09Alex Miller (Clojure team)Still a possibility#2016-12-0123:10bbloomcool#2016-12-0123:12Alex Miller (Clojure team)There is not an obvious place to hang it atm, but there are some options#2016-12-0123:12bbloomsyntax: the cause of and solution to all of our problems#2016-12-0123:17Alex Miller (Clojure team)I don't mean syntax, I mean where to remember it#2016-12-0123:17Alex Miller (Clojure team)Could be easily added to s/def#2016-12-0123:18bbloomoh - i thought the (a?) challenge was no metadata on keywords#2016-12-0123:32Alex Miller (Clojure team)You don't have to provide it as metadata (or store it as metadata)#2016-12-0213:47stathissideris@alexmiller destructuring like that throws an exception: (let [{::stats/keys [foo bar]} {::stats/foo 2}] foo)
#2016-12-0213:47stathissideris:stats/keys
works, but thatās not what I meant#2016-12-0213:48stathissiderisbecause I have this in my require [spec-provider.stats :as stats]
#2016-12-0213:54dergutemoritz@bbloom The relevant ticket to vote on is http://dev.clojure.org/jira/browse/CLJ-1965 š#2016-12-0213:54dergutemoritz@stathissideris That should work AFAICT. What's the exception you get?#2016-12-0213:55stathissiderisspec-provider.trace> (let [{::stats/keys [foo bar]} {::stats/foo 2}] foo)
ExceptionInfo Call to clojure.core/let did not conform to spec:
In: [0 0] val: #:spec-provider.stats{:keys [foo bar]} fails spec: :clojure.core.specs/local-name at: [:args :bindings :binding :sym] predicate: simple-symbol?
In: [0 0 0] val: ([:spec-provider.stats/keys [foo bar]]) fails spec: :clojure.core.specs/seq-binding-form at: [:args :bindings :binding :seq] predicate: (cat :elems (* :clojure.core.specs/binding-form) :rest (? (cat :amp #{(quote &)} :form :clojure.core.specs/binding-form)) :as (? (cat :as #{:as} :sym :clojure.core.specs/local-name))), Extra input
In: [0 0 :spec-provider.stats/keys] val: [foo bar] fails spec: :spec-provider.stats/keys at: [:args :bindings :binding :map :spec-provider.stats/keys :clojure.spec/pred] predicate: map?
In: [0 0 :spec-provider.stats/keys] val: [foo bar] fails spec: :spec-provider.stats/keys at: [:args :bindings :binding :map :spec-provider.stats/keys :clojure.spec/nil] predicate: nil?
:clojure.spec/args ([#:spec-provider.stats{:keys [foo bar]} #:spec-provider.stats{:foo 2}] foo)
clojure.core/ex-info (core.clj:4725)
#2016-12-0213:56stathissiderisIām on clojure 1.9.0-alpha14#2016-12-0213:57bronsaI can't repro:
user=> (let [{::stats/keys [foo bar]} {::stats/foo 2}] foo)
2
#2016-12-0214:00dergutemoritzYeah, same here. Something must be off in your namespace @stathissideris#2016-12-0214:01stathissiderisweird, thanks for trying @bronsa and @dergutemoritz#2016-12-0214:02dergutemoritz@stathissideris You're welcome! If all else fails, try rebooting the REPL š#2016-12-0214:03stathissiderisREPL reboot didnāt help#2016-12-0214:03stathissiderisI think Iāll try in a fresh project#2016-12-0214:04bronsatry *clojure-version*
to confirm you're on alpha14?#2016-12-0214:06bronsa@stathissideris hmm, do you have a ::stats/keys
spec by any chance?#2016-12-0214:06dergutemoritzOooh#2016-12-0214:07dergutemoritzThat'd be nasty#2016-12-0214:07bronsayeah i can reproduce a similar exception by defining a dummy ::stats/keys
spec#2016-12-0214:07dergutemoritzGood thinking @bronsa#2016-12-0214:14bronsainteresting, not sure if this was intentional but one can do this:
user=> (require '[clojure.spec :as s])
nil
user=> (s/def ::keys (s/coll-of '#{foo bar} :kind vector?))
:user/keys
user=> (let [{::keys [foo]} {}])
nil
user=> (let [{::keys [baz]} {}])
ExceptionInfo Call to clojure.core/let did not conform to spec:
In: [0 0] val: #:user{:keys [baz]} fails spec: :clojure.core.specs/local-name at: [:args :bindings :binding :sym] predicate: simple-symbol?
In: [0 0 0] val: ([:user/keys [baz]]) fails spec: :clojure.core.specs/seq-binding-form at: [:args :bindings :binding :seq] predicate: (cat :elems (* :clojure.core.specs/binding-form) :rest (? (cat :amp #{(quote &)} :form :clojure.core.specs/binding-form)) :as (? (cat :as #{:as} :sym :clojure.core.specs/local-name))), Extra input
In: [0 0 :user/keys 0] val: baz fails spec: :user/keys at: [:args :bindings :binding :map :user/keys] predicate: (quote #{bar foo})
:clojure.spec/args ([#:user{:keys [baz]} {}])
clojure.core/ex-info (core.clj:4725)
#2016-12-0214:15bronsaif intentional, your issue is not a bug, otherwise I'd say it's a spec/`:clojure.core.spec/ns-keys` bug#2016-12-0214:15bronsa\cc @alexmiller#2016-12-0214:18stathissideris@bronsa OH! yes, I do have a ::stats/keys
spec#2016-12-0214:20stathissiderisit doesnāt have to be called that, but it was the most natural name for what Iām doing#2016-12-0214:21bronsayeah I'm looking at the bindings spec and I don't think that's intentional behaviour#2016-12-0214:30stathissideris@bronsa good spot!#2016-12-0214:33bronsayeah looks like a bug with s/keys :opt-un
#2016-12-0214:33bronsaminimal repro: user=> (s/def ::foo nil?)
:user/foo
user=> (s/def ::bar (s/keys :opt-un [::foo]))
:user/bar
user=> (s/valid? ::bar {:a/foo 1})
true
user=> (s/def :a/foo nil?)
:a/foo
user=> (s/valid? ::bar {:a/foo 1})
false
#2016-12-0214:35dergutemoritz@bronsa How's that wrong?#2016-12-0214:35Alex Miller (Clojure team)You need to redef bar for that to take effect#2016-12-0214:35Alex Miller (Clojure team)Specs are lazily compiled on use#2016-12-0214:36dergutemoritzNote that bit from the s/keys
docstring: "In addition, the values of all namespace-qualified keys will be validated (and possibly destructured) by any registered specs."#2016-12-0214:36Alex Miller (Clojure team)And cached#2016-12-0214:36bronsa@dergutemoritz ah, then it's not a bug :)#2016-12-0214:36Alex Miller (Clojure team)Oh, that too#2016-12-0214:37bronsaleads to surprising results tho#2016-12-0214:37dergutemoritzI think this is unrelated to the keys destructuring issue @stathissideris ran into#2016-12-0214:37bronsa@dergutemoritz no, that's the same issue @stathissideris is having#2016-12-0214:37Alex Miller (Clojure team)There are some weird corners with unqualified keys #2016-12-0214:38dergutemoritzOK then I didn't get the connection, yet š#2016-12-0214:38bronsathe let binding destructuring specs are defined in terms of s/keys
, which is why he's getting ::stats/keys
validated against his ::stats/keys
spec#2016-12-0214:38dergutemoritzAh, I see#2016-12-0214:38Alex Miller (Clojure team)Yeah, that's pretty subtle :)#2016-12-0214:39dergutemoritzI wonder if this is intended behavior in this particular situation, though š#2016-12-0214:39bronsait feels a bit weird that that happens purely because of how those specs are defined (i.e. it depends on an implementation detail)#2016-12-0214:40Alex Miller (Clojure team)We've talked about this a bit and I think this is a case where some effort could be taken to detect and warn (or maybe error)#2016-12-0214:40bronsarealistically this means using ::keys
or ::strs
as spec is going to lead to those issues most of the times (unless your ::keys
spec is literally a coll of simple-syms)#2016-12-0214:41Alex Miller (Clojure team)It is in the ballpark of being open in your map specs or kwarg options#2016-12-0214:41dergutemoritzAn option to disable this behavior for particular keys
specs might be useful, too#2016-12-0214:42Alex Miller (Clojure team)I'll think about it#2016-12-0214:42Alex Miller (Clojure team)That's already a pretty tricky hybrid map spec#2016-12-0214:42bronsacool, I'll open a ticket later to track this#2016-12-0214:45dergutemoritzGreat find everyone, I'm happy I probably won't have to run into this case myself some day and scratch my head in confusion š#2016-12-0215:25bronsahttp://dev.clojure.org/jira/browse/CLJ-2074#2016-12-0215:25bronsaattached a proposed solution#2016-12-0215:34Alex Miller (Clojure team)We are absolutely not doing that#2016-12-0215:35Alex Miller (Clojure team)I don't expect to make any adjustments to keys for this, but rather alter how the destructuring spec is written#2016-12-0215:36bronsafair enough#2016-12-0300:43bblooms/merge makes me so happy.#2016-12-0300:44bbloombeing able to just mix in key sets for various stages of the pipeline#2016-12-0300:44bbloombeautiful.#2016-12-0301:01bbloomthe go type system has āstruct embeddingā which i fell in love with when working in go - lets you accomplish something similar by merging struct fields#2016-12-0318:48bbloomiām not quite sure yet i grok the every vs coll-of, or every-kv vs map-of stuff yet#2016-12-0318:48bbloomis the general idea to use coll-of and map-of by default and only switch to every if your specs are too slow and you donāt need conformance#2016-12-0318:48bbloom?#2016-12-0318:50bbloomalso - iām now running in to a situtation where a spec is not-conforming and itās just consuming a ton of memory producing the explanation data#2016-12-0319:14arohner@bbloom I havenāt run into that situation, but thatās my guess#2016-12-0319:16bbloomyeah, i get the feeling explain needs to become lazy somehow - not necessarily with thunks, but maybe with something like a ā¦
that you can ask it to expand further#2016-12-0319:47bbloomhmm, yeah - i have a structure with LOTS of structural sharing and a ton of redundant data - not unlike the clojurescript ast#2016-12-0319:48bbloomand explain is starting to become non-useful due to the shear volume of output#2016-12-0319:48bbloomand seemingly exponential time to return#2016-12-0319:48bbloomor print or whatever#2016-12-0319:48bbloomnot quite able to pin down a reproduction yet#2016-12-0321:07Alex Miller (Clojure team)You can set a custom explain printer if that becomes useful#2016-12-0321:08Alex Miller (Clojure team)Just setting print-length will affect the default one though#2016-12-0321:08Alex Miller (Clojure team)In case that helps#2016-12-0321:09Alex Miller (Clojure team)Re your collection question, yes that's a good first approximation#2016-12-0321:09bbloomre: print-length - cool! thanks.#2016-12-0321:10Alex Miller (Clojure team)You can also set coll-check-limit to further tweak what every and every-kv check#2016-12-0321:10bbloomsuuuuuppper nit-picky feedback:#2016-12-0321:11bbloomif that first-approximation is true, that you should default to coll-of or map-of, then it seems like the more informative doc-strings should be on those, rather than every and every-kv#2016-12-0321:11bbloomfeel free to totally ignore i even said that š#2016-12-0321:13Alex Miller (Clojure team)Rich always thought people should default the other way#2016-12-0321:13Alex Miller (Clojure team)But in general most people seem to disagree :)#2016-12-0321:13bbloompreferring the sampling kind?#2016-12-0321:13Alex Miller (Clojure team)Yeah#2016-12-0321:13bbloomhm, i was going back and forth on it in my head#2016-12-0321:14bbloomseeming like sampling is the right thing if the specs are not intended to be used for contract-style checks#2016-12-0321:14bbloomwhich seems true#2016-12-0321:14bbloombut it feels wrong to get non-determinstic behavior by default#2016-12-0321:14Alex Miller (Clojure team)I would actually prefer that the doc for both covered all the details, but you know#2016-12-0321:14Alex Miller (Clojure team)Rich isn't big on duplication like that#2016-12-0321:14bbloomthe slippery slope of excessive doc strings - i like the short strings#2016-12-0321:15Alex Miller (Clojure team)I have a spec reference page in the works for the docs which will help too#2016-12-0321:15bbloomi wish the see also was metadata tho, so the cross referencing was more automatic#2016-12-0321:15bbloomand maybe even let you see-also a spec#2016-12-0321:15bbloomso like the options that are specified could be generated from common doc strings#2016-12-0321:15Alex Miller (Clojure team)If wishes were fishes something something #2016-12-0321:16bbloomheh, indeed - the simple strings are generally fine#2016-12-0321:16Alex Miller (Clojure team)Later, shutting down for takeoff#2016-12-0321:16bbloomaaaaannny way, i guess iāll lean on the sampling ones & see how that feels#2016-12-0321:16bbloomcya, have a good flight#2016-12-0321:52paytonrulesHow do you spec a multi-method? Multi-spec seems like it should do it, but it only talks about specāing a hash with types. More specifically I have the following - but it never validates the :box
parameter.
(s/fdef colliding?
:args (s/cat :invader map? :box ::box/box)
:ret boolean?)
(defmulti colliding? :character)
(defmethod colliding? ::small [invader box]
#2016-12-0322:13Alex Miller (Clojure team)You can't right now#2016-12-0322:14Alex Miller (Clojure team)Probably fixable but I haven't looked at it yet#2016-12-0322:15paytonrulesOh really?#2016-12-0322:16paytonrulesKinda shocked - I had assumed it was me the whole time.#2016-12-0322:17paytonrulesMight wanna make it clearer in the docs that itās not supported. All I could find was some confusion on multi-spec.#2016-12-0322:17paytonrulesThanks for the help tho!#2016-12-0322:20Alex Miller (Clojure team)Yeah, multimethod, protocols, primitive typed fns, and inlined fns will not work right now #2016-12-0322:21Alex Miller (Clojure team)Inlined and protocols are probably not easily fixable#2016-12-0322:37bbloomuser=> (binding [s/recursion-limit 1 s/coll-error-limit 1 s/coll-check-limit 1] (clojure.test/run-tests āambiparse.calc-test))
That normally returns in ~1 second, but when instrumented, it seems to be infinitely looping. jstack says: https://gist.github.com/brandonbloom/dfa559f81ad1b6d0980c120d86a2e18e#2016-12-0322:37bbloomsomething seems to not be respecting the limits#2016-12-0322:37bbloomor iām missing a limit#2016-12-0322:38bbloomamusingly, iām working on a parser, soooo iām running in to exactly the same kinds of infinite loop problems in the program iām trying to debug š#2016-12-0322:58derwolfeHi - I'm experimenting with spec and am getting an error that I'm not understanding RuntimeException Var clojure.test.check.generators/large-integer is not on the classpath clojure.spec.gen/dynaload (gen.clj:21)
. Any tip as to what could be the problem? This is happening when I'm trying to exercise a simple spec I've written.#2016-12-0323:02bbloomlooking at the spec impl, it seems like there isnāt on spec checking level/depth - which might be what i want/need, not sure tho#2016-12-0323:27Alex Miller (Clojure team)It's entirely possible some spec impl is not respecting the limit#2016-12-0323:27Alex Miller (Clojure team)@derwolfe: you need test.check on your classpath#2016-12-0323:30derwolfe@alexmiller thanks!#2016-12-0410:08tslockeIām guessing this comes up a lot - the fact that conformed values flow through the validation process seems to make it difficult to compose specs. Say Iām writing some preconditions for a function, I have to look at the spec definitions to know if Iām going to get, say, raw values, or [tag value] pairs. Am I missing something?#2016-12-0410:18tslockeOK just discovered s/unform. Adds some verbosity to some specs, but at least removes the dependency.#2016-12-0412:47tslockeHmmm doesnāt seem to work with s/coll-of#2016-12-0420:03gfredericks@tslocke I saw some speculation a while back that spec will eventually have two versions of s/and
; not sure if that's exactly what you were referring to#2016-12-0421:14bbloomyeaaah, still not quite sure whatās causing an infinite loop or otherwise such bad perf it might as well be infinite - best i can offer at the moment is these stack traces: https://gist.github.com/brandonbloom/dfa559f81ad1b6d0980c120d86a2e18e#2016-12-0421:14bbloomdefinitely related to the collection stuff tho b/c every makes it random, coll-of makes it every time#2016-12-0421:42bnoguchi@bbloom: ran into the same issue recently as well#2016-12-0421:42bbloom@bnoguchi were you able to shrink the reproduction down at all? i havenāt been able to yet#2016-12-0421:47bnoguchiNot quite. I had to hit the brakes on debugging it but am taking a closer look again at it this week.#2016-12-0504:43jfntnFinally have a good example of an issue I keep running into with s/keys
. Here is a polymorphic payload for a couple of websocket messages:
{:msg/id :foo/msg, :msg/data { ... foo/data ... }}
{:msg/id :bar/msg, :msg/data { ... bar/data ... }}
My basic intuition is to have a multi-spec that dispatches on :msg/id
:
(defmulti msg-spec :msg/id)
But then I have a problem, because methods need to deal with a polymorphic :msg/data
too:
(defmethod msg-spec :foo/msg [_] (s/keys :req [:msg/data !?]))
(defmethod msg-spec :bar/msg [_] (s/keys :req [:msg/data !?]))
All I can think of is to have the multi-spec on :msg/data
instead. But that dispatch function will be much more complex and will need to infer, from the :msg/data
alone, the value that was right there in :msg/id
.
Am I missing something?#2016-12-0504:50Alex Miller (Clojure team)You can base your polymorphism on more than just a key - it's an arbitrary function#2016-12-0504:51Alex Miller (Clojure team)So you could base it on both id and something in msg#2016-12-0504:52Alex Miller (Clojure team)Or you could do more than one level of multispec#2016-12-0504:54jfntnRight, I think I understand that and it seems to be the problem. The :msg/id
is all I need to determine the type of :msg/data
but itās only available at that root level#2016-12-0504:55jfntnWith a multi-spec for :msg-data
Iād need to look at what keys are in there, and in some cases what values, just to derive a tag was just there in the parent map#2016-12-0505:14jfntnI've heard Rich's rebuttal of this kind of "contextual polymorphism" , but this example feels like something that will come up in practice, especially as people adopt qualified keywords in their apis.
An ad-hoc binding of a map-key to a spec would be a bulletproof one-liner alternative to what would now have to be a complex, possibly buggy dispatch function.#2016-12-0505:50Alex Miller (Clojure team)So are you saying that :mag/data has more than one spec? That seems wrong.#2016-12-0514:38jfntnYes indeed. The websocket library is embedding the msg/tag and the msg/data in its payload. It takes a [msg/tag msg/data]
and we get this map.#2016-12-0514:38jfntnSo we have multiple specs for both the msg itself (omitted other keys that need specs) and the msg/data, both of which ultimately depend on the msg/tag.#2016-12-0514:40jfntnCurious whatās your intuition on why this is wrong? This is something that comes up a lot in my experience whit what are now unqualified library apis, but it seems bound to happen more as people adopt namespaced kws?#2016-12-0514:49jfntnIn a nutshell, I can define that :msg/id
is either a :foo/msg
or a :bar/msg
and that :msg/data
is either a ::foo/spec
or ::bar/spec
but I canāt enforce that relationship at the msg level#2016-12-0516:26Alex Miller (Clojure team)Qualified names should have meaningful stable semantics. Using qualified names should make this happen less if people are using sufficiently qualified names (which they should)#2016-12-0516:27Alex Miller (Clojure team)Could you not wrap an s/and around specs on both of these to add a constraint?#2016-12-0516:30Alex Miller (Clojure team)Are ::foo/spec and ::bar/spec just s/keys specs? If so, do they really need to be different or is just (s/keys) to validate all attrs sufficient?#2016-12-0517:06lvhIs the lack of docstrings on spec/def intentional?#2016-12-0517:07lvhI could of course just add the metadata to the var#2016-12-0517:13jfntnalexmiller: yes we have different s/keys specs for the data itself#2016-12-0517:43Alex Miller (Clojure team)lvh: it was not part of the initial impl. itās a highly rated request in the jira system and Rich mentioned it as an idea to me long ago. itās not obvious to me how to best implement it (where to put the meta) for all types of specs (when you consider things like (s/def ::a ::b) as kws donāt have meta). Do you have a var to add meta to?#2016-12-0517:43Alex Miller (Clojure team)jfntn: it would help me to see a more detailed example#2016-12-0517:44jfntnalexmiller, happy to put something more detailed together and ping you later today#2016-12-0517:48Alex Miller (Clojure team)cool, I will be in and out today#2016-12-0520:17lvhalexmiller: Sometimes I do by accident, yes ā but much to your point; thereās no real vars in spec most of the time; so presumably thatās not where tooling would want to go look#2016-12-0521:12thegeez=> (s/conform
(s/+ (s/cat :one (s/+ #{1 2 3})
:alpha (s/? #{:a})))
[1 2 :a 2])
[{:one [1 2], :alpha :a} {:one [2]}]
=> (s/conform
(s/+ (s/cat :one (s/+ #{1 2 3})
:alpha (s/? #{:a})))
[1 2 :a 2 3])
[{:one [1 2], :alpha :a} [{:one [2 3]}]]
#2016-12-0521:13thegeezIn the second example I am surprised with the extra vector around {:one [2 3]}#2016-12-0521:14thegeezThis doesn't seem consistent with a longer example:#2016-12-1210:22dergutemoritzThat would require the keys themselves to be present in the input sequence, too#2016-12-1210:22dergutemoritzSo yeah, you'll have to roll your own wrapper macro#2016-12-1210:24alexisvincent@dergutemoritz Thanks š I wonder though if tuple is not what I want actually#2016-12-1210:25dergutemoritz@alexisvincent YW! Yeah, s/tuple
would get rid of the redundancy but it wouldn't conform into a map#2016-12-1210:25dergutemoritzBut perhaps you can live with that#2016-12-1210:25alexisvincentOh right.#2016-12-1210:25alexisvincentI wonder if s/keys
maintains order?#2016-12-1210:26alexisvincentBecause then I could write a function that first accepts req keys and then optional keys#2016-12-1210:26alexisvincentin order#2016-12-1210:28dergutemoritzFWIW, here's a wrapper macro of the kind I'm talking about:
(defmacro keys-cat [& keys] `(s/cat
#2016-12-1210:28dergutemoritzOr maybe keys-tuple
would be a more fitting name#2016-12-1210:33alexisvincent@dergutemoritz Thanks š#2016-12-1210:33alexisvincentmaybe cat-keys#2016-12-1210:34dergutemoritzhttps://s-media-cache-ak0.pinimg.com/originals/c1/19/8f/c1198f0990a37c100437b8c31309c4e8.jpg#2016-12-1210:49alexisvincentxD#2016-12-1211:35alexisvincentis the general consensus to prefer namespaced keys in maps?#2016-12-1212:50dm3it seems so, for best synergy with spec#2016-12-1213:07alexisvincentAre people generally using ::
as a helper#2016-12-1213:08alexisvincentIts a pain when wanting to use the spec in another nsā¦ You have to specify the full mylonglongname.namespace.thing/name
#2016-12-1213:09alexisvincentbecomes so long windedā¦#2016-12-1213:09alexisvincentOr are people just registering thing/name
#2016-12-1213:14alexisvincentOH š ::
can also be used for aliasing#2016-12-1213:15dm3also (require [some.ns :as n]) (is (= ::n/kw :some.ns/kw))
#2016-12-1213:15dm3I guess that's what you meant#2016-12-1213:54alqvistWhat is the best way of namespacing an existing map? Using a function, not the reader#2016-12-1214:26alexisvincent@dm3 yep š#2016-12-1215:15Alex Miller (Clojure team)@alqvist what do you mean? adding a namespace to the keys of a map?#2016-12-1215:16Alex Miller (Clojure team)there is no function to do that right now#2016-12-1215:34alqvist@alexmiller exactly that - I made a small function for it. Posted it in #clojure#2016-12-1215:34Alex Miller (Clojure team)cool#2016-12-1217:34sparkofreasonIs there some preferred way of writing simulation-testable specs for functions with constraints between parameters? I have a function that takes a map and a key. The key must exist in the map for both the input and output. I can make it work by writing a custom generator for :args
using bind
, was wondering if there was a preferred method.#2016-12-1218:28Alex Miller (Clojure team)that is the preferred method#2016-12-1218:29Alex Miller (Clojure team)https://www.youtube.com/watch?v=WoFkhE92fqc#2016-12-1218:29Alex Miller (Clojure team)^^ is about that#2016-12-1218:44sparkofreasonThanks, that's basically how I handled it.#2016-12-1218:46sparkofreasonI notice in the video that the generator override was specified directly in the test, rather than in the spec. Don't know that it makes much difference, though giving it in the spec would allow you to reuse the override if that spec was composed with others.#2016-12-1218:48bbloomis there a spec or predicate for objects that have strict value equality? looking for something like a clojure.core/edn?
or something like that#2016-12-1218:49bbloomedn? isnāt quite right, since thatās about serializability, but would be close enough for my needs#2016-12-1218:52hiredmanthat would be nice#2016-12-1218:52hiredmanany? is so close, but fails with NaNs#2016-12-1218:53bbloomno, any? isnāt close - it includes functions#2016-12-1218:53bbloomi have a procedure that will go in to an infinite loop if the function isnāt pure & iāve been bitten a few times by including a function in it#2016-12-1218:54bbloomso iām doing like add-watch! does, which is force key values#2016-12-1218:54hiredmanI am pretty sure the generator for any? doesn't generate functions#2016-12-1218:54bbloomgeneration aside, the spec is wrong if i allow functions š#2016-12-1218:55bfabrythe generator for any? was way simpler than preferred last time I checked, there was a bug on test.check about it#2016-12-1218:55bbloomi need a predicate like value-equality?#2016-12-1218:56bbloomwhich i guess i could make, but it seemed like it might have been something that existed or should exist#2016-12-1218:56bbloomi didnāt feel like exhaustively listing whatās in edn in a spec#2016-12-1218:56Alex Miller (Clojure team)I think there are some things in test-check with "printable" in their name?#2016-12-1218:57Alex Miller (Clojure team)I know we spent a lot of time building up stuff like that in the old data.generative#2016-12-1218:57Alex Miller (Clojure team)Certainly open to ideas#2016-12-1218:57bbloomhmm āprintableā is a good hint, iāll look - thanks#2016-12-1218:58hiredmanthe problem with the printable stuff is it generates NaNs#2016-12-1218:59hiredmanand those are generators, not predicates#2016-12-1218:59bbloomalexmiller: this looks more or less like what i need. would be cool to have a predicate for it#2016-12-1219:00Alex Miller (Clojure team)Jira it up#2016-12-1219:00Alex Miller (Clojure team)Seems like we could also fix test.check re NaN (which is not currently printable)#2016-12-1219:05bbloomhttp://dev.clojure.org/jira/browse/CLJ-2083#2016-12-1219:05bbloomi think clojure.core/edn?
would be the least objectionable formulation of this request#2016-12-1219:06bbloomwell, actually - may not#2016-12-1219:06bbloomb/c that would be a ādeepā predicate#2016-12-1219:06bbloomi donāt think any such deep predicates exist - it would have O(N) runtime š#2016-12-1219:39Alex Miller (Clojure team)I think it's probably unlikely we would make that predicate#2016-12-1219:40bbloomyeah - after realizing it was O(N), i realized that - but a spec makes more sense, right?#2016-12-1219:41Alex Miller (Clojure team)Well, definitely a generator, maybe a spec#2016-12-1219:41bbloombesides the stuff in clojure.core.specs, are there any āstandardā specs anywhere? i donāt think so, right? just predicates...#2016-12-1221:06jeremyrainesIn this example from the spec guide, is the ability to key into the arguments with :start and :end created by the s/cat above the anon function? Does the destructuring happen such that the first expression given to s/and has an effect on whatās passed to the other?
(s/fdef ranged-rand
:args (s/and (s/cat :start int? :end int?)
#(< (:start %) (:end %)))
ā¦
#2016-12-1221:07bfabry@jeremyraines yes, the s/cat conforms the data. s/and threads the conformed data#2016-12-1221:07jeremyrainesok, thanks#2016-12-1221:07zaneSuper useful for conformers that coerce between types.#2016-12-1304:39j-poI'm trying to spec something like s-expressions, and my current flailing in that direction involves a recursive s/tuple
(something like
(s/def ::sexp
(s/alt :arg integer?
:expression (s/tuple fn?
::sexp
(s/* ::sexp))))
), but I'd like for the input not to have to be a vector of vectors. Am I better off just turning the input into such a vector as part of the checking process, or is there another way?#2016-12-1305:29Alex Miller (Clojure team)s/cat?#2016-12-1305:32Alex Miller (Clojure team)then you can just replace ::sexp (s/* ::sexp) with (s/+ ::sexp) too#2016-12-1305:34Alex Miller (Clojure team)(s/def ::sexp (s/alt :arg integer? :expression (s/cat :f fn? :args (s/+ ::sexp))))
#2016-12-1309:22j-po@alexmiller Thanks! When I gen/generate
from an s/cat
approach, though, I don't get a sequence of sequences, though, but just one flat one.#2016-12-1309:25vandr0iyI was asking stuff in the beginners channel, but probably this is a better place for doing that.
I posted this snippet of code, claiming that it doesn't work:
(let [arst {:a [1 2 3]
:b {:c #{::z ::x ::c}}}]
(s/keys :req-un (get-in arst [:b :c])))
And I was told that :req-un wants a literal vector of namespaced keywords, not something that evaluates to one, because of it being a macro (obviously) - which makes sense.
So I went a step further, and macro'd my stuff like this:
(def my-bunch-of-stuff {:a {:b [:x :c :d]}
:z {:b vector?}}
(defmacro arst [type]
(let [sp# (get-in my-bunch-of-stuff [type :b])]
(if (fn? sp#)
`(s/spec ~sp#)
`(s/spec (s/keys :req-un ~(vec (map #(->> % str keyword) sp#)))))))
and it still doesn't work OOB; while, if I copy-paste the result of macroexpand '(arst :a)
in the REPL - it works flawlessly. What's going on here? I'm afraid that there's some rookie mistake here somewhere...#2016-12-1309:28mpenetyou need to use either a macro or eval:#2016-12-1309:28mpenet(let [arst {:a [1 2 3]
:b {:c #{::z ::x ::c}}}]
(eval `(s/keys :req-un ~(get-in arst [:b :c]))))
#2016-12-1309:28mpenet(s/keys is a macro)#2016-12-1309:30mpenetand you might actually need to call seq
or vec
on the result of the get-in, not sure s/keys will happily take a set as :req-un value#2016-12-1309:33mpenetyour macro should work (kinda, you need namespaced keys in your bunch-of-stuff and there's a paren missing there too)#2016-12-1309:40vandr0iyI'm sorry for adding the macro after the request, I accidentally pressed enter
before completing my post.
My keys aren't namespaced - I "namespace" them with the (map #(->% str keyword) sp#)
thing.
What puzzles me is: why does this macro work if I try to do (arst :z)
- which takes a function from the bunch-of-stuff and spits a spec out of it -
and doesn't if I get a vector of un-namespaced keywords, "namespace" them with the dirty hack described above???
I (macroexpand '(arst :a))
, and copy-paste what I get in REPL - and it works, but it doesn't if I just try to do (arst :a)
.
I'm pretty sure that ~(vec (map #(->> % str keyword) sp#))
yields a vector of namespaced keywords...#2016-12-1309:43mpenetit doesnt I think, it creates keys with ":foo" content with a leading :, so yeah ::foo but not namespaced per say#2016-12-1309:43mpenettry running your spec, it will complain about this. Try changing your keys in bunch-of-stuff with ns keys you ll see#2016-12-1309:52vandr0iyOh... I see. Was able to make it work using this other horrible hack:
(defmacro arst [type]
(let [sp# (get-in settings [type :b])]
(if (fn? sp#)
`(s/spec ~sp#)
`(s/spec (s/keys :req-un ~(vec (map #(keyword (str *ns* "/" (name %))) sp#)))))))
#2016-12-1309:57trisshow do I write a spec for an instance of Comparable
?#2016-12-1310:01vandr0iy(s/valid? (s/spec #(instance? Comparable %)) "arst")
#2016-12-1310:02trissthanks @vandr0iy . Iāll give that a try#2016-12-1310:43trissthat worked. Brilliant thanks.#2016-12-1310:44trissIf I want a generator that produces a wider range of vaues than the ones provided for double-in and int-in how would I go about getting one of those?#2016-12-1310:44trissall the random numbers Iām getting are pretty small and I need to test against some big ones.#2016-12-1310:50mpenettest.check has generators that allow you to specify a range#2016-12-1310:50mpenetlarge-integer* and double* I think#2016-12-1310:51trisscheers @mpenet Iāll have alook#2016-12-1310:53triss(gen/sample (gen/double* {:min 1 :max 200}))
still produces really small values#2016-12-1310:53trissIād like to see some values closer to 200. Is this possible?#2016-12-1311:00mpenetTry sampling more values#2016-12-1311:13trissah ok.. looks generators probably arenāt going to be efficient enough for the way I was abusing them.#2016-12-1311:50trissIf I wanted a more evenly distributed generator would it be possible to write one?#2016-12-1312:13luxbock@triss yes but you need to use the generators in test.check
#2016-12-1312:14gfredericks@triss what's a uniform distribution over doubles?#2016-12-1312:14gfredericks@triss oh I think this is a sizing issue, sorry I didn't read back far enough#2016-12-1312:15triss@gfredericks ah I can see why that might be awkward. rand
seems more even than this though#2016-12-1312:15gfredericks@triss you should get lots of values close to 200, but gen/sample
is deliberately only showing you small examples#2016-12-1312:15gfredericks@triss try (gen/sample ... 200)
#2016-12-1312:15luxbockis there any reason for why s/spec
couldn't accept a third optional argument for defining a custom generator? I thin it would make it easier to statically analyze specs and might end up with some cool tooling use cases#2016-12-1312:16gfredericks@triss test.check has some subtleties with sizing that need to be documented better, and I have 45 minutes free right now so I think I will start on that#2016-12-1312:17triss@gfredericks I still see a similar distribution. biased to low numbers#2016-12-1312:17trissah many thanks @gfredericks will be much appreciated#2016-12-1312:17gfredericks@triss it's not going to be uniform, but you should definitely get some larger numbers#2016-12-1312:18gfredericks@triss one reason I asked my original question about what a uniform distribution is, is because doubles themselves aren't uniform -- there are a lot more numbers between 1 and 50 than 50 and 200#2016-12-1312:18gfredericksso uniform might mean different things in different contexts#2016-12-1312:18trissoh thatās interesting. because of the way they are represented in the machine?#2016-12-1312:18gfredericksyou could say that#2016-12-1312:19gfredericksit's kind of inherent in the idea of floating point#2016-12-1312:19trissI guess Iād like a distribution of Real numbers? does that make more sense?#2016-12-1312:19triss^a more even distribution of real numbers#2016-12-1312:19gfredericksnot really, but I think I know what you're trying to get at#2016-12-1312:19gfredericks@triss you could use large-integer to get this pretty easily#2016-12-1312:20trissIād love to see a flatter histogram!#2016-12-1312:20gfredericksor at least something close#2016-12-1312:20gfredericksyes that's a good way of putting it :)#2016-12-1312:20trissah brilliant will look at large-integer
#2016-12-1312:20gfredericks(gen/let [x (gen/large-integer {:min 0 :max 200000000})] (/ x 1000000.0))
#2016-12-1312:21gfredericks@triss ā something like that#2016-12-1312:21gfredericksit won't get you every double in that range, but it might be okay for what you're doing#2016-12-1312:21gfredericksand it wouldn't be hard to make it fancier so that it hits most things#2016-12-1312:22trissthanks. this will probably do nicely.#2016-12-1312:22gfredericksnp#2016-12-1312:27gfredericksoh ha yes I forgot that#2016-12-1312:27gfrederickssorry#2016-12-1312:28trisslooks like I wonāt be generating my population of genotypes with spec or generators.#2016-12-1312:28trissor hang on I can write a custom one?#2016-12-1312:28gfredericks@triss try (gen/let [x (gen/choose 0 200000000)] (/ x 1000000.0))
#2016-12-1312:29gfredericks@triss ā these essentially are custom generators#2016-12-1312:29gfredericksit's all about composing lower-level generators to get what you want#2016-12-1312:29gfredericksgen/choose
is one of the lowest-level ones, and gives you a uniform distribution over a range of integers#2016-12-1312:29trissgen/choose is perfect for what Iām doing I think....#2016-12-1312:29gfredericks@triss now that I think about it, I might recommend the large-integer
approach anyhow, but it's tricky to explain why#2016-12-1312:30mpenetworst case you can cheat the thing with (gen/fmap #(do whatever you want) (gen/return nil))#2016-12-1312:30trissok - I need to scratch my head about composing these things for a while....#2016-12-1312:31gfredericks@triss this might be useful: https://github.com/clojure/test.check/blob/master/doc/generator-examples.md#2016-12-1312:32trisswonderful thanks.#2016-12-1312:42mpenetI have this one bookmarked š https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#2016-12-1312:44gfredericks@triss w.r.t. choose
vs large-integer
, the short explanation is that test.check has a strategy where it tries small things first and slowly tries larger and larger things; by using choose
you're overriding that strategy, and that strategy is the reason you saw an uneven distribution with large-integer
#2016-12-1312:45gfrederickswhich is best depends on your goals, but I'd say large-integer
would be a good default#2016-12-1312:46trisswould you consider using generators outside of testing code foolish?#2016-12-1312:47trissit seems very much tailored for testing.... (as the namespace indicates I suppose)#2016-12-1312:49gfredericks@triss clojure.spec uses them for a bit more than test.check does, but still for testing purposes; if you're using them for not-testing-at-all, it will be a little weird but not terrible; understanding the sizing subtleties could be more important though#2016-12-1312:50gfredericksI suppose based on that jungle music talk at the conj I should expect people will be using generators for all sorts of things#2016-12-1312:53gfredericks@triss the gen/generate
function will probably be useful for you, since it accepts a size
parameter#2016-12-1312:53gfrederickse.g., (repeatedly 200 #(gen/generate (gen/let ...) 200))
should give you the distribution you wanted even using large-integer
#2016-12-1322:53agGuys, can someone help me with this: I have a long string of lorem ipsums, something like this "pellentesq dapib suscip liguldon posue augquaeti v tort ..." and now I need a spec with a generator that would generate a name of 3 to 5 words long by randomly pulling those words from that lorem-ipsum string. How do I do that?#2016-12-1323:21adambros(def s āpellentesq dapib suscip liguldon posue augquaeti v tortā)
(clojure.string/join " " (take 3 (shuffle (clojure.string/split s " ā))))
;=> "augquaeti posue vā
@ag does this look right?#2016-12-1323:21adambrosoh you said generatorā¦ hmm#2016-12-1323:26j-poYou could make a generator for sequences fitting that format. Composing in string/join
is the unknown bit, but it seems doable.#2016-12-1323:28j-poYou could do that in a custom generator, for instance, but then you'd have a custom generator š#2016-12-1323:29adambros(gen/sample (gen/bind (gen/shuffle (str/split "foo bar baz qux asd wet ab" #" ")) #(gen/frequency [[5 (gen
/return (take 3 %))] [5 (gen/return (take 5 %))]])))
;=> (("foo" "baz" "bar" "ab" "wet") ("foo" "ab" "baz") ("wet" "bar" "baz") ("baz" "bar" "ab") ("bar" "ab" "baz") ("foo" "
ab" "baz") ("foo" "wet" "asd" "ab" "baz") ("asd" "wet" "ab") ("qux" "wet" "foo" "bar" "baz") ("ab" "bar" "asd" "foo"
"baz"))
#2016-12-1323:29agyeah but I canāt make this work, I got this far:
(gen/generate
(s/with-gen string? #(gen/generate (gen/shuffle (clojure.string/split lorem-ipsum #ā\s")))))
#2016-12-1323:30adambrosgot it to alternate between take 3 and take 5#2016-12-1323:30adambrosi think you see the pattern#2016-12-1323:30agoh, let me try your snippet#2016-12-1323:30adambrosjust add an entry for (take 4 %)
with w/e freq you want#2016-12-1323:32adambrosi think the only reason i came up with that so fast was the time i spent in haskell š
#2016-12-1323:32adambrosbind man...#2016-12-1323:36ag@adambros oh this is awesome, now I think how could be gen/frequency part can be generalized#2016-12-1323:38gfredericksum#2016-12-1323:38gfrederickshow about#2016-12-1323:38gfredericks(gen/vector (gen/elements the-words) 3 5)
#2016-12-1323:38gfredericks@ag ā#2016-12-1323:39gfredericksshould be more efficient than the shuffler too since it doesn't do lots of shuffling work just to throw it away#2016-12-1323:39gfredericksonly difference is it won't give you distinct elements#2016-12-1323:42ag@gfredericks oh, yeah, makes sense#2016-12-1323:42agthanks#2016-12-1323:42adambrosthats interesting, so i guess it depends on if you want/need distinct elements#2016-12-1323:45gfredericksif you need distinct, wrapping it in such-that
might be efficient, as long as the collection is large enough#2016-12-1323:47agerrrhā¦ now how I actually emit strings? str/join
ed? I am trying to wrap it in gen/fmap - doesnāt work ;(#2016-12-1323:54agnevermind got it!#2016-12-1323:54ag(gen/fmap
#(clojure.string/join " " %)
(gen/bind (gen/shuffle (clojure.string/split lorem-ipsum #" "))
#(gen/vector (gen/elements %) 1 5)))
Thanks everyone!#2016-12-1400:02gfredericks@ag that does a bunch of unnecessary shuffling on each generation#2016-12-1400:03gfredericksI'd do (gmap/fmap #(clojure.string/join " " %) (gen/vector (gen/elements (clojure.string/split lorem-ipsum #" ")) 3 5))
#2016-12-1400:03gfredericksno need for a shuffle at all#2016-12-1400:18agso gen/elements
guarantees that it would pull elements randomly? then yeah, makes sense#2016-12-1400:18agthanks!#2016-12-1400:21agdamn it.. now Iām struggling with turning it into a spec
(def ^:private l-ipsum-gen
(gen/fmap #(clojure.string/join " " %)
(gen/vector (gen/elements (clojure.string/split lorem-ipsum #" ")) 3 5)))
(s/gen (s/with-gen (s/and string?) l-ipsum-gen))
#2016-12-1400:21agdoesnāt work#2016-12-1400:23hiredmanif gen/ is clojure.spec.gen then you need to wrap things in no argument functions#2016-12-1400:23hiredmanor, I should say, you need to wrap generators in#2016-12-1400:26aghmmm, ya, canāt figure out right syntax#2016-12-1400:27agokā¦ I think Iāve figured it:
(def ^:private l-ipsum-gen
(gen/fmap #(clojure.string/join " " %)
(gen/vector (gen/elements (clojure.string/split lorem-ipsum #" ")) 3 5)))
(gen/sample (s/gen (s/with-gen (s/and string?) (fn [] l-ipsum-gen))))
#2016-12-1409:16artemyarulinHello. Where I should put specs: next to functions, next to tests, one global file with all specs for my project. Once Iāve seen [module-name]-spec.clj separate file for each module.#2016-12-1410:14pyrHI! How do you go about generating related values?#2016-12-1410:14pyrA simple case would be an indexed map where keys need to reappear in values#2016-12-1412:18gfredericks@pyr are you familiar with gen/let
and/or gen/fmap
?#2016-12-1413:37bronsaI was toying earlier with implementing split-with
using spec
user=> (require '[clojure.spec :as s])
nil
user=> (defn s-split-with [pred?] (s/cat :left (s/* pred?) :right (s/* any?)))
#'user/s-split-with
user=> (s/conform (s-split-with int?) (range 1e5))
StackOverflowError clojure.lang.RT.get (RT.java:751)
#2016-12-1413:38bronsaanything obviously "wrong" or is this stack overflow a real issue?#2016-12-1413:39bronsaby contrast, if the pred fails early and the right branch is used, it doesnt overflow#2016-12-1413:39Alex Miller (Clojure team)It's greedy so I'd expect all 1e5 to go left#2016-12-1413:39bronsae.g. (s/conform (s-split-with ident?) (range 1e5))
returns fine#2016-12-1413:39bronsa@alexmiller yes, I'd expect that to happen, not at stack overflow#2016-12-1413:40Alex Miller (Clojure team)Yeah just thinking through#2016-12-1413:41bronsahttp://sprunge.us/SPDT#2016-12-1413:41bronsahere's the overflow stack trace if that's helpful#2016-12-1413:41Alex Miller (Clojure team)What if you don't have right?#2016-12-1413:42bronsait runs fine#2016-12-1413:42bronsa(this is ok (s/conform (s/cat :left (s/* int?)) (range 1e5))
)#2016-12-1413:43bronsauhm, so is (s/conform (s/cat :left (s/* int?) :right (s/* ident?)) (range 1e5))
#2016-12-1413:44bronsabut (s/conform (s/cat :left (s/* int?) :right (s/* int?)) (range 1e5))
overflows#2016-12-1414:53Alex Miller (Clojure team)the regex derivative is keeping track of all possible solutions - in the last case the split could occur anywhere so there are a large number of those (the others do not have that ambiguity)#2016-12-1414:54Alex Miller (Clojure team)so these are (algorithmically) very different specs#2016-12-1415:00Yehonathan Sharvitwhat is ident?
#2016-12-1416:48Alex Miller (Clojure team)Keyword or symbol#2016-12-1417:13ddellacostafolks Iām a bit confused by something right now, and apologies if this is documented somewhere that I missed:
If I have a spec keyword in namespace x
(`(def ::foo ā¦)`) and then I refer to it via a ref in a separate namespace (`[x :as y]`), I canāt seem to use that refāed keyword in a defmethod
(`(defmethod foo :y/foo ā¦)`)#2016-12-1417:13ddellacostadoes that make sense? Am I doing something obviously wrong?#2016-12-1417:28hiredman::y/foo
#2016-12-1417:39josesanchThis is not working because s/keys is a http://macro.Is there any way to do this?#2016-12-1418:16bbloomjosesanch: i donāt really think spec wants to be used in that way dynamically. i think itās somewhat designed to ensure some things are static at useful times.... can you do get-mandatory-fields in a more static way and then validate with a dynamic spec NAME rather than a dynamic spec?#2016-12-1418:17bbloomsomething like (s/explain-data (get-spec-name llambda) llambda))#2016-12-1418:17bbloomotherwise, your choices are eval (probably not a good idea) or macros (which are going to be static anyway)#2016-12-1418:18josesanchThank you @bbloom. I was thinking about that. In this case I'm validating each field individually#2016-12-1418:18bbloomin that case, i suggest using the per-field specs (which presumably already exist?) and a simple loop or map or reduce on top#2016-12-1418:19bbloomie do custom validation in addition to the spec validation#2016-12-1418:19bbloombut others may have other suggestions ĀÆ\(ć)/ĀÆ#2016-12-1418:21josesanchYes. This way. Thank you very much.#2016-12-1418:22bbloomnice. this is one of the great features of spec vs a traditional type system: you can use it as a building block for other dynamic validations#2016-12-1418:28Alex Miller (Clojure team)also, (s/keys) will validate all keys#2016-12-1418:28Alex Miller (Clojure team)you could combine that with a check for whether a map contains all the required keys#2016-12-1418:29Alex Miller (Clojure team)that could be a custom predicate that built from a set of keys#2016-12-1418:30bbloomhm, are you saying like youād make a wrapper object {:required #{::foo ::bar}, :object {::foo ā¦ ::bar ā¦}}
and then validate that with a custom predicate?#2016-12-1418:31Alex Miller (Clojure team)no#2016-12-1418:32Alex Miller (Clojure team)Iām saying you could (s/and (s/keys) #(every? req-keys-set (keys %)))#2016-12-1418:32Alex Miller (Clojure team)I probably typoed that, but you get the idea#2016-12-1418:34bbloomah ok#2016-12-1418:56ddellacosta@hiredman yeah I figured it out after a little whileā¦PBKAC. Thanks!#2016-12-1520:01sveriI wonder if there is a way to make sense of such a spec message: https://pastebin.com/QrU49Thr#2016-12-1520:02sveriits hard to figure out if I am just missing a key somewhere or if the structure in general (coll-of (coll-of ...)) is broken#2016-12-1520:21Alex Miller (Clojure team)I think there might be more than one thing wrong too#2016-12-1520:22Alex Miller (Clojure team)this: val: 16 fails spec: :de.sveri.getless.service.off/id at: [:args :foods :id] predicate: string?
seems pretty straightforward#2016-12-1520:22Alex Miller (Clojure team)the :id key is an int not a string#2016-12-1520:24Alex Miller (Clojure team)the foods arg also seems to be missing some keys
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :image_small_url)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :image_thumb_url)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :lang)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :code)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :rev)
#2016-12-1520:24sveri@alexmiller Yea, thats right, uhm, ok, I am, surprised#2016-12-1520:24sveriCan you tell me how you extracted the important information so quick?#2016-12-1520:25Alex Miller (Clojure team)looking at the line breaks#2016-12-1520:25sveriIt took me a lot of time figuring out the first error, then reload and look for the second one. My problem is, its just a wall of text.#2016-12-1520:25Alex Miller (Clojure team)each line is a problem - those are the ends of the lines#2016-12-1520:25sveriI see, turning off soft wraps would have helped here.#2016-12-1520:25Alex Miller (Clojure team)the (large) data value is the distracting part in each line#2016-12-1520:26Alex Miller (Clojure team)you can install a custom explain printer too if you want (could actually hide the val, or limit itās size)#2016-12-1520:27sveriThat sounds good, my application makes use of an external service, that returns a lot of data. Is there an example somewhere on how to make a custom printer?#2016-12-1520:27Alex Miller (Clojure team)(set! s/*explain-out*
(fn [explain-data]
(binding [*print-length* 3]
(s/explain-printer explain-data))))
#2016-12-1520:27Alex Miller (Clojure team)*explain-out*
is a dynamic variable holding the function that prints explain data#2016-12-1520:28Alex Miller (Clojure team)^^ that will work at the repl, but you might need to use with-bindings
to make it work in your code#2016-12-1520:28Alex Miller (Clojure team)s/explain-printer
is the default print function#2016-12-1520:29Alex Miller (Clojure team)this example limits *print-length*
to 3 when printing big collections#2016-12-1520:29Alex Miller (Clojure team)(thatās the most common source of large printed values)#2016-12-1520:29Alex Miller (Clojure team)for example (s/explain empty? (range 100))
with the above will not print the full seq#2016-12-1520:34sveriAwesome š
First question, may I put that on a gist with your comments and
second: when I try your code it throws the following exception during runtime: java.lang.IllegalStateException: Can't change/establish root binding of: *explain-out* with set
#2016-12-1520:49Alex Miller (Clojure team)Yes and that's what I mentioned #2016-12-1520:50Alex Miller (Clojure team)It works at the repl because the repl binds explain-out#2016-12-1520:50Alex Miller (Clojure team)But in your code you need to set it in a dynamic scope with something like binding#2016-12-1520:51Alex Miller (Clojure team)Or alter-var-root it#2016-12-1520:51Alex Miller (Clojure team)I should probably just write a quick blog on it#2016-12-1520:56sveri@alexmiller Great, I got it now š thank you very much#2016-12-1600:49noprompti think thereās a bug with :clojure.core.specs/binding-form
. itās defined as
(spec/or :sym :clojure.core.specs/local-name
:seq :clojure.core.specs/seq-binding-form
:map :clojure.core.specs/map-binding-form)
but this is a problem if you actually want to use the data from conform
ing against it because
(spec/conform
:clojure.core.specs/binding-form
'{})
;; => [:seq {}]
which is the smallest example that demonstrates the problem (`{:as m}` conforms to a [:seq ,,,]
value). in order to fix this problem the spec needs to be reorganized to place :map
spec above the :seq
spec.
(spec/or :sym :clojure.core.specs/local-name
:map :clojure.core.specs/map-binding-form
:seq :clojure.core.specs/seq-binding-form)
hereās an example of the updated behavior
(spec/conform
(spec/or :sym :clojure.core.specs/local-name
:map :clojure.core.specs/map-binding-form
:seq :clojure.core.specs/seq-binding-form)
'{})
;; => [:map {}]
(spec/conform
(spec/or :sym :clojure.core.specs/local-name
:map :clojure.core.specs/map-binding-form
:seq :clojure.core.specs/seq-binding-form)
'[])
;; => [:seq {}]
#2016-12-1600:53noprompti guess this is another good case for specs for specs. š#2016-12-1600:53nopromptcc @alexmiller#2016-12-1600:54Alex Miller (Clojure team)There's already a ticket and a patch for this I think#2016-12-1600:56Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2055#2016-12-1600:56Alex Miller (Clojure team)Same thing I think?#2016-12-1601:03nopromptah nice! yep, thatāll do it too! š#2016-12-1601:03noprompt(although my patch would have been smaller) š#2016-12-1601:05noprompt(brandonās is better though) š#2016-12-1601:11Alex Miller (Clojure team)We really need a vcat which would be better than either#2016-12-1601:14Alex Miller (Clojure team)In particular to c#2016-12-1601:14Alex Miller (Clojure team)Conform to a vector#2016-12-1615:59devthwhere does spec store registered specs? can i list / inspect them?#2016-12-1616:14bronsa@devth (s/registry)
#2016-12-1616:15devth@bronsa thanks!#2016-12-1616:15Alex Miller (Clojure team)you might also find stest/instrumentable-syms
or stest/checkable-syms
of interest re functions#2016-12-1616:16devthah, very cool#2016-12-1700:52bbloomiām trying to get a feel for when it makes sense to use multispec vs or/and/merge/etc.... at first i thought it might be the same tradeoff as with multimethods vs if/case/etc. ie: open vs closed primarilyā¦ however iām not quite sure what the generator behavior will be for and/or/merge w/o the retag function#2016-12-1700:53bbloomwhatās folks experiences with small/closed alternatives? like say 3 cases where all of them share some ābaseā fields and each has a couple unique fields#2016-12-1701:06bbloomi think in this case iām looking at now, i can just ignore the ābaseā notion, but not quite sure#2016-12-1705:00tjtoltonare videos from the clojure-spec workshop at the recent conj up online anywhere?#2016-12-1705:01tjtoltonthe datomic workshop videos are incredible, and depending on how my meeting on monday goes, might mean a new enterprise datomic customer!#2016-12-1713:07luxbockI started working on a convenience macro for defining a function spec and its definition within the same defn
like macro#2016-12-1713:09luxbockand I'm trying to decide it it would make more sense to infer the spec of the arguments from the argument destructuring syntax, or to allow you to destructure the output of conform
on the input parameters#2016-12-1713:11luxbockso an example of the first would be something like [{:keys-req [foo ::a-foo]}]
(the argument vector of the function) expanding to (s/cat :thing-with-foo (s/keys :req [::a-foo]))
in the :args
part of the fdef
form#2016-12-1713:15luxbockand the second one would look something like [({:keys [xxx yyy]} ::a-foo)]
which binds xxx
and yyy
to be things pulled from (s/conform thing-which-is-a-foo)
by wrapping the body of the function definition in a let form that does the destructuring for you#2016-12-1713:15luxbockI guess these things are not mutually exclusive, so I could just write two separate macros#2016-12-1713:17luxbockboth macros expand to (do (fdef ...) (defn ...))
at the top level#2016-12-1713:18luxbockmaybe I should postpone this until I have more experience with using spec the usual way#2016-12-1713:28luxbockI suppose you could even combine them by always grouping bindings and specs in parentheses, so you could do [{:keys-req [({:keys [xxx yyy]} ::a-foo)]}]
which defines the :args
as (s/cat :arg-a (s/keys :req [::a-foo]))
and assuming ::foo is something like (s/cat :xxx int? :yyy int?)
then you can refer to xxx
and yyy
as such inside the function body#2016-12-1713:30luxbockif anyone can think of reasons for why this is a bad idea that'd be great so that I won't have to discover this by trying š#2016-12-1715:58paytonrules@luxbock I would love to see this macro on github when itās ready#2016-12-1718:50bbloombeen using update-in, get-in, etc happily for quite some time, but seeing such heavy usage of paths in spec makes me wish Rich would someday give a talk including a rant about xpath/xquery š#2016-12-1718:51Alex Miller (Clojure team)Why would he rant about that?#2016-12-1718:52bbloomi donāt know. i would just like to hear his insights/thoughts about it#2016-12-1718:52Alex Miller (Clojure team)I've done a bunch of xquery and I think it's a pretty reasonable functional query language with some good stuff in it#2016-12-2313:47Yehonathan SharvitBTW the function I want to spec is a function that receives a collection of numbers and calculates their average#2016-12-2313:58Yehonathan SharvitYou probably meant this: (s/def ::seq-of-numbers (s/coll-of ::not-nan-number))
#2016-12-2313:59Yehonathan Sharvit(s/def ::not-nan-number (s/and number? #(not (Double/isNaN %))))
#2016-12-2313:59Alex Miller (Clojure team)Actually I would use s/and to make a spec, not a pred#2016-12-2313:59Alex Miller (Clojure team)Well, you could do it either way#2016-12-2313:59Alex Miller (Clojure team)There are tradeoffs#2016-12-2314:00Alex Miller (Clojure team)Depends whether you care about gen too#2016-12-2314:02Yehonathan SharvitYeah. I need gen#2016-12-2314:03Yehonathan Sharvitdo you know if there is a isNaN that is cljs compatible?#2016-12-2314:04Yehonathan Sharvitothrewise I would need to make my own #?(:clj Double/isNaN :cljs js/isNaN)
#2016-12-2315:12bhagany@viebel in cljs, double-in
takes a :NaN?
parameterā¦ Iām kind of surprised this wasnāt in clojure first? (Iāve only done spec in cljs)#2016-12-2315:13bhaganyah, it is in clojure, too: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/double-in#2016-12-2315:14bhaganyso, looks like you can (s/double-in :NaN? false)
and get correct check and gen behavior in both clj and cljs#2016-12-2316:12bhaganyDidnāt have time to complete this thought earlier - Iād probably end up with something like (s/or :int int? :double (s/double-in :NaN? false))
. This way also has the advantage of easily including ratios, if you wanted to do that. (`number?` wonāt gen ratios)#2016-12-2318:31sophiagogoing to try this again during the daylight...and hopefully before too many people have left for the weekend, if not already. i'm looking for some advice speccing out this project:#2016-12-2318:34sophiagoessentially i'm just wondering if there's any way to automate pairing return types based on input values like this or whether i'm just being far too ambitious about what spec can do#2016-12-2318:36sophiagoit was also suggested i might be able to simplify how i've specified all the combinations of types, but i'm not quite sure how to go about that#2016-12-2318:42sophiagoi suppose one way would be to use gen with the type constructors instead of using the selectors to define all these types? then feeding those into the input and of the functions i'm using fdef with? i suppose when it comes to checking cases where it should reduce i may have to settle for just enumerating a large number of tests i can go over visually rather than having it flag the return values as invalid for me, although that would be ideal#2016-12-2319:37assoc-inDoes anyone know if when I instrument a function with a spec that defines :ret as int? and the function I am testing returns a string if the spec error should show up in the repl? I was able to get the error of the return not being an int? when I used spec/check but expected it with instrument as well. Thanks!#2016-12-2319:38assoc-inAlso I am able to get the :arg spec error to show up with invalid inputs so I believe I am instrumenting the function correctly.#2016-12-2319:38seancorfieldinstrument
only verifies that functions are passed valid arguments, i.e., that the calling code āworksā. If you want to verify a function āworksā ā testing the return and invariants ā you need to look at clojure.spec.test/check
.#2016-12-2319:39seancorfieldTwo different types of testing.#2016-12-2319:40assoc-inInteresting I figured it would be a nice addition to the development workflow being able to check the return types were correct on the fly like the arguments are checked#2016-12-2320:04sophiago@seancorfield i was confused by your comment on my snippet last night. were you suggesting i use s/valid
with the type constructors rather than specifying so many s/def
s using the selectors and instance?
#2016-12-2320:06seancorfieldI was suggesting reducing the number of specs so that you didnāt have separate cases for int + int / complex + int / int + complex / etc ā I think youāre over-specifying things and making life harder for yourself.#2016-12-2320:07seancorfield@assoc-in Probably a good idea to watch some of the videos online about clojure.spec
so you understand the drivers behind its design ā since expecting instrument
to run :ret
and :fn
checks is common when folks first start using spec unless theyāve read / watched a bunch of the design justifications.#2016-12-2320:09seancorfield@assoc-in Not sure if youāre in the #beginners channel? Thereās some discussion right now about spec and someone just linked to a great talk by Stu Halloway on it.#2016-12-2320:12sophiago@seancorfield what i'm really concerned with testing are subtyping + coercion. so i do need those cases specified for it to be useful and at least it's something i can do with relative ease. the latter...i'm less sure about. it may come down to just visually checking a large search space to see if any aren't reduced or if they cause the coercion functions themselves to throw errors. but at least for that i need a variety of subtype relationships#2016-12-2320:14seancorfieldAh, I seeā¦ Then I donāt really have any suggestions...#2016-12-2320:15sophiagoah ok. thank anyway. i'm just going to start by having it enumerate a certain number of each type per each function i'm testing with no :fn
key or more particular restrictions on :ret
#2016-12-2320:16sophiagofor the future...if i do want to use a helper function for :fn
do you know the syntax for passing the arg values to that? as opposed to what i tried in that snippet?#2016-12-2320:17sophiagoor ret
rather#2016-12-2320:18sophiagoi was trying to do something like this : :ret (coerce-types #(:args :a %) #(:args :b %)))
#2016-12-2323:46hlshipIs it allowed to instrument protocol methods? That is, I define a protocol, and want to instrument the created dispatch function.#2016-12-2323:46hlshipSo far, this is not working; Iād imagine that it would instrument the function to verify arguments & etc., then delegate to the original function to actually dispatch on type to an instance of the protocol.#2016-12-2323:47hlshipNever mind, it works. Problem w/ a test.#2016-12-2323:51sophiago@hlship i was going to say...that's exactly what i'm working on right now!#2016-12-2323:54hlshipItās odd; I get the expected failures from the REPL, but inside tests, I can pass non-confirming values without failure.#2016-12-2323:54hlshipI suspect this would work if it was a normal function, but Protocol methods are their own beasts.#2016-12-2323:55sophiagohmm...that's odd#2016-12-2323:55sophiagothis is my first go at fully testing a project and i'm having trouble getting the syntax correct it seems#2016-12-2323:56hlshipI can imagine any number of load-order or bytecode generating things that could be getting in the way.#2016-12-2323:56hlshipIād hate to have to add a function in front of the method, just to get the desired test-time spec help.#2016-12-2323:57sophiagoyeah, i'm not planning on doing that either#2016-12-2400:18sophiagoi still can't figure out how to specify a list of possible s/def
s to pass to :args
and :ret
š i keep getting a "No value supplied for key" error#2016-12-2402:28seancorfield@hlship see http://dev.clojure.org/jira/browse/CLJ-1941 -- known places where instrument
doesn't work (includes primitive returning fn and protocols)#2016-12-2402:33sophiagoah, that says only primitives are affected tho?#2016-12-2402:34seancorfieldProtocols are mentioned in the comments.#2016-12-2402:34seancorfieldCan you share the problematic code re: :args
:ret
?#2016-12-2402:35sophiagoi'm just trying to pass a list of specs to each:#2016-12-2402:35sophiagocan't imagine this is the correct way to do it?#2016-12-2402:36seancorfieldremove the [a b]
s-exp#2016-12-2402:37seancorfieldThen it should work.#2016-12-2402:37seancorfieldAs for ::sym
-- it can't be all those things at once -- shouldn't that be s/or
?#2016-12-2402:37seancorfieldand int?
to make it a predicate#2016-12-2402:38sophiagooh yeah. i was playing around with several expressions trying to get it to work#2016-12-2402:38seancorfieldhttps://clojure.org/guides/spec#_spec_ing_functions#2016-12-2402:39sophiagowow. i can't believe i even had the guide open and misread speccing functions as taking an argument list#2016-12-2402:39sophiagothat means it's time to go to sleep soon š#2016-12-2402:39seancorfieldHeh, I've read that thing over and over and I still write nonsense for specs at times š#2016-12-2402:40sophiagoso i should just be able to call (s/exercise
add)`?#2016-12-2402:40sophiagooh that doesn't place nicely with slack#2016-12-2402:40sophiago(s/exercise `add)
#2016-12-2402:41seancorfieldIf your specs are generatable... which yours won't be since you can't generate from instance?
I believe...#2016-12-2402:41sophiagoah no wait#2016-12-2402:41sophiagos/exercise-fn
right?#2016-12-2402:41seancorfieldHmm, actually I'd have to try s/exercise
to check whether it only uses the :args
spec#2016-12-2402:43sophiagofor functions defined in protocols, but dispatched on defrecords not primitives so hopefully it should work#2016-12-2402:43seancorfieldAh, yes, it must generate data to even pass stuff into :args
...#2016-12-2402:44sophiagoah ok. so would i use s/gen
for the list of types?#2016-12-2402:48seancorfieldYou'll need to write generators for your rational and complex types I suspect...#2016-12-2402:49seancorfieldWhat happens if you try (s/exercise ::rational)
?#2016-12-2402:52sophiagoi get the same error: "Unable to construct gen..."#2016-12-2402:58seancorfieldRight... you need something like this: (defrecord Foo [a])
(require '[clojure.spec :as s] '[clojure.spec.gen :as gen])
(s/def ::foo (s/with-gen #(instance? Foo %)
(fn [] (gen/fmap ->Foo (s/gen int?)))))
(s/exercise ::foo)
#2016-12-2402:59seancorfieldBasically you need to specify a generator that maps your record constructor over a generator for the values it accepts#2016-12-2402:59sophiagoah i see#2016-12-2402:59sophiagoi truly did make this much more difficult by trying to go that granular#2016-12-2403:00sophiagoit wasn't something obvious in the guide i missed going over this all day#2016-12-2403:00sophiagook, thanks. going to give it a try#2016-12-2403:01seancorfieldYeah, that was my first thought... a very ambitious spec! https://clojurians.slack.com/archives/clojure-spec/p1482548398003597#2016-12-2403:02seancorfieldMaking specs generative can be quite the process... we try to do that with all of ours but sometimes it takes... ingenuity...#2016-12-2403:06sophiagowell i was trying to get it to somehow enumerate tests on the correct output types automatically...#2016-12-2403:06sophiagothis is actually much simpler than i imagined#2016-12-2403:06sophiagobut still not covered in the guide#2016-12-2403:07seancorfieldGary Fredericks has created a good companion library for spec that helps generate all sorts of things BTW (we use it, specifically, for regex generation)#2016-12-2403:07sophiagooh i think i saw that#2016-12-2403:08sophiagotest.chuck#2016-12-2403:08sophiagoi just glanced over the functions and don't think it would help in this case though#2016-12-2403:09sophiagotbh this is not so complicated. protocols are generally really easy to debug#2016-12-2403:09sophiagoi actually thought since it's so systematic it would be good to learn how to spec a whole project so i could apply it to code where i really can't figure out what's going on#2016-12-2403:14seancorfieldFYI, s/exercise-fn
doesn't check :ret
-- only :args
#2016-12-2403:15seancorfield(defn foo [a b] (+ a b))
(s/fdef foo :args (s/cat :a int? :b int?) :ret string?)
(s/exercise-fn `foo)
doesn't complain that foo
is supposed to return string?
values#2016-12-2403:28sophiagowell i passed it the whole list anyway š#2016-12-2403:28sophiagoi'll have to check it manually#2016-12-2403:29sophiagobut it will tell me if any subtyping combinations are throwing errors, which i believe was the case before#2016-12-2403:34sophiagohuh. i'm getting the same error...#2016-12-2410:25sveriI get this error: RuntimeException Var clojure.test.check.generators/keyword-ns is not on the classpath clojure.spec.gen/dynaload (gen.clj:21)
when running this code: (gen/generate (s/gen number?))
in the REPL. Any ideas what I am missing here?#2016-12-2411:59gfredericks@sveri do you have a test.check dependency declared?#2016-12-2411:59sveri@gfredericks No, I dont think so#2016-12-2411:59gfredericksyou need one to use generators#2016-12-2411:59sveriSo this is needed whenever I want to use generators#2016-12-2411:59sveriok š#2016-12-2411:59gfredericks[org.clojure/test.check "0.9.0"]
#2016-12-2411:59sveriThank you very much#2016-12-2411:59gfredericksnp#2016-12-2418:30sophiagowhat is clojure.spec/Specize
? i'm getting the following error: IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil
#2016-12-2418:34donaldballIIRC itās the protocol by which values are coerced into specs, notably by which keywords resolve themselves in the spec registry#2016-12-2418:48sophiagoah, so it does have to do with trying to exercise protocol functions? because i think i may be experiencing this issue, but am having trouble isolating it to be sure: http://dev.clojure.org/jira/browse/CLJ-1941#2016-12-2418:49sophiagoto be fair, that's the not the error reported in that ticket#2016-12-2418:53sophiagomy case is complicated because the protocol functions i'd like to test all dispatch on records so theoretically should be unaffected. however, i went so granular as to define arguments to their type constructors using s/with-gen
and gen/fmap
. so i can't tell if some of the errors are actually what i'm looking for, known subtyping bugs, because i'm not sure if the generators that don't do that are affected and am currently getting three different error codes dependent on whether the arguments to the type constuctors are primitives or other records and also (the really odd part) the function itself, which shouldn't make a difference as far as i can tell#2016-12-2509:18naomarik@bbloom > whatās the guidance for spec-ing pseudo-namespaced keywords? iām thinking like datomic attributes when you do :customer/name
or something like that
Iāve actually been using it like this pretty much exclusively and I feel reading and referring to the spec becomes pretty trivial especially when you have multiple types that have the same name but the namespace disambiguates it (like multiple name
fields for instance :customer/name :business/name
)#2016-12-2510:03dvcrnHey guys! Using clojure-spec for the first time to describe a db schema. Does this look okay or am I doing something obvious wrong?
(s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::username string?)
(s/def ::picture (s/nilable string?))
(s/def ::relationship-type int?)
(s/def ::friend (s/keys :req-un [::username ::last-name ::first-name ::picture ::id ::relationship-type]))
(s/def ::friends (s/coll-of ::friend :kind vector?))
#2016-12-2510:03dvcrn(no error, just want to check in)#2016-12-2510:04dvcrnI think I don't completely understand why :req-un
is necessary#2016-12-2510:29naomarik@dvcrn :req-un
so when you pass in the data to be validated you can do {:username "",
:last-name "",
:first-name "",
:picture "",
:relationship-type -1}
otherwise with :req youād have to qualify the keywords.#2016-12-2510:31naomarikalso try out (s/exercise ::friend)
to test out your specs š#2016-12-2510:34dvcrn@naomarik I can't follow. You mean with req-un I can leave out keys?#2016-12-2510:35naomarikwithout :req-un you do something like this: {:friend/username "",
:friend/last-name "",
:friend/first-name "",
:friend/picture "",
:friend/relationship-type -1}
#2016-12-2510:35dvcrnah, I see!#2016-12-2510:35naomarikso these are fully qualified keywords#2016-12-2510:35naomarikwhich is very useful imo when dealing with multiple entities in the same map#2016-12-2510:36naomarikhelps disambiguate whose ID or name youāre talking about#2016-12-2510:36dvcrnright š#2016-12-2510:37dvcrnso I guess fully qualified / namespaced keywords are better#2016-12-2510:38naomarikiām not qualified to give an answer on that but i can say at least in my extremely limited usage on an immature project it has helped me discern what data iām looking at#2016-12-2510:38dvcrnyeah completely makes sense#2016-12-2511:24noogaunqualified keywords would usually come from/go on wires#2016-12-2616:59roelofI have this error message : (clojure.spec.gen/generate (clojure.spec/gen :paintings2.routes.home/page))
RuntimeException Var clojure.test.check.generators/large-integer is not on the classpath clojure.spec.gen/dynaload (gen.clj:21)
#2016-12-2617:01roelofon this code : (ns paintings2.routes.home
(:require [paintings2.layout :as layout]
[compojure.core :refer [defroutes GET]]
[ring.util.http-response :as response]
[ :as io]
[compojure.route :refer [resources]]
[environ.core :refer [env]]
[paintings2.api-get :as api]
[clj-http.client :as client]
[clojure.spec :as s]))
(defn ->long [s] (try (Long/parseLong s) (catch Exception _ ::s/invalid)))
(s/def ::page (s/and (s/conformer ->long) (s/int-in 1 471)))
(s/def ::optional-page (s/nilable ::page))
(defn page-check [page] (let [page page page-num (or (s/conform ::optional-page page) 1)] page-num))
(s/fdef page-check
:args (string? (::page :page))
:ret (number? (::page :page))
:fn (s/valid? true? (::page :page) )
)
#2016-12-2617:01joshjonesand did you put [org.clojure/test.check "0.9.0"]
in your project dependencies
?#2016-12-2617:01roelofand I have this in my projects.clj : :profiles {:dev {:dependencies [[org.clojure/test.check "0.9.0"]
]}}
#2016-12-2617:02joshjonesWhy not put it in your main :dependencies
? Why in a profile?#2016-12-2617:02roelofsee my last response, @joshjones#2016-12-2617:02roelofo, then I misunderstood you#2016-12-2617:02roelofmoment, I will change it#2016-12-2617:03joshjones(defproject your-project "1.0"
:description "spec playground"
:url ""
:license {:name "Eclipse Public License"
:url ""}
:dependencies [[org.clojure/clojure "1.9.0-alpha14"]
[org.clojure/test.check "0.9.0"]])
#2016-12-2617:04roelofoke, now I see another error message : ExceptionInfo Unable to construct gen at: [] for: (conformer ->long) clojure.core/ex-info (core.clj:4725)
#2016-12-2617:06joshjonesnow you're getting somewhere -- i get why you want to use a conformer, but i've not used a conformer to generate data before, nor have i seen any info on doing so -- only seen info on conforming/validating using a conformer#2016-12-2617:08joshjonesso you just want to generate numbers in a certain range?#2016-12-2617:08roelofyes, I want to check if the url parameter is between the 1 and 471 or is not there#2016-12-2617:08roelofis the last case the pagenumber schould be set to 1#2016-12-2617:09joshjonesand the input can be either a string or a number, right?#2016-12-2617:11joshjonesthe way you've written the spec for ::page
, I do not think there's enough info there for it to do any data generation#2016-12-2617:12roelofoke, what I try to do it this : The user can use this url : http://localhost:3000/#2016-12-2617:12roelofand then my code schould take that the pagenumber is 1#2016-12-2617:13roelofor the user can use this url : http://localhost/?page=n#2016-12-2617:13roelofn will be a integer where my code makes a long of it#2016-12-2617:13roelof@joshjones so far, clear ?#2016-12-2617:14roelofNow I want to make a generator which makes several cases for n#2016-12-2617:15roelofso n could be "a" then the outcome is invalid or "124" then the outcome is 124 or "500" then the outcome is also invalid#2016-12-2617:15roelofso I want testcases that test this#2016-12-2617:16joshjonesok -- what you want to actually generate is the entire URL?#2016-12-2617:18joshjonesyou are using something already for routing, and for parsing the query string?#2016-12-2617:44joshjones(defn gen-page-num
[]
(gen/fmap #(str %) (s/gen (s/int-in 1 471))))
(s/def ::page
(s/spec (s/and (s/conformer ->long) (s/int-in 1 471))
:gen gen-page-num))
#2016-12-2617:45seancorfieldYou might want to keep this level of back and forth in the #beginners channel#2016-12-2617:57roelof@joshjones my code for the url parsing with the pagenumber could be find here : https://github.com/rwobben/paintings/blob/master/src/clj/paintings2/routes/home.clj#2016-12-2617:58roelof@seancorfield The last question is about spec because josh thinks that my fdef for testing and generating test-data is not good#2016-12-2617:59joshjones@roelof Use the above generator and revised def for ::page
and you'll be able to generate data using ::page
#2016-12-2618:01roelofand I can use that data for checking the page-check function#2016-12-2618:03roelof@joshjones Am I right that I cannot check what happens if a user enters false data like this : ' /?pag="a"#2016-12-2618:07seancorfield@roelof Although this is about clojure.spec
, there are still a lot of beginner-level issues at play here so Iād suggest keeping to the #beginners channel where folks have opted into helping folks with this level of discussion, rather than swamping the other channels with long back and forth about syntax and basic errors.#2016-12-2709:37curlyfryHi, I have the following scenario:
(s/def ::name string?)
(s/def ::person (s/keys :req-un [::name]))
(s/def ::name (s/and string? #(str/starts-with? % "Jean")))
(s/def ::french-person (s/keys :req-un [::name]))
I want to keep these specs in the same file, but I also want to disambiguate the different ::name specs, without giving them different names. Is there any way to do this? Something like ::french/name
vs just ::name
, but that isn't working.#2016-12-2710:13mpenet@curlyfry you ca use create-ns and alias for that#2016-12-2710:22curlyfry@mpenet Cool, thanks! Can I create a concatenation of the "current" ns with a symbol using create-ns
?#2016-12-2713:31roelofI hope this is not a beginner question.#2016-12-2713:31roelofI want to test a function that has as input something like this : [{ :name "Roelof", :place "internet"} {:name "Jane Doe" :place "unknown" }]
and as output {"Roelof", "Jane Doe"}
. If I want to test this , can I use the same idea as this : (def email-regex #"^[a-zA-Z0-9._%+-]
#2016-12-2713:33roelofor what must I fill in if a use a fdef with :args and :ret and :fn#2016-12-2716:47joshjonesPerhaps your question is constructed in a way that makes answering it a bit difficult. For example, āinput something like thisā is ambiguous ā are there always two maps in the input vector, or might there be more? And, do you really mean to have a single k/v pair map as the output, where both the key and value are strings? Or did you mean to make that a vector of the :name
s from the input? Itās clearing up this confusion that results in the things the admins donāt want in the channel, I think.
But hereās some info that might be valuable to anyone new to spec, just a sample of how to construct what youāre asking for. What you want to do is ask, āwhat am I trying to specā? In your case, itās a function which takes a vector of maps, and returns a vector of strings. Those are your :args
and :ret
, and a decent sanity check for :fn
is that the number of names you return should equal the number of maps you were given. So, hereās a spec based on that:#2016-12-2716:47joshjones(ns clj.core
(:require [clojure.spec :as s]
[clojure.spec.gen :as gen]
[clojure.spec.test :as stest]))
(defn get-names [v]
(reduce #(conj % (:name %2)) [] v))
(s/def ::name string?)
(s/def ::place string?)
(s/def ::get-names-args (s/coll-of
(s/keys :req-un [::name ::place])
:kind vector?
:gen-max 5))
(s/fdef get-names
:args (s/cat :v ::get-names-args)
:ret (s/coll-of string? :kind vector?)
:fn (fn [{{input-vector :v} :args, return-vector :ret}]
(= (count input-vector) (count return-vector))))
#2016-12-2716:48joshjonesYou can now exercise and check the function with:
(s/exercise-fn `get-names)
(stest/check `get-names {:clojure.spec.test.check/opts {:num-tests 100}})
#2016-12-2820:53joshjonesIām a little confused on fmap
vs bind
ā according to the docs for clojure.test.check.generators/bind
: Create a new generator that passes the result of gen into function k. k should return a new generator.
fmap
also takes a generator, and applies some function to it. The only difference I can see is that in bind
, the function is responsible for returning a generator, and in fmap
, the function just returns data.
Is there some other difference Iām not seeing?#2016-12-2821:07gfredericksThat's exactly the difference#2016-12-2821:08gfredericksbind is more powerful than fmap#2016-12-2821:20sophiagoso the idea is this is somewhat like Haskell with generators analogous to monads? that comparison doesn't make the most sense to me tho#2016-12-2821:56gfredericksGenerators are a monad, yes#2016-12-2821:56gfredericksI expect that's literally true in quickcheck, which test.check is largely based on#2016-12-2822:15sophiagoi suppose in the sense that they're bound to a certain type? but i don't see how the monad laws could even possibly apply to generators#2016-12-2822:34gfredericksThe first three make sense to me. I'll translate them to generators when I get to a keyboard#2016-12-2823:02gfredericksā I haven't grokked the fourth one but it probably works ĀÆ\(ć)/ĀÆ#2016-12-2823:02gfredericks@sophiago#2016-12-2823:05sophiagoi don't think that SO post is the best source. i've always only known three laws: left identity, right identity, and associativity. https://wiki.haskell.org/Monad_Laws#2016-12-2823:06gfrederickslooks like it just skips the 3rd one#2016-12-2823:06sophiagothe associativity one in your example is unclear to me#2016-12-2823:06gfredericksso the first two are equivalent#2016-12-2823:06sophiagothe first two make sense. i didn't know there was a gen/return
either!#2016-12-2823:07gfrederickswe could experiment to see if the two seem to do the same thing š#2016-12-2823:08sophiagoi'm confused about the purpose of gen/return
from reading the api docs#2016-12-2823:08gfrederickswhat it does, or when it's useful?#2016-12-2823:08sophiagojust what it does#2016-12-2823:09sophiagoi suppose returns a generator? lol#2016-12-2823:09sophiagoso maybe i should have asked the second question...#2016-12-2823:09sophiagolike in what case would you bind a generator and then return it?#2016-12-2823:12gfredericksbind+return is just fmap#2016-12-2823:12gfredericksso it's easier to use fmap#2016-12-2823:12gfredericksreturn is more often useful on its own#2016-12-2823:12gfredericksI'll go grep my source code to see when I use it#2016-12-2823:14gfrederickshere's an example that uses it as the base case in a recursive generator: https://github.com/gfredericks/chess-clj/blob/89b2e5543534f82915a1aab67e026a326a0ed0a5/test/com/gfredericks/chess/generators.clj#L194#2016-12-2823:16gfredericksthe associativity law looks right to me#2016-12-2823:16gfredericksit's just double-binding in two different ways; looks suspiciously trivial actually#2016-12-2823:16sophiagowoah, that's a really cool application of generators!#2016-12-2823:17gfredericksš#2016-12-2823:17sophiagoyes...because >>= is the bind operator š#2016-12-2823:18sophiagoso if you can associate that then you've met the third monad law...really the important one in that it lets you compose monads that encapsulate different typeclasses#2016-12-2823:19sophiagothis would make a good blog post or something to that effect#2016-12-2823:19gfredericksI would need an example of such a composition to see what that means#2016-12-2823:20sophiagoit doesn't have a corollary in dynamic typing. that's why i always thought algo.monad was quite odd...#2016-12-2823:20sophiagobut it makes more sense with generators because they do in a sense have type signatures#2016-12-2823:20gfredericksyeah I've never seen the point of a monad library in clojure#2016-12-2823:21sophiagoit's really just a library of macros that needlessly replicate the functionality of a number of common haskell monads#2016-12-2823:21gfrederickstoo much you can't do and not enough help from the compiler#2016-12-2823:22sophiagobut this makes sense. you need bind in order to compose generators with different types#2016-12-2823:24sophiagowow, now i'm really interested in uses of spec beyond things like quickcheck. i haven't seen other examples actually using generators to write code, although i think rich touched on it in his talk i saw#2016-12-2823:24gfredericksI'm not even sure what that means#2016-12-2823:24gfredericksgenerating code?#2016-12-2823:25sophiagowell, i only glossed over your chess example, but it seems you're using them to actually generate moves rather than to just test functions you've written?#2016-12-2823:27gfredericksno it's still for tests#2016-12-2823:27gfredericksit is generating moves, but only in service of generating legal positions, which is for testing#2016-12-2823:28sophiagoyeah i looked at it again. at first i thought you were using it to play chess#2016-12-2823:28gfredericksI don't think test.check generators are necessarily the best fit for non-testing purposes#2016-12-2823:28gfredericksgenerally they'll give you distributions that are a bit weird outside of testing#2016-12-2823:29sophiagoyeah, i've only noticed that now that i've started getting them working for whole projects...are they purposefully non-random or that's just a consequence of their implementation?#2016-12-2823:30gfredericksthey are random#2016-12-2823:30gfredericksjust skewed in their distribution, as a heuristic for catching bugs#2016-12-2823:32sophiagoi mean, for example, if i call (gen/sample (s/gen int?) n)
the size of the ints increase with n. initially they're quite small...which, as you said, makes sense for testing#2016-12-2823:32gfredericksyeah, that's half of the weirdness#2016-12-2823:33gfredericksthat's demonstrating that gen/sample
uses an increasing size
parameter#2016-12-2823:33sophiagooh#2016-12-2823:33gfredericksbut even if you fix size
, which you can do with gen/generate
, there are still elements of the "smaller things are worth trying more often" heuristic#2016-12-2823:34gfredericksmore about sizing: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2016-12-2823:36sophiagoah, these are the docs i should have been looking at this whole time! i've been using this: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html which just says everything is a wrapper of test.check macros#2016-12-2823:37gfredericksI'm trying to do a big documentation overhaul, so I'll try to look for ways to make discovery better too#2016-12-2823:41sophiagoyeah, your test.check docs are most useful for what i personally needed than most of what's in alex's guide. like i could have used gen/nat
earlier to avoid divide by zero errors and instead rolled my own#2016-12-2823:47sophiagoanyway, i think it'd be interesting to have something about the generator combinators and category theory. it's an interesting consequence of how they're typed that we're not used to in a dynamic language#2016-12-2823:48sophiagoi'd consider writing something like that for a wiki if i thought i could both make it accurate and substantial enough for anyone to care about š#2016-12-2823:50gfredericksthat would be fun to read#2016-12-2823:50sophiagomaybe...#2016-12-2907:00seako@sophiago carin meier wrote a blog post about using spec to generate code that you might find interesting http://gigasquidsoftware.com/blog/2016/07/18/genetic-programming-with-clojure-dot-spec/#2016-12-2907:08seakoalso, i would like chime in that i also think it would be very interesting to read about generator combinators and category theory#2016-12-2914:43stephenmhopperI have a spec that I've written with clojure.spec and I can use it to generate valid Clojure data structures. Can I use it to generate invalid structures?#2016-12-2914:45gfredericksgenerating things in a negative way is hard to do in general#2016-12-2914:45gfredericksI expect clojure.spec makes no attempt to support that, since it's not obvious what strategy it would use#2016-12-2914:46gfredericksyou can use unit testing for that, or custom generators to cover more specific cases#2016-12-2914:46gfredericksfor a lot of cases you might do fine by using gen/any
and filtering out things that match the spec#2016-12-2914:50stephenmhopperI did not know about gen/any
, but that should do the trick. I was considering just creating a version of my schema where all required keys are instead marked as optional while also changing the data type / generators for some of the keys, but gen/any
is simpler#2016-12-2914:50stephenmhopper@gfredericks thank you#2016-12-2914:51gfredericksnp#2016-12-2915:06naomarikare you guys converting database records field names to namespaced keywords?#2016-12-2915:07naomarikor just specing database stuff with :req-un
#2016-12-2915:13gfredericksI think converting would be cooler, but it takes more work and I haven't worked on something like that yet anyhow#2016-12-2915:14gfrederickse.g., a namespace per table#2016-12-2915:14naomarikyes this is what iād like#2016-12-2915:14naomariki spent some time specing all my stuff like that and while i can just use req-un i would rather see it qualified#2016-12-2915:15naomarikwould be great if i didnāt have to rename joined tables either, ie multiple id
fields#2016-12-2915:24gfredericksright#2016-12-2916:16naomarikthereās anyway to conform a spec given keys with :req-un
into the matched qualified keywords?#2016-12-2916:55joshjonesI do not think so; do you have a particular use case for this?#2016-12-2916:57seancorfieldFWIW clojure.java.jdbc supports producing qualified names via the :qualifier
option. Doesn't handle joins tho'. #2016-12-2916:59joshjonesYou could use s/describe
on the spec to get the vector of :req-un
s, and then pull the key with the name
of the key you want to qualify, grab its namespace
, and assoc a ānewly createdā fully qualified key onto the conformed data#2016-12-2917:00joshjonesIf you really wanted to. š#2016-12-2917:13naomarik@seancorfield ya saw that, will probably do that for non joined data and use an identifier that will handle select AS
statements#2016-12-2917:41joshjonesjust as an exercise @naomarik :
(defn qualify-keys
[spec conformed]
(let [unq-to-qual (as-> (s/describe spec) $
(second (drop-while (partial not= :req-un) $))
(zipmap (map name $) $)
(reduce-kv #(assoc %1 (keyword %2) %3) {} $))]
(reduce-kv #(assoc (dissoc %1 %2) %3 (get %1 %2))
conformed unq-to-qual)))
(s/def :ns.one/requnkey string?)
(s/def :ns.two/reqkey string?)
(s/def ::mymap (s/keys :req-un [:ns.one/requnkey] :req [:ns.two/reqkey]))
(s/conform ::mymap {:ns.two/reqkey "required!" :requnkey "not required!"})
=> {:ns.two/reqkey "required!", :requnkey "not required!"}
(qualify-keys ::mymap *1)
=> {:ns.two/reqkey "required!", :ns.one/requnkey "not required!"}
#2016-12-2918:08joshjones@seancorfield you just mentioned in the #clojure channel about using with-redefs
to stub functions for testing; whatās your take on taking a specād function and now being able to do this, versus the with-redefs
approach?:
(stest/instrument `my-func {:stub #{`my-func}})
#2016-12-2918:09seancorfield@joshjones I havenāt experimented with stubs in clojure.spec
yet so I donāt have an opinion.#2016-12-2918:10seancorfieldMostly weāre specāing data structures and using s/conform
etc ā weāre not specāing functions much.#2016-12-2918:14joshjonesok ā I have not tried it for production-level testing either, but it looks useful:
(defn get-data-from-db [] nil)
(s/fdef get-data-from-db
:args empty?
:ret (s/int-in 1 10))
(stest/instrument `get-data-from-db {:stub #{`get-data-from-db}})
(get-data-from-db)
=> 6
(get-data-from-db)
=> 7
#2016-12-2918:17seancorfieldSo it stubs the named function to return generated data per spec rather than actually calling it? Interesting. Wouldnāt work for anything that actually required mock state, rather than just stubbed data, but I can see that being useful sometimes.#2016-12-2918:18seancorfieldI havenāt generally found stubs to be as useful as mocks so I donāt know how much Iād use it.#2016-12-2918:18joshjonesright, thatās a good point#2016-12-2918:19seancorfieldI can see it being useful as youāre developing top-down where you stub functions as placeholders until you actually write them.#2016-12-2918:20seancorfieldRight now, Iād actually write the physical stubā¦ with the spec approach you could just get instrument
to āwrite the stubā for you, but then youād have to re-instrument or un-instrument as you actually wrote each function.#2016-12-2918:21joshjonesis there something you typically use for mocking state? or it just depends on your particular application?#2016-12-2918:21roelofHow can i check in a spec for a key in a nested map ? so this is valid (s/explain ::objectNumber-list [{:body{:artObjects{:objectNumber "sk-c-5"}}}])
#2016-12-2918:21seancorfieldSince mocks tend to be pretty tied to your implementation, I generally just hand-roll them.#2016-12-2918:22seancorfield@roelof Please keep working in the #beginners channel on that.#2016-12-2918:22seancorfieldYou need to think carefully about the elements of that list, is all Iāll say.#2016-12-2918:24seancorfield@joshjones When we mock things, we tend to need to coordinate data across a series of calls ā or even just record data across a series of calls ā so custom functions that use atoms or refs to track that is our āgo toā approach.#2016-12-2918:25seancorfieldFor example, we mock a payment provider to track calls and amounts, and pass or fail the transaction based on state.#2016-12-2918:25joshjonesI guess the amount of data you mock is pretty small though right? i.e., the purpose of the mocking at this stage is not for load-testing, which is done in a completely different way.#2016-12-2918:27joshjonesfor example, we load test our elasticsearch code using dedicated aws servers, so itās purely a load test ā for testing correctness, etc., small amounts of mock data is used locally to be sure everything works as expected.#2016-12-2918:29seancorfieldMocking is to verify that a function calls some other function(s) in a particular way and/or that a series of calls to the mocked functions returns data in a particular way.#2016-12-2920:23henriklundahl@roelof, you spec each value which is associated with a specific key and then specify which keys should be included in each map. Start with the innermost map(s) and work your way out.#2016-12-3009:00sophiago@seako thanks! i've been busy so just getting around to reading this, but it looks really interesting!#2016-12-3009:03sophiagoand i'll think a little more about writing the post. it's not so complicated from the category theory side, so i don't think i need to worry about my level of knowledge with that...more so with test.check. maybe if i've worked with it a bunch more by the time @gfredericks revamps his docs i can add it to the wiki there#2016-12-3100:11sophiagowow, so i finally got around to going through this: https://github.com/gigasquid/genetic-programming-spec and now i'm wondering how it could be used for actual testing#2016-12-3100:14sophiagofor example, in the program i just specced out i originally wanted to actually match the correctness of args and return values, which would have meant a quadratic number of sfdefs
. i had a list of i think 11 types and five functions...so if i'm doing the math right that would have meant 605 different definitions? now i'm thinking it could actually be possible to enumerate that entire testing space if i learned to have the specs generate themselves... something to think about#2016-12-3122:09bbloomis there some way to get more info from this error: "Couldn't satisfy such-that predicateā ? the ex-info map is empty#2016-12-3122:09bbloomiām curious what predicate is failing#2016-12-3122:09gfredericksit's um#2016-12-3122:10gfredericksprobably from an s/and
somewhere#2016-12-3122:10gfrederickstest.check master has a mechanism for spec to add more info in this case, but spec would have to be updated as well#2016-12-3122:10gfredericksit's something rich asked for#2016-12-3122:11bbloomok cool - will just bin-search it like i do practically all my debugging š#2016-12-3122:11gfredericksyep š¢#2017-01-0111:31tianshudoes clojure.spec.test/instrument support check :ret
and :fn
for common function call?#2017-01-0111:31tianshucurrently it seems only work for test.check.#2017-01-0116:22settingheadI noticed spec tolerates ambiguity (say (s/cat :first (s/* number?) :second (s/* number?))
can interpret [1 2 3]
in 4 different ways, but it returns the first way it could find). is there any way to disallow ambiguity?#2017-01-0116:54english@doglooksgood from the spec guide: "Note that the :ret and :fn specs are not checked with instrumentation as validating the implementation should occur at testing time."#2017-01-0118:45mingpThose of you who use a lot of spec, what would you say to concerns that spec makes you code a certain way to be able to work with it, e.g. namespaced keywords for map keys? This is something that has worried me and made me hesitant to start using spec, but of course I say this as someone who hasn't used it, so what would those of you who have say?#2017-01-0118:46mingpSomething else I've been wondering. To what extent are you all using function specs vs explicit conform
? Do you leave both/either on in production?#2017-01-0118:47notanon@mingp what is the concern with namespaced keywords?#2017-01-1917:30lmergen@seancorfield: i would be very interested to learn how you use spec in real world projects #2017-01-1917:30lmergenwe're introducing it right now, running alpha 14#2017-01-1917:31lmergenand it's a bit tough to find the right "pattern" that makes sense to us#2017-01-1917:31seancorfieldFor example, we have API input specs that accept either a date, or a string that can be parsed to a date using two allowed formats. s/valid?
on such a string will just return true
(and we still have a string; or false
for an illegal string). s/conform
on such a string will return a date (or ::s/invalid
for an illegal string).#2017-01-1917:33joshjonesso in this case is it accurate to say that you are using spec to actually transform the data? (string -> date)#2017-01-1917:34seancorfieldAnother use case: we allow members to search within specific distances (dating site). If they are a subscriber, they can specify distances of 20, 30, 40, 50. All members can specify distances of 150, 500, 1000, and -1 (āanyā). We use a spec with s/or
to get back either [:basic distance]
or [:platinum distance]
so we can validate both the actual value and which category it belongs to.#2017-01-1917:35seancorfield@joshjones Yes, some coercions. As Alex often cautions, that conforming / coercing means all clients of those specs get the coercion if they use s/conform
but in this case thatās what we want.#2017-01-1917:35ddellacostato me that seems like a spec is somewhere in between a value and a statement about a value#2017-01-1917:35ddellacostaI find this pretty confusing, fundamentally#2017-01-1917:35ddellacostaseems like it makes s/conform
of limited use#2017-01-1917:35ddellacostaor at least, limited in the case of using s/or
#2017-01-1917:36joshjonesyes yesterday @alexmiller said: āspec works best when you primarily use it to speak truthful, assertive statements about what your data is
and not to transform your data or make statements about what your data is notā
https://clojurians.slack.com/archives/clojure-spec/p1484765212007116 https://clojurians.slack.com/archives/clojure-spec/p1484765279007117#2017-01-1917:36ddellacostaagain, Iām talking about s/conform
here @joshjones#2017-01-1917:37ddellacostaand it seems to violate exactly what those comments are saying, in fact#2017-01-1917:37seancorfieldIf you need to know which āpathā of conformance your value takes, s/or
is very useful. Since otherwise youād need to code up both the validation logic and the parsing logic in order to determine that.#2017-01-1917:37joshjonesIām not referring to anything youāre talking about @ddellacosta , only to what @seancorfield said about his use of spec#2017-01-1917:37ddellacosta@joshjones ah, apologiesāmisunderstood#2017-01-1917:37joshjonesno problem, I am also seeking a better understanding of best practice š:skin-tone-2:#2017-01-1917:40ddellacostayeah, along those lines this is all really helpfulāthanks for linking to those comments @joshjones#2017-01-1917:42seancorfieldItās taken us a while to settle on this approach ā given that Alex has repeatedly cautioned against specs that coerce data š ā and we also have the added complication that for a given API, some of our input parameters must be looked up and āconformedā dynamically (i.e., we have a database table listing possible inputs and each row identifies a spec that should be applied). So we canāt just write static specs for input arguments, we have to have somewhat generic specs and conform each input argument based on a lookup.#2017-01-1917:43seancorfield(and weāre still wading through cleaning that up)#2017-01-1917:43seancorfieldWe could probably simplify this by dynamically generating specs directly from our metadata at this point ā and we may well do so.#2017-01-1917:44ddellacostaso question for you, @seancorfield āwhy not simply write something explicit that does not rely on spec to add the metadata about your distance (platinum vs. basic)? It seems to me that this is a value, not a spec.#2017-01-1917:44ddellacostaIām asking selfishly to clarify my understanding of spec, to see how you are thinking about this, as I still donāt fully get it#2017-01-1917:45seancorfieldBecause using spec allows us to remove a lot of logic that we had before.#2017-01-1917:47seancorfield(s/def ::platinum-distance #{20 30 40 50})
(s/def ::basic-distance #{150 500 1000 -1})
(s/def ::distance (api-spec ->long (s/or :basic ::basic-distance
:platinum ::platinum-distance)))
(and the api-spec
macro) replaces a bunch of explicit code that attempted to coerce string to number and validate it and categorize it as basic vs platinum. Now we can just have (let [[level distance] (s/conform ::distance input)] ā¦)
and weāre done.#2017-01-1917:48seancorfieldHaving the declarative specs means we can show those to business and they can āread the codeā without having to actually read real code full of conditionals and transformations.#2017-01-1917:49seancorfieldWe can also have the business rules in one place (for the most part) and they drive not only our actual business logic but can also be used to generate test data (and test functions extensively).#2017-01-1918:46ddellacostaokay, Iāll have to think about that a bit. Thanks a lot @seancorfield , this has been very helpful#2017-01-1920:50scriptorhow would one spec varargs?#2017-01-1920:51scriptorcould I just use the vararg's name and use coll-of?#2017-01-1920:55schmeescriptor I think the idea is to spec the args as a sequence with spec/cat
and use spec/?
for the optional args#2017-01-1920:56joshjones@scriptor yes, compose the var args the same way youād compose any sequence.. only it doesnāt have to be coll-of
ā¦ can be k/v pairs, any sequential spec. for example:
(s/def ::someopt nat-int?)
(s/def ::vargs (s/cat :optargs (s/keys* :opt-un [::someopt])))
(defn myfunc [& optargs] nil)
(s/fdef myfunc
:args ::vargs)
(stest/instrument `myfunc)
(myfunc :someopt 55 :otheropt 99)
#2017-01-1920:56joshjonescan be a coll-of
, use spec regexes, anything ā¦ the above just shows a common use case for var args: k/v pairs in a sequence (as opposed to a map)#2017-01-1920:59scriptorso :rest
indicates the following spec is for the optargs
parameter?#2017-01-1920:59scriptorwell, not indicates, you're just using the name :rest
#2017-01-1921:00joshjonesyes, should probably have called that optargs
.. will change it for clarity#2017-01-1921:00scriptorperfect, thanks @schmee and @joshjones#2017-01-1921:24joshjones@scriptor I need to clarify something after doing another example ā my example above works but I wanted to use another common case because I was wrong about just using coll-of
for cases where you have one required and then an optional argument. for example:
(s/def ::vargs (s/cat :req string?
:optargs (s/* int?)))
#2017-01-1921:24joshjonesif you use coll-of
for :optargs
it will not work because youāre giving it multiple items, not an actual collection#2017-01-1921:25scriptorah, interesting, I guess it's because of the regexy nature of s/cat#2017-01-1921:26joshjoneswell, s/cat gives the actual argument vector spec. but inside that, you then have several elements, and one of those is not a collection if itās for example: [āabcā 42 43 44]
#2017-01-1921:26scriptorthe s/* greedily consumes the rest of the arglist, which as you said is multiple separate items and not a single collection#2017-01-1921:26joshjoneswhat you have there is a string, an int, an int, and an int .. no collections#2017-01-1921:27joshjonesyes thatās correct. i feel my answer above was misleading so i wanted to clarify#2017-01-1921:43joshjonesanother example that will match (fn [s & opts])
:
[āabcā]
[āabcā 3/4]
[āabcā 3/4 42.9]
[āabcā 3/4 33.3 10.5 99.9]
(s/def ::vargs
(s/cat :req string?
:optargs (s/? (s/alt :ratioonly ratio?
:ratio-double (s/cat :s ratio?
:i (s/+ double?))))))
#2017-01-1921:44joshjoneswanted to show that the optional argument portion of the sequence can be more than just a simple homogeneous concat of elements#2017-01-1921:54Alex Miller (Clojure team)you should really use s/keys*
for this - that is itās reason for existence#2017-01-1921:55Alex Miller (Clojure team)(s/def ::varargs (s/cat :req string? (s/keys* :opt-un [::ratioonly ::ratio-double])))
etc#2017-01-1922:06joshjones@alexmiller that was in my first example above ā but doesnāt that require k / v pairs?#2017-01-1922:09Alex Miller (Clojure team)yes, isnāt that what youāre specing?#2017-01-1922:10Alex Miller (Clojure team)oh, I see you have a :ratio-double with multiple vals#2017-01-1922:10joshjonesno, in this case i was showing a spec that would match all of the following:
(myfunc ārequired stringā)
(myfunc ārequired stringā 4/3)
(myfunc ārequired stringā 4/3 33.2 99.9)
#2017-01-1922:11Alex Miller (Clojure team)Iām going to go out on a limb and say thatās weird :)#2017-01-1922:11Alex Miller (Clojure team)and you shouldnāt write a function signature that way in the first place :)#2017-01-1922:11joshjonesyes it is ā my first example showed what you said ā i was just showing how the optional arguments portion of a function need not be a homogeneous group of items .. not endorsing it, just demonstrating the versatility š#2017-01-1922:12Alex Miller (Clojure team)ok#2017-01-1922:12joshjonessaying āhey, if it can do this, it can definitely do something much simplerā š:skin-tone-2:#2017-01-1922:32gdeer81is there a way to do multiple specs at once like you have a bunch of things that should be a string so instead of doing (s/def ::foo string?) (s/def ::bar string?) you can just say [::foo ::bar :baz ::fname ::lname ::email] string?#2017-01-1922:54bbloomhas somebody done a richer explain function yet? i find that when you have some ors or alts w/ larger maps involved, the output becomes unwieldy#2017-01-1923:01Alex Miller (Clojure team)@gdeer81: no#2017-01-1923:02Alex Miller (Clojure team)@bbloom Drogalis started porting some of his onyx stuff to work over spec explain-data (and using fipp) - really more of a poc stage#2017-01-1923:02bbloomcool#2017-01-1923:02bbloomš#2017-01-1923:02Alex Miller (Clojure team)I actually picked that up and hacked on it a bit but found that explain-printers are missing some crucial data#2017-01-1923:02Alex Miller (Clojure team)namely, the original root value and root spec#2017-01-1923:02Alex Miller (Clojure team)thatās missing from the explain-data map#2017-01-1923:03Alex Miller (Clojure team)I have a pending enhancement + patch to add that#2017-01-1923:03bbloomah yes, those things and the immediate parent or some chain of anscestors should be passed down#2017-01-1923:03bbloomthis way explainers can be context sensitive#2017-01-1923:03Alex Miller (Clojure team)everything else is discoverable within the explain-data path#2017-01-1923:03bbloomah right, can just pop on the path#2017-01-1923:03bbloomthat makes sense#2017-01-1923:03Alex Miller (Clojure team)but you donāt have the root value#2017-01-1923:03bbloom(get-in root (pop path))#2017-01-1923:04bbloomgotcha. well looking forward to that š#2017-01-1923:04bbloomthanks#2017-01-1923:04Alex Miller (Clojure team)https://github.com/onyx-platform/empathy is Mikeās stuff#2017-01-1923:04bbloomlol @michaeldrogalis love the name#2017-01-1923:04Alex Miller (Clojure team):)#2017-01-1923:04bbloomthis brings up one other issue that iām running in to now#2017-01-1923:04michaeldrogalisHeh. Never did get around to finishing that.#2017-01-1923:04bbloomin fact, itās one that i ran in to in the past#2017-01-1923:04Alex Miller (Clojure team)Iāve done some more work on it#2017-01-1923:05Alex Miller (Clojure team)but not in a public repo#2017-01-1923:05bbloomhttps://github.com/plumatic/schema/pull/134#2017-01-1923:05Alex Miller (Clojure team)Iāve shelved it for the moment until the patch above gets through as then it will be easier to demo#2017-01-1923:05bbloomiām running in to a situation now where i have a recursive spec that is giving me useless explain output b/c itās producing LOTS of output#2017-01-1923:06bbloomb/c the recursive spec fails to match#2017-01-1923:06bbloomone thing you could do with explain-data is find the CLOSEST match#2017-01-1923:06michaeldrogalis@alexmiller Curious as to how you attempted it. Took me ages to do it with Schema, not really looking forward to trying it again with Spec.#2017-01-1923:06Alex Miller (Clojure team)I seem to recall Colin made something similar in the macro grammar stuff he did#2017-01-1923:07Alex Miller (Clojure team)^^ @bbloom#2017-01-1923:07bbloomyeah, itās kinda critical i think#2017-01-1923:07bbloomotherwise spec falls down on ASTs and other recursive structures after data gets too big#2017-01-1923:07Alex Miller (Clojure team)@michaeldrogalis well, I mostly brought it up to more recent spec and extended it just slightly further#2017-01-1923:07bbloomat least explain does#2017-01-1923:08michaeldrogalisAh, got it.#2017-01-1923:08bbloomunfortunately the literature on error detection and recovery in parsers seems to be kinda ad-hoc#2017-01-1923:08Alex Miller (Clojure team)@bbloom I also have a patch that sorts the longest errors first, which generally is a good strategy#2017-01-1923:08bbloomyeah, thatās a good start#2017-01-1923:08Alex Miller (Clojure team)except with recursion when you probably want the reverse :)#2017-01-1923:08bbloomheh#2017-01-1923:09bbloomis that longest path to an error?#2017-01-1923:09Alex Miller (Clojure team)yeah#2017-01-1923:09bbloomor just BIGGEST error?#2017-01-1923:09bbloomok#2017-01-1923:09bbloomi asked about this on twitter a while ago in the context of text parsing#2017-01-1923:10bbloomhttps://twitter.com/jordwalke/status/805921251004821504#2017-01-1923:10bbloomreference manual is here: http://gallium.inria.fr/~fpottier/menhir/manual.pdf#2017-01-1923:10bbloomchapter 10 is about error handling, which may be worth studying#2017-01-1923:10bbloomthere seems to be some attempt at a sensible system for producing good error messages#2017-01-1923:11Alex Miller (Clojure team)I think cleaning up fanout is another big area for improvement #2017-01-1923:11Alex Miller (Clojure team)For removing repetition#2017-01-1923:11bbloomyou mean like representing the output as a tree?#2017-01-1923:12Alex Miller (Clojure team)Nah, just reporting all errors at common path with single context#2017-01-1923:12bbloomah ok#2017-01-1923:12Alex Miller (Clojure team)An ide might do a tree though#2017-01-1923:12bbloomanother idea is a sort of two column view#2017-01-1923:12bbloomie pretty print (or just indent) on the left#2017-01-1923:12bbloomand messages on the right#2017-01-1923:13Alex Miller (Clojure team)I suspect recursion will just suck regardless :)#2017-01-1923:13bbloomyeah, well recursion always sucks when it comes to printing stuff š#2017-01-1923:13bbloomas a man who does a lot of recursion and a lot of printing, believe me, i know#2017-01-1923:13bbloomheh#2017-01-1923:13bbloomwhatever happened to that data structure visualizer? is that still built in? lol#2017-01-1923:13Alex Miller (Clojure team)it is :)#2017-01-1923:14Alex Miller (Clojure team)it still doesnāt work very well :)#2017-01-1923:14bbloomyeah, thatās the thing about text based UIs#2017-01-1923:14bbloomwhen they are bad, they are kinda bad#2017-01-1923:14bbloomvs guis#2017-01-1923:14bbloomwhen they are bad, THEY ARE VERY BAD#2017-01-1923:14Alex Miller (Clojure team)gotta run, later#2017-01-1923:14bbloomcya#2017-01-2001:23bbloomjust curious: why does s/keys check the map? predicate? seems like ILookup is the minimum requirement, or am i missing something?#2017-01-2001:26seancorfieldI believe there's a JIRA issue open for that...#2017-01-2001:27bbloomfound it: http://dev.clojure.org/jira/browse/CLJ-2041 thx#2017-01-2003:10Alex Miller (Clojure team)It has to iterate through the entries so ILookup is not sufficient#2017-01-2003:31bbloomMakes sense. I forgot about how it handles present keys even if not specified as optional.#2017-01-2003:33bbloomOne solution would be to only provide that functionality if iterable or seqable. Not sure if that's a great idea though.#2017-01-2009:26joost-diepenmaatmaybe this is a FAQ, but is there a way to automatically re-run clojure.spec/instrument after defining new functions or redefining already specced ones? Iād be fine if it would work in CIDER only.#2017-01-2009:52souenzzoCan I use it in "production"? Or call s/conform at beginning of function?
https://clojure.org/guides/spec#_instrumentation#2017-01-2009:56mpenetinstrumentation is not meant to be used in production#2017-01-2009:56mpenetit's actually spelled out explicitly on that page#2017-01-2009:57mpenetas for conform, yes you can#2017-01-2009:59mpenetIn production what I do is leverage :pre/:post + s/valid? in functions. and you can always turn it off with a compile time toggle. It works quite well#2017-01-2014:49timgilbertHey, I have a kind of polymorphic map where some keys should be present depending on the value of one of the other keys, so it's like either {:x/type :t/foo :foo/val 32}
or {:x/type :t/bar :bar/baz "47"}
#2017-01-2014:49timgilbertIt seems like this is difficult to describe with spec since the map specs only look at keys, not values. Am I missing something?#2017-01-2014:50potetm@timgilbert https://clojure.org/guides/spec#_multi_spec#2017-01-2014:51potetmIf I understand right, that's what you're looking for.#2017-01-2014:51timgilbertAha! Exactly what I was looking for. Thanks @potetm.#2017-01-2014:52potetmnp š#2017-01-2021:14viestiPosed a question in the general Clojure channel, but realised that I need to think about predicates that reference their environment a little more. This was the snippet that I was pondering: user> (let [limit 100] (s/form (s/spec #(< % limit))))
(clojure.core/fn [%] (clojure.core/< % limit))
#2017-01-2021:14viestispecifically the limit
in there#2017-01-2021:15bbloomsadly, clojureās quotations donāt capture environments#2017-01-2021:15bbloomthereās really nothing you can do other than eager substitution#2017-01-2021:16bbloomie a macro#2017-01-2021:16viestiwas using s/form for unit tests on spec that I generate from data#2017-01-2021:17Alex Miller (Clojure team)I just answered in #clojure#2017-01-2021:17viestithinking that the serialization thing is probably still on itās way to clojure.spec#2017-01-2021:17Alex Miller (Clojure team)s/form is there for that purpose (with some known bugs)#2017-01-2021:18viestithanks for the answer @alexmiller and sorry for posting on wrong channel š#2017-01-2021:18tbaldridgeserialization with captured environments can get a bit tricky anyways.#2017-01-2021:19bbloomindeed#2017-01-2021:19bbloomitās among the crazy ideas Iāve been studying for a number of years š#2017-01-2021:20viestiš#2017-01-2021:45viestiwas actually using s/form for unit testing specs that I generate from sql ddl statements, just toying around for now#2017-01-2208:32andrewzhurov#2017-01-2208:32andrewzhurov#2017-01-2208:32andrewzhurovfirst#2017-01-2208:33andrewzhurovI more than sure that those questions appear often here, so, some links ?#2017-01-2214:45settingheadfor those interested in seeing another javascript port of clojure.spec (in additon to js.spec), here is one I've made: https://github.com/clausejs/clausejs. Regex specs and fspecs are currently working. Data generation comes next.
I've also made a spec documentation generator called clausejs-docgen
(still WIP), which I have been using to generate Clause.js's own documentation.
Lastly, there is a syntax
function which takes in an fspec, enumerates over all of its arguments' possibilities, and outputs a concise list of lambda-ish syntax references (you can find examples in the doc site here: https://clause.js.org)#2017-01-2214:46Yehonathan Sharvitmetalclj#2017-01-2223:24mrgDoes anyone have any examples on how clojure.spec/conformer works? I am trying to coerce a json map (including UUIDs and instants, and seqs of keywords) and I cannot find any good examples anywhere#2017-01-2223:46Oliver GeorgeI don't think it's really intended to use conform in that way#2017-01-2223:49Oliver Georgeas best I understand it, spec is intended to describe static data rather than the coerce data#2017-01-2223:50Oliver GeorgeTechnically that looks a bit like coercion in that the conformed data is a different shape. #2017-01-2223:50Oliver GeorgeHope that helps. #2017-01-2223:55seancorfield@mrg I answered in #clojure but I'll respond to @olivergeorge's comment: yes, you generally want to be very wary of specs that coerce data because then every "client" of that spec has to accept the coercion. So it's only good for a spec where you always want all "clients" to see the same coercion too.#2017-01-2300:12mrgThanks @olivergeorge and @seancorfield . I was basing that off a comment of Timothy Baldridge on reddit that was suggesting that conformer was the way to go if one wanted to use a spec to coerce data#2017-01-2300:12mrgWhat would be the right way to do something like that then though?#2017-01-2300:13mrgSay I have a spec (s/def ::myspec #{:foo :bar})
#2017-01-2300:13mrgand then i have a deeply nested json map that at multiple places uses ::myspec#2017-01-2300:14seancorfieldWell, yes, s/conformer
is the way to go -- but it's a "concern" to have a spec do data transformation.#2017-01-2300:14mrgsorry, there would also be (s/def ::myspecs (s/coll-of ::myspec :min-elements 1 :max-elements 5))
or something like that#2017-01-2300:14mrgI don't really like it either#2017-01-2300:15mrgBut I can't think of a better way that doesn't have me re-implement half of spec for every branch of the tree#2017-01-2300:16seancorfieldI'm not sure what you're asking -- and I think you're misunderstanding what we're saying...?#2017-01-2300:16mrgQuite likely#2017-01-2300:17mrgMy actual use case is this: I have a service that 1) generates random data that conforms to a spec and spits it out as json 2) is supposed to take such a json map and check its conformance to the spec#2017-01-2300:19mrg1) works mostly, using generators and gen/generate#2017-01-2300:19mrgMy problem is in step #2 when I try to read the generated json back in#2017-01-2300:20mrgthe uuids have become strings, the enums (like the ::myspec) above as well, and so have the #inst datetimes#2017-01-2300:20mrgAnd I can't figure out how to coerce them back into the format that my spec expects#2017-01-2300:32seancorfieldHere's an example of some specs etc we use that accept strings and coerce them to dates: (defn coerce->date
"Given a string or date, produce a date, or throw an exception.
Low level utility used by spec predicates to accept either a
date or a string that can be converted to a date."
[s]
(if (instance? java.util.Date s)
s
(-> (tf/formatter t/utc "yyyy/MM/dd" "MM/dd/yyyy"
"EEE MMM dd HH:mm:ss zzz yyyy")
(tf/parse s)
(tc/to-date))))
(defn ->date
"Spec predicate: conform to Date else invalid."
[s]
(try (coerce->date s)
(catch Exception _ ::s/invalid)))
(defmacro api-spec
"Given a coercion function and a predicate / spec, produce a
spec that accepts strings that can be coerced to a value that
satisfies the predicate / spec, and will also generate strings
that conform to the given spec."
[coerce str-or-spec & [spec]]
(let [[to-str spec] (if spec [str-or-spec spec] [str str-or-spec])]
`(s/with-gen (s/and (s/conformer ~coerce) ~spec)
(fn [] (g/fmap ~to-str (s/gen ~spec))))))
(s/def ::dateofbirth (api-spec ->date #(dt/format-date % "MM/dd/yyyy") inst?))
#2017-01-2300:33seancorfieldThis accepts an instant or a string that will coerce to an instant. It auto-generates as well based on inst?
and converts it to a MM/dd/yyyy
string.#2017-01-2300:34mrghey, thank you @seancorfield , that is super useful!#2017-01-2300:34mrgAnd I appreciate you helping me out on a Sunday evening š#2017-01-2313:31rickmoynihan@brownmoose3q: Interestingā¦ Iāve asked similar questions before, but never had any responsesā¦ Iām not sure if itās because there isnāt an answer for it yetā¦ Iād certainly like to know how to integrate clojure.spec with a clojure.test runner.#2017-01-2313:32rickmoynihannot sure what others doā¦ but might be nice to have some clojure.test wrappers for tools like exercise etc...#2017-01-2313:33rickmoynihanobviously you can also use instrument#2017-01-2313:58andrewzhurovI just find out that lein test
is giving that wierdo error and refind-out for myself this issue: https://github.com/technomancy/leiningen/issues/2173
turned off monkeydispatch
and whoop - all is good.#2017-01-2313:59andrewzhurovSo... seems to be a way of testing š#2017-01-2317:20lsnapeCan anyone suggest how to simplify the following:
(s/def ::created inst?)
(s/def ::updated inst?)
(s/def ::foo* (s/keys :req-un [::created ::updated]))
(s/def ::foo
(s/with-gen ::foo*
#(gen/such-that (fn [{:keys [created updated]}]
(>= (.getTime updated) (.getTime created)))
(s/gen ::foo*))))
(s/exercise ::foo)
I would like to ditch ::foo*
, but if I reference the ::foo
generator from within s/def ::foo
, then understandably I get a SOE#2017-01-2320:22souenzzoThere is option in s/keys
that fails/remove on unwanted keys??
(s/valid? (s/keys :req [:a/b]) {:a/b 0 :c/d 1}) ;=> false
or something like
(s/"filter????" (s/keys :req [:a/b]) {:a/b 0 :c/d 1}) ;=> {:a/b 0}
#2017-01-2320:32seancorfield@souenzzo spec is designed to be open for extension ā read the guide for more details.#2017-01-2320:33seancorfieldIf you want a spec to fail for unwanted keys, you need to explicitly s/and
a predicate that checks the set of keys in the map.#2017-01-2320:33seancorfield(but you probably shouldnāt do that)#2017-01-2404:32bastiHi, Iām pretty new to Clojure and I am playing around with Spec. Iāve got following error message
[{:type java.io.FileNotFoundException,
:message "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath.",
while evaluating
(ns foo-test
(:require [clojure.spec :as s]
[clojure.spec.test :as stest]
[clojure.spec.gen]
[clojure.pprint :as pprint]
[clojure.test :as t]))
;; Sample function and a function spec
(defn fooo [i] (- i 20))
(s/fdef fooo
:args (s/cat :i integer?)
:ret integer?
:fn #(> (:ret %) (-> % :args :i)))
(stest/check `fooo)
Whatās going on here? ^^#2017-01-2404:33Alex Miller (Clojure team)generators use the test.check library. this is optional so itās not a required compile-time dependency#2017-01-2404:34Alex Miller (Clojure team)but if you want to use it at test time, youāll need to include test.check as a test dependency#2017-01-2404:36Alex Miller (Clojure team)[org.clojure/test.check ā0.9.0ā]
#2017-01-2404:37Alex Miller (Clojure team)youād want that in the :dev profile for it to be test-scoped only (not a dependency that downstream consumers would get)#2017-01-2404:39bastiokay I c, sweet now it works, thank you for explaining it to me so clearly @alexmiller - really appreciated š#2017-01-2404:42Alex Miller (Clojure team)np - this is in the spec guide too - https://clojure.org/guides/spec#2017-01-2404:42Alex Miller (Clojure team)if you havenāt seen that yet#2017-01-2404:46bastino I havenāt, thanks for pointing me to it!#2017-01-2409:26lsnapeHi, I am having trouble writing a spec that has the following constraints:
{:id 123
:foo [{:id 456 :parent-id 123}
{:id 789 :parent-id 123}]}
See that :id
and :parent-id
are required to have the same value.#2017-01-2409:34lsnapeAha, a simple predicate s/and
ed with s/keys
would do nicely :duck:
(s/and (s/keys :req-un [::id ::foo])
(fn [{:keys [id foo]}]
(every? #{id} (map :parent-id foo))))
#2017-01-2413:29luxbockI'm specced a function which performs a deep-merge via addition on some maps with longs at the leafs, and I wanted to create a generative test for it#2017-01-2413:30luxbockhowever my generative test actually fails because I run into integer overflow, which is fair enough, that's good to know in theory#2017-01-2413:30luxbockbut in practice none of the values I use are ever going to get that large as to cause an overflow#2017-01-2413:31luxbockso in this case, should I be handling the bounding of these values via instrument
by using :gen
overrides?#2017-01-2413:32luxbockor should I handle it at the site of the spec definition by adding some arbitrary bound as a predicate to go with pos-int?
#2017-01-2414:44luxbockI tried working around this issue by using stest/instrument
and overriding the gen that is causing the overflow via :gen
, and then running stest/check
on the function again but this did not seem to have any effect#2017-01-2414:45donaldballMaybe you want something like a ::smallish-long
that describes a domain consistent with your expected values in production?#2017-01-2414:49luxbock@donaldball yeah I tried that approach as well, but it has the following problem: the function I'm testing is essentially taking a collection of values of ::map-with-smallish-int
and summing all of the ::smallish-int
together, so I specced the function to use the same return value#2017-01-2414:50luxbockand this of course fails because the largest values of what ::smallish-int
generates summed together are going to be larger than the bound I'm using#2017-01-2414:50luxbockso of course I could use use a new spec for the return value of the function under test, which uses a higher bound#2017-01-2414:50luxbockbut that feels less than ideal#2017-01-2414:51donaldballAnnoying, albeit correct š#2017-01-2414:51donaldballYouā¦ could merge with +ā
#2017-01-2414:52luxbockyeah that's another option but then I'm changing the definition of my function to use a slower function just to appease my tests#2017-01-2414:52luxbockbut I'm guessing that I don't understand how instrument
with :gen
overrides is supposed to be used#2017-01-2414:54luxbockit would be nice if during tests I could just override the generators of my specs to match to the domain values I'm expecting during normal use, and then I write the test to perform inside this context#2017-01-2415:18luxbockusing +'
doesn't really help either as once the values get high enough to coerce to BigInt they will fail nat-int?
#2017-01-2415:53Alex Miller (Clojure team)@luxbock the approach of using instrument with a :gen override is what I would recommend and should work#2017-01-2415:53Alex Miller (Clojure team)if you can a repro, would be interested#2017-01-2416:34luposlipIs there any chance that you guys could make the clojure.spec guide (PDF) printable? š
https://clojure.org/guides/spec#2017-01-2416:45qqqDoes Chrome -> Print -> Save as PDF work? Or do you need something else?#2017-01-2416:49luxbock@alexmiller this is what I'm doing, is this supposed to work?
https://gist.github.com/luxbock/ecb263750f61cc6fab7a736e5bd96008#2017-01-2417:11luposlip@qqq The result is just a lot of empty pages (I'm using Chromium BTW)#2017-01-2417:14qqq@luposlip: my bad, I also get only empty pages in chrome / safari#2017-01-2417:14qqqtime to edit some HTML#2017-01-2417:15qqqhmm, I get something printable#2017-01-2417:15qqqbut it loses all the formatting#2017-01-2417:16luposlipSounds just great @qqq, because printable pages would make http://clojure.org so much more useful (for users that want to read the guides etc. on ereaders etc.)#2017-01-2417:16qqqI don't know how to fix this, sorry -- it seems like the CSS I delete to make it printable also deletes the nice CSS formatting that makes it pretty and has nice code layout.#2017-01-2417:17luposlipWould be really nice with a epub export button for the long pages š#2017-01-2417:17qqqis the main clojure site on github ?#2017-01-2417:17qqqI wonder if you can just print the github rendering of the page#2017-01-2417:17luposlipI don't think so [UPDATE: I stand corrected]#2017-01-2417:17qqqhttps://github.com/clojure/clojure-site#2017-01-2417:17qqqhttps://github.com/clojure/clojure-site/blob/master/content/guides/spec.adoc#2017-01-2417:18qqqcan yo print that page?#2017-01-2417:18qqqhttps://github.com/clojure/clojure-site/blob/master/content/guides/spec.adoc <-- is printable for me, think this is probably the best for now#2017-01-2417:18luposlipgreat idea, didn't know it was on github - thanks for the input @qqq!!#2017-01-2422:35mac@luposlip Also Firefox does a decent job with the page if you print to pdf.#2017-01-2422:50Alex Miller (Clojure team)there is an issue for adding print support at https://github.com/clojure/clojure-site/issues/99#2017-01-2422:52Alex Miller (Clojure team)Making that work is probably not something Iām going to work on, but if someone else wanted to help, that would be cool#2017-01-2503:31bbloom@alexmiller i think you could get 90% of the way there with a smaller change than a custom stylesheet - Safari etcās reader mode is broken on the current site b/c div#preamble doesnāt match div.sect1#2017-01-2503:32bbloomi think that if you just make all the seconds have the same class, then reader modes will treat it all as body text and thatās a de facto print mode#2017-01-2503:32Alex Miller (Clojure team)most of that did not actually mean anything to me :)#2017-01-2503:33bbloomeach page in the reference has multiple divs, but only the first div shows up if you use an article reading filter#2017-01-2503:33Alex Miller (Clojure team)Iām like a monkey banging rocks together in web programming#2017-01-2503:33bbloomlol#2017-01-2503:33bbloomyou on a mac? go to any reference page in safari and youāll see a little paragraph icon in the top left corner of the url bar#2017-01-2503:34bbloomthereās various plugins for all the other browsers that do that too#2017-01-2503:34bbloombut since the āpreambleā in the generated pages have different structure and style classes applied, only that first section gets treated as body text#2017-01-2503:35Alex Miller (Clojure team)so just some tweaks might fix that#2017-01-2503:36bbloomyup, and then people can print that view just fine#2017-01-2503:36Alex Miller (Clojure team)ok, if I get a chance Iāll see if I can muggle my way through it#2017-01-2503:36bblooman aria hint might do the trick super quick too#2017-01-2503:36Alex Miller (Clojure team)is that a font?#2017-01-2503:37bbloomš ARIA = accessible rich internet applications#2017-01-2503:37bbloomitās about accessibility#2017-01-2503:37bbloomscreen readers etc#2017-01-2503:37Alex Miller (Clojure team)š#2017-01-2503:38Alex Miller (Clojure team)sadly, no rock emoji#2017-01-2503:38bbloomthe simplest thing to try:#2017-01-2503:38bbloomwrap the entire content of the page in a <article> tag#2017-01-2503:39bbloomspecifically, change the div with class clj-content-container
from div
to article
- and see if that does it#2017-01-2503:39bbloomi bet that it does, no impact to styles at all#2017-01-2503:40Alex Miller (Clojure team)cool#2017-01-2503:41Alex Miller (Clojure team)thanks#2017-01-2503:41bbloomno problem. hopefully that helps š#2017-01-2503:41bbloomi abuse reader mode and use huge fonts for most stuff i read š i have good eye sight and hope to keep it that way!#2017-01-2503:46Alex Miller (Clojure team)I match my font size to my age#2017-01-2503:47bbloomheh, i like it#2017-01-2504:34luxbock@alexmiller could you take a look at this and confirm if I'm using instrument
correctly or if there might actually be an underlying issue with :gen
https://gist.github.com/luxbock/ecb263750f61cc6fab7a736e5bd96008#2017-01-2504:38Alex Miller (Clojure team)I will take a look in the morning#2017-01-2505:59nickMaybe this is silly but say I have a spec like this (s/def ::dog (s/keys ::req [::name ::type ::sound]))
but I also want to include a key that's always something specific like :feet 4
- so if I generate that I'll get like {:name "red" :type "golden retriever" :sound "woofwoof" :feet 4}
#2017-01-2506:01nickI want to do something like this:
(defmulti dog-type :dog/type)
(defmethod dog-type :golden-retriever [_]
(s/keys :req [::name ::sound]))
But also have in the keys a type key that is dog-type and the actual key.#2017-01-2506:56tengIs it possible to use spec to validate maps that has keys that are not namespaced, like :name instead of :user/name?#2017-01-2507:54seancorfield@teng Yes, with :req-un
and :opt-un
. Although they require namespaced keys for the specs, the actual map keys are unqualified.#2017-01-2507:55seancorfield(see the guide for more details)#2017-01-2507:55tengok, thx.#2017-01-2507:55seancorfield@nick Do you mean you'd want (s/def ::feet #{4})
as the spec for the :feet
key?#2017-01-2512:04odinodin(clojure.spec/explain string? 123)
val: 123 fails predicate: :clojure.spec/unknown
How can I get the predicate to show string?
#2017-01-2512:12dominicm@odinodin I think you need to do (s/def ::string? string)
and then you can (s/explain ::string? 123)
#2017-01-2512:12odinodin@dominicm thanks, makes sense š#2017-01-2513:20Alex Miller (Clojure team)This unknown is a bug that has a pending patch btw #2017-01-2513:23Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2068#2017-01-2513:23Alex Miller (Clojure team)So that will work eventually#2017-01-2513:41odinodin@alexmiller that is awesome š#2017-01-2515:13Alex Miller (Clojure team)@luxbock I agree that Iād expect something like this to work. I did just notice this line in the instrument docs though: ":gen overrides are used only for :stub generation.ā which honestly surprises me. it seems like they should be used also for instrumented :args gen too.#2017-01-2515:14Alex Miller (Clojure team)although maybe you should really be passing those :gen overrides in your call to check
#2017-01-2515:24Alex Miller (Clojure team)yeah, you should move your gen overrides map from instrument into the check options map. also, in the :gen map, the values should be no-arg functions that return the generator, not the actual generator, so just prefix all of them with #
to turn them into anonymous no-arg functions#2017-01-2515:28Alex Miller (Clojure team)once I did that, it was running, but I started getting OOMEs#2017-01-2515:29Alex Miller (Clojure team)to address that, I narrowed the collection generators which default to max size 20. I added :gen-max 3
to both the fdef :args spec and the ::operations
override gen.#2017-01-2515:29luxbock@alexmiller ah yeah I think I tried :gen
with check but I didn't realize that I need to wrap them in thunks#2017-01-2515:29Alex Miller (Clojure team)Then it worked#2017-01-2515:29luxbockthanks, I will play around with it a bit more later today#2017-01-2515:30Alex Miller (Clojure team)these days I generally preemptively add :gen-max 3
to all collection specs reachable by fdef :args or used in recursion#2017-01-2515:30Alex Miller (Clojure team)I think it would be a good idea to change the default there from 20 to 3#2017-01-2515:32luxbockyeah that's good to know as well#2017-01-2515:36Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2102#2017-01-2515:36Alex Miller (Clojure team)if youād like to vote :)#2017-01-2515:36luxbockit feels that all the spec related testing functionality one needs to write these types of tests is quite spread out, both in terms of namespaces and where the different options for customizing the generators go#2017-01-2515:37luxbocknothing that a good utility library won't fix, but I have a hard time keeping in my head where everything is supposed to come from and how it ties together#2017-01-2515:49Alex Miller (Clojure team)well the stuff in spec vs spec.test is intentionally split to emphasize the parts that are designed specifically for testing. and spec.gen is the same - split out the part that dynamically loads and depends on test.check generators.#2017-01-2515:49Alex Miller (Clojure team)those are at this point pretty well factored in my mind#2017-01-2515:50Alex Miller (Clojure team)the gen overrides stuff is a bit messier and not consistent enough between s/exercise, stest/instrument, stest/check etc and we may still make some changes in those#2017-01-2516:02luxbockI'm writing code which I can't fully test in use until it hits the production server so I'm trying to cover as many edge cases as possible with spec and generative tests to avoid the long feedback loop of deploy + test, and it's definitely giving me a lot more confidence in that I got rid of most of the silly bugs before trying it in actual use#2017-01-2516:05Alex Miller (Clojure team)cool, thatās good to hear#2017-01-2520:52agquick noob question how do I limit the size of a number in spec like: (s/and int? pos?)
#2017-01-2521:12qqq@ag: you can pass in arbitrary function#2017-01-2521:12qqqif you also need to generate, you need to write you rown generator#2017-01-2521:37Alex Miller (Clojure team)@ag (s/and int? pos-int? #(< % 100)
#2017-01-2521:37Alex Miller (Clojure team)but once you start going there, consider integer ranges#2017-01-2521:37Alex Miller (Clojure team)(s/int-in 1 100)
#2017-01-2521:38Alex Miller (Clojure team)the range specs come with a better generator by default#2017-01-2521:41agah, rightā¦ thanks @alexmiller#2017-01-2523:03aguhā¦ so how do I make a spec for a key like this :options {:on-click `~(fn [r] ,,,,)}
it contains an fn but in quoted form?#2017-01-2523:16agI guess I need s/fdef, how do I make a spec for something like {:on-click (fn [e],,,}
- function that takes one argument of anything and returns whatever?#2017-01-2600:00agoh, another total noob question.. I need a spec that generates vector of 1 to 5 elements, each element is of another spec (map), something like this:
(s/def ::col (s/keys :req-un [::id ::title]))
(gen/generate (s/gen (s/+ ::col)))
but it has to be 1) a vector 2) of 1 to 5 elements#2017-01-2600:02agoh, nevermind I think I got itā¦#2017-01-2600:02agdammit specs and test.check makes me feel stupid#2017-01-2600:05agmeh, it seems I only got the generator part#2017-01-2600:06ag(s/with-gen vector? #(gen/vector (s/gen ::col) 2 5))
aināt good, I need spec to conform to vector of ::col#2017-01-2601:42joshjones@ag this?
(s/def ::id (s/int-in 1 100))
(s/def ::title string?)
(s/def ::col (s/keys :req-un [::id ::title]))
(s/def ::cols (s/coll-of ::col :kind vector?
:min-count 1
:max-count 5))
(gen/generate (s/gen ::cols))
#2017-01-2602:02Alex Miller (Clojure team)@ag for your prior question, see s/fspec#2017-01-2616:51mrcncis it preferable to use s/assert
over s/valid?
so you can disable it at runtime?#2017-01-2616:53mrcncthey seem pretty similar#2017-01-2617:17seancorfieldTo me they have very different uses: we use spec for input validation (and some coercion) so we have error handling that says (if (s/valid? ::spec value) (do-good-stuff) (handle-error))
ā thatās not the same as (s/assert ā¦)
#2017-01-2617:18seancorfieldI personally donāt much like asserts ā Iāve never liked them in any language ā and part of that is because folks tend to turn them off in production, which is when itās even more critical that your code doesnāt attempt to operate on invalid data. Seems backwards to me...#2017-01-2617:44mrcncgood points @seancorfieldā¦thx!#2017-01-2618:48spiedensame ā usually check valid?
then throw an ex-info
with explain-data
#2017-01-2620:37moxajspecs cannot be created at runtime, right? since the api is mostly macros#2017-01-2620:58schmeeI think you can get it done with quoting and eval
#2017-01-2621:01moxajwell, maybe as a last resort .. eval
is evil#2017-01-2621:03bfabryspecs being macros is what gives them reasonable reporting on failures, a spec created at runtime would have to either do away with a lot of the error description, or have a very clunky api. there was some talk of some non-macro versions of some of the spec api maybe in the future though#2017-01-2621:06spiedenseems to guide the intended usage as something static, too#2017-01-2621:06spieden.. in a world of dynamism =)#2017-01-2621:15tbaldridge@moxaj why is eval
evil?#2017-01-2621:18moxaj@tbaldridge well, most of the time when you use it, there's a better solution (there are exceptions of course)#2017-01-2621:19tbaldridgeThat doesn't make it evil though š#2017-01-2621:19moxaji'm pretty sure there's a workaround for my issue which does not include eval
#2017-01-2621:19moxajwell yeah, it's perfectly fine, but sometimes abused#2017-01-2621:20tbaldridgeI say this because the "eval is evil" trope is mis-applied in Lisps. Eval is evil in a language like Javascript (where the phrase comes from) where eval means concat-ing strings together. In Clojure eval is akin to something like stored procs in SQL. It makes code injection really hard#2017-01-2621:21tbaldridgeSo all that being said, macros + eval works pretty well for dynamic specs, and done correctly you could even get pretty good source code mappings#2017-01-2621:26moxajalso, i'm targeting cljc, and eval
only works in bootstrapped cljs afaik#2017-01-2621:27tbaldridgefair enough#2017-01-2703:58weicould someone point me to some real world examples of conform? I have a hunch that it would be very useful for me, especially with specs like s/or
, but I havenāt found the output to be especially easy to adapt, for example in a cond
statement.#2017-01-2704:08weihereās an example that I think is overly verbose: https://gist.github.com/yayitswei/fc60da9271452716ae2a197c945a4b3e, would appreciate any tips on trimming this down#2017-01-2707:30luxbockis there some existing function that gives me the same value that the predicates/specs in :fn
receive?#2017-01-2707:30luxbockI am assuming no, but it's probalby not too difficult to create#2017-01-2708:28luxbockthis seems to work#2017-01-2712:35nblumoeHey, what are uses for the :opts
key spec on maps? As additional keys are allowed anyways and all present keys are being validated even if they are not in the explicit spec there does not seem to be any relevance for validation. Is it for generation? Something else Iām missing?#2017-01-2713:54schmeenblumoe AFAIK itās for generation and as documentation#2017-01-2713:57nblumoeThe guide says this: ā When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. Weāll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.ā I do not know what āWeāll see later where optional attributes can be usefulā is referring to#2017-01-2713:58nblumoehttps://clojure.org/guides/spec#_entity_maps#2017-01-2714:45Alex Miller (Clojure team)They are used in generators (as well as serving as doc)#2017-01-2714:51Alex Miller (Clojure team)@luxbock the :fn spec receives a map {:args ā¦, :ret ā¦}
where the values are the conformed values of the args and ret specs. you donāt need that get-fspec-attr - you can call (s/get-spec sym)
to get the fspec and that supports ILookup, so you can (:args (s/get-spec sym))
to get (for example) the args spec.#2017-01-2714:52luxbock@alexmiller: thanks, yeah figured there might have been a better way#2017-01-2800:12adambros@alexmiller getting weird behavior when trying to conform this:
(s/def ::thing (s/cat
:a (s/? string?)
:b (s/+ number?)))
(s/def ::seq-of (s/+ ::thing))
(s/conform ::seq-of '(āfooā 1 ābarā 2 3 āquxā 4))
;=> [{:a "foo", :b [1]} [{:a "bar", :b [2 3]} {:a "qux", :b [4]}]]
;expected
;=> [{:a "foo", :b [1]} {:a "bar", :b [2 3]} {:a "qux", :b [4]}]
;; ONLY 2nd thing matters?
(s/conform ::seq-of '(āfooā 1 2 ā bar 3))
;=> [{:a "foo", :b [1 2]} {:a "bar", :b [2]}]
;; NO OPTIONAL
(s/def ::thing (s/cat
:a string?
:b (s/+ number?)))
(s/def ::seq-of (s/+ ::thing))
(s/conform ::seq-of '(āfooā 1 ābarā 2 3 āquxā 4))
;=> [{:a "foo", :b [1]} {:a "bar", :b [2 3]} {:a "qux", :b [4]}]
This also only shows up if there are 2+ numbers in the 2nd or later ::thing
AND the problem goes away if I make :a
not optionalā¦
is this at all related to http://dev.clojure.org/jira/browse/CLJ-2003 ?#2017-01-2800:33agis it possible to āredefā the spec? if I have a spec that depends on other spec and that one uses a function as a predicate and I want to predicate to be something else in the test#2017-01-2800:34hiredman@adambros that looks like what phil brown mentions in the comment on that, but alex says it isn't actually related to that issue and he should open a new issue for it#2017-01-2800:38hiredmana cons somewhere there should be a concat#2017-01-2800:42Alex Miller (Clojure team)@adambros not sure if thatās the same or not, feel free to log#2017-01-2803:05uwoIād be grateful for feedback š https://groups.google.com/forum/#!topic/clojure/wwJVJKtxps8#2017-01-2809:39nblumoeI ran into some issues with spec.test/check
not terminating when using a coll-of
spec with :kind
:
(s/def ::bar (s/coll-of number? :kind vector?))
(defn foo [a]
(reduce + 0 a))
(s/fdef foo
:args (s/cat :a ::bar)
:ret number?)
(first (stest/check `foo))
The last call does not terminate, just keeps heating my CPU. When removing the :kind vector?
constraint from the spec ::bar
everything works fine. With :kind list?
I also get the erroneous behaviour.#2017-01-2809:42nblumoeWondering If I am doing something wrong, if this is a bug and if it was filed on Jira already#2017-01-2809:44schmeenblumoe try adding :gen-max 3
to the ::bar
definition#2017-01-2809:44schmeeit could be that it is generating some crazy big collections#2017-01-2809:45nblumoethanks, but still the same issue#2017-01-2823:22leongrapenthin@nblumoe If you don't provide :into []
to s/coll-of
spec will generate a vector every time just to call empty
on it to then generate the stuff into it. Apparently these "generated constructor vectors" explode and you can't control their sizing. So you have to provide :into
. @alexmiller Is this intended or ticketworthy?#2017-01-2823:27gfrederickssounds like a ticket to me#2017-01-2823:29Alex Miller (Clojure team)Not sure I understand all that without looking at the code but doesn't sound right#2017-01-2823:51leongrapenthin@alexmiller @nblumoe @gfredericks created http://dev.clojure.org/jira/browse/CLJ-2103#2017-01-2915:48nblumoeGreat, thanks @leongrapenthin #2017-01-3020:32mrgIs there a good way to generate an increasing number as id in a variable-length collection?#2017-01-3020:32mrgi.e. i have this:#2017-01-3020:33mrgnow if i generate subsegments, how would I get the first one to have the ::id 1, the second one (if there is one) the ::id 2, and the third one (if there is one) the ::id 3 ?#2017-01-3020:40Alex Miller (Clojure team)you canāt make that happen automatically, but you could build a generator that did that#2017-01-3020:41Alex Miller (Clojure team)I would create a custom generator for ::subsegments
using gen/fmap
- use the default generator to create, then in the fmap function replace the ::id
values to have the index#2017-01-3021:29mrgI'll give that a shot @alexmiller .#2017-01-3021:30mrgHow would you keep the index state? There doesn't seem to be a fmap-with-index, nor a multi-arity version where I could pass in a range or something and zip them together#2017-01-3022:03Alex Miller (Clojure team)Yeah, you can just map over the generated value and range#2017-01-3022:25mrgDo you have an example or a pointer to somewhere that would explain that?#2017-01-3022:25mrgI don't quite grok generators yet#2017-01-3022:45jrIs there a way to spec a mapargs fn? for example:
(defn foo [& {:keys [bar]}] bar)
(foo :bar "value of bar")
#2017-01-3022:50joshjones(s/def ::bar string?)
(s/def ::baz int?)
(s/def ::mapargs (s/keys* :req-un [::bar ::baz]))
(defn foo [& {:keys [bar baz]}] bar)
(s/fdef foo :args ::mapargs)
(stest/instrument `foo)
(foo :bar "value of bar" :baz 42)
@jr#2017-01-3022:54jr@joshjones wow that's great. thanks!#2017-01-3022:56jrI didn't realize that keys*
is different in that it transforms a sequence of values into a map#2017-01-3023:02joshjonesjust modified the above example to remove cat
which was not necessary fyi#2017-01-3023:03jrno worries I got the gist of it. I was specing a macro with the following structure
(defui foo []
:tracks [name [:path :to :value]]
[:h1 name])
#2017-01-3023:04jrwhere the options like :tracks
are optional#2017-01-3102:00rmuslimovHello, I have newbie question for spec. Letās suppose I have map {"first-name" ānameā}
, can you please show the spec which check key (= āfirst-nameā) and if the is string. Thanks#2017-01-3102:08rmuslimovThing I cannot understand here, that if I had keyword instead of āfirst-nameā everything would be trivial:
(s/def :first-name string?)
(s/valid? (s/keys :req-un [:first-name]) {:first-name ānameā}) ;=> true
But if āfirst-nameā I cannot attach spec to it#2017-01-3102:45joshjonesif you are doing this just for experimentation -- then
(s/def ::str-str-map #(string? (get % "first-name")))
(s/valid? ::str-str-map {"first-name" "josh"})
but in practice, this only makes life difficult. there's not a way i know of to do this cleanly, as spec is not trying to promote this type of behavior#2017-01-3103:40rmuslimov@joshjones got it, thanks!#2017-01-3109:09luxbockit's quite difficult to figure out what other options stest/check
accepts in the :clojure.spec.test.check/opts
key#2017-01-3109:09luxbockthe doc string says: "::stc/opts opts to flow through test.check/quick-check" but it's not really clear which function this refers to#2017-01-3109:12luxbockI can go to the source and see it calls stest/check-1
, which calls a private function stest/quick-check
which calls clojure.spec.gen/quick-check
which calls clojure.test.check/quick-check
#2017-01-3109:13luxbockwhich tells me the keys are :seed
and :num-tests
, so as far as I can tell :seed
is the only other option it accepts#2017-01-3109:14luxbockI wish the doc-string for stest/check
just told me this right away because I might not have clojure.test.check
required and so I need to dig quite deep to find the information#2017-01-3109:15luxbockthe same goes for s/coll-of
which tells me it accepts the same options as every
, rather than just telling what those options are#2017-01-3111:16thhellerso I have a hard time composing specs due to its macro nature#2017-01-3111:16thhellerie. anything that creates a spec must be a macro#2017-01-3111:17thhellerI can't have a function (validate-a-thing TypeOfThing)
that returns a spec#2017-01-3111:18thhellerit must be a macro that generates (s/spec #(instance? TypeOfThing %))
#2017-01-3111:18thhelleror am I missing something obvious? using instance?
as a basic example my spec is actually a bit more complex than that#2017-01-3111:29thheller(defn validate-a-thing [type] (s/spec #(instance? type %))
works well enough but the error suffers as it is always fails predicate: (instance? type %)
#2017-01-3111:31thhellercan't find an obvious way to provide custom explain
logic, don't care about the gen
part in this case#2017-01-3111:34nblumoeShouldnāt my generator be good enough to be used to check
the corresponding function against itās spec? (frequencies (map (partial s/valid? (s/and ::matrix
::non-zero-row-sums)) (gen/sample (matrix-gen) 100)))
;; => {true 27, false 73}
I end up with ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
again and again.#2017-01-3115:00dergutemoritz@thheller Yeah you are right, validate-a-thing
needs to be a macro to achieve what you want#2017-01-3115:00dergutemoritz@thheller I don't think you're missing anything#2017-01-3115:03dergutemoritz@thheller See this paragraph for the rationale: https://clojure.org/about/spec#_code_is_data_not_vice_versa#2017-01-3119:13uwoIād really appreciate some feedback on this: https://groups.google.com/forum/#!topic/clojure/wwJVJKtxps8#2017-01-3120:05joshjonesthis exact situation has come up several times here @uwo ā unfortunately, thereās not much you can do other than what youāve already identified, at least, none that Iām aware of. The reality is that you do not have one set of billed-charges
; you have two, which means you will need to spec two, whether thatās through a multi-spec or explicit naming#2017-01-3120:07joshjonesdo you have more levels of nesting than what youāve shown?#2017-01-3121:39uwo@joshjones thanks, and yes, many many levels of nesting#2017-02-0107:32Yehonathan SharvitWhat are the conceptual and practical differences between s/or
and s/alt
?#2017-02-0107:40mike_ananevhi! if i need to transfer spec via network, how to get "source" of spec by its name?#2017-02-0107:40Yehonathan Sharvitsomething like s/describe
?#2017-02-0107:42mike_ananev@viebel yeah, but any anonymous fn in spec is not readable. i need to get source of spec, like I see in IDE#2017-02-0108:29dergutemoritz@viebel s/alt
is a regex op, s/or
is not#2017-02-0108:30Yehonathan Sharvit@dergutemoritz But if I put s/alt
inside an s/cat
expression it seems to work as expected.#2017-02-0108:30dergutemoritz@viebel You mean s/or
?#2017-02-0108:32Yehonathan Sharvityeah sorry for the typo: But if I put s/or
inside an s/cat
expression it seems to work as expected.#2017-02-0108:34dergutemoritz@viebel The difference is that a s/cat
within an s/alt
will be spliced whereas within an s/or
it won't#2017-02-0108:34dergutemoritzLet me cook up an example#2017-02-0108:37dergutemoritz@viebel Here you go - hope it's clear, couldn't come up with a more concise example š#2017-02-0108:57Yehonathan SharvitThanks a lot @dergutemoritz !#2017-02-0108:58Yehonathan SharvitNow itās 100% clear!#2017-02-0108:58dergutemoritzYou're welcome @viebel, glad it helped!#2017-02-0109:08Yehonathan SharvitAnd to make it 200% clear here is your code snippet @dergutemoritz inside klipse for a live demo http://app.klipse.tech/?eval_only=1&cljs_in.gist=viebel/6bdefe58f4a38591399f0628fb775418#2017-02-0114:27souenzzoOh...
(spec/valid? :my.spec/base lista)
=> false
(spec/explain :my.spec/base lista)
Success!
=> nil
#2017-02-0114:38souenzzoThere is some debug Tool?
lista
is a deep nested map
My Spec has 10+ referentes/ns and some recursion#2017-02-0114:44souenzzoI'm using 1.8 backport#2017-02-0114:47joshjoneswell that changes things ā does alpha14 exhibit the error?#2017-02-0116:29souenzzoNo bugs on 1.9.0-alpha14
š š#2017-02-0116:33pesterhazyanyone have a reading/watching/listening list for spec? I'm just getting started#2017-02-0116:33pesterhazyother than https://clojure.org/about/spec of course which I'll start with#2017-02-0116:34mpenet@pesterhazy for gen related stuff I really like https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#2017-02-0116:59gdeer81does anyone fdef their conformer functions?#2017-02-0119:41Alex Miller (Clojure team)@pesterhazy well https://clojure.org/guides/spec of course#2017-02-0119:41Alex Miller (Clojure team)and then there are some vids Stu has done at https://www.youtube.com/watch?v=nqY4nUMfus8&list=PLZdCLR02grLrju9ntDh3RGPpWSWBvjwXg#2017-02-0119:42Alex Miller (Clojure team)and some at http://blog.cognitect.com/?tag=clojure.spec#2017-02-0121:11pesterhazyThanks!#2017-02-0221:14villesvHi, I am wondering if maybe I have stumbled across an issue in clojure.spec
. I have perused the issue list but have found nothing that I think describes the issue at hand for me.#2017-02-0221:15villesvA small example:
(s/def :some.ns.1/k string?)
(s/def :some.ns.1/m (s/keys :req [:some.ns.1/k]))
(s/def :some.ns.2/k string?)
(s/def :some.ns.2/m (s/keys :req-un [:some.ns.2/k]))
(s/explain-str :some.ns.2/m {:k "1"
:some.ns.1/m {}})
;; => "In: [:some.ns.1/m] val: {} fails spec: :some.ns.1/m at: [:some.ns.1/m] predicate: (contains? % :some.ns.1/k)\n"
#2017-02-0221:17villesvMy point being, although I am checking (doesn't matter if I use s/explain-str
, s/assert
etc) for :some.ns.2/m
- clojure.spec ends up also validating another spec.#2017-02-0221:18villesvThat is, although :some.ns.1/m
is in there and not conforming to spec - I did not expect it to be checked for. I also noticed that this happens both when I use :req-un
and :req
for :some.ns.2/m
#2017-02-0221:18villesvI am thinking this is a bug#2017-02-0221:37villesvHmm, no seems I am mistaken, I found this in the spec source docs for clojure.spec/keys
:
> In addition, the values of all namespace-qualified keys will be validated
> (and possibly destructured) by any registered specs. Note: there is
> no support for inline value specification, by design.
How have I not stumbled across this before?#2017-02-0223:30ghadidunno -- it's in the guide too , I'm pretty sure. Best way to understand is that namespaced keys have program-global meaning#2017-02-0302:38joshjones@fnil, it's always been this way...#2017-02-0307:32luxbockfinally starting to get a hang of how to use instrument
effectively while doing interactive development via the REPL, :replace
is very handy in isolating parts of a long pipeline of functions that build up the computation step by step#2017-02-0307:38luxbockhaving a version of enumerate-namespace
that recursively enumerates all the internal functions called by some top-level function would be quite handy#2017-02-0307:39luxbocknot sure if that already exists in some library, if anyone knows let me know#2017-02-0309:08villesv@ghadi @joshjones You are correct and I am embarrassed š I think the case at hand may have contributed to the confusion. I was applying specs to a deep, previously unspec:ed map where both spec:ed and unspec:ed keys were present.#2017-02-0309:50chrisblomi'm converting the json schema for AWS cloudformation to clojure.spec#2017-02-0309:51chrisblomthis schema has a few reflexive references#2017-02-0309:52chrisblomi'm running into problems when converting it to spec#2017-02-0309:53chrisblomi have a spec of form (s/def :foo/bar ...)
where :foo/bar
is used somewhere in the ...#2017-02-0309:54chrisblomthis gives me an Unable to resolve spec
error, what's the recommended way to do such circular definitions in spec?#2017-02-0310:38villesvOh, I would love to have some sanity brought on to cloudformations json. Love the service but I stumble on the JSON details all the time#2017-02-0311:09villesvHow is :foo/bar
used recursively? Doesn't something like:
(s/def :foo/bar (s/keys :opt [:foo/bar]))
(s/valid? :foo/bar {:foo/bar {:foo/bar {}}})
work? Or am I being naive?#2017-02-0314:55frankthat seems to work in the repl#2017-02-0314:58joshjones@chrisblom What @fnil has suggested works for a map. A more fleshed out example:
(s/def ::k string?)
(s/def ::m (s/keys :req [::k] :opt [::m]))
(s/valid? ::m {::k "abc" ::z 42
::m {::k "def"
::m {::k "ghi"}}})
#2017-02-0314:59joshjonesbut I don't know if you're spec-ing a map, a collection, or what ? you can also do recursive definitions for collections, etc., but if you can be more specific about the structure you're trying to spec..#2017-02-0315:49chrisblomhmm thanks guys, it's working now, not sure what i was doing wrong#2017-02-0315:55preporHello. I think that fully namespaced keywords without full application's namespace should be considered as very bad practice (:message/state for example) at least in library code. But why the official guide is full of such usages?
(s/def :event/type keyword?)
(s/def :event/timestamp int?)
(s/def :search/url string?)
(s/def :error/message string?)
(s/def :error/code int?)
#2017-02-0315:59villesv@prepor Why do you think so?#2017-02-0316:00joost-diepenmaatI think I agree @prepor. since specs are global I would say that libraries should never use keys outside of the namespaces already in the lib#2017-02-0316:01joost-diepenmaatto prevent clashes with application code and other libs#2017-02-0316:01frankI think it makes it a tad harder to find the namespace where the s/def
lives. Also, if you're referring to the specs in other namespaces, it's harder to figure out which namespace you need to require in order to have those specs available#2017-02-0316:02prepor@fnil because of @joost-diepenmaat explanation, yes#2017-02-0316:08villesvAll true, although I find the flexibility great.#2017-02-0316:09joost-diepenmaatIām using single-keyword namespaces in an application right now and itās very neat. Writing libraries generally means sacrificing some easyness for interoperability#2017-02-0316:09villesvI definitely agree that it can be hard to locate where a certain keyword was defined (or sometimes redefined)#2017-02-0316:21prepor@joost-diepenmaat a boundary between application and library is often not clear. today it's application code, and tomorrow you can decide to split it into libraries to use inside your microservices.#2017-02-0316:22joost-diepenmaatTrue #2017-02-0316:23preporso, personally I decided to not use "short namespaces" at all. and for me it's strange that official guide forces this usage. maybe I don't understand something...#2017-02-0316:35seancorfieldThe guide uses ::stuff
in most places tho` which is qualified by the full namespace. I think the other places in the guide are just examples -- certainly not "forcing" any particular usage on you. #2017-02-0316:36joshjonesThe spec guide seems to be aimed at learning how spec works, and as such, brevity and simplicity are key -- adding too much info on namespaces makes it more difficult to learn. ::mykey
is short, and simple ... at any rate, if you want to do something like this, I think it's a good way to shorten the code while still being namespace-safe:
(create-ns 'my.app.events)
(alias 'event 'my.app.events)
(s/def ::event/type keyword?)
#2017-02-0316:39preporWhen something is used in guide without red warnings ā it's "forcing" )#2017-02-0316:42prepor@joshjones yes, I know about create-ns / alias#2017-02-0316:42seancorfieldIt's a guide not a set of rules. You're reading too much into it. #2017-02-0316:43seancorfieldIf anything I'd say the core team don't provide enough guidance. They really don't force their views on anyone. #2017-02-0316:47prepor@seancorfield will you think the same when you catch a bug because two of your dependencies have name conflicts without any warnings because the official guide guided their authors to not always use fully qualified keywords? š#2017-02-0316:50preporIt's near the only way how you can broke others code in clojure ecosystem in non tricky way, I think#2017-02-0316:56seancorfieldWhen the core team talk about spec, they're pretty clear about using namespaces to prevent exactly that bug. Again, you're treating a learning guide as a rule book. There doesn't seem much point in this discussion if you're set on blaming the core team for your mistakes šø #2017-02-0317:05preporhm. wrong tools force me to make my mistakes. is there no chance that core team did something wrong? don't think so. I'm sure that its ok, then somebody asks questions about core team decisions.#2017-02-0317:06preporI'm trying to find some big real world applications on github which uses core.spec but can't š#2017-02-0317:50sattvikOne option I have found that can work is using the :full.ns.no.alias/kw
form can help work around circular dependencies.#2017-02-0318:18seancorfield@prepor We use spec in production fairly heavily ā and I know there are other companies already doing so ā but of course you wonāt find commercial application code on GitHub š#2017-02-0318:19seancorfield@prepor And remember the saying: āItās a poor craftsman that blames his tools.ā (not sure whether that saying originates but I remember hearing it a lot growing up)#2017-02-0318:54fentoncan you spec a variadic function?#2017-02-0318:56sattvikSure. You use the regex specs to do so.#2017-02-0319:00sattvikYou can do something like:
(s/fdef hello
:args (s/cat (s/? string?))
:ret string?)
(defn hello
([] (hello āworldā))
([name] (str āHello, ā name \!)))
#2017-02-0319:04fentonok... I haven't looked into this much but wondering about doing pattern matching with core.match and spec to be like a guard or something...#2017-02-0319:16Alex Miller (Clojure team)@prepor the guide uses shorter names and ::
kws for readability. it would be reasonable to add a side bar explaining this.#2017-02-0319:35Alex Miller (Clojure team)@prepor sidebar added https://github.com/clojure/clojure-site/commit/cf0a744c2e59fcd0c6459d55dba429577399e6c0#2017-02-0319:50prepor@alexmiller cool, thank you! the only thing which I imagine to catch (and debug) such errors automatically is "production mode" for clojure.spec which forbids redefinition of specs in explicit way#2017-02-0319:59spieden@fnil we generate cloudformation templates as EDN and serialize to JSON, works great#2017-02-0320:00spieden@chrisblom that sounds like a great project. huge surface area, though#2017-02-0320:08villesv@spieden very cool! I have had such a thought but have not fully grasped cloudformation and its template language yet#2017-02-0320:09spiedenitās a bit of a curve but has been a great workhorse for us#2017-02-0320:09spiedencanāt imagine maintaining a template by hand, though#2017-02-0320:10spiedenwe have lots of functions that build up common structures, etc.#2017-02-0320:10spiedenwithout DRYing things out like this itād be crazy town#2017-02-0320:10villesvSpec should be great for it#2017-02-0320:13spiedenyeah for sure. they have their own (hosted) validation operation, but it misses things and works against the JSON representation#2017-02-0320:25villesvSo I noticed š#2017-02-0322:38kassapoCould you help a Newbie: I am going through http://blog.cognitect.com/blog/2016/10/5/interactive-development-with-clojurespec and cannot get past codebreaker=> (s/exercise (:args (s/get-spec `score)))
FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:458)
I am using Clojure 1.9.0-alpha14 in lein repl#2017-02-0322:41jradd [org.clojure/test.check "0.9.0"]
to your (dev) dependencies#2017-02-0517:32nicolaHello, i need some verification from experts! Iām thinking about refactor my json-schema library to clojure.spec generator from json-schema definitions. The problems i see: this will be generation based on macros (or there is another way?) and some json-schema could come from users - but specs are not garbage collected (some sandbox?) ? Does anybody have spec generation experience?#2017-02-0517:34nicolaOr may be this is a bad idea at all?#2017-02-0606:22Yehonathan SharvitIām trying to understand the difference between s/coll-of
and s/every
.
> Note that 'every' does not do exhaustive checking, rather it samples coll-check-limit elements. Nor (as a result) does it do any conforming of elements. 'explain' will report at most coll-error-limit problems. Thus 'every' should be suitable for potentially large collections.#2017-02-0606:23Yehonathan SharvitWhat does it mean that every
doesnāt do any conforming of elements?#2017-02-0606:23Yehonathan Sharvit(s/conform (s/every number?) (range 1000))
seems to do the conforming pretty well!#2017-02-0608:52dergutemoritz@viebel Try with a spec that conforms to a different value#2017-02-0608:52dergutemoritzLike s/or
#2017-02-0608:54Yehonathan SharvitLike this:
(s/conform (s/every (s/or :n number?
:s string?))
(concat (range 1000)))
#2017-02-0608:55dergutemoritzYeah that should work#2017-02-0608:55Yehonathan SharvitIf I understand correctly s/every
leaves the data as-is#2017-02-0608:55dergutemoritzYup!#2017-02-0608:55Yehonathan Sharvitwhile s/coll-of
conforms it#2017-02-0608:55dergutemoritzIt will only check whether the values match the spec#2017-02-0608:56dergutemoritzRight#2017-02-0608:56Yehonathan SharvitThx @dergutemoritz#2017-02-0608:56dergutemoritzYW!#2017-02-0609:41kassapoI cannot find the correct way to call clojure.spec.test/check
on my function that might read a file
`#2017-02-0609:41kassapo(s/def :read/file #(instance? File %))
(s/def :read/content string?)
(s/def :read/fail nil?)
(defn read-file [^File f] (when (.isFile f) (slurp f)))
(s/fdef read-file
:args (s/cat :f :read/file)
:ret (s/or :read/content :read/fail)
:fn (fn [{{f :f} :args ret :ret}]
(if (.isFile f)
(s/valid? :read/content ret)
(s/valid? :read/fail ret))))
(defn gen-file []
(gen/elements (map (fn [^String f] (File. f))
["./project.clj" "foo"])))
(s/exercise :read/file 4 {:read/file gen-file})
#2017-02-0610:47Yehonathan SharvitIām reading (with enthusiasm) "CREATING A SPEC FOR DESTRUCTURINGā http://blog.cognitect.com/blog/2017/1/3/spec-destructuring. Everything is pretty clear except one point related to s/merge
at the end of the article#2017-02-0610:47Yehonathan Sharvit;; collection of tuple forms
(s/def ::map-bindings
(s/every (s/or :mb ::map-binding
:nsk ::ns-keys
:msb (s/tuple #{:as :or :keys :syms :strs} any?)) :into {}))
(s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))
#2017-02-0610:48Yehonathan SharvitI donāt understand why inside ::map-bindings
, we have the :msb
part#2017-02-0610:48Yehonathan SharvitAFAIU, the :msb
part relates only to ::map-special-binding
#2017-02-0612:16Yehonathan Sharvitany idea @dergutemoritz or @alexmiller ?#2017-02-0612:24dergutemoritz@viebel It makes sure that no other keywords than the ones in the set are present#2017-02-0612:24dergutemoritz::map-special-bindings
is defined in terms of s/keys
which is an open key set by design#2017-02-0612:25dergutemoritzAt least that's what I surmise is the motivation. Being the author of that post, @alexmiller might have more insight š#2017-02-0612:25Yehonathan SharvitThat makes sense. But I think it would be clearer if (s/tuple #{:as :or :keys :syms :strs} any?)
was part of ::map-special-binding
. What do u think @dergutemoritz ?#2017-02-0612:31dergutemoritzNot sure I would find it clearer that way#2017-02-0612:32dergutemoritzThe way it is now makes the ::map-special-bindings
spec a tad more widely usable I guess#2017-02-0612:35Yehonathan Sharvitok. makes sense#2017-02-0618:23Alex Miller (Clojure team)making ::make-special-bindings
reusable is not really a goal#2017-02-0618:23Alex Miller (Clojure team)the tuple check canāt be part of ::map-special-binding
b/c that spec is a map spec which is merged with the rest#2017-02-0618:28Alex Miller (Clojure team)I think itās probably a reasonable criticism that this spec is too restrictive (from a future-looking point of view) in narrowing to just this set of options as we know it now.#2017-02-0618:30Alex Miller (Clojure team)the intention is really to segment the space of possible tuple forms to bindings (which are simple symbols, vectors, or maps), qualified keywords whose names are ākeysā or āsymsā, or special options. Here Iāve listed a concrete set of options but lost the open-ness of something like keys
. That could be addressed by replacing the #{:as ā¦}
with something more generic like simple-keyword?
.#2017-02-0618:31Alex Miller (Clojure team)^^ @viebel @dergutemoritz#2017-02-0619:11bbloomthat makes some sense to me ^^ another option would be to use conformers to rewrite the :as in to a ::as node, so that itās value can be properly checked to be a simple-symbol, for example#2017-02-0619:12bbloomthat seems like the the way to recover the openness with all the benefits of the global namespaced keys validation#2017-02-0619:21Yehonathan Sharvit@alexmiller I donāt understand why in this case, the open-ness is desirable. I mean if someone passes :wrong-keys
(instead of :keys
) to a let
statement, it should be invalidated...#2017-02-0619:22Alex Miller (Clojure team)@bbloom no desire to use conformers for something like this#2017-02-0619:22Alex Miller (Clojure team)@viebel the same reason openness is desirable elsewhere (see Richās keynote from the conj)#2017-02-0619:23Alex Miller (Clojure team)that is, if you specify a constraint, then removing the constraint later is a breaking change, not an additive change#2017-02-0619:23bbloom@viebel just to make up some straw man extension, consider if you wanted to add some metadata to the match#2017-02-0619:23bbloom{:keys [x y] :types {x int}} ;-)
#2017-02-0619:24bbloomyou donāt want the spec to reject that types#2017-02-0619:26bbloom@alexmiller sorry, i was thinking about :as in vectors - i think conformers make sense there#2017-02-0619:26bbloomfor maps, itās already a map š#2017-02-0619:27bbloomthe :msb form is interesting tho - why not just use a :req-un for the :as key?#2017-02-0619:28bbloomi must be missing some subtlety#2017-02-0619:29bbloomoooh, you also do that, this is about accumulating extra information in the parse - ok, i think that makes sense.... sorry for thinking aloud here#2017-02-0619:29Alex Miller (Clojure team)yes#2017-02-0619:30Alex Miller (Clojure team)this is a particularly tricky instance of what has been called a hybrid map#2017-02-0619:30Alex Miller (Clojure team)which has both a k/v spec + a keys-type spec#2017-02-0619:31Alex Miller (Clojure team)the tricky part being the :<ns>/keys and :<ns>/syms which are like keys but only semi-fixed#2017-02-0619:31bbloomyeah, my experience w/ hybrid maps has been that they are more trouble than they are worth except for very limited syntax use cases#2017-02-0619:31Alex Miller (Clojure team)nonetheless, they exist :)#2017-02-0619:31bbloomindeed#2017-02-0619:31bbloomiāve run in to the ::keys bug#2017-02-0619:31Alex Miller (Clojure team)yeah#2017-02-0619:31bbloomit is not obvious what to do about that š so i just renamed my thing, heh#2017-02-0619:32Alex Miller (Clojure team)well, I have some ideas about that but havenāt had a chance to put a patch on there#2017-02-0619:32Alex Miller (Clojure team)but basically it needs to not use s/keys#2017-02-0619:32Alex Miller (Clojure team)as that has behavior that we donāt want in this particular case#2017-02-0619:33Alex Miller (Clojure team)namely, matching all keys as specs#2017-02-0619:33bbloomyeah, i think the patch i saw there was like a flag to disable the nice open validation of keys - which seems like the wrong way to go about it. easier to factor out the parts you want and then call the underlying part directly#2017-02-0619:33bbloomie composition over parameterization#2017-02-0619:33Alex Miller (Clojure team)that patch is dead, not doing that#2017-02-0619:33bbloomi assumed that š#2017-02-0619:33Alex Miller (Clojure team)and I think I rewrote the ticket to remove that as an option#2017-02-0619:34bbloomcool - i have only been following at a distance#2017-02-0621:47hiredmanit seems like you can't use spec to spec a map like {::results ... ::type ...}
where the value associated with the key ::type
determines what spec is used against the value associated with the ::results
key#2017-02-0621:49hiredmanit seems like the best you could do would be something with multi-spec, with a different ::results
spec (registered under a different key) for each possible value of ::type
#2017-02-0621:57hiredmanI am trying to spec what I have been thinking of as a result set, a map with a structure like {::results [...] ::next ...}
where ::next
is used for pagination, and the items under ::results will vary with the query that was run. my first thought was to slap a type tag on the result set, and use a multi-spec, but that doesn't work.#2017-02-0622:15bbloomhiredman: weāve talked about context sensitivity a bunch of times here in the past - you have a number of options#2017-02-0622:15bbloomnone of them particularly great#2017-02-0622:16bbloomthe simplest one is to just call the spec you want yourself#2017-02-0622:16hiredmantoo opaque#2017-02-0622:16bbloomwell it depends on your use case#2017-02-0622:16hiredmanyeah, for mine it is too opaque š#2017-02-0622:16bbloomheh, ok#2017-02-0622:17bbloomi mean, itās just an s/conform call#2017-02-0622:17hiredmanI think I will just have a bunch of slightly different specs, remove any kind of polymorphism#2017-02-0622:17bbloomyeah, so thatās the next simplest solution#2017-02-0622:17hiredmanfor now, or until something better comes up#2017-02-0622:17bbloomone key for each possible type and then use multi-spec#2017-02-0622:18bbloomso you have like ::foo-results and ::bar-results#2017-02-0622:18hiredmanI am generating documentation and json schema from these specs#2017-02-0622:18hiredmanyeah#2017-02-0622:18hiredmanwell, easiest, since consumers consume json, is to differ based on namespace, which just disappears in the json encoding#2017-02-0622:18bbloomheh, well i guess that works š#2017-02-0622:32bronsawhat about something like
(require '[clojure.spec :as s])
(s/def ::type keyword?)
(s/def ::value any?)
(defmulti t-val identity)
(defmethod t-val :a [_] int?)
(defmethod t-val :b [_] string?)
(s/def ::tagged (s/and (s/keys ::type ::value)
#((t-val (::type %)) (::value %))))
(def foo {::type :a ::value 1})
(def bar {::type :b ::value "2"})
(def baz {::type :b ::value 1})
(s/valid? ::tagged foo) ;-> true
(s/valid? ::tagged bar) ;-> true
(s/valid? ::tagged baz) ;-> false
#2017-02-0622:34hiredmanonce you start building using functions instead of the combinators (like s/keys) explain is less useful, and any kind of treatment of specs as data becomes more of pain (like if you are walking them to generate something else)#2017-02-0710:01p-himikIs it possible, given some data, a keyword for its spec and a keyword for a simple spec that is a part of the data spec, extract all values conforming to the simple spec?#2017-02-0710:02p-himikE.g. I have a map id->val and three specs - for id, for val and for the map itself. And I'd like to extract all vals from the map.#2017-02-0714:42frankso not all of the map entries in this map conform to the map spec, but you want to find the ones that do?#2017-02-0720:42ddellacostahow to specify a map that may be recursive?#2017-02-0721:07joshjonesactually, any map spec is recursive, since a map spec allows keys not specified in the spec#2017-02-0721:08joshjonesbut for clarity's sake, you may want to put the spec name in the :opt
portion of the map spec#2017-02-0721:08ddellacostaitās recursive perhaps, but not useful if it doesnāt validate anything about the map, is it?#2017-02-0721:09joshjonesit will validate anything given in :req
etc#2017-02-0721:09hiredmanuser=> (s/def ::foo (s/or :a number? :b ::bar))
:user/foo
user=> (s/def ::bar :req [::foo])
ArityException Wrong number of args (3) passed to: spec/def clojure.lang.Compiler.macroexpand1 (Compiler.java:6832)
user=> (s/def ::bar (s/keys :req [::foo]))
:user/bar
user=> (s/valid? ::bar {::foo 1})
true
user=> (s/valid? ::bar {::foo {::foo 1}})
true
user=> (s/valid? ::bar {::foo {::foo :a}})
false
user=>
#2017-02-0721:09ddellacostaas it is, I have
(s/or (s/map-of string? keyword?) (s/map-of string? map?))
but that is only recursing one level#2017-02-0721:10ddellacosta@hiredman thanks, that looks like it#2017-02-0721:12ddellacostaoh hrm maybe not with the map I have here#2017-02-0721:12hiredmanuser=> (require '[clojure.spec :as s])
nil
user=> (s/def ::foo (s/map-of string? (s/or :value keyword? :rec ::foo)))
:user/foo
user=> (s/valid? ::foo {"a" :a})
true
user=> (s/valid? ::foo {"a" 1})
false
user=> (s/valid? ::foo {"a" {"b" :b}})
true
user=>
#2017-02-0721:12ddellacostahaha, on cue#2017-02-0721:12joshjonesyou don't even need to do that#2017-02-0721:13joshjones(s/def ::a string?)
(s/def ::mymap (s/keys :req [::a]))
(s/valid? ::mymap {::a "abc" ::mymap {::a "def" ::mymap {::a "xyz"}}})
#2017-02-0721:13ddellacostathat works if your keys are namespaced keywords#2017-02-0721:13ddellacostabut I have string keys here#2017-02-0721:14joshjonesi see#2017-02-0721:14ddellacostabut, Iām filing all of this away for the time I do need such a constructionā¦thanks @joshjones and @hiredman , very helpful#2017-02-0723:48agHow do I generate map with keys and values that correspond to a given (s/keys)
spec?
let's say I have (s/keys :req-un [::id ::name] :opt-un [::money])
based on that I should generate vector like:
[{:id 1 :name "Greg" :money 0.02} {:id 2 :name āAliceā}]
upd: I actually want the keys to be randomly selected out of that spec, not precisely the same keys#2017-02-0723:50bfabry@ag https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/exercise#2017-02-0723:50hiredmanhaving the same key in req and opt doesn't make sense#2017-02-0723:50bfabry^ that is true#2017-02-0723:51ag@hiredman sorry my mistake. fixed it#2017-02-0723:52hiredmanmaybe close the paren too while you are at it#2017-02-0723:53hiredmanit depends sort of on what purpose you are generating them for, but if you generally just want to poke at them, exercise is a way to generate data for a give spec, s/keys created or otherwise#2017-02-0723:55agoh, I guess I got it all wrongā¦ hold on..#2017-02-0723:57agI need to generate map where the keys are ārandomly selectedā out of the s/keys
spec and the corresponding values also generated according to the spec#2017-02-0723:58bfabrysounds like you want another s/keys spec with all the keys optional#2017-02-0723:59agwell, can I read the keys out of the previously defined s/keys spec?#2017-02-0800:00agor somehow tell test.check.generator/map
or hash-map
that I want keys to be out of the spec?#2017-02-0800:02bfabryyou should be able to read the keys out using s/form, assuming it's a simple spec#2017-02-0800:03bfabrycljs.user=> (require '[clojure.spec :as s])
nil
cljs.user=> (s/form (s/keys :req-un [::foo] :opt-un [::bar]))
(cljs.spec/keys :req-un [:cljs.user/foo] :opt-un [:cljs.user/bar])
#2017-02-0800:04bfabryofc if it's a more complicated spec you'll need to walk the form finding the key specs
cljs.user=> (s/form (s/and (s/keys :req-un [::foo] :opt-un [::bar]) identity))
(cljs.spec/and (s/keys :req-un [:cljs.user/foo] :opt-un [:cljs.user/bar]) cljs.core/identity)
#2017-02-0800:05bfabry(being able to drop into a 1.9 repl in sub-seconds rules, thank you planck)#2017-02-0800:09aghmmm, interesting. thanks!#2017-02-0800:24ag@bfabry I did not know about s/form
, just played with it - itās nice. thanks again!#2017-02-0800:25qqqwhat is the standard way of memoizing spec checks? I have a recursive data structure (say a tree), and I don't want it to have to verify the property holds on subtrees every time -- I want to somehow 'cache' "this node passed spec xyz" somehow#2017-02-0800:26qqqwhen memoizing, can we somehow do it with "weak references" so that instead of having gc leaks, the object is thrown away when it's no longer needed ?#2017-02-0800:28qqqI suspect one nice thing about meta data over memoization is that with meta data, with the object is no longer needed, and the gc gcs it, it takes away the metadata too#2017-02-0800:28qqqwith memoization, I fear it'll prevent the gc from doing its work because the spec function will keep th eobject around#2017-02-0800:32hiredmanwhy are you running spec checks over and over on a large structure? like, just don't do that#2017-02-0800:36qqqif I have input argument spec validation on, wouldn't it run the spec every time the function is called?#2017-02-0800:37qqqnow, if this structure is recursive (say: this is a tree, where the sum of every subtree is a multiple of 3), then it ends up checking the entire tree every time#2017-02-0800:38hiredmanmy first concern with that question is it sounds like you are planning to turn on spec instrumentation outside of your test suite#2017-02-0800:38qqqthat is true; is it bad to have spec instrumentation turned on in production code? if it's a costant time hit, I would not mind#2017-02-0800:38hiredmanit is bad, spec is not designed for that#2017-02-0800:39hiredmanI should say#2017-02-0800:39hiredmanthe instrument stuff in spec is not designed for that#2017-02-0800:40hiredmanthe intended use, as far as I understand, is to turn instrumentation on when running your test suite#2017-02-0800:40hiredmanand that is it#2017-02-0800:40qqqI think assertions are good in dev code (not just in test quite). I'm trying to use spec as "ultra powerful assert" in dev code. You're saying this is bad, and I should not use spec as an assert ?#2017-02-0800:43qqqFor some functions, I want to assert that their input satisfies certain pre-conditions. These pre-conditions can be expensive (checking all subtrees); therefore, I'd like a way to cache this.
Now, given that I want to do the above, can I do this with spec, or would spec be a bad match?
I'm hoping that spec can do this, as I really like spec's language.#2017-02-0800:43bfabryimo many many people will use spec the way you describe, but there are some things to be wary of: it will be slow. if you spec any functions that you pass as arguments those functions will be called multiple times to check that they pass the spec as part of instrumentation. instrumentation does not check return values#2017-02-0800:44qqq@bfabry: we can assume I don't need to "run specs on functions passed as inputs" -- and my only conern at the moment being simple data structures and recurisve data sturcutres // functions are way too hard#2017-02-0800:45hiredmanif you don't have any automatic checking turned on, checking will only happen when you ask for it, so just don't ask for expensive checks more than once#2017-02-0800:46bfabryfwiw there's no reason you can't use spec's data description language to validate data without using instrumentation. just call s/valid or s/conform directly. you could even wrap your calls to s/valid and s/conform in something that memoizes based on the object's reference id or whatever if you really want to tweak performance of that validation#2017-02-0800:48qqq(defn my-valid [spec data] (or (contains? (meta data) spec) (s/valid spec data))#2017-02-0800:48qqqor something of that nature, this is clever#2017-02-0800:50bfabryyeah exactly, then you can stick that in :pre if that's your preferred mechanism#2017-02-0800:50joshjonesif the function is in clojure.spec.test
, don't use in prod#2017-02-0800:50bfabryyou could even still associate the spec with the function, so you get the documentation, generation stubbing when wanted etc, just don't call s/instrument#2017-02-0800:51qqqhttp://blog.fogus.me/2009/12/21/clojures-pre-and-post/ <-- this existed 7 years ago, finally starting to use it now#2017-02-0800:52joshjonesthe spec guide under "Using Spec for Validation" gives the :pre and :post example for reference#2017-02-0800:55qqq@joshjones: found it, thanks!#2017-02-0804:20onetom@ag ^^^#2017-02-0804:27onetomIs it expected behaviour to get a clojure.spec/unknown
explanation if a non-conforming spec has a custom generator, like this:
(s/def ::a (s/with-gen string? identity))
(s/def ::some-map (s/keys :req [::a]))
(s/explain ::some-map {::a nil})
In: [:app.deals/a] val: nil fails spec: :app.deals/a at: [:app.deals/a] predicate: :clojure.spec/unknown
#2017-02-0804:28onetomInsead of
(s/def ::a string?)
...
In: [:app.deals/a] val: nil fails spec: :app.deals/a at: [:app.deals/a] predicate: string?
#2017-02-0804:46seancorfieldWhat sort of generator is identity
?#2017-02-0804:46seancorfield(hmm, and doesn't with-gen
take a nilary function that returns a generator?)#2017-02-0804:47seancorfieldYup, "Takes a spec and a no-arg, generator-returning fn and returns a version of that spec that uses that generator" -- identity
does not satisfy that.#2017-02-0804:48seancorfieldSo (s/with-gen anything identity)
doesn't make sense... you're not going to get a valid generator from that?#2017-02-0804:52seancorfieldYeah, if you (s/exercise ::a)
you'll get boot.user=> (s/exercise ::a)
clojure.lang.ArityException: Wrong number of args (0) passed to: core/identity
which is what I'd expect.#2017-02-0804:52seancorfieldSo the error from s/explain
isn't surprising but it is perhaps a bit misleading @onetom#2017-02-0804:53onetomah, sorry, i just threw identity
in there because it's an irrelevant detail#2017-02-0804:54onetomi observed the very same behaviour in our actual app with this real generator:
(s/def :deal/name (-> string?
(s/with-gen #(gen/fmap
(fn [s] (str "<DEAL-NAME-" s ">"))
(gen/string-alphanumeric)))))
#2017-02-0804:56onetom(s/def ::a (s/with-gen string? gen/string-alphanumeric))
(s/def ::some-map (s/keys :req [::a]))
(-> ::some-map s/gen gen/generate)
(s/explain ::some-map {::a nil})
=> #:boot.user{:a "lybK4CU4teXKCnErk9h5ajdGBu"}
In: [:boot.user/a] val: nil fails spec: :boot.user/a at: [:boot.user/a] predicate: :clojure.spec/unknown
#2017-02-0804:56seancorfieldOK, that does s/exercise
...#2017-02-0804:57seancorfieldYeah, that sounds like it's worth a JIRA issue...#2017-02-0804:57seancorfieldI'd expect a better message, at least.#2017-02-0804:59onetomand this is the very first time i wrote a custom generator for an actual real-world use case...
thats my generic experience with software...
sometimes im wondering im just unlucky.
what comforts me slightly is that i have a friend who is an order of magnitude "unluckier" š#2017-02-0804:59onetomok, i will make a JIRA issue. (this will be my first JIRA issue... im already worried what will happen ;)#2017-02-0805:03seancorfieldFunctionally-linked people are very useful in QA'ing software š#2017-02-0805:35onetomI've created http://dev.clojure.org/jira/browse/CLJ-2107 but it seems I can't edit it to correct the markdown syntax in it š#2017-02-0805:48seancorfieldHow's that? (edited)#2017-02-0805:49seancorfield(uses {code}
around code not three backticks)#2017-02-0805:50seancorfieldand {{
}}
instead of backticks for inline code.#2017-02-0807:20hiredmanwith-gen is a function, so it's arguments are evaluated, so the string? argument to with-gen is a function object, when spec is trying to find the name to report it does some stuff, which for symbols and keywords reports a good name, but for other Objects (including function objects) you get :clojure.spec/unknown#2017-02-0807:21hiredmanuser=> (s/def ::a (s/with-gen (s/spec string?) identity))
:user/a
(s/def ::some-map (s/keys :req [::a]))
:user/some-map
user=> (s/explain ::some-map {::a nil})
In: [:user/a] val: nil fails spec: :user/a at: [:user/a] predicate: string?
nil
user=>
#2017-02-0807:21hiredmanwrapping with s/spec allows the spec macro to capture the meaningful, the symbol before evaluation#2017-02-0808:41mpenets/spec also takes :gen so you can avoid calling with-gen separately, I find it nicer personally#2017-02-0808:41mpenet(s/def ::a (s/spec string? :gen identity))
#2017-02-0808:42mpenetI almost never use with-gen
because of this now that I think of it#2017-02-0810:41luxbockI have some data which I can neatly define a spec for using the regexp combinators of spec, which is great because I can generate example data for free, but for the functions I'm testing it's quite important that the returned sequences are vectors rather than list#2017-02-0810:43luxbockcoll-of
and every
allow you to define the type of the sequence, but they don't know about the structure of the sequence#2017-02-0810:44luxbockis there any easier way to force the generators to produce vectors than using a custom generator that calls vec
on them?#2017-02-0810:46linussHey guys, I'm trying to use the simple-type
generator in one of my specs, but I can't seem to get it to work. I currently have (s/def ::any (s/spec (fn[] (true)) :gen #(gen/simple-type)))
but it seems that whatever permutation of the :gen
value I use it returns an error#2017-02-0811:53dergutemoritz@linuss Can you paste the error you get? At any rate, the parens around (true)
look wrong, that means you are calling true
as a function#2017-02-0811:56linuss@dergutemoritz Ah, thanks! Yeah, that doesn't help. However, I'm still getting the following error: java.lang.IllegalArgumentException: No value supplied for key: (fn* [] (gen/simple-type))
#2017-02-0811:57linussoh, hold up#2017-02-0811:57linusstypo on my end#2017-02-0811:57linussjava.util.concurrent.ExecutionException: clojure.lang.ArityException: Wrong number of args (1) passed to: specs/fn--10638
That's the error I keep getting#2017-02-0812:03dergutemoritz@linuss Ah, right, your spec predicate function needs to accept a single argument.#2017-02-0812:04dergutemoritzSo (fn [x] true)
would do the trick. You can use any?
instead, too, which is the core function with the same behavior.#2017-02-0812:05dergutemoritzPlus that clojure.spec
has a default generator for it#2017-02-0812:05linussOh wow, thanks!#2017-02-0812:06dergutemoritzYou're welcome!#2017-02-0812:07linussOh, hm, I can't seem to find any?
. Could you point me to the right location?#2017-02-0812:08dergutemoritz@linuss It's in clojure.core
since 1.9#2017-02-0812:08linussah!#2017-02-0812:33trisshey all, Iām trying to run stest/check
against some functions#2017-02-0812:33trissbut i get the following error:#2017-02-0812:33triss...
{:result #error {
:cause "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath."
:via
[{:type java.io.FileNotFoundException
:message "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath."
:at [clojure.lang.RT load "RT.java" 458]}]
:trace ...
#2017-02-0812:35trissdo i need a particular dependency?#2017-02-0812:36not-raspberryYes. [org.clojure/test.check "0.9.0"]
#2017-02-0812:37not-raspberryAs in the docs https://clojure.org/guides/spec#_project_setup#2017-02-0812:56pbailleHi, i'm curious about how/should spec can be used as dispatching system?#2017-02-0813:45pbailledoes this question even make sense? š#2017-02-0813:59dergutemoritz@pbaille Depends on what you mean by dispatching sytem š#2017-02-0813:59pbaillesomething like multimethods#2017-02-0814:00dergutemoritzNot really.. you can certainly use s/conform
in a multimethod's dispatch function, though#2017-02-0814:00dergutemoritzWhich seems like it could be a useful thing#2017-02-0814:01pbaillei've done a little gist about this, doesn't look really nice... https://gist.github.com/pbaille/b1bc0d05c2ec428e220fa28d28c8354f#2017-02-0814:03pbaillelooks like performance is an issue here, and the try catch stuff is ugly...#2017-02-0814:04pbaillebut that illustrate what i am trying to acheive#2017-02-0814:06pbaillethat's probably a terrible idea š#2017-02-0814:30linussWould it be possible to write a spec for a function with side-effects, like slurp
?#2017-02-0814:38Timmaybe if it takes input?#2017-02-0816:08cryptoratI am trying to write a definition for āweeks in a yearā and having difficulty checking that the integer is < 53.
(s/def ::valid-weeks-in-year (s/and ::non-negative-integer #(< % 53)))
It is failing with java.lang.ClassCastException: clojure.lang.MapEntry cannot be cast to java.lang.Number
. What am I missing?#2017-02-0816:09cryptoratIn a general case, how do I check that a number falls in a range?#2017-02-0816:12linussI think your error is somewhere else, this works without issue on my end#2017-02-0816:13linuss(s/def ::non-negative-integer (s/and int? #(>= % 0)))
(s/def ::valid-weeks-in-year (s/and ::non-negative-integer #(< % 53)))
(s/valid? ::valid-weeks-in-year 10) => true
#2017-02-0816:14ghadi@cryptorat int-in
?#2017-02-0816:14ghadihttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/int-in#2017-02-0816:15cryptoratoh, int-in. That looks like it will work.#2017-02-0816:15cryptoratI wonder if my non-negative-integer is the problem. Let me try with yours.#2017-02-0816:16cryptoratYup, that is it.#2017-02-0816:16cryptoratWell I learned two things. Thank you.#2017-02-0816:23cryptoratThe problem seems to lay in using
`(s/def ::non-negative (s/or :positive pos?
:zero zero?))`
instead of
#(>= % 0)
#2017-02-0816:28cryptoratAnd since or returns a map entry....#2017-02-0816:28cryptoratWell that all makes sense now.#2017-02-0816:54joshjones@cryptorat a non-negative integer is also known as a natural integer. 1.9 has a predicate for this, so you can just use nat-int?
as your predicate, although for your case, as @ghadi said, int-in
is probably more appropriate since you need an upper bound#2017-02-0816:57cryptoratMy understanding is that whether or not zero is included in the set of natural numbers can be debated. I figured best to avoid that in case someone changes their mind. Too cautious perhaps.#2017-02-0816:58joshjonesthis is a good point ā i doubt the definition will change in the clojure universe but good catch nonetheless. but as you have already seen, (s/int-in 0 53)
is much better anyway for your case#2017-02-0817:18akielIf you have two maps where map :a
has :a/id
and map :b
likes to reference a particular map :a
. How would you call the key in :b
? Would you just use :a/id
or would you create a new key called something like :b/a-ref
?#2017-02-0817:33joshjones@akiel an example of the two maps would help clarify#2017-02-0817:36akieleither {:a/id 1 :b/name āfooā}
references {:a/id 1 :a/name ābarā}
or {:b/a-ref 1 :b/name āfooā}
references {:a/id 1 :a/name ābarā}
#2017-02-0817:37joshjoneswhat do you mean āreferencesā ?#2017-02-0817:38akiela big disadvantage of using :a/id
also for such kind of references is, that not every map containing an :a/id
can be considered to be an :a
.#2017-02-0817:38akielby references I mean the same what in relational databases forein keys do#2017-02-0817:39akielI canāt embed the referenced map directly, because that would blow up the data#2017-02-0817:44akielTo make it more spec relevant - I ask because in spec keywords are used as names for something like types and that keywords are also used in maps to name something like attributes.#2017-02-0817:47dergutemoritz@akiel I'd go with a different name, i.e. the :b/a-ref
version. Because a reference to a thing is not the same as the thing itself after all.#2017-02-0817:51dergutemoritzThen again, embedding the data directly shouldn't really have that much of an impact if you have the thing that is pointed to in memory anyhow#2017-02-0817:55akiel@dergutemoritz I lean also towards using :b/a-ref
. Regarding embedding directly you are right, it wonāt cost memory. There I was wrong.#2017-02-0817:56akiel@dergutemoritz But it would cost on wire. I need to transport that data over wire.#2017-02-0817:56dergutemoritzI see#2017-02-0817:59dergutemoritzAnother reason to name it :b/something
is that you can then use a name that describes the relationship. E.g. if :a
is :person
and :b
is :book
then you could have (s/def :book/author :person/id)
which I'd say would even justify leaving off the -ref
suffix.#2017-02-0817:59dergutemoritz@akiel ^#2017-02-0818:00akiel@dergutemoritz Yes you are right - role names.#2017-02-0818:03akiel@dergutemoritz A related thing: Would you go for all keys in a person to start with the namespace :person
or would you also use other common keys in a person? Like :person/name
, :book/name
vs. just :common/name
.#2017-02-0818:09dergutemoritz@akiel I don't know, I think that's subject to an ontological debate you have to have with your domain experts š#2017-02-0818:11dergutemoritzNote that you could have both in a way: (s/def :common/name string?) (s/def :person/name :common/name)
#2017-02-0818:11dergutemoritzThat way you could attach additional meaning to :common/name
and also have it influence :person/name
#2017-02-0818:12akiel@dergutemoritz The domain is given in my case. But I just think about map keys in relation to specs. With 20 different names, you end up with 20 specs for names which are all the same. Iām not sure if thats a bit of an antipattern now regarding to spec.#2017-02-0818:13dergutemoritzIt really depends on your domain#2017-02-0818:13akiel@dergutemoritz Than you have specs like :common/name
which are never used as keys in a map. That doesnāt have to be a problem - just thinking about it.#2017-02-0818:14dergutemoritzYeah, that would be an abstract spec probably#2017-02-0818:15dergutemoritzI guess the question is whether the concept of a name is universal in your domain or specific to each entity. Or maybe a mixture of those.#2017-02-0818:15dergutemoritzBut I don't feel like I've fully figured this out, yet, either#2017-02-0818:19akielFor example I have a transaction type which can have values like :insert
or :update
. It has the same meaning and the same values for each entity. Should I have a transaction type for every entity or just one?#2017-02-0818:26dergutemoritzIf it has the same meaning in all contexts, then I'd go with a single one#2017-02-0818:27akiel@dergutemoritz Thanks, that sounds reasonable. Same name for the same thing. š#2017-02-0818:27dergutemoritzš#2017-02-0818:28dergutemoritzAt least that's my current understanding of things š If anyone else has another interpretation, I'm all ears#2017-02-0818:28akielThat's why I liked to discuss a bit on that matter.#2017-02-0823:00seancorfieldWas that meant for a different channel @zane ?#2017-02-0823:10zaneSure was!#2017-02-0911:17mike_ananevHello! If i have regular fn which returns some number, how to wrap it into generator for spec?#2017-02-0911:37not-raspberryLike, random number? This is pretty much not the point of generators. To my understanding, you're supposed to extend some existing generator by using it as a seed for your generator.#2017-02-0911:39not-raspberrylook at clojure.spec.gen/fmap#2017-02-0911:44mike_ananev@not-raspberry i know about fmap. Can you provide some example how to use it in my question? I have fn (e.g. block-number-fn) which can produce block numbers. I have a spec for block. How to express block-number-fn as gen for block spec?#2017-02-0911:45not-raspberrywhat's a block number?#2017-02-0911:45not-raspberryan id?#2017-02-0911:45mike_ananevit is objects#2017-02-0911:45mike_ananevmore complex structure#2017-02-0911:46not-raspberryand without clojure.spec, how would you generate it? {:www (random) :zzz (random)}
?#2017-02-0911:47mike_ananevi have a Java constructor#2017-02-0911:47mike_ananev(Block. "bla" "bla")#2017-02-0911:48mike_ananevmore simple example#2017-02-0911:48not-raspberrycan you generate it based on 1 random integer/double?#2017-02-0911:49mike_ananevforget blocks#2017-02-0911:49mike_ananevanother example#2017-02-0911:50mike_ananevi need gen for uuid#2017-02-0911:50mike_ananevi know that gen for uuid already in spec#2017-02-0911:50mike_ananevbut if there no one?#2017-02-0911:50mike_ananev(UUID/randomUUID) fn return uuid#2017-02-0911:51mike_ananevhow to wrap fn with (UUID/randomUUID) into gen?#2017-02-0911:51not-raspberryLet's limit ourselves to textual uuids.#2017-02-0911:52not-raspberryyou write a spec for a character allowed in uuid, e.g. a set of those, you get a gen for free#2017-02-0911:52mike_ananevi understand it, this sounds like workaround#2017-02-0911:52not-raspberrythen you write a spec for a coll of those chars with rexactly 24 chars (let's say uuids have 24 chars, i dunno)#2017-02-0911:53mike_ananevhow to wrap fn above into gen?#2017-02-0911:53not-raspberrythen you write your own genetrator with fmap that maps the previous generator with a function that inserts hyphens blocks with#2017-02-0911:54not-raspberryyou miss the point - don't use null-ary functions in specs. That makes them impossible to reproduce.#2017-02-0911:54mike_ananevok#2017-02-0911:55mike_ananevbe reproducable is mandatory?#2017-02-0911:58not-raspberry(defn gen-string-thats-somewhat-funny []
(sgen/fmap
do-something-funny-with-string
(sgen/string-alphanumeric)))
where do-something-funny-with-string
is not funny itself but the results of it should be. Also it's pure.#2017-02-0911:59not-raspberryIs reproducible mandatory? Nothing is pretty much. But it's inconvenient to stand out.#2017-02-0912:00not-raspberryI haven't found an easy way to write "non-pure" generators.#2017-02-0912:01not-raspberryyou can always discard the argument the function passed to fmap gets.#2017-02-0912:01not-raspberryBut the API overall seems to have been designed to discourage that.#2017-02-0912:09gfredericksreproducibility and shrinking are the two big features you get by constructing generators using the combinators#2017-02-0912:24mike_ananev@not-raspberry thanks. went to "hammock"#2017-02-0912:25gfredericks@mike1452 to address your specific example, I would generate a UUID by generating a pair of longs and calling the UUID constructor#2017-02-0912:26gfrederickswhich is essentially what the builtin UUID generator does#2017-02-0912:27mike_ananev@gfredericks can you provide source of builtin gen?#2017-02-0912:27mike_ananevlink i mean#2017-02-0912:27mike_ananevi saw macros name in sources but can't find particular code for this#2017-02-0912:27gfredericks@mike1452 https://github.com/clojure/test.check/blob/test.check-0.9.0/src/main/clojure/clojure/test/check/generators.cljc#L1256#2017-02-0912:28mike_ananevthanks#2017-02-0912:29gfredericksas the comments say, the jvm code there is lower level because it's faster; that wouldn't be a normal way to build generators#2017-02-0912:40mike_ananev@gfredericks thank you, I will try it#2017-02-0915:04ericnormandHello!#2017-02-0915:04ericnormandI have a question about the clojure.spec guide#2017-02-0915:04ericnormandit says:
> The map spec never specifies the value spec for the attributes, only what attributes are required or optional.#2017-02-0915:05ericnormandbut then it does look like the values are being specified#2017-02-0915:05ericnormandthe next sentence is:
> When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value.#2017-02-0915:06not-raspberryyou mean s/keys
?#2017-02-0915:06not-raspberryit specifies keys that specify values#2017-02-0915:07joshjones@ericnormand so what is the question eric?#2017-02-0915:08ericnormandthey seem to contradict#2017-02-0915:08ericnormanddoes the s/keys specify values or not?#2017-02-0915:09ericnormandit seems that it does#2017-02-0915:10joshjonesno, it does not, only keys. however, when a map itself is validated or checked, the check is done to ensure that the keys specs are present, and that the values that those key specs specify are correct#2017-02-0915:10ericnormandok, that's what I'm saying#2017-02-0915:10ericnormandthe first sentence says it doesn't specify the value#2017-02-0915:10joshjonesit doesn't#2017-02-0915:10ericnormandit's a direct contradiction#2017-02-0915:11ericnormandit does#2017-02-0915:11ericnormandyou just said it does#2017-02-0915:11joshjonesiāll give an example, just a min#2017-02-0915:11ericnormandhere's my example:#2017-02-0915:11ericnormand(s/def ::name string?)
(s/def ::person (s/keys :req-un [::name]))
(s/explain ::person {:name 3})
#2017-02-0915:11ericnormandthis fails with this message: In: [:name] val: 3 fails spec: :foo.core/name at: [:name] predicate: string?
#2017-02-0915:12ericnormandthe value is clearly expected to be a string#2017-02-0915:12ericnormandperhaps it's just a problem with the wording of that sentence#2017-02-0915:13ericnormand> The map spec never specifies the value spec for the attributes, only what attributes are required or optional.#2017-02-0915:13ericnormandto me, that means that s/keys
only deals with existence of keys, not their values#2017-02-0915:15joshjonesyou quoted the guide which says: āThe map spec never specifies the value spec for the attributesā ā is this the part that you say is confusing?#2017-02-0915:15tbaldridge@ericnormand it means that s/keys never contains specs for the values, aside from the values spec'd by the keys themselves#2017-02-0915:16ericnormand@joshjones yes#2017-02-0915:16ericnormand@tbaldridge I see, I totally misunderstood#2017-02-0915:16joshjones(s/def ::mymap (s/keys :req [::mykey]))
(s/def ::mykey number?)
::mymap
does not specify the value spec for the attribute above. It only specifies which key is required.#2017-02-0915:16tbaldridgeMy favorite analogy: A "Car" is (s/keys [::wheels ::seats ::engine]), doesn't say what a ::wheel is, it just says "A car contains these things"#2017-02-0915:16ericnormandbut it implicitly says the value has to be a number#2017-02-0915:16joshjones::mykey
can change all day long, and the spec for ::mymap
doesnāt change#2017-02-0915:17joshjonesthe spec for the KEY says the value has to be a number, NOT the spec for the map#2017-02-0915:17ericnormandok, thanks#2017-02-0915:17ericnormandbut I still think it's unclear enough to warrant a rephrasing#2017-02-0915:17tbaldridgeBut if you say "Is this a car?", it has to check to make sure that what you say is a ::wheel is actually a ::wheel#2017-02-0915:17ericnormandif I had trouble with it, others probably will too#2017-02-0915:19tbaldridge@ericnormand isn't that all covered in the spec guide? From the section on entity maps:#2017-02-0915:19tbaldridge"Clojure programs rely heavily on passing around maps of data. A common approach in other libraries is to describe each entity type, combining both the keys it contains and the structure of their values. Rather than define attribute (key+value) specifications in the scope of the entity (the map), specs assign meaning to individual attributes, then collect them into maps using set semantics (on the keys). This approach allows us to start assigning (and sharing) semantics at the attribute level across our libraries and applications."#2017-02-0915:20ericnormandthat combined with the sentence I quoted allows for the interpretation I made#2017-02-0915:20ericnormandthat keys are what matter#2017-02-0915:20ericnormandand that if you want to conform the value yourself, you can#2017-02-0915:21ericnormandplease take my suggestion for what it is#2017-02-0915:21ericnormandI think it's unclear and would help others if it were rephrased#2017-02-0915:21ericnormandI don't want to argue about it š#2017-02-0915:21joshjonesI can understand why itās confusing ā (s/valid ::somemapspec {ā¦})
could only check that the keys are there. then another (s/validextra ::somemapspec {ā¦})
could validate both that the keys are present, and that the keys themselves are valid. i get it ā¦ but thatās not how it works#2017-02-0915:22tbaldridgeI don't understand: https://clojure.org/guides/spec already explains this about 5 different ways in the section labeled "Entity Maps". I'm not trying to argue, I'm just not sure how it could be any more explicit.#2017-02-0915:22ericnormandit explains it five different ways, but the first way it is explained can be read the wrong way#2017-02-0915:22ericnormandand it can color how the rest are interpreted#2017-02-0915:23joshjonesiāve not heard anyone ask this question before, but Iām sure some people have wondered. more people are confused about how keys that are in the map are checked, even if theyāre not given in :req
and :req-un
#2017-02-0915:23ericnormandthanks for your help!#2017-02-0915:24tbaldridgeBut I mean....that's the 3rd paragraph in the section. And your initial question about "the map spec never specifies" is tied to a code example as context.#2017-02-0915:29tbaldridgeTo be frank, I think "read from top to bottom taking context into account" is assumed as part of the guide. At least that's my opinion.#2017-02-0915:30mpenetyeah I got surprised by this one once (s/def ::foo string?) (s/valid? (s/keys) {::foo 1})
-> false#2017-02-0915:31mpenetkinda makes sense after all, but I guess doesn't come to mind at first#2017-02-0915:38tbaldridgeYeah, that one is a bit surprising#2017-02-0915:38tbaldridge(didn't know about it till I read the guide just now)#2017-02-0915:39mpenetit makes namespaced kw very specific to specs#2017-02-0915:39mpenetmaybe too much, but that's a matter of opinion I guess#2017-02-0915:41mpenetone could argue it goes against the "specify what you want in there and ignore the extras" direction spec took#2017-02-0915:43not-raspberryspecifying against extra keys makes it impossible to write backwards-compatible specs#2017-02-0915:44mpenetyeah the argument you'll hear is "create new keys"#2017-02-0915:44mpenetwhich is again, arguably, a bit odd#2017-02-0915:45not-raspberry(actually, future-proof, not backwards-compatible - it forces you to synchronize 2 services when adding an extra key)#2017-02-0915:50mpenet@not-raspberry not sure we're talking about the same thing tho. I was referring to the example I mentioned, not to the fact that s/keys allows extra (unespecified) keys to be valid#2017-02-0915:51not-raspberryI was just guessing the rationale for s/keys behaviour.#2017-02-0915:51mpenetthat is actually a good thing as you mention. the example I mentioned, is, odd.#2017-02-0915:52tbaldridgeIt is an interesting topic. One one hand I do like that any value of my map will be valid after I've run it through conform.#2017-02-0915:52joshjones@mpenet why is your example surprising?#2017-02-0915:52tbaldridgeOn the other hand it is silent behavior (code that is executed not based on any specs I wrote in this context).#2017-02-0915:53mpenetit ran valid? on a key that's not present in (s/keys) (the spec)#2017-02-0915:53mpenetand failed#2017-02-0915:53tbaldridge@joshjones because the overall language around spec has been "specify what you want to check, and allow the rest to be whatever".#2017-02-0915:53mpenetin my book, this is a bit too surprising. Unless you read the doc very carefully you wont know about this behavior#2017-02-0915:54joshjonesbut the spec guide clearly states that all keys in a map will be checked for conformance#2017-02-0915:54mpenetAs I said, it's arguable.#2017-02-0915:54joshjonesitās definitely a āgotchaā but itās something that is pretty quickly found, if youāre writing specs, IME#2017-02-0915:55mpenetwell the fact that @tbaldridge didn't know about it might be revealing š#2017-02-0915:55tbaldridgeFunny enough I've been writing specs for months, and although I've encountered this, I've never understood it until now.#2017-02-0915:55tbaldridgethat's really my fault though for never reading the guide š#2017-02-0915:56joshjonesthe usual case is, write a key spec, write a map spec that doesnāt include that key, and put an invalid value for that key ā thatāll uncover the āgotcha"#2017-02-0915:56tbaldridgeFor better or worse, I tend to skim guides. So I like my APIs to be explicit. I can handle (s/keys :req [...] :also-check true), since I can go look up what :also-check does. It's silent behavior that can become a "gotcha"#2017-02-0915:57mpenetI never encountered it personally, we have probably thousand of lines of spec and it's 99% un-* keys#2017-02-0915:57joshjonesi do agree that the behavior is not the most intuitive ā but because iām dense, i usually read the guides very thoroughly š#2017-02-0915:57mpenetI think I saw it mentioned here#2017-02-0915:57tbaldridgealthough all of this is also in the doc string, so double bad on me for not reading that either š#2017-02-0915:57joshjoneswell, normally when you write specs for regular situations, youāre not going to put a key in the map that is not specād in the map spec. but when youāre learning and ātinkering aroundā, trying to figure things out, itās pretty normal thing to do imo#2017-02-0915:59mpenetit makes kinda useless (other than for doc) :opt for instance#2017-02-0915:59mpenetI think that's actually what the doc says#2017-02-0915:59joshjonesyes š#2017-02-0915:59mpenetbut odd nonetheless#2017-02-0915:59mpenetah, and gen#2017-02-0915:59mpenetso not 100% useless#2017-02-0916:03mpenetIt has one big drawback tho, if you have a "giant map" and need to only validate 1 key it will still try to do its thing on all the others#2017-02-0916:03mpenetthis won't show on synthetic benchmarks like the one that's on github, but i am not sure the other frameworks do this#2017-02-0916:04mpenet(pretty sure they don't actually)#2017-02-0916:19tbaldridge:opt is also useful for gen, as I don't think s/keys gens other keys, only :req and :opt#2017-02-0916:21mpenetyep#2017-02-0919:21bbloomJust a random note on the open-maps discussion: I discovered that TypeScript allows open js objects but has an interesting feature called āexcess property checksā - essentially, it only disallows extra keys at compile-time for literal values. This way it catches typos etc, but doesnāt prevent future growth. A pretty neat solution, I think.#2017-02-0921:13onetomWe just had an awesome introduction to spec
at the Clojure Remote 2017 conference!
Adoption will increase for sure š#2017-02-0921:54Alex Miller (Clojure team)@ericnormand just skimming the backchat, is there something that needs to be clarified in the guide? if so, let me know or file an issue at https://github.com/clojure/clojure-site/issues#2017-02-0922:03Alex Miller (Clojure team)issue welcome even if not sure#2017-02-0922:21ericnormandthanks, alex!#2017-02-1000:13bsimacan I use a multispec on a vector of tags, instead of just a single keyword?#2017-02-1000:13bsimahow would I write such a defmulti
?#2017-02-1000:28bsima(defmulti event-type
(fn [m] (->> m :event/type (into #{}))))
#2017-02-1000:28bsimahere's what I came up with#2017-02-1000:29bsimaturns the vector of tags into a set, so the multispec can use the set as a predicate function for the multimethod dispatch#2017-02-1000:31bsimathis way I can combine multiple defmethods
. If we take the example of :event/type
from the official clojure spec guide, then I could do:
{:event/type [:event/search :event/error]
:event/timestamp 1463970123000
:error/message "Invalid host"
:error/code 500
:search/url ""}
#2017-02-1000:32bsimaand so both :event/search
and :event/error
would be valid specs#2017-02-1001:28Alex Miller (Clojure team)Yeah, there are no constraints on multi-spec as long as you follow the pattern#2017-02-1002:05dragoncubeIn docs for s/keys there is āNote: there is
no support for inline value specification, by design.ā. Does some expanded explanation of this exist somethere?#2017-02-1002:41dragoncubeis it possible to specify spec for value referred by specific key in the map? I.e I want to say that {:id āmy-idā} āmy-idā should be validated by spec ::resource-id#2017-02-1002:43dragoncubecurrently if you use s/keys there is only implicit assignment of the spec in case the keys are same#2017-02-1002:53Alex Miller (Clojure team)@dragoncube there is some additional info about that at http://clojure.org/about/spec#2017-02-1002:54Alex Miller (Clojure team)re rationale that is#2017-02-1002:54Alex Miller (Clojure team)re the second question, the key must match the spec name#2017-02-1002:54Alex Miller (Clojure team)the idea here (also in the rationale) is that we are assigning enduring semantics to an attribute that can be used throughout the system#2017-02-1002:55Alex Miller (Clojure team)it is admittedly a different philosophy than something like Schema#2017-02-1002:57dragoncubeyeah, but it is quite hard to work with the external data which shape you donāt control#2017-02-1002:58dragoncubefor example, {:id āres-1ā :name āMy resource 1ā :sub {:id āsub-res-1ā āSub-resource 1"}}#2017-02-1003:00dragoncubeshould I use synthesized keys for registering specs or I need to mimic data structure with packages structures?#2017-02-1003:01dragoncubeIt is so tempting to use :: for specs registration#2017-02-1003:04dragoncubefor example above I would like to register two specs: ::resource-id and ::sub-resource-id in the same namespace#2017-02-1003:04Alex Miller (Clojure team)well you can certainly do that along with s/keys and :req-un which will only match the name part, not the namespace part#2017-02-1003:08dragoncubeyes, it is possible but in this case you need either give it synthesized (not existing) namespace part like :rest-resources/id and :rest-resources.subresource/id or literally create āsubresourceā package and define ::id spec there#2017-02-1003:10dragoncubeboth of these are far from ideal#2017-02-1013:27luxbockhaving s/vcat
would definitely be quite helpful#2017-02-1013:29tcouplandespecially if you could unform back to a vector#2017-02-1013:47Alex Miller (Clojure team)Agreed :)#2017-02-1013:47Alex Miller (Clojure team)Rich is as well I think but hasn't decided how it should be implemented yet #2017-02-1013:57tcouplandhow do people feel about something like this:
(defn coerce
[spec data]
(let [conformed (s/conform spec data)]
(if (= :clojure.spec/invalid conformed)
conformed
(s/unform spec conformed))))
I've been avoiding conform == coerce, after getting bitten, but this has got me thinking of dabbling again.#2017-02-1015:19luxbockI wish the :gen
overrides of stest/instrument
behaved the same way as the :gen
of stest/check
#2017-02-1015:20luxbockright now you can only use it for functions which have been stubbed#2017-02-1017:03msshey all, quick q about how instrument
and check
work in cljs.
Iām running a figwheel repl in cursive. the repl is choking and timing out when I eval an instrument
or check
call for an fdef
āed function . further evaluations ā even of simple forms (e.g. (+ 1 1)
ā then result in an eval timeout.
posted in the figwheel channel already but feel like I might just be misunderstanding a core spec concept and how Iām supposed to use it#2017-02-1018:19Alex Miller (Clojure team)that sounds weird#2017-02-1018:20Alex Miller (Clojure team)instrument should not take a long time to execute#2017-02-1018:20Alex Miller (Clojure team)check certainly can take a while as it runs 1000 tests by default - you can change that by passing it additional options. although most often adjusting your generators is what really needs to happen.#2017-02-1018:33mssyeah it feels like I must be doing something wrong.
fwiw my spec is just a set of a few dozen string values (i.e. (cljs.spec/def ::my-spec #{āsomethingā āsomething-elseā ā¦})
.
my fdef
looks something like
(cljs.spec/fdef my-bool-returning-func
:args (s/cat :my-spec-val ::my-spec)
:ret boolean?)
simply calling "cljs.spec.test/instrument `my-bool-returning-funcā
or ācljs.spec.test/check `my-bool-returning-funcā causes the same choking behavior#2017-02-1018:33mssdonāt know if thatās idiomatic/correct or not#2017-02-1021:42assoc-inI am looking at using spec for validation for a SPA app. I am hoping that I can use the same specs for both the front and backend. I am guessing that I would place them in a cljc file, but I was wondering how the s/def would work with s referring to clojure.spec and cljs.spec. Any ideas?#2017-02-1021:53bbloomwhatās the preferred idiom to āoptionalizeā some map with :req keys? specifically, something like (s/merge ::foo (s/or ::bar (s/keys)))
#2017-02-1021:54bbloomthat is, defining a new spec that is a ::foo and optionally also a ::bar#2017-02-1022:05Alex Miller (Clojure team)@assoc-in the new cljs feature for automatic aliasing means that I think you can just use the clojure namespace in the cljc file and it will work for both#2017-02-1022:06Alex Miller (Clojure team)but you should check with David in #clojurescript#2017-02-1022:07Alex Miller (Clojure team)@bbloom not sure I understand what youāre asking?#2017-02-1022:09bbloom@alexmiller to make a simplification of what iām doing concrete: imagine i have some spec (s/def ::syntax (s/keys :req [::line ::column]))
and i have some other AST nodes that might or might not have line/column information on it: (s/def ::ast (s/keys :req [::type ::op ::whatever] :opt [::line ::column]))
#2017-02-1022:09bbloomhowever, i have itās not just two fields, itās quite a few and i want to basically say āan ast node is these keys and optionally a ::syntax"#2017-02-1022:09bbloomis that clearer?#2017-02-1022:10Alex Miller (Clojure team)you could define ::syntax
with :opt
instead and just s/merge
?#2017-02-1022:10bbloomideally not, since the parser requires everything be a ::syntax with proper line and column info#2017-02-1022:10bbloomright now iām doing s/or with an empty (s/keys)#2017-02-1022:10Alex Miller (Clojure team)then why did you say optionally above?#2017-02-1022:11Alex Miller (Clojure team)in that case, just merge ::syntax
and ::ast
?#2017-02-1022:11bbloomthereās two parts of the system#2017-02-1022:11assoc-in@alexmiller okay thanks I'll ask#2017-02-1022:11bbloomthe parser requires all the output have ::syntax, and the analyzer allows it#2017-02-1022:11bbloomie not every node that comes out of the analyzer has line/col info on it - but in the parser, it is required#2017-02-1022:12bbloomi donāt really want :opt tho, since if there is a ::line, i expect there to also be a ::column#2017-02-1022:12bbloomthey arenāt individually optional#2017-02-1022:12bbloomhence (s/def ::ast (s/merge (s/keys :req [.....]) (s/or ::syntax (s/keys)))
#2017-02-1022:12Alex Miller (Clojure team)why do you need to do anything in the analyzer? the s/keys
you already have in ::ast
will validate those#2017-02-1022:13bbloomb/c it only analyzes them independently#2017-02-1022:13bbloomso if i do (assoc foo ::line 1) that should be a problem b/c thereās no column#2017-02-1022:13bbloomthe s/or doesnāt really solve that problem tho#2017-02-1022:14bbloomthe s/or does help generation tho#2017-02-1022:15Alex Miller (Clojure team)I donāt think there is a way to do it in keys, so youād need an extra constraint#2017-02-1022:16bbloomi figured as much š was wondering if there was a particular pattern that has been successful for this sort of thing#2017-02-1022:16Alex Miller (Clojure team)donāt think Iāve seen it#2017-02-1022:17Alex Miller (Clojure team)I think I would leave the :opt in ::ast to get gen and and that with a custom predicate#2017-02-1022:18Alex Miller (Clojure team)the pred should filter any generated maps that donāt satisfy the pred#2017-02-1022:18Alex Miller (Clojure team)in gen that is#2017-02-1022:18bbloomhm ok#2017-02-1022:19bbloomit feels like a symmetric operation to s/?#2017-02-1022:19bbloomor maybe s/nilable#2017-02-1022:19bbloomoooh#2017-02-1022:19bbloomwould that work? (s/merge (s/keys ā¦) (s/nilable ::syntax))
#2017-02-1022:21bbloomah, sadly not#2017-02-1022:21bbloomwell waitā¦ hmmm maybe it sorta does? just doesnāt seem to do what iād want in gen#2017-02-1022:23assoc-in@alexmiller Thanks I asked David and he said I can just use clojure.spec everywhere in the cljc file and I'll be good to go. I am really happy to see that spec's will be usable across both clojurescript and clojure so easily#2017-02-1022:26bbloom@alexmiller hereās what i want to work š https://gist.github.com/brandonbloom/5d89fe87cb6fe34844423f9e88939085#2017-02-1022:27bbloomthatās a pretty minimal repo of a shortcoming of generate on s/merge with s/nilable#2017-02-1112:41trissok this is a long shot I think but just double checking...#2017-02-1112:41trissso iāve got a vector of vectors#2017-02-1112:42trissand i want to ensure that the integers within it are no greater than the number of vectors in the outer one#2017-02-1112:43trisscan i say this with spec?#2017-02-1112:44triss(s/coll-of (s/coll-of (s/and int? #(>= % 0))
:kind vector?
:count 8)
:kind vector?
:min-count 2)
#2017-02-1112:46trissis there a way of referencing the size of the outer vector here?#2017-02-1113:05dergutemoritz@triss Only by wrapping the check around the outer s/coll-of
with s/and
AFAICT#2017-02-1113:05triss@dergutemoritz thanks man. just realized this!#2017-02-1113:06dergutemoritzYW!#2017-02-1118:07Yehonathan SharvitWhat is the proper way to integrate spec.test/check
with clojure.test
?#2017-02-1205:23zaneSee the pinned items in this channel.#2017-02-1208:23Yehonathan SharvitItās not clear from the guide how to use spec inside deftest#2017-02-1208:22nblumoeDoes anyone have experience with using clojure.spec in a scenario where numerical precision becomes an issue? I would like to have generative testings for a function whose output would match the input (except for some numerical imprecision due to matrix calculations happening). I struggle to come up with a good :fn
spec which does not require crippling the generator. A simple :fn #(m/equals (:ret %) (-> % :args :matrix) 1e-6)
does not work, because the generator ends up using ālargeā doubles which break the max. diff of 1e-6
. I tried a spec using ratios but this ends up with āCouldnāt satify such-thatā issues: #(-> (m/div (:ret %) (-> % :args :matrix))
(m/sub 1.0)
m/abs
m/maximum
(< 1e-8))
#2017-02-1208:23nblumoeI am pretty sure I could constrain the generator in a way to make the simple check work but of course I would like to avoid doing that.#2017-02-1208:23nblumoeideas?#2017-02-1213:58gfredericks@nblumoe what does the function you're trying to spec do?#2017-02-1215:25nblumoe@gfredericks a simplified answer would be: Principal component analysis and then reconstruction of the original data from scores and loadings. Some special twists due to the specific domain of the problem though (chemistry).#2017-02-1215:35gfredericks@nblumoe so your problem is that the algorithm breaks down at extreme values, but it would be arbitrary and artificial to impose constraints on the values in the spec?#2017-02-1215:36nblumoeno, the algorithm works fine. the only issue is specing it reliably without making the generator too specific (with āspecingā I mean including generative testing)#2017-02-1215:36nblumoe(well, if by āalgorithmā you mean the :fn
spec predicate then YES š )#2017-02-1215:37gfredericksokay, so it's not that the algorithm doesn't work at extreme values, just that the arbitrary equality threshold of 1e-6
breaks?#2017-02-1215:38nblumoeyes exactly. because this criterion is easy to break for example when using doubles in the range of 10e100
or whatever large enough#2017-02-1215:38gfrederickscan you make the equality threshold a function of the input data?#2017-02-1215:38gfredericksthat might have the advantage of giving you a tighter threshold at the other end as well#2017-02-1215:39nblumoeI was trying that too but ended up with unsatisfied āsuch-thatā issues. would also need to set the threshold element wise (e.g. if a matrix contains 1e100 as well as 1e1 those should have different thresholds then)#2017-02-1215:40gfrederickswhat does such-that
have to do with it?#2017-02-1215:40nblumoeusing ratios of the inputs and returns was how I tried to circumvent the issues....#2017-02-1215:41nblumoequote from above: āI tried a spec using ratios but this ends up with āCouldnāt satify such-thatā issues"#2017-02-1215:41gfredericksasserting the ratio should be close to 1?#2017-02-1215:41gfredericksare you using such-that directly, or is it generated by spec because of a spec/and
?#2017-02-1215:42nblumoethe such-that
issue basically means that the generator was not able to find valid data within 100 tries, which can happen when the spec is quite specific and the search space for data is rather large#2017-02-1215:42nblumoe(lol pun not intended)#2017-02-1215:43nblumoeindeed I am using s/and
#2017-02-1215:43gfredericksI'm just not seeing why the "using ratios" idea entails modifying the input spec; I'd imagine it just means modifying the :fn
on the spec#2017-02-1215:43gfredericksā edited#2017-02-1215:44nblumoeyeah I was also surprised that changes to :fn
could result in those issues#2017-02-1215:44gfredericksI'm 80% sure changes to :fn
should be independent of the probability of such-that errors; what does your s/and
look like?#2017-02-1215:45nblumoeI would have that that only :args
could have such effect. is anything generated for the :fn
itself maybe?!?#2017-02-1215:45nblumoe:args
is a bit complicated tbh: :args (s/and (s/cat :matrix (s/and ::matrix
::range-transformable
::non-zero-row-sums
::non-zero-length-rows
#(s/valid? ::non-zero-length-rows (range-transformation %)))
:const-row-sum (s/int-in 1 100)
:num-eigenvectors (s/with-gen pos-int?
#(s/gen (set (range 0 20)))))
; upper limit of eigenvectors
; should not be hardcoded
#(= (:num-eigenvectors %)
(min (count (first (:matrix %)))
(count (:matrix %))))
#(s/valid? ::non-zero-row-sums (-> (:matrix %)
range-transformation
(initial-loadings (:num-eigenvectors %)))))
#2017-02-1215:47gfredericksthe such-that errors might be nondeterministic -- are you sure you don't get them with the non-ratio code?#2017-02-1215:47nblumoeusually that does not cause any such-that
issues on generation, works pretty flawlessly accept for the aforementioned changes to :fn
#2017-02-1215:47gfredericksmaybe your test fails before it's likely to encounter them?#2017-02-1215:47nblumoecan test againā¦ how many tries should I give it? š#2017-02-1215:47gfredericksI'm not sure what you're asking#2017-02-1215:48nblumoesry, edited#2017-02-1215:48gfredericksI think just try s/exercise
on the args spec#2017-02-1215:50gfredericks(s/exercise (s/and (s/cat ...) ...) 10000)
#2017-02-1215:50gfredericksprobably wrap that in (count ...)
š#2017-02-1215:50gfredericksto avoid printing big things#2017-02-1215:56nblumoeindeed! it occasionally fails, so the issue is rather the :args
spec and the generator....#2017-02-1216:02gfredericksit could be either of the s/and
s, or both#2017-02-1216:03gfredericksthe way s/and
works as a generator is to generate something from the first spec and then filter on the rest of them; so the generator of the first spec needs to be highly likely to generate something that passes all of the predicates, or you get the such-that error#2017-02-1216:04gfrederickssometimes you can simply restructure the s/and
to tailor to that, and it works out; other times the only way to get a good generator is to write a custom one for the s/and
#2017-02-1216:04gfrederickssince you have nested s/and
s it's not obvious which one is the problem#2017-02-1216:04gfredericksI'd try exercising the inner one and see what you get#2017-02-1216:06nblumoeok thanks, that is valuable information! I already have a custom generator for ::matrix
but that is not really tailored in any way to satisfy the following predicates. will do some exercise
s š#2017-02-1216:20nblumoegreat! the outer s/and
was the issue: the second predicate for :num-eigenvectors
was apparently failing too often#2017-02-1216:21nblumoethanks a lot @gfredericks !#2017-02-1216:25gfredericksnp#2017-02-1216:25nblumoethat specific predicate also does only cover one case where the number of eigenvector matches either the number of columns or number of rows. that is only one specific case though (in which the input data will be reproduced). I need to think about how to cover the usage with less eigenvectors than thatā¦ the invariant for the generative testing seems to be quite tricky though then#2017-02-1308:28dbushenkohi all!#2017-02-1308:28dbushenkohow to test side-effected functions with clojure.spec?#2017-02-1308:55Oliver GeorgeThat complicates things but there are some features you might like to check out. Pretty sure you can stub out functions. That might let you isolate you're logic for testing. #2017-02-1308:55Oliver GeorgeI haven't used that feature myself. #2017-02-1310:52not-raspberry@dbushenko Attach the spec as a validator to the var/ref/agent/atom, maybe... https://clojuredocs.org/clojure.core/set-validator!#2017-02-1311:22dbushenkothanks!#2017-02-1312:44joshjones@dbushenko @olivergeorge regarding stubbing a function ā after fdef
ing the function, use (stest/instrument `your-func {:stub #{`your-func}})
to use the :ret
in the fdef
instead of calling the real function, fyi#2017-02-1313:25cgrandevery
doesnāt use res
on its element predicate argument:
=> (s/form (s/every string?))
(clojure.spec/every
string?
:clojure.spec/kind-form
nil
:clojure.spec/cpred
#object[user$eval8666$fn__8668 0x5ba5f0bd ā
Iāve found http://dev.clojure.org/jira/browse/CLJ-2035 which covers that but with a larger scope.
@alexmiller Would it be sensible to open a ticket just for the resolution of the predicate?#2017-02-1313:33Alex Miller (Clojure team)No, please add to 2035 if it's not already covered by the patch there#2017-02-1313:34Alex Miller (Clojure team)I think there may actually be another ticket for this already though#2017-02-1313:36Alex Miller (Clojure team)Nah, I was thinking of 2059 which is in explain-data#2017-02-1313:38cgrandok thanks#2017-02-1314:50mssare js objects spec-able without converting to cljs data structures?#2017-02-1315:24gfredericksyou can always write arbitrary predicates that check whatever you want#2017-02-1315:30sveriHi, I have a list of maps and each map has an :idx key which is an int. Now what I want is that given one list every :idx is a unique int. Is there a way to enforce that?#2017-02-1315:36frank(= (count my-list) (-> (map :idx my-list) distinct count))
#2017-02-1315:37sveriCool, even exercise respects that function, thank you @frank#2017-02-1315:39franknp!#2017-02-1316:46Alex Miller (Clojure team) @sveri the collection spec have a :distinct option#2017-02-1316:46Alex Miller (Clojure team)Although I guess that goes farther than you want#2017-02-1316:56piotr-yuxuanHi here! Letās say Iāve got a map of user. The key ::password
must be present if and only if a user is new (the key ::new?
is bound to a truthy value). Is it possible to express this with clojure.spec
?#2017-02-1317:00sveri@alexmiller I have seen that, but I suspect it means distinct for the whole map and not only for the :idx key#2017-02-1317:04Alex Miller (Clojure team)Yep#2017-02-1317:04Alex Miller (Clojure team)@piotr2b: sure, use a custom predicate#2017-02-1319:41Oliver George@mss if you can convert to clojure data structures first then you'll get access to more spec features. In particular s/keys & all the standard generators #2017-02-1323:42gfredericks@sveri test.check has distinct-by
generators for that sort of thing; I don't know if spec exposes it in the collection specs or not#2017-02-1409:42dbushenko@joshjones: thanks for your answer!#2017-02-1410:06kirill.salykinHi guys,
I am new to clojure spec, It looks like it can be used for data validation and coercion.
eg checking if if username and password provided, or if phone number in correct format.
But I donāt know how I can show proper error message for failed spec? I was not able to attach any meta data to specs.
Any advice on this? Maybe someone got similar ideas?
Thanks!#2017-02-1410:16tcoupland@kirill.salykin try explain, explain-str or explain-data#2017-02-1410:50kirill.salykin@tcoupland contains failed predicate,
but what I want is to have a custom message in explain-data
something like, (s/def ::year integer?, :message āwrong formatā)
I tried to attach it as meta-data - but no luck#2017-02-1410:57Oliver GeorgeI don't think that's possible. #2017-02-1410:58Oliver GeorgeWould a lib focused on user input validation be more appropriate?#2017-02-1410:58Oliver George(Vs data structure validation)#2017-02-1411:00kirill.salykinBut clojure.spec looked so good to describe form objectsā¦
I guess Iāll find some validation library, thanks#2017-02-1411:02Oliver GeorgeI agree it's great for that. #2017-02-1411:04sveri@kirill.salykin I was at a local clojure meetup lately and @akiel presented this library: https://github.com/alexanderkiel/material-comp
I totally liked the approach and think this is the direction to go.
It basically normalizes a spec and returns a generic error message for it. Everything is based on explain-data.#2017-02-1411:05sveriThere is not much in it right now, but it should get you started.#2017-02-1411:05kirill.salykinthanks a lot!#2017-02-1411:33akiel@kirill.salykin I would be happy to hear what you think about it. Iāll certainly separate the validation aspect from the material-ui lib. There are also some slides from the meetup: http://www.slideshare.net/alexanderkiel/form-validation-with-clojure-spec#2017-02-1414:32witekWhat is the right way to provide a predicate which is always true? I want a spec which is a tuple of a keyword and anything. How to spec the anything?#2017-02-1414:33thhellerany?
was added for this purpose#2017-02-1414:58witekAnd how to spec a tuple with an optional element?#2017-02-1414:59witek[:a] or [:b :c] should be valid. Is this possible with a simple tuple? Or do I need an or?#2017-02-1415:09joshjones@witek one way is:
(s/def ::a-or-bc (s/or :a (s/tuple #{:a})
:bc (s/tuple #{:b} #{:c})))
#2017-02-1415:09joshjonesdepending on whether it's nested, you can also use cat
instead of tuple
#2017-02-1417:54pesterhazya wiki of "how do I spec X?" snippets would be useful#2017-02-1418:40tbaldridgeoooh...something like that talk at the last Clojure/Conj where you write example data and the computer starts generating specs until it finds one that matches all your example input?#2017-02-1419:07richiardiandreaIf I have:
(s/fdef resource->relationship
:args (s/cat :from-fn (s/fspec :args (s/cat :resource :resource/entity)
:ret qualified-keyword?))
:ret :relationship/entity)
and my from-fn
, which is :name
runs fine:
(:name {:name "item-definition-components", :description "The list of item definition components.", :uri "{item-definition}/components", :list-of "item-definition-component", :family-id :itemdefinitions, :id :itemdefinitions/item-definition-components})
Why is that if I instrument
I get:
Call to #'rest-resources-viz.extractor/resource->relationship did not conform to spec:
In: [0] val: "" fails at: [:args :from-fn :ret] predicate: qualified-keyword?
:clojure.spec/args (:name {:name "item-definition-components", :description "The list of item definition components.", :uri "{item-definition}/components", :list-of "item-definition-component", :family-id :itemdefinitions, :id :itemdefinitions/item-definition-components})
:clojure.spec/failure :instrument
#2017-02-1419:08richiardiandreait looks like it runs it in a different way when instrumented#2017-02-1419:10richiardiandreaI am now doubting, is instrument
good for use at the repl?#2017-02-1419:38Alex Miller (Clojure team)instrumentation of an fdef with an fspec arg will check that arg by invoking the function you pass it with args generated from the fspec :args#2017-02-1419:39Alex Miller (Clojure team)here itās invoking the function you passed to resource->relationship with a generated :resource/entity and getting back empty string rather than a qualified keyword#2017-02-1419:40Alex Miller (Clojure team)instrument is primarily designed for use at the repl (or during tests), so it should be good for that#2017-02-1419:42Alex Miller (Clojure team)if from-fn
is a keyword, then it will return a string based on the example you have above, not a keyword#2017-02-1419:42Alex Miller (Clojure team)looks like a bug in either your code or your spec to me#2017-02-1419:45richiardiandreano well, I forgot to include the code of the function, but it "keywordizes" the result#2017-02-1419:45richiardiandreain fact, when I launch it with instrument
the result is the above, but without it it runs fine, same example...of course I need a repro case š#2017-02-1419:45richiardiandreathanks for confirming though#2017-02-1419:46hiredmanwell, (qualified-keyword? (keyword "")) is going to be false#2017-02-1419:50hiredmanthe input you pass in doesn't matter#2017-02-1419:50hiredmanthat error is from the generated data being used to test the spec#2017-02-1419:50richiardiandreaoh really#2017-02-1419:50richiardiandreaok so I was completely missing that point#2017-02-1419:50hiredmanyep#2017-02-1419:50richiardiandreaoh wow that explains everything#2017-02-1419:51hiredmanyep#2017-02-1419:51richiardiandrealol thanks...weirdly enough, printlns in resource->relationship
where not printing as well#2017-02-1419:52hiredmanI would have to look at the source for instrument, because if a spec for a function argument fails, it never actually runs the function#2017-02-1419:52richiardiandreayeah it exactly looks like it#2017-02-1419:53richiardiandreathanks (hired) man! You saved my chickens šø#2017-02-1501:34seantempestaWhatās the best practice for validating data submitted to a server with spec (for an API)? Should I use just check with s/valid?
and use code to return an error? Or maybe use an assert and throw an exception?#2017-02-1501:52Oliver GeorgeYour first suggestion sounds sensible to me. #2017-02-1501:54Oliver George(In general people try to avoid relying on exceptions in pure functional code since it breaks the input->output contract. )#2017-02-1502:12seantempestaRight. That makes sense. Thanks. #2017-02-1502:15agI need to have either bool
or string
, why this generates single value wrapped in a vector? (gen/generate (s/gen (s/alt :s string? :b boolean?)))
#2017-02-1502:16ageh, nevermind. I forgot that thereās s/or
#2017-02-1511:09smoggHowdy.
Assuming I'm writing a spec looking like this:
(s/def ::labels (s/coll-of string?))
(s/def ::values (s/coll-of integer?))
(s/def ::chart (s/keys :req-un [::labels ::values]))
how would I go about validating that both :labels
and :values
are of equal length in :chart
?#2017-02-1511:11dergutemoritz@smogg You could do it like this (s/def ::chart (s/and (s/keys :req-un [::labels ::values]) #(= (count (::labels %)) (count (::values %)))))
#2017-02-1511:14smoggok, so a custom fn#2017-02-1511:14smoggthank you @dergutemoritz#2017-02-1514:10cryptoratSo today I am trying to generate a list.
(s/def ::team (list? (s/+ ::team-member)))
I have verified that ::team-members can be generated. What obvious thing am I missing this morning?#2017-02-1514:17cryptorathmm.. Looking at what smogg wrote (s/def ::team (s/coll-of ::team-member))
#2017-02-1514:17cryptoratSeems to work for generating.#2017-02-1514:17smogg@cryptorat yeah, if you want a coll of something you'd use coll-of
#2017-02-1514:19cryptoratDoes that mean there is no way to determine if it is a list vs a map?#2017-02-1514:20cryptoratOr is a map not considered a collection? hmmā¦let me read up on coll-of#2017-02-1514:26smoggif you want a map you can use s/keys
to specify the keys#2017-02-1514:26smoggif you want to specify what collection you want to get with s/coll-of
you'd pass a :kind
#2017-02-1514:27smoggfor example: (s/coll-of integer? :kind vector?)
#2017-02-1514:27cryptorat:kind
yeah. I just found that.#2017-02-1514:28cryptoratFantastic. It is all coming together nicely. Thank you for the help.#2017-02-1514:28smoggyw#2017-02-1516:01lsnapeInteresting semantic differences between s/merge
and clojure core merge:
(s/def :foo/bar string?)
(s/def :baz/bar nat-int?)
(s/def ::bar-map (s/merge (s/keys :req-un [:foo/bar])
(s/keys :req-un [:baz/bar])))
I would expect :baz/bar
to override :foo/bar
, but AFAICT it and
s the specs together for that key.#2017-02-1516:01Yehonathan Sharvits/merge
behaves like s/and
#2017-02-1516:02Yehonathan SharvitThe term merge might be misleading#2017-02-1516:02Yehonathan Sharvitmerge
macro
Usage: (merge & pred-forms)
Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs. Unlike 'and', merge can generate maps satisfying the
union of the predicates.
#2017-02-1516:04lsnapeThanks! What exactly is meant by union of the predicates here?#2017-02-1516:04Yehonathan SharvitIt means: union of required keys, union of optional keys#2017-02-1516:05Yehonathan SharvitAnd I guess that there is some rule for handling cases where a key appears in one spec as required and in the other as optional#2017-02-1516:05lsnapeGotcha#2017-02-1516:05Yehonathan SharvitCould you test it?#2017-02-1516:05lsnapeyup I reckon#2017-02-1516:06Yehonathan SharvitYou can create an interactive snippet with klipse e.g : http://app.klipse.tech/?cljs_in=(require%20%27%5Bclojure.spec%20%3Aas%20s%5D)%0A%0A(s%2Fdef%20%3Afoo%2Fbar%20string%3F)%0A(s%2Fdef%20%3Abaz%2Fbar%20nat-int%3F)%0A%0A(s%2Fdef%20%3A%3Abar-map%20(s%2Fmerge%20(s%2Fkeys%20%3Areq-un%20%5B%3Afoo%2Fbar%5D)%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20(s%2Fkeys%20%3Areq-un%20%5B%3Abaz%2Fbar%5D)))%0A%0A(s%2Fvalid%3F%20%3A%3Abar-map%20%7B%3Abaz%2Fbar%201%7D)#2017-02-1516:07Yehonathan Sharvit(Sorry for the long url)#2017-02-1516:08joshjonesinteresting, i did not think merge behaved this way because i havenāt used it much#2017-02-1516:09lsnapeIt looks like the :req-un
wins#2017-02-1516:10joshjones(s/def :foo/bar #(< 1 % 10))
(s/def :baz/bar #(< 5 % 20))
(s/def ::bar-map (s/merge (s/keys :req-un [:foo/bar])
(s/keys :req-un [:baz/bar])))
(s/valid? ::bar-map {:bar 2})
=> false
(s/valid? ::bar-map {:bar 11})
=> false
(s/valid? ::bar-map {:bar 7})
=> true
#2017-02-1516:10joshjoneshad no idea#2017-02-1516:10lsnape(require '[clojure.spec :as s])
(s/def :quux/flib int?)
(s/def ::bar-map (s/merge (s/keys :req-un [:quux/flib])
(s/keys :opt-un [:quux/flib])))
(s/valid? ::bar-map {}) ;; false
#2017-02-1516:10Yehonathan SharvitNo matter whatās the order inside the merge?#2017-02-1516:11lsnapeOrder doesnāt matter. I guess this behaviour is consistent with the and
ing of the value specs.#2017-02-1516:11Yehonathan Sharvitmakes sense#2017-02-1516:12lsnapeYeah, good to know š#2017-02-1516:14Alex Miller (Clojure team)Merge means: the value must pass each merged spec#2017-02-1516:15Alex Miller (Clojure team)The specs are not combined or unions though#2017-02-1516:18lsnapeWhat I was hoping for was someway to relax an existing spec that Iāve defined. For example, a new map that has blank strings initially ready for a user to fill out.#2017-02-1516:20lsnapeThe spec on the server requiring that these strings be filled out in the map.#2017-02-1517:43vikeriIs there a spec that specifies a sorted-map
?#2017-02-1518:37spieden@vikeri not built in i donāt think, but you could just have a type checking predicate i suppose#2017-02-1518:39vikeri@spieden Alright thanks, in the end I decided to use a vector instead, better when serializing#2017-02-1518:47spiedenyeah that may make your life easier overall#2017-02-1519:56Alex Miller (Clojure team)@vikeri you can do something like (s/map-of int? string? :kind sorted?)
#2017-02-1519:59vikeri@alexmiller That sure looks like it, didnāt think of that you could use :kind
for broader things than vector?
, list?
etc.#2017-02-1520:06Alex Miller (Clojure team)itās an arbitrary function#2017-02-1520:06Alex Miller (Clojure team)however, it does affect gen - if you use, either it should gen or you should also use :into
#2017-02-1520:09Alex Miller (Clojure team)Iām not sure there is some way to get the sorted map constraint in there and make map-of gen (without supplying a custom gen for the overall coll)#2017-02-1520:12spiedeni seem to be hitting an infinite recursion trying to gen for a part of my spec#2017-02-1520:12spiedenitās not the only recursive definition i have ā other one works fine#2017-02-1520:15viestihum, trying to figure out this: (s/exercise (s/keys :req #{:lol/id} :opt #{:lol/name}))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
#2017-02-1520:16hiredmansomewhere in the specs for id and name or in a spec they rely on you are using s/and#2017-02-1520:16hiredmanand somewhere in that s/and is a predicate that is very picky#2017-02-1520:17viestiwhere :lol/id is int? and :lol/name is (s/and string? #(<= (.length %) column_size))
#2017-02-1520:17hiredmanthe way generates work for s/and is it uses the generator for the first predicate and then filters the output using such-that#2017-02-1520:17hiredmangenerators#2017-02-1520:17hiredmanright, so you should hook up a custom generator#2017-02-1520:18viestihum, s/exercise works individually for both specs but not for the key spec#2017-02-1520:18hiredmansure#2017-02-1520:19viestishould I write a custom generator for the key spec#2017-02-1520:19hiredmanthat is going in to the weeds a little bit on what happens as you compose test.check generators#2017-02-1520:19hiredmanfor :lol/name#2017-02-1520:20viestifirst thought that the problem was strings limited by length, but itās probably combination of :req and :opt, since having both key specs as :opt passes s/exercise#2017-02-1520:20hiredmanbecause you can generate random strings for a long time before you get one that is less than or equal to column_size#2017-02-1520:21viestisince this seems to work: (s/exercise (s/keys :req #{} :opt #{:lol/name :lol/id}))
([{} {}] [{} {}] [{} {}] ...)
#2017-02-1520:22hiredmanthis is the nature of randomness, sometimes things work that don't always work#2017-02-1520:22viestiš#2017-02-1520:22viestihave to take a shower now, introducing some randomness š#2017-02-1520:23hiredmanI don't recall the details of how the generator for s/keys works, but it may ignore :opt most of the time, or some large enough amount of the time for it to mostly work when you move the keys to opt#2017-02-1520:30spiedenany thoughts on infinite recursion during spec generated gen? only difference i can see is the ācall depthā where the recursion happens ā itās deeper in the case thatās going into the infinite loop#2017-02-1521:26tbaldridge@spieden you have to always give the generator a "way out"#2017-02-1521:26spiedenah like a terminal node in the recursion?#2017-02-1521:27tbaldridgeso one of the s/or or s/keys needs to have a nil, or fully optional params, etc.#2017-02-1521:27tbaldridgeyeah#2017-02-1521:27spiedeni thought maybe thereād be a way to control depth#2017-02-1521:27tbaldridgethere is, but the generator needs something to bottom out at, if you say "go 2 levels deep", what's it going to gen at the bottom level? Has to be a empty map, or a nil, or something.#2017-02-1521:28tbaldridgeor basically a scalar value.#2017-02-1521:28spiedenok thanks, iāll take a look at my specs again#2017-02-1521:28spiedenseems like there ought to be, actually#2017-02-1521:30spiedenyeah thereās an s/or on the path that has both a recursive and non recursive case#2017-02-1521:31tbaldridgeany hint from the stacktrace where the problem is?#2017-02-1521:32spiedenthereās no stack trace, it just spins endlessly and eats CPU when i try to sample one from it#2017-02-1521:33spiedenhereās the code in case you feel like taking a look: https://gist.github.com/spieden/49b6df8c2dae9f8e659edac3b974bb94#2017-02-1521:33spiedenitās ::sgr/parallel thatās the problem while ::sgr/condition is fine#2017-02-1521:36spieden::sgr/parallel -> ::sgr/branches -> ::sgr/states -> ::sgr/state -> ::sgr/parallel is the only cycle i can find#2017-02-1521:37tbaldridgeyeah, I don't think that's the problem#2017-02-1521:37tbaldridgeif you have a loop like taht you'd get a stack overflow#2017-02-1521:37spiedenoh hereās the other spec ns https://gist.github.com/spieden/58f0fac97f725522860f2b68e25f3769#2017-02-1521:37tbaldridgethe problem here is that spec creates the generators very eagerly.#2017-02-1521:37spiedenah ok, SO#2017-02-1521:38tbaldridgeand so I've seen really long times to create generators.#2017-02-1521:39spiedenhmm iāll try giving it longer to see if it ever completes#2017-02-1521:44spiedennope just cooks my cpus#2017-02-1521:46Alex Miller (Clojure team)one thing worth trying is to add :gen-max 3
to all of your coll-of generators#2017-02-1521:46Alex Miller (Clojure team)thatās frequently a problem for me in composite gens as the default is 20 and it doesnāt take much nesting for that to be big#2017-02-1521:49spiedenok thanks, iāll try that!#2017-02-1521:50spiedenboom, results#2017-02-1521:51spiedenyeah i guess the branching was just going out of control#2017-02-1521:51spiedenthanks @alexmiller @tbaldridge !#2017-02-1521:51Alex Miller (Clojure team)I have a pending enhancement to change the default to 3 :)#2017-02-1521:54spiedensounds like a sensible default =)#2017-02-1521:58qqqI'm wondering if it's possible to statically analyze, for each non-trminal, the smallest structure it can generate -- then, after gen hits a certain size, it tells everything to use it's smallest choice#2017-02-1521:59qqqthus, instead of controlling depth, gen merrily does its thing until it hits say 1000 nodes, then it goes "oh -%&#$&" and it tries to finish as quickly as possible#2017-02-1522:06aghey guys, I forgot how do you select random multiple elements out of a vector? something like gen/elements
but for a random number elements instead of just one#2017-02-1522:07agehm, nevermind, I guess I could combine (gen/sample) and (gen/elements)#2017-02-1522:22agā what a spec would look like for a map where values depend on each other? e.g. date range with keys :from
and :to
where :to
always should be bigger that :from
?#2017-02-1522:29agis it possible at all?#2017-02-1522:31spieden@ag using a predicate anything is possible!#2017-02-1522:32spiedeni.e. (s/and (s/keys ā¦) (fn my-pred [the-map] ā¦))#2017-02-1522:33agoh, rightā¦ let me try that#2017-02-1522:53agwowā¦ Spec truly is amazing!#2017-02-1522:57spiedenso satisfying when spec finds allthebugs =)#2017-02-1522:57spiedenfrom hundreds of lines of test code to three#2017-02-1523:06gfredericks@ag don't use sample to build new generators#2017-02-1523:06gfredericks(gen/vector (gen/elements ...)) should do what you were asking#2017-02-1600:45ag@gfredericks Iām using test.chuck, but canāt get my head around this. How do I generate two dates - one always smaller than the second?#2017-02-1600:46agIām using test.chuck.generators/datetime#2017-02-1600:49gfredericks@ag fmap+sort, such-that+not=#2017-02-1600:50agare you saying I should generate many, sort and then grab first two?#2017-02-1600:50agmakes sense#2017-02-1600:50gfredericksNo#2017-02-1600:51gfredericksGenerate exactly two using tuple or vector#2017-02-1600:51gfredericksThen fmap and filter that#2017-02-1600:51agfilter? what filter for?#2017-02-1600:53gfredericksThe filter is to make sure they're not equal#2017-02-1600:54gfredericks("filter" meaning such-that
)#2017-02-1600:54agoh, rightā¦ thanks!#2017-02-1603:18Alex Miller (Clojure team)or generate one date and a quantity and add the quantity to the date#2017-02-1607:38viestiFor the record, problem that I had yesterday was actually due to using set (#{}) in :req/:opt for s/keys: user> (s/exercise (s/keys :req #{:lol/id2} :opt #{:lol/id}))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
user> (s/exercise (s/keys :req [:lol/id2] :opt [:lol/id]))
([#:lol{:id2 0} #:lol{:id2 0}] [#:lol{:id2 0} #:lol{:id2 0}]ā¦
#2017-02-1607:40viestithe s/keys generator also obeys :req and :opt, so generates maps that have only :req keys, which is neat š#2017-02-1607:49viestianother thing, Iām tinkering with generating specs from a source outside code (database schema), and doing something like this: user> (let [k :lol/id]
(s/def k int?))
user/k
#2017-02-1607:50viestiwhich is not actually what I wanted, so Iām working around it by doing this: user> (let [k :lol/id]
(eval `(s/def ~k int?)))
:lol/id
#2017-02-1607:53viestiWhich seems a bit odd. Am I missing something or is the idea that specs are defined firstly in Clojure code and not derived from outside sources?#2017-02-1611:29Oliver GeorgeI faced similar problem when I wanted to generate specs based on an sql db schema by inspection. In that case I generated clojure code. #2017-02-1611:29Oliver GeorgeThe macro based API complicates things. #2017-02-1611:31Oliver GeorgeIt must be possible to do. I'd love to see examples. #2017-02-1615:34Alex Miller (Clojure team)you can do that, or use eval#2017-02-1615:34Alex Miller (Clojure team)in the future there may be a lower-level function-based api, however it will not be able to automatically capture forms for explain reporting#2017-02-1615:58mpenetlovely idea -> http://dev.clojure.org/jira/browse/CLJ-2112#2017-02-1615:58mpenetthat'd make the life of all the spec to <whatever-format> projects a lot easier#2017-02-1616:05ikitommithumbed that one up, thanks @mpenet. Hacked around that just yesterday to get dynamic conformers ādrop extra keysā & āfail on extra keysā working for map specs.#2017-02-1616:20Alex Miller (Clojure team)@mpenet has been in the plan from the beginning, but has been blocked behind the various form errors#2017-02-1617:22eraserhdIt would be cool to use s/exercise
to generate a sample structure for documentation, but I would need to tweak a few things: Make all keys required, ensure all collections have >= 1 element.#2017-02-1617:22eraserhdI realize this would probably be hacky, but any thoughts on how to do this?#2017-02-1619:28gfredericksI'd probably write a function doc-sample
that has the special cases you described, calls itself recursively, and falls back on gen/generate
when it doesn't know what to do#2017-02-1623:01calvisIām writing a macro that is essentially a wrapper around defn
, so I thought it would be cute to use defn
ās spec to conform for easy access to the args & body. it works on some, but not when the body of the macro is a map, because..
user> (s/conform :clojure.core.specs/defn-args
'(foo [bar] {:baz 42}))
{:name foo, :bs [:arity-1 {:args {:args [[:sym bar]]}, :prepost {:baz 42}}]}
the body conforms as :prepost
rather than :body
. is there any way to make it backtrack and choose the path where :prepost
is empty and :body
is correct?#2017-02-1623:08gfredericksoh man; does that mean defn
is essentially unspecable?#2017-02-1623:08gfredericksI mean I guess you could rewrite it with an or
#2017-02-1623:09calviswell I donāt think itās un-specable, just that the conform result does not align with the semantics#2017-02-1623:10gfredericksthat's presumably because the spec is ambiguous#2017-02-1623:10gfredericksand I don't think it should be#2017-02-1623:11calviswell yeah, I think the spec as written is a bug / oversight#2017-02-1623:11bronsayeah#2017-02-1623:11bronsai'd open a ticket about that#2017-02-1623:11calvisjust hoping for a way around it#2017-02-1623:11gfrederickschange the spec yourself š#2017-02-1623:11calvismore work than I was hoping to do#2017-02-1623:13gfredericksI think it'd be a pretty simple tweaking of this bit here https://github.com/clojure/clojure/blob/c0326d2386dd1227f35f46f1c75a8f87e2e93076/src/clj/clojure/core/specs.clj#L76#2017-02-1623:14gfredericksyou'd wrap the :prepost
and :body
in an or
or alt
or whatever, where the body uses +
instead *
in the branch with the prepost#2017-02-1623:14calvisyeah I agree#2017-02-1623:18bronsa(s/def ::args+body
(s/cat :args ::arg-list
:rest (s/alt :body+attr (s/cat :prepost (s/? map?)
:body (s/+ any?))
:body (s/* any?))))
#2017-02-1623:18bronsathis works#2017-02-1623:18bronsauser=> (s/conform :clojure.core.specs/defn-args '(foo [bar] {:baz 42}))
{:name foo, :bs [:arity-1 {:args {:args [[:sym bar]]}, :rest [:body+attr {:body [{:baz 42}]}]}]}
user=> (s/conform :clojure.core.specs/defn-args '(foo [bar] {:baz 42} 1))
{:name foo, :bs [:arity-1 {:args {:args [[:sym bar]]}, :rest [:body+attr {:prepost {:baz 42}, :body [1]}]}]}
#2017-02-1623:19gfredericks@bronsa no need for the s/?
at that point, right?#2017-02-1623:20bronsawhat about (foo [bar])
#2017-02-1623:20gfredericksthat would go through the second branch#2017-02-1623:21bronsaah sorry, I read s/?
and thought s/*
#2017-02-1623:21bronsayeah no need for that s/?
indeed#2017-02-1623:22bronsabetter name for :rest
?#2017-02-1623:23bronsa:body-or-body+prepost
doesn't look too good :)#2017-02-1623:24gfredericks:body+
:body*
:body'
dunno too many names#2017-02-1623:24calvisseems like the spec should be named ::args+prepost+body
and the :rest
is ::prepost+body
#2017-02-1623:25calvisfurther broken down into :prepost+body
or :body
#2017-02-1623:27bronsa:?prepost+body
-> :prepost+body/:body
#2017-02-1623:27bronsa:P#2017-02-1623:28bronsaI guess an argument could be made that prepost is part of the body#2017-02-1623:28calvisgiven the original name of the spec that seems to be the direction they went#2017-02-1623:29bronsaaight, I'll make a ticket + patch and just keep it as
(s/def ::args+body
(s/cat :args ::arg-list
:body (s/alt :prepost+body (s/cat :prepost map?
:body (s/+ any?))
:body (s/* any?))))
and rich/alex can figure out some better naming#2017-02-1623:33calvisthanks!#2017-02-1623:36bronsahttp://dev.clojure.org/jira/browse/CLJ-2114#2017-02-1623:38bronsahmm, maybe prepost should be stricter than just map?
#2017-02-1623:39bronsathe actual grammar for defn only accepts a map with :pre [fn*] :post fn
#2017-02-1623:40bronsashrug#2017-02-1623:41calvisstill an improvement ĀÆ\(ć)/ĀÆ#2017-02-1623:43gfredericksit does?#2017-02-1623:44bronsawell#2017-02-1623:44bronsait ignores any other key#2017-02-1623:44gfredericksit looks to me like the only requirement is :pre
's value is a collection#2017-02-1623:44bronsaan throw at runtime if you don't provide [fn*] for :pre or fn for :post#2017-02-1623:45gfredericksI thought they were arbitrary expressions#2017-02-1623:45bronsaarbitray predicates#2017-02-1623:45bronsai'm using fn to mean IFn
instance#2017-02-1623:46bronsawait#2017-02-1623:46bronsano you're right I'm stupid#2017-02-1623:46gfrederickshave you been using pre-post wrong for years and didn't notice because the functions are always truthy š#2017-02-1623:47bronsayes, absolutely#2017-02-1623:47gfredericksdynamic languages are a magic surprise#2017-02-1623:52bronsa@gfredericks the few slack users that use the slack->irc bridge will know what we've been up to#2017-02-1623:52bronsawe need to find them and silence them#2017-02-1623:52gfredericksglad I stopped using that when the threads feature was released#2017-02-1623:52bronsaoh god I don't want to know how that looked#2017-02-1623:53gfredericksI think it just looked like regular messages#2017-02-1623:54gfredericksso I would've ended up unknowingly replying to thread messages in the main channel#2017-02-1623:54bronsalol#2017-02-1623:55gfredericksI can put up with all sorts of nonsense for the sake of technological idealism, but unknowingly looking like a crazy person is not one of them#2017-02-1623:55bronsaone thing I really miss about the #clojure irc: slack makes me feel so much more guilty when I derail a conversation/channel into offtopic banter#2017-02-1623:56gfredericksonly because threads exist? or because topics are more specific?#2017-02-1623:56bronsathe latter I think#2017-02-1700:04gfredericksI miss the jovial bots#2017-02-1700:05bronsaand the internet points#2017-02-1700:05gfredericksoh yeah I had a lot of those I think#2017-02-1700:05gfredericksshould've sold'em before the market crashed#2017-02-1700:06bronsano point in crying over what could've been now#2017-02-1700:06bronsathat was a terrible pun i apologize#2017-02-1700:07gfrederickscan we start threads based on thread messages#2017-02-1700:07bronsaI've frequently complained to my coworkers about the lack of sub threads#2017-02-1700:07gfredericksseems not#2017-02-1700:08gfredericksI miss trying to get the bots to talk to each other#2017-02-1700:09bronsawas it you who came up with that brilliant double quoting thing to bypass their "security" measures?#2017-02-1700:10bronsai remember being very amused#2017-02-1700:10gfredericksI don't think I ever achieved an infinite loop personally#2017-02-1700:12bronsaI might spend the next 3 hours going over old irc logs to find the one I'm thinking about#2017-02-1700:13gfredericksthat would be fun to see; I don't recall anything like that#2017-02-1700:14gfredericksthis feels deceptively like a private conversation#2017-02-1700:16hiredmandeceptive indeed#2017-02-1700:16bronsasomebody's gonna chime in real soo-#2017-02-1700:17gfredericksit lists names at the top, but I doubt that includes lurkers#2017-02-1711:27borkdudeClojure IRC still exists, right? Or is it effectively dead now#2017-02-1711:28bronsait exists but i don't think it's nowhere as active as it used to be#2017-02-1711:29bronsaalso i don't join it anymore so from my perspective it's dead and I'm mourning#2017-02-1713:56shemit's there and still useful and enjoyable#2017-02-1715:27Alex Miller (Clojure team)still in regular use#2017-02-1716:59ggaillardHi everybody !
I'm having a hard time with cljs.spec
(1.9.473). I want to instrument my spec-ed functions, but I can't find the
instrument
and instrument-all
macros it seems they have been removedā¦ do they still exist somewhere ? I can't find any blogpost about it ā¦#2017-02-1717:03gfredericksdid you check under the .test
namespace @ggaillard#2017-02-1717:05ggaillardI'm checking right now#2017-02-1717:08ggaillardGreat ! I had a caching problem with figwheel preventing me to :require cljs.spec.test. Cider was showing me an other file for some reason ā¦
Thank you @gfredericks !#2017-02-1723:03bnoguchiAnyone here figure out how to constrain a multi-spec generator to generate maps constrained to always generate maps with only one of the tagged values (i.e., to always follow only one multimethod branch during generation)?#2017-02-1723:04bnoguchie.g., in the guide example, only generating events of type :event/search
#2017-02-1723:27Alex Miller (Clojure team)You can just invoke the generator for the result of invoking one of the method specs#2017-02-1723:31bnoguchi@alexmiller You mean if mm is the multimethod, then (s/gen (mm {:the/tag tag-val})
?#2017-02-1723:32Alex Miller (Clojure team) Yeah, or just invoke the spec directly if you van#2017-02-1723:32bnoguchiWon't that not constrain the tag to the given value?#2017-02-1723:33bnoguchii.e., :the/tag
is open ended as keyword?
, so will generate infinite sample of possibly non-matching keywords#2017-02-1723:35bnoguchie.g., for :event/type example in the guide, this would effectively be for :event/search
:
clojure
(s/gen (s/keys :req [:event/type :event/timestamp :search/url]))
#2017-02-1723:35bnoguchiAh, I guess, I can pass gen overrides for [:event/type]
to (s/gen spec overrides)
#2017-02-1723:41bnoguchiIt would be cool if multi-spec generators supported overrides, to force a particular generated type, e.g.,:
(s/gen :event/event {[:event/type] #(gen/return :event/search)})
#2017-02-1801:12weiwhat happened to spectrum? is it still being developed?#2017-02-1908:08Yehonathan SharvitIs there a way to instrument by default while in dev environment?#2017-02-1914:17gfredericks@viebel I think you could pull that off with leiningen profiles; not sure how that would interact with code reloading though#2017-02-1915:02nblumoeIs there an established standard to run spec tests (`stest/check`) from a clojure.test
suite (e.g. on lein test
)?#2017-02-1915:21tbaldridge@nblumoe not a standard, but I would probably reach for test.check's defspec
: https://github.com/clojure/test.check#clojuretest-integration#2017-02-1915:21tbaldridgeCall that and use (s/gen ::my-spec)
to get the gen to us in prop/for-all#2017-02-1915:25gfredericksIf it's just the builtin spec tests, then just using deftest and wrapping the stest/check call in an assertion might make more sense#2017-02-1915:25gfredericksdefspec would be more useful for custom assertions/tests#2017-02-1915:32nblumoe@gfredericks something like (deftest specs (is (stest/check)))
would not be sufficient as it seems, what would you check for?#2017-02-1915:33gfredericksExamine the return value. Is there a result
key?#2017-02-1915:34Yehonathan SharvitThis might help#2017-02-1915:35Yehonathan SharvitItās for cljs but not too hard to adapt to clojure#2017-02-1915:38nblumoethx. If I understand correctly all three solutions require looking at the :failure
key manually / with a helper. Okay then I will do that too. Was hoping for a generic integration into clojure.test#2017-02-1916:03nblumoeok works from the REPL and via Cider, but not with lein test
. Any ideas? Here is the error (I donāt get it so far tbh): ERROR in (foo) (FutureTask.java:122)
foo
expected: (nil? (-> spec-check first :failure))
actual: java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
at java.util.concurrent.FutureTask.report (FutureTask.java:122)
java.util.concurrent.FutureTask.get (FutureTask.java:192)
clojure.core$deref_future.invokeStatic (core.clj:2290)
clojure.core$future_call$reify__9377.deref (core.clj:6849)
clojure.core$deref.invokeStatic (core.clj:2310)
clojure.core$deref.invoke (core.clj:2296)
clojure.core$map$fn__6881.invoke (core.clj:2728)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:56)
clojure.lang.LazySeq.first (LazySeq.java:71)
clojure.lang.RT.first (RT.java:682)
clojure.core$first__6404.invokeStatic (core.clj:55)
clojure.core/first (core.clj:55)
geomix.calc.pva_test$check_SINGLEQUOTE_.invokeStatic (pva_test.clj:244)
#2017-02-1916:14nblumoethis happens whenever I call some function with (stest/check
my-fn)` as an argument within a deftest
body....#2017-02-1917:48nblumoehad to require [clojure.test.check.clojure-test]
#2017-02-1918:50Alex Miller (Clojure team)You're running into the lein test monkey patch problem#2017-02-1918:51Alex Miller (Clojure team)There are tickets in both lein and test.check about it, but you can work around it by turning it off in project.clj#2017-02-1918:52Alex Miller (Clojure team)@nblumoe ^^#2017-02-1918:54Yehonathan SharvitWhat is the flag that needs to be set in order to instrument all the namespaces (while in dev environment)?#2017-02-1918:54Alex Miller (Clojure team)There is no such flag#2017-02-1918:54nblumoe@alexmiller I guess that would be monkeypatch-clojure-test
https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L351 ?#2017-02-1918:54Alex Miller (Clojure team)@nblumoe yep#2017-02-1918:55nblumoethanks a lot, good to know! :+1:#2017-02-1918:55Alex Miller (Clojure team)@viebel you can call instrument with no args though#2017-02-1918:56Yehonathan SharvitWhere would I put this call?#2017-02-1918:57Yehonathan SharvitIs there a dynamic var that I could set from leiningen like *assert*
?#2017-02-1921:03richiardiandrea@viebel I do a classic when
at the top of my files to instrument everything, I haven't seen anything fancier #2017-02-1922:00frankyou can set up a :once
fixture that calls (s/instrument)
before running tests and (s/unstrument)
after running tests#2017-02-1922:01frankhttps://clojuredocs.org/clojure.test/use-fixtures#2017-02-2010:06akielI have a question regarding required keysets in combination with om.next or pull queries, where required attributes might be skipped. I have formulated my questions in the following gist: https://gist.github.com/alexanderkiel/81daedb133f7939d0c88ff28dd457442#2017-02-2014:33mike_ananevHi! I have ns with spec's. how to load all spec names from another ns?#2017-02-2014:34mike_ananevI want to get list of def's from particular ns#2017-02-2014:35akiel@mike1452 you can just require the ns with specs#2017-02-2014:35mike_ananevI want to print on screen available specs#2017-02-2014:35mike_ananevnames#2017-02-2014:35mike_ananevin order to do it I need to get list of spec names#2017-02-2014:36akielthere is (s/registry)
#2017-02-2014:37mike_ananev@akiel wow! yeah, sure! thanks!!!#2017-02-2014:39akiel@mike1452 no problem š#2017-02-2015:22dacopareI found that some of my spec check were running slowly and causing timeouts. If you find the same thing happening, you can use the options to specify the number of tests you want to run.
(s.t/check `my-ns/my-func {:clojure.spec.test.check/opts {:num-tests 10}})
I hope this helps one of you.#2017-02-2015:26gfredericksprobably had more to do with the volume of data generated than the number of tests#2017-02-2015:44akiel@dacopare As @gfredericks said already: keep an eye on your generators. You can also overwrite them in tests or control them globally with something like :max-gen
in coll-of
.#2017-02-2015:51dacopare@gfredericks: yes, the datastructures that are input and output are very large. Limiting the number of tests is the only way I can see to mitigate this.#2017-02-2015:56gfredericks@dacopare there are better ways. The :max-gen option mentioned above might do it, I'm not sure. Otherwise there is gen/scale to manipulate the size parameter that a generator sees#2017-02-2016:48lwhortonheya guys ā¦ iām curious if thereās an idiom for nil-able keys in (s/keys ā¦)
wrt other namespaces? what I want to do is something like this:
(ns entity-a)
(s/def ::entity-a (s/keys :req [::b]))
(s/def ::b (s/nilable :entity-b/entity-b)) <ā probably wont work
ā¦
(ns entity-b)
(s/def ::entity-b (s/keys .... (more stuff)))
1) seems redundant to have to define :req [::b]
, where ::b is defined locally but just points to another ns
2) ::b should be nilable only from aās point of view ā¦ so I donāt want to put nilable down inside the spec of entity-b
ā¦
so should this really just be
(s/def ::entity-a (s/keys :req [::b]))
(s/def ::b (s/or :nil nil :val :entity-b/entity-b)
?#2017-02-2016:52gfredericks@lwhorton did you consider making the key optional instead of making the value nilable?#2017-02-2016:54lwhortoni did, but in this particular case the domain is really shoddy. i donāt have time to go refactoring (yet), so entity-a can have the key, but the value might be nil. I suppose that leads to another question - is it more common to just ignore such cases with an opt key, or to capture all possible nil states?#2017-02-2016:56gfredericksI don't know if there's any sense of what's more common with spec yet#2017-02-2016:56gfredericksone of spec's core ideas is that the meaning of a key is the same everywhere, which is why you're having to define a different key#2017-02-2016:57gfrederickswhich I imagine implies that somewhere you'll have code that translates something to that new key. and if that's the case, then it shouldn't be much more difficult to instead remove the key when the value is nil#2017-02-2016:58lwhortongood point. or we could just have a proper domain model thought out without monkey patching and quick fixes š. thanks @gfredericks#2017-02-2020:09lvhIf I have a fn with 2 arguments that are related for correct operation (itās a fn-var and an argument name), can I use clojure.spec.test/check
to check it, or should I write a more traditional clojure.test.check prop?#2017-02-2020:10gfredericksdepends on if you want to make that relationship part of the arg spec or not#2017-02-2020:10gfredericksvia s/and
probably#2017-02-2020:14lvhIām not sure I understand the suggestion; if I do something like (s/and ... is-a-valid-arg-name?)
I still need to know the value of the method (and random strings are extremely unlikely to be arg names) ā is there something else Iām missing?#2017-02-2020:19gfredericksin that case you'd need to add a custom generator too#2017-02-2020:21zaneThere's no way to use clojure.spec/or
or clojure.spec/multi-spec
to do what lvh wants here?#2017-02-2020:22lvh@gfredericks just one that applies to the entire arg spec. Makes sense. Thanks š#2017-02-2110:47mike_ananevHi! If I need (gen/sample ::somespec) about 1M or more, how to parallelize this operation? is there build-in tools? or just pmap or something?#2017-02-2113:14gfredericks@mike1452 what do you need them for? #2017-02-2113:43mike_ananev@gfredericks for data sample generation. I want to give samples to other team, which needs shape of our data for dev process.#2017-02-2113:56mpenet@alexmiller what's the extra form
parameter in s/valid?
?#2017-02-2113:56mpenetdoesn't seem to be documented#2017-02-2113:57mpenetwell, it's not used actually#2017-02-2114:06Alex Miller (Clojure team)It's an alternate value to return if not valid#2017-02-2114:07Alex Miller (Clojure team)Or maybe I'm reading that wrong#2017-02-2114:07mpenetmaybe I misunderstand something then: https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L672-L683#2017-02-2114:07Alex Miller (Clojure team)Can't say I've ever used it#2017-02-2114:08mpenetdoesn't seem used at all but in the exception#2017-02-2114:13Alex Miller (Clojure team)thatās not the current version of that code#2017-02-2114:13gfredericks@mike1452 you can parallelize it however you like, that's kind of independent#2017-02-2114:14mpenet@alexmiller it's the one linked from the api docs I think#2017-02-2114:14Alex Miller (Clojure team)@mpenet looks like its used internally via pvalid?#2017-02-2114:14mpenetI ll have a look#2017-02-2114:14Alex Miller (Clojure team)yeah, the autodoc updating was broken by an issue in spec, so they are not the latest#2017-02-2114:14Alex Miller (Clojure team)I have a patch pending to fix the code and get that working again#2017-02-2114:15Alex Miller (Clojure team)that valid? form is used when you have an alternate form to supply for the error reporting#2017-02-2114:16mpenetthat's exactly what I wanted it to be š#2017-02-2114:16mpenetI ll try it out#2017-02-2114:17Alex Miller (Clojure team)I think maybe the only place itās used is from within keys
#2017-02-2114:17Alex Miller (Clojure team)used with that extra form that is#2017-02-2114:18mpenetso it's not supposed to be "public" ?#2017-02-2114:18mpenetI mean subject to changes/removal#2017-02-2114:26mpenetnot sure it helps actually#2017-02-2114:26mpenetI am wishing for a explain-info
I think. Sometimes I need to add some context to explain data#2017-02-2114:27mpenetI know I can wrap the whole spec and add this at a higher level, but it's a bit gross#2017-02-2114:59Alex Miller (Clojure team)the valid? function should be considered public and I donāt think there are any expectation that api will change#2017-02-2115:00Alex Miller (Clojure team)my impression is that Rich doesnāt want to provide extensions to modify explain (other than by writing your own spec and taking care of it there)#2017-02-2115:02mpenetš care about more detail of my use case for a potential suggestion?#2017-02-2115:06mpenetin short: that spec predicate is quite resource heavy, and upon failure the function wrapped by that predicate provides a lot of metadata about the failure (think a query parser output, with line/col num, explain traces etc). Right now not only I loose all this data but I need to re-trigger a parse to get back the metadata (or throw upstream, bypassing spec, which is ugly). In theory, one could just wrap and report the error manually, but this values are "deep" in a (spec validated) map.#2017-02-2115:07mpenetopen to suggestions on how to better handle this#2017-02-2115:11Alex Miller (Clojure team)sounds like a good use case for writing a custom spec#2017-02-2115:12mpenetyou mean implementing the internal protocol?#2017-02-2115:12Alex Miller (Clojure team)yeah#2017-02-2115:12mpenetis that "safe"?#2017-02-2115:12mpenetI mean allowed#2017-02-2115:13Alex Miller (Clojure team)itās available to do but is subject to change while in alpha#2017-02-2115:14mpenetok, I guess I might just do this. That's how it used to work with Schema btw, seems quite an elegant way to do it#2017-02-2115:14Alex Miller (Clojure team)itās something I would avoid doing as much as possible#2017-02-2115:14mpenetyes, same feeling.#2017-02-2115:56mpenetworks like a charm#2017-02-2116:22carocad@mpenet I was under the impresion that explain-data
was meant to provide that functionality: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/explain-data#2017-02-2116:23mpenetno explain-data just returns the data behind a failing spec, doesn't allow you to "customize" said data#2017-02-2116:23mpenetsame diff as ex-info vs ex-data (kinda)#2017-02-2116:27carocadbut isnt the data behind a failing spec the one that your parser already returned? I mean if your parser would be a speced fn then the data that you get already contains that custom information that you want. Or am I missing something?#2017-02-2116:28mpenetno it's the spec form : something like #(try (parse q) (catch ExceptionInfo e ...))
#2017-02-2116:29Alex Miller (Clojure team)well, youāll also get the failing value#2017-02-2116:29Alex Miller (Clojure team)but not all the work that went into evaluating the failing value#2017-02-2116:29mpenethmm I don't think so#2017-02-2116:30mpenetvalid will return false, conform ::invalid#2017-02-2116:30mpenetex-data the form#2017-02-2116:30Alex Miller (Clojure team)each problem in explain-data has the value that failed the spec#2017-02-2116:30mpenetyes, the initial value true, but not the "detail" of the failed predicate (in my case an ex-info instance)#2017-02-2116:31Alex Miller (Clojure team)right#2017-02-2116:31Alex Miller (Clojure team)maybe the exception case in particular is interesting#2017-02-2116:32mpeneteven then, that wouldn't make it as easy as it is not with the custom spec, it just feeds that extra data to the explain map#2017-02-2116:32mpenetyes, imho it's something that might make sense in spec itself.#2017-02-2116:32Alex Miller (Clojure team)well, what if when conforms fails with an exception, the ex-info was conveyed#2017-02-2116:32Alex Miller (Clojure team)normally it just returns a truthy/falsey value (and nil/false canāt convey additional info)#2017-02-2116:33Alex Miller (Clojure team)but exceptions can#2017-02-2116:33mpenetthat would be nice, that's what I meant with explain-info
, it could return that#2017-02-2116:33mpeneteither accept a explain-info return, or the kw#2017-02-2116:34mpenetbut the reify Spec option is actually nicer in my case (for other reasons)#2017-02-2116:34carocadhow about (or (s/explain-data spec x) ::ok )
#2017-02-2116:35carocadreturns ::ok when everything when well or the ex-info from your spec if it failed#2017-02-2116:41mpenet@alexmiller using explain-info you would basically write your own conformer( try/catch potential exception in my case) and return the (explain-info {...}) on failure. I am not sure doing by default for any exception (or ex-info instance) is desirable#2017-02-2116:42Alex Miller (Clojure team)perhaps not#2017-02-2119:22tjtoltonIs core.spec able to use something else as keyword qualifiers besides the current ns? As I say that out loud, I realize how silly it would be if that were not the case, but I see a lot of the example code using the double colon keywords, which auto expand to current ns#2017-02-2119:23Alex Miller (Clojure team)any ns is fine#2017-02-2119:23tjtoltonI don't suppose there's some kind of dynamic binding that would change the way the double colon autoexpands?#2017-02-2119:24tjtolton::keyword => :com.mycompany/keyword#2017-02-2119:24tjtoltonetc#2017-02-2119:25Alex Miller (Clojure team)you can use aliases#2017-02-2119:26Alex Miller (Clojure team)(alias āa ācom.mycompany)
::a/keyword
#2017-02-2119:26Alex Miller (Clojure team)the tricky part is that right now, com.mycompany
must be an actual namespace#2017-02-2119:26tjtoltonš®#2017-02-2119:26tjtoltonthat's great!#2017-02-2119:27Alex Miller (Clojure team)you can work around that by doing
(create-ns ācom.mycompany)
#2017-02-2119:27Alex Miller (Clojure team)that is something weāve talked about changing though#2017-02-2119:28Alex Miller (Clojure team)https://www.deepbluelambda.org/programming/clojure/know-your-keywords covers this#2017-02-2120:38agguys, Iām kinda stuck again. Iāve done it before (I think) now I canāt remember the right way of doing that. How do I properly generate things based on vector? So If I have generated vector of maps, and then for each item I need to generate something else#2017-02-2120:40agpassing vector generator with fmap into a function with for
kinda works, but I think this isnāt right approach, because I need to call gen/generate
inside for
#2017-02-2120:41agsince it canāt return generator#2017-02-2120:42zaneLike this, @ag? https://github.com/clojure/test.check/blob/master/doc/intro.md#bind#2017-02-2120:43zane> For example, say we wanted to generate a vector of keywords, and then choose a random element from it, and return both the vector and the random element.#2017-02-2120:43ag@zane here in fn in just grabs random element from generated vector and thenā¦#2017-02-2120:44agactually that may work for me#2017-02-2121:38gfredericksRandom element? #2017-02-2121:39gfredericksWhy did you generate a whole vector of them just to pick one out? #2017-02-2122:19Alex Miller (Clojure team)well one reason you might do that is if you need a vector AND an element from that vector#2017-02-2123:27agactually @gfredericks comment in my case make sense.#2017-02-2123:48agok, thereās still something I canāt figure out for the second case, where I have a predefined vector of maps and for each item I need to generate (something based of that map) and attach in a key to that map and finally return that modified vector#2017-02-2123:50agmy solution requires to call gen/generate
within for
, which I donāt like.#2017-02-2123:50agI though test.chuckās for
would work for me, but I canāt get it right#2017-02-2123:54gfredericksCalling gen/generate in general is not a proper way to build a generator#2017-02-2123:55gfredericksAnd it shouldn't ever be necessary#2017-02-2123:56gfredericksIf you want to describe what you're doing I'm happy to help work it out#2017-02-2123:59agso I have list-of-accounts (predefined list) which is a vector of maps, e.g. [{:id 1 :type :customer} {:id 2 :type :investor},,,]
. Now, for each item in that vector I need to generate a balance-update
data and add it in :bal
key and return that modified vector#2017-02-2200:01agletās say balance-update-generator for the sake of exercise generates vector of ints#2017-02-2200:06aghold on. gen-balance-update generates data based on the information of each individual item in the original vector i.e.: (gen-balance-update m)
#2017-02-2200:07gfredericksk, modifying...#2017-02-2200:07agit returns a vector, that vector should go to m
under the :bal
key#2017-02-2200:09agaha!#2017-02-2200:10agI think this is what I need. Iām not sure yet#2017-02-2200:10aglet me try#2017-02-2200:10gfredericksan equivalent approach would be to generate the whole map from gen-balance-update
, and then avoid doing the map
and assoc
on line 15#2017-02-2200:10agGary, you are my hero!#2017-02-2200:25agbtw, has anyone encountered that - very often when I run (gen/generate)
in clojurescript it just kills the websocket repl#2017-02-2200:26gfredericksis it possible it's generating something Very Big?#2017-02-2200:31agnot too big thoughā¦#2017-02-2200:31agstill must be doing something wrong#2017-02-2200:34gfredericksthere are certain combinations of generators that have unfortunate distributions where they are Not Too Big most of the time but occasionally Very Big#2017-02-2200:34gfredericksbut I don't know anything about the cljs repl so I don't know if that's even a reasonable guess#2017-02-2200:40agitās ok, not critical, at least right now for me. I have specs in .cljc files and thatās pretty nice#2017-02-2200:40agI think it should work fine with tests running in headless browser#2017-02-2200:41agbut in general I try to avoid generating large items, clojurescript struggles#2017-02-2200:47gfredericksI would like it to be difficult to accidentally generate Very Large things, but unfortunately I don't think there's a good general solution for the existing pitfalls#2017-02-2207:56mpenet@alexmiller Should I bring up my case earlier in a jira ticket? or is it something that should be discussed with Rich first. Happy to provide a patch if there's interest.#2017-02-2213:33ikitommiStarting to be quite happy with my custom 3-arity conformer: easy to selectively conform on runtime (json, string, fail-on-extra-keys, drop-extra-keys etc). Is this something that could go into clojure.spec itself? ping @alexmiller. If so, should I just write an issue/proposal of this into Jira? And the idea is allow separating āwhatā (the spec) from āhowā (the conformer). e.g.
(s/def ::name string?)
(s/def ::gender keyword?)
(conform
(s/keys :req-un [::name ::gender])
{:name āTommiā, :gender āmaleā, :weight 92}
(merge json-conformers strip-extra-keys-conformer))
; => {:name āTommiā, :gender :male}
Requires some spec inspection, but the http://dev.clojure.org/jira/browse/CLJ-2112 should make it hack-free. Spec as Records would help even more here (could just add a custom new Protocol for this).#2017-02-2213:36mpenetclj-2112 could exist as a library until it's finalized#2017-02-2213:36mpenetas long as the final thing doesn't end up too far from what's in the ticket it'd be a decent solution#2017-02-2213:36mpenetparsing clj forms is a bit gross otherwise imho#2017-02-2213:37mpenet(there's also https://github.com/uswitch/speculate doing that, monkey patching spec along the way to reach its goals)#2017-02-2213:37mpeneta bit too convoluted to my taste#2017-02-2213:38ikitommiwe have similar patching around.#2017-02-2213:38ikitommiwhile waiting for things to resolve in the clojure.spec side.#2017-02-2213:41ikitommiand 2112 could be a separate lib, but I would still need the 3-arity confrom for the specs. currently need to wrap all specs to make it workā¦ https://github.com/metosin/spec-tools#2017-02-2213:42ikitommi(and the dynamic binding hack)#2017-02-2213:43mpenetthat's the one thing that I don't like so far with spec-tools#2017-02-2213:44ikitommiwell that was my question about getting help from the spec, would like to dissolve the hacks on our side.#2017-02-2213:44mpenetI'd rather have the conformers as an arg at another level (spec maybe).#2017-02-2213:45ikitommime too, the 15:33 comment š#2017-02-2213:45mpenetunderstood#2017-02-2213:47mpenetspeculate goes further in the monkeypatch'iness -> https://github.com/uswitch/speculate/blob/master/src/clojure/spec/override.clj#2017-02-2213:47mpenetš#2017-02-2213:49mpenetthat said I am dying for swagger/spec in compojure-api#2017-02-2213:49Alex Miller (Clojure team)@mpenet ticket would be fine - important thing is a good problem statement#2017-02-2213:50mpenet@alexmiller I'll try#2017-02-2213:50Alex Miller (Clojure team)@ikitommi I don't understand what you're proposing #2017-02-2213:54ikitommia 3-arity conform that would a third argument, a callback which the Specās conform would call with the sepc and the value as arguments, the callback would return either a new conformer function of nil. If it returned, it would be used instead of the installed normal conformer.#2017-02-2213:54Alex Miller (Clojure team)It seems unlikely Rich would want to add that#2017-02-2213:55Alex Miller (Clojure team)But starting with a good problem is the way to win his heart :)#2017-02-2213:55Alex Miller (Clojure team)This is a solution, not a problem#2017-02-2213:56ikitommiIf I just write the problem statement into Jira?#2017-02-2213:58Alex Miller (Clojure team)Yes, that's ok, although I will close it if I don't think it's a problem we are trying to solve#2017-02-2214:03ikitommiok, will do. thanks.#2017-02-2214:23mpenetI hope it's not too opaque: http://dev.clojure.org/jira/browse/CLJ-2115#2017-02-2214:41Alex Miller (Clojure team)Iāll take a look, thx#2017-02-2214:46mpenetthe proposal "only" covers the need to have more info for failures the rest is probably too specific to our stuff (even tho the example mentions it)#2017-02-2214:47tbaldridge@mpenet it's unclear from the ticket, are you parsing strings with spec?#2017-02-2214:48mpenetI want conform/valid to tell me if a spec value is a valid string representation of a "Query"#2017-02-2214:49mpenetso it does parse under the hood, and right now (with our patch) conform returns the parsed AST#2017-02-2214:49mpenetbut this is a detail of our use case I think#2017-02-2214:51mpenetthe important bit is about the need to preserve the metadata from the failure to extend what's returned by explain* later#2017-02-2214:51tbaldridgeYeah, that'd be nice, imo. But it also seems that passing a parse tree into spec would get your around the problem.#2017-02-2214:52tbaldridgetreating Spec like a parser, is a good way to get yourself into trouble, in my experience.#2017-02-2214:52mpenetI am not using spec as a parser at all#2017-02-2214:53mpenetthe "parse" function returns either a valid ast or an ex-info, the ast is not consumed by spec in any way ( it's basically a (conformer #(try (parse s) (catch ExceptionInfo :clojure.spec/invalid))
)#2017-02-2214:53mpenetthe goal ultimately is to have our schemas (that contains these query string representations + tons of other stuff), validated by spec#2017-02-2214:54mpenetand have error reporting/handling at a single level#2017-02-2214:54mpenetotherwise yes, we could do another pass just for query validation#2017-02-2214:54mpenet(outside of spec)#2017-02-2216:12tjtoltonOkay, finally managing to get my company to use spec! great stuff! Here's one thing we're stuggling to internalize - our public API uses json that isn't namespaced, and our internal spec's, obviously, use namespaced keys.
Is there some way to say "this incoming data is of the same structure as the spec, but the keys are unqualified, so handle all the keys as though they are qualified to this namespace"?#2017-02-2220:08zaneDid you ever get an answer to this question, @U37NPE2H0?#2017-02-2220:10tjtolton:req-un is actually the answer, yeah#2017-02-2220:10tjtoltonthanks for following up#2017-02-2216:12tjtoltonor does one have to generate a spec-specific view of the data before passing it into a spec'd function --
essentially (clojure.core.walk/prewalk #(keyword *desired-ns* %) data)
#2017-02-2216:16tjtoltonfurthermore, I'm struggling to get my company to accept namespaced keys. They essentially view namespaced keys as an implementation detail of spec that should not be complected with our data model or public API.
I think the big idea with spec is that namespacing keys is actually just a different way to represent data which spec has chosen to support and encourage. But without having enough experience using namespaces and qualified data, I'm pretty sympathetic to the "don't mix implementation details with data" argument.#2017-02-2216:21not-raspberry:req-un?#2017-02-2216:23not-raspberryIf I understand correctly. I only tried to answer the first part of the question.#2017-02-2216:23tjtoltonyeah, I think that's outstanding.#2017-02-2218:29freromHi, I have a small problem with spec that I would like to get some thoughts on.
So the problem is spec name conflicts. Since s/keys couples key and spec names together conflicts can arise:
;; Pet
(s/def ::name string?) ;; CONFLICT
(s/def ::pet (s/keys :req-un [::name]))
;; Person
(s/def ::last string?)
(s/def ::first string?)
(s/def ::name (s/keys :req-un [::first ::last])) ;; CONFLICT
(s/def ::person (s/keys :req-un [::name ::animal]))
(s/valid? ::person {:name {:first "John"
:last "Smith"}
:pet {:name "Buddy"}})
How would your handle this without:
- Separating these into different files
- Combining the specs, e.g. with s/or#2017-02-2218:41not-raspberry(s/def : string?)
#2017-02-2218:41joshjonesWell, you can either use two different files, each having its own namespace, in which case ::name
will resolve to the namespace it's in, or, you can skip the auto-resolve ::
and just name them differently, in the same file:
(s/def :some.namespace/name string?)
(s/def :other.namespace/name number?)
@frerom#2017-02-2218:44freromI would love to do the above. But s/def only accepts qualified keywords no?#2017-02-2218:47tbaldridge@frerom sure, but you can do:
(s/def :person/name ...)
(s/def :pet/name ...)#2017-02-2218:52freromBut s/keys only accept qualified keywords#2017-02-2218:53joshjones(s/def :person/name string?)
(s/def ::yourmap (s/keys :req-un [:person/name]))
#2017-02-2218:54joshjones(s/valid? ::yourmap {:name "Fred"}) ;; true
#2017-02-2218:56freromHuh. So this is exactly what I want, but it doesn't seem to work in ClojureScript? All I get is
Assert failed: all keys must be namespace-qualified keywords
#2017-02-2218:57tbaldridgecan you copy your code?#2017-02-2218:57tbaldridge:person/name is a qualified keyword, I think you might have a typeo#2017-02-2219:00freromSorry about that, it was a typo. It works perfectly fine. Thanks a lot for the help š#2017-02-2219:05freromI guess the risk with this solution is that because the spec doesn't have the namespace in the qualified keyword you might actually get spec name conflicts for different reasons? If :person/name is defined in difference files. So I should probably keep that in mind and not be too generic with the naming.#2017-02-2219:07not-raspberry@joshjones Doing (s/def :person/name string?)
is a good idea until someone does the same.#2017-02-2219:12freromYes, what I would want to do (If I can make up syntax) is something like ::person/name
which expands to :my.namespace.person/name
#2017-02-2219:13joshjones(create-ns 'my.namespace.person)
(alias 'person 'my.namespace.person)
(s/def ::person/name string?)
(s/valid? ::person/name "fred") ;;true
::person/name
=> :my.namespace.person/name
#2017-02-2219:22joshjones@not-raspberry I agree, although nothing prevents anyone from naming any spec anything they want, including any namespace, so it's just making a potential problem less likely, not eliminating it, and assumes good behavior#2017-02-2219:24jasonjcknis there a way to create an s/keys spec from data? e.g. (s/keys :req-un value)
#2017-02-2219:25tbaldridge@frerom right, the names of specs should be bounded to the problem at hand. So if I'm writing a compiler I may use :expr.if/condition while if I was writing a public facing api for a company I may use :com.mycomp.department.person/name#2017-02-2219:25tbaldridge@jasonjckn macros or eval are your options, in that order normally.#2017-02-2219:26jasonjcknalright, i guess macros it is#2017-02-2219:26jasonjckni have never used eval in 7 years of clojure#2017-02-2219:31frerom@tbaldridge I agree#2017-02-2223:12ddellacostawhatās the idiomatic way to write something that is essentially an enum of keywords?#2017-02-2223:13ddellacostaI was looking for something like (s/enum :foo :bar)
but nothing like that exists; should I simply use (s/or :foo #(= :foo %) :bar #(= :bar %))
?#2017-02-2223:13ddellacostaseems overly verbose, to say the least#2017-02-2223:14zane@ddellacosta: #{:foo :bar :baz ā¦}
#2017-02-2223:15ddellacosta@zane š I feel dumb#2017-02-2223:15ddellacostathanks!#2017-02-2223:15zaneDon't!#2017-02-2223:15zaneNo worries!#2017-02-2304:36not-raspberry@ddellacosta watch out - set-based specs dont work for false and nil#2017-02-2304:38Oliver GeorgeIs anyone using caching/memoize type techniques to avoid instrumentation cost? Say keeping a cache of hash values for data which has been checked so that future checks are cheap.#2017-02-2304:51seancorfield@olivergeorge Given that instrument
should be a dev/test-only activity, does performance matter?#2017-02-2304:53Oliver George@seancorfield in this case it causes odd performance#2017-02-2304:54Oliver GeorgeLaggy data input etc. In my case: checking the full definition of a complex form at each function/api call.#2017-02-2304:55seancorfieldWell, yes, instrument
has an overhead. Of course.#2017-02-2304:55Oliver GeorgeSomething you keep an eye out for when developing. Nice not to have false positives.#2017-02-2304:55not-raspberry@seancorfield not if performance doesnt matter.#2017-02-2304:56Oliver GeorgeYeah. I appreciate that. Love spec. Caching would work. Just wondering if anyone has looked into using it.#2017-02-2304:57seancorfieldIf you have an implementation function and then a wrapper -- and the wrapper is memoized -- then instrumenting the implementation should only apply the overhead for each new set of arguments (and this seems the right way to deal with it).#2017-02-2304:58seancorfieldI'd consider very slow/erratic behavior of an instrumented function to be a good indicator that I might need to cache/memoize the function anyway...?#2017-02-2304:59seancorfield(happy to be proved wrong but that's my initial intuition)#2017-02-2305:00Oliver GeorgeThanks. I see what you mean about wrapping a specific fn. That would do the trick for specific cases.#2017-02-2305:08Oliver George@seancorfield any chance you know of a fifo memoize implementation for cljs?#2017-02-2305:09Oliver George(worried that a long running SPA would gobble memory with memoize in the wrong place)#2017-02-2305:10seancorfieldWe can hope that core.cache and core.memoize get converted to .cljc
perhaps?#2017-02-2305:11Oliver GeorgeThat does sound ideal.#2017-02-2305:11seancorfield(and we just got that functionality in the CI support for contrib projects so...)#2017-02-2305:12seancorfieldAs to your original Q, no, sorry, I've no idea about memoization for cls otherwise, sorry š#2017-02-2307:41ikitommito support selective conforming (or protocol extensions) for the spec: http://dev.clojure.org/jira/browse/CLJ-2116#2017-02-2315:03ddellacosta@not-raspberry I was looking for the equivalent of an enum of keywords, so think Iām safe here...#2017-02-2321:39Oliver George@alexmiller I'm trying to understand if memoizing s/conform is likely to cause problems.
(set! s/conform (memoize (fn [spec x] (s/conform* (s/specize spec) x))))
Biggest risk I see is a spec changing causing the cached results to be invalid.
Seems like the spec arg may vary in nature (keywords vs implementations). Perhaps it's better to memoize after calling (specize spec) ?#2017-02-2322:47Alex Miller (Clojure team)same problems as usual with memoize:#2017-02-2322:47Alex Miller (Clojure team)1) itās an unbounded cache, which only fills up at 3 am in production#2017-02-2322:48bbloomaw cāmon now, sometimes it fills up at 4am š#2017-02-2322:48Alex Miller (Clojure team)2) you wonāt see effects of any stateful changes (if in your repl, probably from changing specs). but if you made calls to anything stateful, you wonāt see changes.#2017-02-2322:48Alex Miller (Clojure team)@bbloom only on daylight savings change#2017-02-2322:48bbloomgood point.#2017-02-2322:49Alex Miller (Clojure team)but specs are built with a delay anyways, which is forced on first use#2017-02-2322:49Alex Miller (Clojure team)so they effectively already cache the spec conform implementation and wonāt see changes if you change specs after use anyways#2017-02-2322:49Alex Miller (Clojure team)so thatās maybe only a small downside#2017-02-2323:28agso I have a s/keys
spec with :opt
fields. When I use (gen/generate (s/gen ::my-spec)
it generates them with :req fields and skips :opt fields. Is there a way to force :opt fields to be added?#2017-02-2323:29schmeeif you generate a bunch of them it will add in optionals#2017-02-2323:29schmeegenerate starts out with the simplest possible values and gets more complicated as it goes#2017-02-2323:32agoh, wait, it actually does not skip :opt fields all the time, but now if I have (gen/vector (s/gen ::my-spec))
some items will have some of :opt fields and some would not. Is there a quick and easy way to fix that?#2017-02-2323:32schmeeag thatās what I meant above#2017-02-2323:33schmeeif you want the field all the time, why are they optional?#2017-02-2323:34agbecause, in the runtime they are.#2017-02-2323:34agmeh, I guess itās actually fine.#2017-02-2323:34agI thought for a moment itās just not generating :opt fields at all#2017-02-2323:34agI was wrong#2017-02-2323:37schmeeyou can use a custom generator if you want to add them all the time#2017-02-2400:12Oliver George@alexmiller Thanks Alex, that's helpful. Sounds like some of those issues could be addressed with a LRU/TTL version of memoize. Shame CLJS doesn't have one packed up at this point.#2017-02-2400:13Oliver George(or flushing the memo cache when the spec registry changes)(#2017-02-2400:13Alex Miller (Clojure team)Yeah, core.cache and core.memoize have those, but not cljs right now#2017-02-2400:14Alex Miller (Clojure team)@ag there is a pending patch to fix keys gen for opts#2017-02-2400:19Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2046#2017-02-2401:09agif my vector contains maps with a key thatās value is a function, how do I check for it? is there a predicate like function?
. Actually what I have there is a clojurescript function. (type)
returns something like this:
#object[Function "function Function() { [native code] }ā]
how can I get a predicate that checks for that?#2017-02-2401:16bnoguchiIf we don't want specs' conformed values to thread in an (s/and spec-1 spec-2
from spec-1
to spec-2
to etc , can we substitute s/merge
for s/and
? And can spec-*
be a non-map spec or just a predicate when called with merge
?#2017-02-2401:22danielcompton@ag: cljs.core/ifn?
#2017-02-2401:28ag@danielcompton something tells me that this is right, but for some reason not really working when placed into .cljc file
(s/map-of keyword? ifn?)
#2017-02-2401:29agoh, I think my problem not that, spec works, but needs a generator. hold on#2017-02-2401:31agUnable to construct gen at: [:options 1] for: ifn?]
#2017-02-2401:32agthis worked: (s/with-gen (s/map-of keyword? ifn?)
#(gen/return {:on-click (fn [e] nil)}))
#2017-02-2401:40danielcomptonOh, maybe cljs.core/fn?
would be more appropriate, as maps, keywords, e.t.c. are also ifn?
#2017-02-2402:49agan hour ago I didnāt know that either of them existed. ĀÆ\(ć)/ĀÆ#2017-02-2402:51agyup, youāre right! Thank you @danielcompton#2017-02-2421:15j-poShould (require '[my-ns :as n])
allow you to then address :my-ns/kw
as :n/kw
? I'd been functioning under the impression that yes, but now I'm trying to address a spec defined in another file in the repl and I'm getting "unable to resolve spec".#2017-02-2421:16Alex Miller (Clojure team)No, would be ::n/kw#2017-02-2421:16j-poAha!#2017-02-2421:16j-poThanks#2017-02-2421:17Alex Miller (Clojure team)The :: means "auto-resolve" which will use aliases#2017-02-2502:44thedavidmeisteris there a way in spec to define relationships between attributes, like #(> ::foo :bar)
for example?#2017-02-2504:01joshjones@thedavidmeister
(s/def ::foo int?)
(s/def ::bar int?)
(s/def ::keyspec (s/and (s/keys :req [::foo ::bar])
#(> (::foo %) (::bar %))))
#2017-02-2504:03thedavidmeister@joshjones so... is %
in that context going to be the thing being validated?#2017-02-2504:08joshjonesyes -- for example (s/valid? ::keyspec {::foo 2 ::bar 3})
the map is the argument to the function#2017-02-2504:13thedavidmeister@joshjones oh, cool#2017-02-2504:14thedavidmeisterso then my next q is#2017-02-2504:15thedavidmeisterif i have :req
, how do i get a list of the keys that are missing?#2017-02-2504:25joshjonesthis is not meant to be a 'best practices', just to show how you can get the data:
(->> (s/explain-data ::keyspec {::foo 2})
:clojure.spec/problems
(map :pred)
(filter #(= 'contains? (first %)))
(map last))
#2017-02-2504:46thedavidmeister@joshjones oh ok#2017-02-2504:46thedavidmeisterlooks a bit weird but i guess it does the job#2017-02-2504:46thedavidmeistercheers š#2017-02-2505:23Alex Miller (Clojure team)Call explain-data to get a list of all problems#2017-02-2505:40joshjones@thedavidmeister hmm, "weird" is about all I got late Friday after evening activities š but here you go, same as above but rewritten in a function. cheers to you too
(defn keys-missing
[spec data]
(let [explanation (s/explain-data spec data)
problems (:clojure.spec/problems explanation)
xf (comp (map :pred)
(filter #(= 'contains? (first %)))
(map last))]
(into [] xf problems)))
#2017-02-2505:41joshjonesit relies on things you possibly shouldn't rely on, like contains?
implying that a key is missing, works for now but it's not "public" ... but it's enough to get you going I think š#2017-02-2507:04thedavidmeister@joshjones yup, looks good enough for me to start swapping out my hand-rolled validation code#2017-02-2518:45bbloomi forget - was there a collection of attempted/in-progress specs for all the core functions somewhere?#2017-02-2519:34Alex Miller (Clojure team)from core, no (beyond the one ticket out there that slightly extends whatās committed)#2017-02-2519:34Alex Miller (Clojure team)there is another project someone started, but Iāve intentionally not looked at it#2017-02-2601:01agHey guys, so if I have s/keys
spec, that enforces a predicate on one of the keys (so it actually requires a generator, since it cannot satisfy predicate after so many tries), how the generator part would look like for the spec like that?
trivial example:
(s/with-gen
(s/and
(s/keys :req [:foo :bar :baz :qux :zap :dap])
(fn [m]) (= (:foo m) 42))
(fn []
;; how the generator would look like?
))
Do I have to generate a hashmap with (s/gen)
for each key and then some logic that satisfies the constraint? Is there a better way?#2017-02-2601:05agI guess I would create a spec without the field(s) that I want to constraint and then merge it with another spec (with the constraint(s) and generator)#2017-02-2601:09seancorfieldYeah, splitting it in two would be my suggestion.#2017-02-2601:10seancorfieldSince then you can (g/fmap #(assoc % :foo 42) (s/gen ::base-spec))
or something like that...#2017-02-2622:24Oliver GeorgeIt seems like I can't unform collections. What am I missing?
(require '[clojure.spec :as s])
(s/def ::text
(s/and string?
(s/conformer
(fn [s] {:type ::text :value s :errors []})
(fn [m] (:value m)))))
; GOOD: string conforms correctly
(s/conform ::text "blah")
=> {:type :user/text, :value "blah", :errors []}
; GOOD: map unforms back to string
(s/unform ::text (s/conform ::text "blah"))
=> "blah"
; Bad: collection unform has no effect
(s/unform (s/coll-of ::text) (s/conform (s/coll-of ::text) ["blah" "blah"]))
=> [{:type :user/text, :value "blah", :errors []} {:type :user/text, :value "blah", :errors []}]
#2017-02-2622:45Oliver GeorgeAh, it's a known bug. http://dev.clojure.org/jira/browse/CLJ-2076#2017-02-2701:27bbloomhey @gfredericks et al - have you explored concurrent (ie non-monadic, maybe applicative) generation in any way?#2017-02-2701:31gfredericks@bbloom what would you use that for#2017-02-2701:31gfredericksI'm assuming you're not talking about "running my tests in parallel"#2017-02-2701:31bbloomno, iām talking about conceptual concurrency#2017-02-2701:32bbloomnot necessarily parallelism#2017-02-2701:32bbloomiām mostly thinking about the effort required to make generators that behave well#2017-02-2701:32bbloomthe generate & test approach is kinda cumbersome for some more interesting properties#2017-02-2701:33gfredericksOh you're talking about that awkward separation#2017-02-2701:33gfredericksFor when you wanna do some testy things and then generate some more stuff? #2017-02-2701:34bbloomtrying to come up with a good example, way to explain my thought here#2017-02-2701:34bbloom1 sec#2017-02-2701:38bbloomok so if i do something like:#2017-02-2701:38bbloom(s/exercise (s/and int? #(> % 50000000000)))#2017-02-2701:38bbloomthis still works#2017-02-2701:38bbloombut only b/c the int generator is producing lots of values and then the s/and generator is filtering them#2017-02-2701:39bbloomiām wondering if thereās any experimentation done where the filter can feed information back to the generator#2017-02-2701:39bbloomrather than being a strictly forward pipeline#2017-02-2701:41gfredericksOhright. I feel like rich might have wanted something like that. But I have no idea what direction you would go for a general solution to that. Did you have something specific in mind when you said concurrent? #2017-02-2701:41bbloomsomething earlier had reminded me of the art of the propagators stuff#2017-02-2701:42bbloomand i re-read the section in that report about the truth maintenance systems and the search problem#2017-02-2701:42bbloomonce you add predicates, generation changes from an enumeration problem in to a search problem#2017-02-2701:42bbloomnumbers with inequalities are a pretty easy search space - but can get rough if you did something like āis primeā or whatever#2017-02-2701:43bbloomso you wind up having to override the generator for the whole thing#2017-02-2701:43bbloomand the more interesting the predicates, the more generator work you need to do#2017-02-2701:43gfredericksYeah, it smells like declarative programming, where you can do much more than you might think, but way less than you need in real life#2017-02-2701:44bbloomheh#2017-02-2701:45bbloomi was just wondering if you guys had thought about this sort of thing much at all - since i keep trying to use generative testing, and it keeps being a lot of work. so far iāve only really been truly successful with it in the form of āsimulation testingā on a service#2017-02-2701:46bbloombut i had to do a lot of work to build a good model / generator / validators / etc - made sense for a whole system, but s/exercise etc is fun and rapidly loses utility as i enhance my specs#2017-02-2701:46gfredericksYeah I agree about all of that#2017-02-2701:48gfredericksI haven't thought about it too much because I haven't had any reason to be hopeful that it would be fruitful. I'm definitely open to being persuaded otherwise. #2017-02-2701:49bbloomi guess thatās why i specifically mentioned āapplicativeā - as it seems like there might be some way to improve generators in a compositional way using binary combinators#2017-02-2701:49bbloomthe two interesting ones being intersect and union#2017-02-2701:49bbloomso like maybe rather than defining a generator override for a spec by name#2017-02-2701:49bbloomyou can specify an override for a tuple of union/intersect and a pair of generators#2017-02-2701:50bbloomwhich is already sorta what s/and and s/or do, but the do it left to right#2017-02-2701:51bbloomalthough itās not quite clear to me how to go about this without immediately winding up in the land of SAT/SMT š#2017-02-2701:54gfredericksyeah; and inevitably it would only work for a few lucky combinations of things and otherwise you'd be on your own#2017-02-2701:55gfredericksif there are common combinations of specs that are easy to generate automatically, that can, I assume, be automated by writing special spec macros for them#2017-02-2701:57bbloomok - well i may noodle on this some more at some point, but for now iāll just resign myself to abandoning generative testing as soon as it gets tough and then re-adopting it as soon as my system gets big enough to justify the simulant sort of thing š#2017-04-0518:06slipsetFollowing http://dev.clojure.org/jira/browse/CLJ-2141, it seems to me that Clojure is missing a predicate namespaced? which returns true if a thing is, well, namespaced. #2017-04-0519:00ikitommiwhat kind of data does a (s/coll-of string? :into {})
expects?#2017-04-0519:00ikitommi(def spec (s/coll-of string? :into {}))
(s/explain-data spec {"1" "2"})
; #:clojure.spec{:problems ({:path [], :pred string?, :val ["1" "2"], :via [], :in [0]})}
(s/explain-data spec ["1" "2"])
; nil
(s/conform spec ["1" "2"])
; CompilerException java.lang.ClassCastException
#2017-04-0519:31Alex Miller (Clojure team)@ikitommi if using :into {}, you should be using a spec for a map entry (like a 2 element tuple)#2017-04-0519:33Alex Miller (Clojure team)Like (s/coll-of (s/tuple string? string?) :into {})#2017-04-0519:34Alex Miller (Clojure team)@slipset I don't think that's generally useful enough#2017-04-0519:34Alex Miller (Clojure team)@ghadi design#2017-04-0519:40ikitommithanks alex#2017-04-0520:11Alex Miller (Clojure team)That's effectively how map-of is implemented#2017-04-0523:20takeoutweightIs there any (idiomatic) way to attach custom s/explain
data for user specs? Use case is a predicate that supplies its own context information that I'd like to have preserved in the failure message.#2017-04-0523:38Alex Miller (Clojure team)no#2017-04-0523:38ghadi@alexmiller can you elaborate around design alternatives for vcat
?#2017-04-0523:39Alex Miller (Clojure team)@ghadi no#2017-04-0523:39Alex Miller (Clojure team)@takeoutweight you can vote at http://dev.clojure.org/jira/browse/CLJ-2115#2017-04-0523:39Alex Miller (Clojure team)or weigh in there#2017-04-0523:39Alex Miller (Clojure team)@ghadi at this point I just donāt remember the details (and Rich did not fully elaborate on them anyways)#2017-04-0523:40Alex Miller (Clojure team)we did talk about it for a while. I mostly remember that he didnāt like any of my ideas. :)#2017-04-0523:40ghadisounds legit#2017-04-0710:08trisshey all. so are specās searchable in anyway?#2017-04-0710:09trissletās say I have (s/def ::a string?)
#2017-04-0710:10trissand (s/def ::b (s/keys :req-un [::a])
#2017-04-0710:11trissand I want to know if ::b
contains any values that are strings?#2017-04-0710:11trissis it serious effort to find this out?#2017-04-0710:28ggaillard@triss I don't think there is a function for that purpose. But (s/form spec)
give you the spec form, so in your case the list (s/def ::b (s/keys :req-un [::a])
. You could walk it like a tree and recursively s/form
any spec you find until you get a predicate that match you search criteria ā¦ I do something similar to extract the set of keys used by nested s/merge
specs. Maybe there is a cleaner way to do it thoughā¦#2017-04-0710:37ggaillardDo you know if there is a way to make spec generate a minimalist sample for a particular spec ?
What I mean by minimalist is "the first simplest value validating the predicate". If I have (s/nilable string?)
I'd like to get nil
, if I have string?
, I'd like to get ""
, if I have (s/coll-of ::foo :kind vector?)
I'd like to get []
.
I don't know (yet) how spec works internally but I would expect it could be able to find the shortest path to the first and simplest value validating the predicate. It would be really useful to generate kind of placeholder data waiting to be filled but still validating the spec. Especially when building forms with Re-Frame or similar frameworks.#2017-04-0711:56borkdudeHas anyone seen a similar error message like this lately?
Attempting to call unbound fn: #āclojure.spec/macroexpand-check
We get it in our ClojureScript build. Weāre using clojure.future.spec alpha15.#2017-04-0712:17Alex Miller (Clojure team)@triss there is no easy way to figure that out right now#2017-04-0712:18Alex Miller (Clojure team)@ggaillard you can call gen/sample and take the first example. Samples "grow" in complexity as you use a generator so the first few are usually "simple"#2017-04-0714:18thomashi, I have a (defspec tester 100....
in my test file... but when I run either lein test
or from the REPL it seems to run only once. Any idea what could cause this?#2017-04-0715:17fossifoohow do you fdef a fn without args?#2017-04-0715:24Alex Miller (Clojure team)(s/fdef foo :args (s/cat))
#2017-04-0715:24Alex Miller (Clojure team)although it may not be worth defining a spec for a no-arg function#2017-04-0715:25Alex Miller (Clojure team)as you canāt check its args via instrumentation and generative testing is likely not too useful#2017-04-0715:27fossifoohmmm#2017-04-0715:27fossifootrue#2017-04-0715:28fossifoowell, this method will grow to support multiple parameters, so that's why i wondered in the first place#2017-04-0715:28fossifooi already found out that s/+ works for that#2017-04-0715:29fossifoojust wondered how to do no parameters#2017-04-0717:07Alex Miller (Clojure team)(s/fdef foo :args (s/* any?))
(with a suitable spec instead of any?) will support 0 or more args#2017-04-0721:24weiis there a way to make a s/keys
take a var in the req
list? (let [k :ns/kw] (s/explain (s/keys :req [k]) {:ns/not-kw "hello"})) => Success!
#2017-04-0721:27Alex Miller (Clojure team)not easily - you can use macro or eval to do so#2017-04-0721:30weicurious why the above doesnāt work?#2017-04-0721:30Alex Miller (Clojure team)because s/keys
is a macro and expects a literal vector of keywords in :req
, not something that is evaluated#2017-04-0721:31Alex Miller (Clojure team)in the example above, everything thatās not a keyword (`k`) is just being ignored#2017-04-0721:32Alex Miller (Clojure team)itās being interpreted as (s/explain (s/keys :req []) {:ns/not-kw "hello"})
#2017-04-0721:33weimakes sense, thanks#2017-04-0823:52twashingConsider this code
(s/def ::reqid number?)
(s/def ::subscription-element (s/keys :req [::reqid]))
(s/def ::subscriptions (s/coll-of ::subscription-element))
(defn next-reqid [scanner-subscriptions] :foo)
(s/fdef next-reqid
:args ::subscriptions
:ret number?)
(s/exercise-fn `next-reqid)
#2017-04-0823:53twashing1. Trying to run s/exercise-fn
passing in N arguments, instead of a collection. How can I get s/exercise-fn
to pass in a collection?#2017-04-0823:53twashing2. Also, how are people using the pinned defspec-test
. Running (defspec-test test-name [sut/myfn])
always passes, even if thereās bad code.#2017-04-0823:59twashingOk, Iāve sorted 1.ā¦ But 2. is still a thorn.#2017-04-0900:01twashingBasically, clojure.spec.test/check
returns an empty sequence, even though :ret number?
should break the function.#2017-04-0901:14twashingOk got it. The invocation needed to be (clojure.spec.test/check <backtick>myfn)
, which I was messing up.#2017-04-0902:44xiongtxDoes it need to be backtick? Regular quote should do#2017-04-0903:40twashingHey Tx#2017-04-0903:41twashingFor whatever reason, the check
macro needs
A) (st/check `ranged-rand)
B) (st/check āranged-rand) will yield an empty result.#2017-04-0903:42twashingPresumably it has to do with how the symbol is expanded?#2017-04-0904:01seancorfieldIt needs to be a fully-qualified symbol name - which back tick does - or you could use my.ns/my-fn #2017-04-0907:03fossifoowhen i stest/check my database access functions, the real db implementation gets called. i don't want that. i currently try to with-redefs-fn those. is that the way to go?#2017-04-0907:09fossifooseems to work, i was just too dumb to get the signiture right. time for more specs i guess š#2017-04-0909:40fossifoohmm. actually i'm still stuck. does sb have working examples of instrument with :stub & :gen or :spec?#2017-04-0916:51joshjones@fossifoo some sample code i have stashed for stubbing https://gist.github.com/joshjones/90f65bb11106053240baf5c6d5a4fc2b#2017-04-0917:03fossifoothanks, but only stubbing is not enough in my case#2017-04-0917:03fossifooi either need to use :gen or :spec too because i need to scope the fn spec differently#2017-04-0917:04fossifoobut in the mean time i also found an actual bug in the spec i wanted to check#2017-04-0917:04fossifooso it might've been that all along#2017-04-0917:13fossifoona, no luck. i always get Couldn't satisfy such-that predicate after 100 tries. {}
#2017-04-0917:13fossifoobut the docs make it really hard to know what should be put into :gen or :spec#2017-04-0917:13fossifooand the code is not very straight forward either#2017-04-0917:34Alex Miller (Clojure team):gen or :spec where?#2017-04-1007:53srihariIs it possible to dynamically (at runtime) define a spec in clojurescript?#2017-04-1011:30rovanionThe macro clojure.spec/keys and clojure.spec/keys* expects a seq of keys as the value of its named arguments :req and :opt. I want to use the same list of keys for two different specs, one using keys and one using keys*. But the macros get passed the quoted symbol for the variable and not the value of it. Is it possible to unpack the value pointed to by a symbol at read time like the common lisp #. syntax?#2017-04-1011:30rovanionI've described the issue in longer form on SO: https://stackoverflow.com/questions/43321732/is-it-possible-to-resolve-a-symbol-to-its-value-before-passing-it-to-macro-in-cl#2017-04-1014:51frankI think that depending on how the macro was implemented, you may be able to#2017-04-1015:48donaldball(eval `(s/def ::record (s/keys :req ~ks)))
is what Iāve been using when generating specs from data#2017-04-1020:51tjtoltonI'm sorry, I'm having trouble finding this in the docs (which leads me to believe core.spec has some kind of strong opinion on this) -- is there a canonical way of saying "a predicate that matches anything". essentially a predicate (fn [thing] true)#2017-04-1020:51tjtoltonlooking for, essentially, (s/.)#2017-04-1020:51donaldballany?
#2017-04-1020:52tjtoltonis that a thing?#2017-04-1020:52tjtoltonare you sure you're not thinking of prismatic?#2017-04-1020:52schmeeit is new in clojure 1.9#2017-04-1020:53schmeehttp://clojuredocs.org/clojure.core/any_q#2017-04-1020:53tjtoltonohh, you mean its like a clojure.core/any? ?#2017-04-1020:53tjtoltonhuh#2017-04-1020:53tjtoltonhow about that#2017-04-1022:07stathissiderisare there any efforts for a spec-based condp
-like macro?#2017-04-1022:07stathissiderissomething that would be a more general core.match
#2017-04-1100:35madstapWoops, that is actually just
(defmacro specondp
{:style/indent 1}
[expr & clauses]
`(condp s/valid? ~expr
Probably doesn't need to be a macro...#2017-04-1101:59athosThis might be useful. https://github.com/lambdaisland/uniontypes#2017-04-1102:51Oliver GeorgeI do look forward to instrumentation on clojure.core#2017-04-1102:52Oliver GeorgePerhaps I'm wishing but imagine it would have saved me this headache. (cljs.reader/read-string (str (keyword "3asdf")))
#2017-04-1102:52Oliver GeorgeThat throws a mysterious error since we're trying to read a keyword which starts with a number.#2017-04-1102:53Oliver George(my use case was using js->clj keywordize-keys to turn json into clojure data structures and later serialising to js/localStorage)#2017-04-1102:53Oliver GeorgeA spec on the keyword function would have picked up the illegal input (I believe)#2017-04-1103:23hiredmanNah#2017-04-1103:24hiredmanThe core team has repeatedly stated that those keywords are not illegal, the keyword function is just able to create a wider range of keywords than the reader#2017-04-1104:09Oliver GeorgeIs that right. Huh. It was a bugger to find my unexpected keyword. I guess I'd be happy for the error thrown in read-string to improve.#2017-04-1104:11stathissideris@madstap yes, that would be the basic version, but I was thinking about something that actually binds local vars in the same way that core.match does #2017-04-1104:17stathissiderisWith var names derived from the keyword names in s/cat #2017-04-1104:18cfleming@olivergeorge Itās worse than that - using the function you can create keywords with spaces in them, or anything else your heart desires.#2017-04-1104:19cflemingFeel like a keyword with a newline in it? No problem.#2017-04-1104:24Oliver GeorgeYeah that's a nasty surprise. Seems like a spec instrument on clojure.core/keywords would help to protect me nicely.#2017-04-1104:24Oliver GeorgeAs it is I'll log a ticket for improving read-keyword so that it's blindly obvious when the specific issue I came up with occurs.#2017-04-1104:26Oliver George@alexmiller I gather adding spec's to the core ns's is complicated by aot compilation (or similar). Does that mean user defined optional core ns specs will be difficult/troublesome?#2017-04-1104:26Oliver George@cfleming have you found any good opportunities for Cursive to integrate/leverage specs?#2017-04-1104:27Oliver George(I wondered if it could be used to improve autocomplete suggestions)#2017-04-1104:28cflemingThere are plenty, but I havenāt had time to investigate it yet. It could definitely be used to improve autocomplete, yes, assuming the spec machinery provides enough hooks to get that info out.#2017-04-1104:29Oliver George@cfleming good stuff#2017-04-1107:40ikitommiHmm.. s/merge
seems to report problems of qualified keys twice:
(require ā[clojure.spec :as s])
(s/def ::a int?)
(s/def ::b string?)
(s/explain-data
(s/merge
(s/keys :req-un [::a])
(s/keys :req-un [::b]))
{:a 1 :b 2})
; #:clojure.spec{:problems ({:path [:b], :pred string?, :val 1, :via [:user/b], :in [:b]})}
(s/explain-data
(s/merge
(s/keys :req [::a])
(s/keys :req [::b]))
{::a 1 ::b 1})
;#:clojure.spec{:problems ({:path [:user/b], :pred string?, :val 1, :via [:user/b], :in [:user/b]}
; {:path [:user/b], :pred string?, :val 1, :via [:user/b], :in [:user/b]})}
#2017-04-1108:05akielAs I read the doc on s/&
this (s/conform (s/& (s/+ char?) #(apply str %)) [\s \p \e \c])
should return "spec"
but returns [\s \p \e \c]
. Am I wrong?#2017-04-1108:41akiel(s/conform (s/& (s/+ char?) (s/conformer #(apply str %))) [\s \p \e \c])
returns "spec"
. I debugged the code and s/&
calls dt
with cpred?
nil
. So maybe it should use cpred?
true
here?#2017-04-1109:18wottisgreetings#2017-04-1109:21akielJuste require shepherd.spec
without any alias and use :principle/type
#2017-04-1109:25wottis@akiel thank you very much!#2017-04-1112:13Alex Miller (Clojure team)@akiel s/& just validates the predicate then returns the conformed value of the spec so that is the expected behavior. You could switch it to s/and though and use s/conformer for the last bit #2017-04-1112:16Alex Miller (Clojure team)@ikitommi it's reporting the same bug from each branch of the merge. Not sure if those should be deduped or not#2017-04-1112:18ikitommiI think it should, as it works with unqualified keys.#2017-04-1112:24Alex Miller (Clojure team)@olivergeorge you can add user-defined specs to core. Works fine. We have no plans to place a spec on keyword any narrower than string? though. What would be a useful enhancement though is to have either a predicate or "safe" version of keyword that accepted only print-read-safe keywords#2017-04-1112:25Alex Miller (Clojure team)keyword-strict or something#2017-04-1112:26Alex Miller (Clojure team)And then you could use that in json parsing etc#2017-04-1112:39akiel@alexmiller The doc of s/&
says:
takes a regex op re, and predicates. Returns a regex-op that consumes input as per re but subjects the resulting value to the conjunction of the predicates, and any conforming they might perform.
especially: but subjects the resulting value
and any conforming they might perform
I read that as if the cvals of the preds should be returned. The same as s/and
is doing, as you say.#2017-04-1113:00Alex Miller (Clojure team)Hmm, not sure. I would need to ask Rich.#2017-04-1113:14akiel@alexmiller Ok. Thanks.#2017-04-1115:50ikitommi@alexmiller wrote an jira issue of that double problems, also another of s/keys*
is returning something that is not a clojure.spec.Spec
#2017-04-1115:51bronsa@ikitommi no regex op return specs tho#2017-04-1115:52bronsasame for s/cat, s/+ etc#2017-04-1116:43ikitommi@bronsa oh, that bad.#2017-04-1117:07ikitommiwell, now that I look at the clojure.spec test suite, no wonder there are (form) bugs. @alexmiller, is there a way to contribute more tests? testing all forms & return values as Spec
s would easily reveal all things that donāt work right now.#2017-04-1118:03ikitommione more form bug:
(require '[clojure.spec :as s])
(s/def ::a int?)
(s/form (s/& ::a))
; (clojure.spec/& #object[clojure.spec$spec_impl$reify__13797 0x47b888f9 "
#2017-04-1118:13bronsa@ikitommi i think that's by design (regex ops not being specs)#2017-04-1119:40Alex Miller (Clojure team)correct, thatās as intended (those respond to regex?
#2017-04-1119:45Alex Miller (Clojure team)regarding &
forms, that problem space is related to the issues with the keys* form (which is implemented with &). would be fine to log though.#2017-04-1121:02Oliver GeorgeThanks @alexmiller I'm happy with those options. #2017-04-1121:06Alex Miller (Clojure team)@olivergeorge Iāve talked about these before with people but I donāt think it was ever actually logged#2017-04-1122:03Oliver GeorgeI wonder where that documentation should live. #2017-04-1122:04Oliver GeorgeAdding a keyfn option to js->clj would make it easier to step safely around the sharp edges#2017-04-1122:42Oliver GeorgeThere is an open ticket to improve cljs.reader/read-keyword to produce a better error here: http://dev.clojure.org/jira/browse/CLJS-1907#2017-04-1122:43Oliver GeorgePlease upvote#2017-04-1123:31waffletowerIf this is a reasonable place to ask about generatorsā¦ I am trying to understand clojure.test.check.generators/no-shrink. I have tried to eliminate shrinking in this generator example:
(def large-flat-natural
(gen/no-shrink (gen/fmap abs gen/large-integer)))
#2017-04-1123:31waffletowerBut it gives shrunken results:
(repeatedly 16 #(println (gen/sample large-flat-natural 24)))
(1 0 1 2 1 15 1 0 5 5 7 76 2 109 3 234 99 41987 18 9 17237 45920 1019 14841)
(0 1 1 0 2 1 5 5 46 3 177 0 6 169 23 25 25 5 367 1420 311458 36483 42628 1)
(0 0 0 0 4 2 1 0 71 1 5 35 22 208 26 11 3 8391 32493 232365 2 2 2750 5576)
(1 1 1 0 0 1 2 2 3 6 6 19 1 55 3 2 33 20263 1877 1 362177 2 6258 15641)
(1 0 1 2 2 2 6 58 0 26 66 39 1860 1 347 4 9073 6 19852 31997 25 56137 117428 23)
(0 1 1 2 0 1 1 0 31 13 1 116 3 105 1 26 8959 710 38835 4 1701 0 91091 27453)
(0 1 0 2 1 8 7 15 2 8 4 9 35 1254 0 22 20463 1 170 2 15412 1 0 23163)
(1 1 0 1 1 1 14 1 2 1 31 141 1 3 409 15 19 7946 49819 55 3831 44642 1 510806)
(0 1 1 1 1 1 8 1 1 21 3 2 32 0 504 1 4 0 1 2737 24695 2701 242 470111)
(1 1 1 0 1 1 3 5 41 37 7 5 250 515 9 0 2 13 566 44053 174 9371 401689 66970)
(1 1 1 0 2 5 4 1 36 1 21 3 106 4 813 542 116 197 19 949 187929 24 6 52)
(0 0 1 1 2 6 3 1 2 0 15 1010 50 30 13 51 15183 0 13328 1885 2 3 2787 1425)
(1 0 2 1 1 2 1 22 0 25 81 0 11 0 113 4975 177 2 15989 12 90275 186 7 2619)
(1 0 0 3 1 3 3 2 7 2 21 957 127 1055 1364 6 15462 2 40 39 16 298275 2985 762)
(0 0 1 1 7 1 2 1 1 200 256 1 1 1412 3913 0 61 1 355 0 4 25 2 292057)
(0 1 0 1 2 5 2 3 0 44 13 2 3 423 2744 381 689 233 260 1324 49733 280 14384 626)
#2017-04-1200:08gfredericks@waffletower those aren't shrunk, they're small-sized#2017-04-1200:09gfredericksBackground: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2017-04-1200:11waffletowerWill check that out. Trying to get a uniform distribution. Am tempted to use the private (make-gen) and the random namespace as you do with the generator uuid.#2017-04-1200:11waffletowerIs there a reason that make-gen is private?#2017-04-1200:11gfredericksUniform among longs?#2017-04-1200:11waffletoweryes#2017-04-1200:11gfredericksIt's private because it's usually not necessary#2017-04-1200:12gfredericksBut we don't have a uniform long generator at the moment#2017-04-1200:12gfredericksThough choose is close#2017-04-1200:12gfredericksI'm considering adding one#2017-04-1200:12waffletowerI know I am young to test.check, and you have been really helpful, but I keep bumping into boundaries.#2017-04-1200:13gfredericksAre you using it for PBT or some other speccy thing? #2017-04-1200:13waffletowerIn this case I am making a generator for dates#2017-04-1200:13gfredericksWhy do you want a uniform long? #2017-04-1200:14gfredericks(I.e., why does that help for dates?)#2017-04-1200:14waffletowerusing clj-time and it is convenient to generate a random date from a long rather than its constiuent parts. Want them to be evenly distributed.#2017-04-1200:15waffletower(def date-8601
(gen/fmap
(fn [x] (f/unparse (f/formatters :date-time) (c/from-long x)))
gen/large-natural))
#2017-04-1200:15gfredericksYou'll get weird dates with >4 digit years that way, won't you? #2017-04-1200:16waffletowerI could use ints instead, but would be nice if they were uniform and not usually centered around 1970#2017-04-1200:17gfredericksI guess I'm trying to figure out why uniformity is important for this, especially since the domain will inevitably end up rather arbitrary#2017-04-1200:18gfredericksE.g., dates in the years -1000000 to 1000000#2017-04-1200:18gfredericksI think date & time generators end up being weird regardless :/#2017-04-1200:19waffletowerIts a question of control. For some data I feel a desire to turnoff the growth and shrinking mechanisms and control the output with localized RNG#2017-04-1200:20gfredericksgen/choose is the classic tool for turning off growth#2017-04-1200:21waffletowergreat lead, I will check it out.#2017-04-1200:21gfredericksIt just has edge cases outside of the 32 bit range#2017-04-1200:21gfredericksI was thinking of making a variant called uniform-integer that world be more robust#2017-04-1200:23waffletoweryou are supremely helpful. thanks again#2017-04-1200:24waffletowerI see gen/scale as wellā¦#2017-04-1200:36gfredericksYep. gen/sized is a more general (ha!) version of that#2017-04-1200:36gfredericksA lot of generators that do nontrivial growth things do it by combining choose and sized#2017-04-1200:37gfredericksI guess I mean that those two things give you the power to completely customize growth#2017-04-1200:49fossifoodoes sb have working examples of instrument with :stub & :gen or :spec?#2017-04-1200:55fossifoohere's a gist of what i try to get to run: https://gist.github.com/FossiFoo/f98f2eaf920c3481d9fdf9bdeb41c420#2017-04-1200:57fossifoothe ":replace" works, but i thought i could just change the result of (db/select) so that it fits the spec when testing the function calling it#2017-04-1200:58fossifooi know that my datomic/adi db/select will return the correct types, but i'm not sure how to put that knowledge into spec#2017-04-1201:27fossifooi think an example of ":gen a map from spec names to generator overrides" in clojure.spec/instrument would help#2017-04-1201:27fossifooi got a version to run that uses ":spec", so that's a little nicer#2017-04-1201:35fossifoofdef docs says "Once registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests," is there such a thing as this runner?#2017-04-1202:42wottisdoes someone know a larger project (present on github) that uses clojure.spec? I want to see how spec is really being used. All of the small, and simple repl example can't answer the questions i have.#2017-04-1203:47seancorfieldAs noted in #clojure, it's kinda hard since most real world Clojure projects have not yet moved to 1.9.0 Alpha builds.#2017-04-1208:36curlyfry@wottis I like this one (in ClojureScript): https://github.com/jrheard/voke/tree/master/src/voke It's a WIP game that uses spec for its entity system. Not super large, but makes good use of fdef
s and has a few generative tests! š Maybe @jrheard can tell us a bit more about it?#2017-04-1208:46rovanionwottis: Here is a list of all the projects on clojars I've found using spec: systems-toolbox, spex, oops, alia-spec, clj-edocu-users, mistakes-were-made, utangled-client, clj-edocu-help, spec, net, clj-element-type, gadjett, datomic-spec, tappit, tick, deploy-static, tick, spectrum, sails-forth, ring-spec, doh, tongue, diglett, rp-query-clj, pedestal.vase, clj-jumanpp, om-html, crucible, ctim, crucible, wheel, merlion, turbovote-admin-specs, odin, pc-message, active-status, rp-json-clj, curd, rp-util-clj, swagger-spec, invariant, functional-vaadin, untangled-client, flanders, uniontypes, dspec, schpeck, macros, replique, specific#2017-04-1208:47wottis@curlyfry @rovanion Thank you very much!#2017-04-1208:47mpenetspandex also has specs#2017-04-1208:48rovanionRight, libraries that are so "popular" that they're on http://clojure-toolbox.com are: alia, cljs-oops, closp, core.typed, graphql-clj, java.jdbc, onyx,
sablono, spandex, tupelo.#2017-04-1208:48rovanionForgot that I had split them into two.#2017-04-1208:50wottishrhr that should be enough for now#2017-04-1209:26rovanionSo perhaps start with some library from the latter list.#2017-04-1209:47rovanionIs it possible to express something like (spec/keys :req-un [(spec/or ::a ::b)])
so that either :a
or :b
is present in the resulting map, but never the two at the same time?#2017-04-1210:00dergutemoritz@rovanion Not with s/keys
alone#2017-04-1210:01dergutemoritzYou gotta s/and
it with a pred of that effect#2017-04-1210:02fossifoothis seems kinda noisy#2017-04-1210:03fossifooi also wondered about that and i have some cases where a thing is either a or b or c or d (or ... some more)#2017-04-1210:03fossifoomight be bad interface design, but it's exploding fast#2017-04-1210:25dergutemoritz@fossifoo "is either a or b or c or d" sounds like s/multi-spec
could be worth a look for you#2017-04-1210:28fossifooah, yeah, i glanced over that. might work out in my case#2017-04-1214:10Alex Miller (Clojure team)@fossifoo regarding your question above about the docstring for fdef - that has already been fixed - should be clojure.spec.test/check. Where are you seeing the version pointing to run-test?#2017-04-1214:12Alex Miller (Clojure team)oh, probably the online clojure doc - there is actually a known problem preventing those docs from being regenerated and they are a few alphas behind. There arenāt many changes to the API since then but thatās one.#2017-04-1214:33fossifoookay#2017-04-1214:33fossifoodo you happen to have an example or clarification for instrument {:gen} ?#2017-04-1214:35fossifoo"a map from spec names to generator overrides" i read that as {:some/spec (gen/string)}
or such , but that didn't seem to work for me#2017-04-1214:36fossifooi always got the "could'nt generate in 100 tries" error with that. don't have the actual error on hand#2017-04-1214:36gfredericks(fn [] gen/string)#2017-04-1214:37gfredericksI think#2017-04-1214:37fossifooah, worth a try#2017-04-1214:37fossifooit's not worded very clearly#2017-04-1214:38fossifooand the source is not really readable either ;D#2017-04-1222:20micahasmithany advice on spec-ing async functions?#2017-04-1222:35micahasmithalso, is this sort of redef-ing inside of a spec bad form?#2017-04-1222:36micahasmith(s/fdef ::shopify-api!
:args (s/cat :opts ::shopify-api-opts)
:ret nil?
)
(with-redefs [org.httpkit.client/get (fn [a b c] nil)]
(s/valid? ::shopify-api! shopify-api!))
#2017-04-1300:37tbaldridgeIMO, with-redefs is always bad form#2017-04-1300:43micahasmithfeel like the more i work through this clojure.spec guide, iām realizing this is more about typing + generative testing than testing#2017-04-1300:44micahasmithwhich is strange given then (s/fdef :fn #())
functionality#2017-04-1300:53micahasmithor does my mocking have to turn into custom generators#2017-04-1309:40rovanionHas any of you ever tried to s/merge
two s/keys*
? I can only seem to get it to work for validation and not for generation.#2017-04-1309:41rovanionWrote a complete example on SO: https://stackoverflow.com/questions/43388710/generator-for-union-of-two-sets-of-keys-for-named-arguments-with-clojure-spec#2017-04-1314:23jfntnGetting some odd validation errors on the :ret
with orchestra.spec.test
so Iām not sure if the spec is wrong?#2017-04-1314:33Alex Miller (Clojure team)comparator fns donāt have to return only -1 / 0 / 1, although a particular one might#2017-04-1314:35Alex Miller (Clojure team)they can return any int - the sign is the important thing#2017-04-1314:36Alex Miller (Clojure team)comparators also typically have constraints around comparing two of the same kind of things#2017-04-1314:36jfntnoh I didnāt know about the sign#2017-04-1314:37jfntnI think the issue is I forget fspec
is generating data from the args spec and calling the comparator with it, but I have some tricky invariants between a and b that are not encoded inside the args spec#2017-04-1314:38Alex Miller (Clojure team)yep#2017-04-1314:38jfntnthat makes sense, thank alexmiller#2017-04-1314:49jfntnTrying to resist the urge to shave that yak but Iām curious what would be the right approach here: a and b only have a partial order, the state thatās being closed over by the comparator is a lookup table thatās used to get a total order. So if I wanted the whole fdef
to work, Iād have to constrain the fspec
:args
generator with the value generated for the fdef
:args
constructor š¤#2017-04-1314:51Alex Miller (Clojure team)yeah, thatās not likely going to be possible to do easily#2017-04-1314:51Alex Miller (Clojure team)but what you can do is to a) build a better args spec that uses values from your lookup and b) write a fn spec that verifies something about the comparator#2017-04-1314:52Alex Miller (Clojure team)for a, I mean the generator#2017-04-1314:54Alex Miller (Clojure team)and then you also need to ask yourself whether the tests you get out of it are worth the effort youāre putting into the spec. itās ok if the answer is no. :)#2017-04-1314:55Alex Miller (Clojure team)higher order stuff is particularly hard to gen test#2017-04-1314:59jfntnI think the answer is no in this case, but I think I understand how this would work: the fspec
for the comparator would be checked in isolation so its args need to have the invariants from the lookup table but we donāt need to somehow ensure itās lexically the same table. So the gen for the lookup table can build the relationships, and the gen for comparable would bind to it and pluck a couple of values out of the table. Is that somehow correct?#2017-04-1316:18Alex Miller (Clojure team)yeah, youād have to close over some of that stuff I guess#2017-04-1414:38fossifoo@gfredericks thanks for the example, instrument {:gen (f [] somegen)}
works indeed#2017-04-1414:58borkdudeBefore I post this to JIRA, could someone take a look if this is bug?
https://gist.github.com/borkdude/95301951e81e3022674279d463783586#2017-04-1415:28moxaj@borkdude reloaded.repl
is not a valid symbol#2017-04-1415:29borkdude@moxaj Why not - and why is this the only symbol in my dependencies causing problems?#2017-04-1415:30borkdude(symbol? āreloaded.repl) ;;=> true
#2017-04-1415:32borkdudeAccording to https://clojure.org/reference/reader#_symbols it is valid I think?#2017-04-1415:34moxajyeah, I was wrong about that š® seems like a bug to me#2017-04-1415:35moxajif you move the quote inside the set it works#2017-04-1415:37borkdudeAh#2017-04-1415:38borkdudeThis also works:
user=> (def deps ā#{[org.clojure/core.async ā0.3.442ā]
[prismatic/schema ā1.1.4"]
[com.stuartsierra/component ā0.3.2ā]
[reloaded.repl ā0.2.3"]})
#āuser/deps
user=> (s/def ::dependency deps)
#2017-04-1415:41borkdudeor the equivalent in a let
, but not inside ->>
. Probably because def
and let
are compiler level things and ->>
is a macro#2017-04-1415:42borkdudeI can live with the let
solution, but if itās a bug, Iām happy to post it to JIRA#2017-04-1417:16Alex Miller (Clojure team)itās not a bug#2017-04-1417:17Alex Miller (Clojure team)the quoted form is expanded to (quote #{ ... })
#2017-04-1417:17Alex Miller (Clojure team)s/def is a macro - it doesnāt expect to find a (quote ...)
form#2017-04-1417:17Alex Miller (Clojure team)@borkdude ^^#2017-04-1417:19moxaj@alexmiller why is it trying to resolve the symbol though?#2017-04-1417:20Alex Miller (Clojure team)where do you see that?#2017-04-1417:21moxajI think this is the line throwing the exception: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L308#2017-04-1417:21Alex Miller (Clojure team)oh, itās seeing a bare symbol with a ā.ā in it, which is interpreted as a class instance#2017-04-1417:21Alex Miller (Clojure team)so itās trying to load the class#2017-04-1417:22Alex Miller (Clojure team)reloaded.repl
is being interpreted like java.lang.String
#2017-04-1417:26Alex Miller (Clojure team)āitā here being the compiler, I donāt think itās ever even getting to spec#2017-04-1417:34moxaj@alexmiller to me it seems like the resolve
call at the line I linked is responsible for the exception#2017-04-1417:35moxajor what do you mean by "not getting to spec"?#2017-04-1417:35borkdudeThen whatās the difference between calling
(-> '#{[org.clojure/core.async "0.3.442"]
[prismatic/schema "1.1.4"]
[com.stuartsierra/component "0.3.2"]
[reloaded.repl "0.2.3"]})
which works and
(s/def ::dependencies '#{[org.clojure/core.async "0.3.442"]
[prismatic/schema "1.1.4"]
[com.stuartsierra/component "0.3.2"]
[reloaded.repl "0.2.3"]})
which doesnāt ?#2017-04-1417:35borkdudeboth are macros#2017-04-1417:36Alex Miller (Clojure team)well the macros do do different things :)#2017-04-1417:36Alex Miller (Clojure team)@moxaj you could be right - itās basically the point at which the form passed to s/def is evaluated inside s/def#2017-04-1417:37Alex Miller (Clojure team)s/def uses the form in both an evaluated and unevaluated context#2017-04-1417:37moxajhttps://gist.github.com/moxaj/d21f8f1db3e4cf32dcb12485a248070e#2017-04-1417:37borkdudebut I quoted the form which should remain the same when evaluated?#2017-04-1417:37moxajreplaced resolve
with my-resolve
to confirm#2017-04-1417:39Alex Miller (Clojure team)during macro-expansion a quoted form is literally (quote ...)
- it has not yet been evaluated#2017-04-1417:39borkdudehmm ok#2017-04-1417:40souenzzo+1 on clojure.spec
use data > functions > macros
.... I also have some problems related to this macros...#2017-04-1417:40Alex Miller (Clojure team)the quote expansion happens during reading#2017-04-1417:41Alex Miller (Clojure team)@souenzzo several parts of spec donāt work without macros#2017-04-1417:41Alex Miller (Clojure team)like explain
#2017-04-1417:44Alex Miller (Clojure team)also, keep in mind that spec forms are data and can be conformed into a map-y form with spec specs (and unformed back)#2017-04-1417:44Alex Miller (Clojure team)they can also be transformed in the middle of that or even constructed straight from the mappy form via unform#2017-04-1417:45mpenetNot yet right?#2017-04-1417:45borkdudeok, so in the case of passing a set, itās best to evaluate it before passing, instead of quoting it#2017-04-1417:45Alex Miller (Clojure team)hard to say ābestā - just that there is an interplay here between reading, macroexpansion, and the macro that is subtle#2017-04-1417:46moxaj@borkdude in your case, the safest I believe would be to def
it, and use the var#2017-04-1417:46Alex Miller (Clojure team)@mpenet patch is on CLJ-2112. itās not done yet but idea is there.#2017-04-1417:46borkdude@moxaj yes, let also works (wrote that earlier already)#2017-04-1417:46mpenetYes#2017-04-1417:46Alex Miller (Clojure team)(defn make-or [keys preds]
(let [or-data {:s 'clojure.spec/or
:args (map #(hash-map :tag %1 :pred [:pred %2]) keys preds)}]
(s/unform ::ss/or-form or-data)))
(make-or [:a :b] [`int? `keyword?])
;;=> (clojure.spec/or :a clojure.core/int? :b clojure.core/keyword?)
#2017-04-1417:46Alex Miller (Clojure team)was just playing with it this morning actually - thereās a function that makes a spec from data#2017-04-1417:47mpenetSo "are" -> "will be" #2017-04-1417:47Alex Miller (Clojure team)well Iād say specs āareā data now, just that you need the specs for that data too#2017-04-1417:48Alex Miller (Clojure team)the ::ss/or-form
in the example above is the only missing piece#2017-04-1423:30standIs there some way of specing a situation where a map has mutually exclusive optional keys? That is neither ::a
or ::b
is required but if ::a
appears, ::b
may not and vice versa.#2017-04-1423:43Alex Miller (Clojure team)You can and a predicate#2017-04-1423:43Alex Miller (Clojure team)Or possibly multi-spec is a better match#2017-04-1423:58standCan you explain what you mean by "and a predicate?"#2017-04-1501:10Alex Miller (Clojure team)(s/and (s/keys ...) #(xor ... %))#2017-04-1501:11Alex Miller (Clojure team)Not that xor is a thing, but you get the idea#2017-04-1501:59seancorfieldA concrete example from our codebase @stand (s/def :criteria/general (s/and (s/keys :req-un [(or :criteria-item/value :criteria-item/values)
:criteria-item/weight])
#(not (and (:value %) (:values %)))))
#2017-04-1502:00seancorfieldA general criteria map must have :weight
but can only have one of :value
or :values
.#2017-04-1502:02standI don't believe that would work in my case. I need a situation where neither :value
or :values
need appear but if one appears the other cannot. I don't think you can do an or
in a keys :opt
, can you?#2017-04-1502:06seancorfieldAh, no, you cannot.#2017-04-1502:07seancorfieldBut they would both just be :opt
keys and you could still s/and
the check that they are not both present#2017-04-1502:07seancorfieldlike this (s/def :criteria/general (s/and (s/keys :opt-un [:criteria-item/value :criteria-item/values
:criteria-item/weight])
#(not (and (:value %) (:values %)))))
#2017-04-1502:08seancorfieldThat would work for the neither case, the either case, and prevent the both case I think?#2017-04-1613:35ikitommiWill there be a utility to turn forms back into specs (without eval)?#2017-04-1615:39mpenetI guess spec form conforming + unform would allow this#2017-04-1702:10xiongtxIs there a plan to make an fn spec part of the fn itself, similar to :pre
and :post
? As a piece of metadata, maybe?#2017-04-1702:11xiongtxIt makes sense to keep contraints on an fn w/ the definition#2017-04-1703:40Alex Miller (Clojure team)No#2017-04-1714:11stathissiderisJust released spec-provider, a library that will infer specs from sample data: https://github.com/stathissideris/spec-provider#2017-04-1716:08mobileinkIs there any way to attach metadata to a spec? I want to encode json specs like the following in spec: "n": {"type": "string",
"readOnly": true,
"description": "Friendly name of the resource"},
#2017-04-1716:13mobileinkthis is best iāve come up with, at least it documents it: (s/def ::n (s/and string?
(comment {:read-only true
:doc "Friendly name of the resource"})))
#2017-04-1716:16mobileinkspec works great for e.g. https://github.com/OpenInterConnect/IoTDataModels/blob/master/oic.core.json except for meta props like readOnly.#2017-04-1716:47ikitommi@mobileink there are at least two libs on top of spec which allow meta-data on specs (and produce JSON Schemas): https://github.com/metosin/spec-tools & https://github.com/uswitch/speculate. Iām working on the first one, in which the meta-data is written to spec forms, so the are really persisted. Last PR before releasing, this week hopefully.#2017-04-1716:48mobileinkthank you. i did take a brief look at spec-tools and saw some spec->json schema stuff; does it go the other way āround?#2017-04-1716:49mobileinkanyway, i take it that means no way to do it in plain spec?#2017-04-1716:51ikitommino, just one-way currently. @mpenet had plans of doing the other way around, would open up a whole new level of interop with js.#2017-04-1716:52mobileinkso in spec-tools, for readOnly I would do :json-schema/readOnly true
?#2017-04-1716:54ikitommiyes#2017-04-1716:55ikitommiIn plain spec, I guess you could add meta-data to registered specs: read the registry value, add meta, re-register.#2017-04-1716:57mobileinkyeah, was thinking about that but I think Iāll take the path of least resistance and give spec-tools a try. thanks.#2017-04-1717:24waffletowerI am trying to make generative testing of a recursive spec more feasible. Consider the following simple example:
(s/def ::throne string?)
(s/def ::monarch string?)
(s/def ::gold int?)
(def kingdom (s/spec
(s/keys :req-un [::throne
::monarch
::gold]
:opt-un [::kingdoms])))
(s/def ::kingdoms (s/coll-of kingdom))
#2017-04-1717:26waffletowerIs there an external way to limit the size of the collection produced by s/coll-of
, or is a custom generator necessary?#2017-04-1717:55nfisher@waffletower :min-count :max-count allows you to control the collection size.#2017-04-1717:56nfisher(s/def ::kingdoms (s/coll-of ::kingdom :min-count 1 :max-count 10))
#2017-04-1717:58nfisherIs that what youāre looking for?#2017-04-1718:06waffletowerThanks!
That limits the size supported in the spec though, correct? I am trying to leave the maximum unbounded but place limits to facilitate generation.#2017-04-1718:08waffletowerI have tried this route:
(s/def ::kingdoms (s/with-gen
(s/coll-of kingdom)
(gen/resize 1 (s/gen (s/coll-of kingdom)))))
this often causes an exception which I am currently tracking down:
ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn clojure.spec/every-impl/reify--13992 (spec.clj:1285)
#2017-04-1718:08Alex Miller (Clojure team)Use :max-gen for that#2017-04-1718:09Alex Miller (Clojure team)Default is 20 but I find 3 to be better usually#2017-04-1718:12waffletowerThanks Alex, need to slow down to see text like: same options as 'every'
š#2017-04-1718:42ikitommimap-of
keys donāt get conformed?
(require '[clojure.spec :as s])
(s/conform
(s/map-of keyword? keyword?)
{"key" :value})
; :clojure.spec/invalid
(s/conform
(s/map-of keyword? keyword?)
{:key :value})
; {:key :value}
(s/conform
(s/map-of (s/conformer keyword) (s/conformer keyword))
{"key" "value"})
; {"key" :value}
#2017-04-1718:59Alex Miller (Clojure team)no, but you can pass :conform-keys option for that if you need it#2017-04-1805:09ikitommithanks!#2017-04-1805:15ikitommistill, I find the below bit unintuitive: without :conform-keys
the first one informs of error but the second one does not:
(s/conform
(s/map-of keyword? keyword?)
{ākeyā :value})
; :clojure.spec/invalid
(s/conform
(s/map-of (s/and (s/conformer keyword) keyword?) keyword?)
{ākeyā :value})
; {ākeyā :value}
#2017-04-1808:43didibus@ikitommi Why would a string conform to a keyword?#2017-04-1809:52ikitommibecause of the conformer. If I set the :conform-keys
it conforms ok as Alex pointed out. But without it, the conform returns invalid data, which feels odd given the s/conform
docs: āGiven a spec and a value, returns :clojure.spec/invalid if value does not match spec, else the (possibly destructured) value.ā.#2017-04-1812:29Alex Miller (Clojure team)Seems like you're ignoring the map-of docstring#2017-04-1812:29Alex Miller (Clojure team)There are several good reasons not to conform the keys by default#2017-04-1812:36pseudwhat's the point of clojure spec's fdef ? If I check against my specs using pre & post-conditions, I will get failures at the point of invocation, but having defined a function spec doesn't really prevent the function returning a non-compliant value (and I assume the same is true for providing wrong input)#2017-04-1812:37pseud(aside from stubbing dependent functions for tests)#2017-04-1812:37Alex Miller (Clojure team)For use with stest/instrument and stest/check#2017-04-1812:38pseudThat's it ? The spec guide hints at fdef providing something for development, I can't temporarily get more feedback during development ?#2017-04-1813:01urbank@pseud Isn't instrument meant for development - you can instrument a function so that it check its input, output when called?#2017-04-1813:03curlyfryinstrument only checks input for fdef if I remember correctly#2017-04-1813:03curlyfryBut yeah, it's meant to give more feedback during development#2017-04-1813:47luxbockthere's https://github.com/jeaye/orchestra if you want to check :ret
and :fn
as well#2017-04-1816:06mobileinkhaving trouble composing a few simple specs. seems to work until I use the range spec listed in the snippet.#2017-04-1816:08mobileinkalso, if I compose :oic/core with something that does not include this range spec, it works ok.#2017-04-1816:10mobileinkbackground: converting json schemata like this to spec: https://github.com/OpenInterConnect/IoTDataModels/blob/master/oic.baseResource.json#2017-04-1816:24arohneris there a way to make check
stop on first failure?#2017-04-1816:27mobileinksame thing happens with (s/def ::step (s/or :i integer? :n number?))
#2017-04-1816:32jfntnIs it possible during tests to have an fdef check its args but skip checking with exercised inputs?#2017-04-1816:48mobileinkis this a bug? (s/def ::foo number?)
(s/def ::bar (s/or :i integer? :n number?))
(s/def ::baz (s/and (s/keys :opt [::foo]) (s/keys :opt [::bar])))
(s/explain ::baz {::foo 1 ::bar 9})
;; In: [:oic.r/bar] val: [:i 9] fails spec: :oic.r/bar at: [:oic.r/bar :i] predicate: integer?
;; In: [:oic.r/bar] val: [:i 9] fails spec: :oic.r/bar at: [:oic.r/bar :n] predicate: number?
#2017-04-1817:59seancorfieldNo. s/and
flows the conformed value through subsequent forms.#2017-04-1817:59seancorfield(s/conform (s/keys :opt [::foo]) {::foo 1 ::bar 9})
;=> #:boot.user{:foo 1, :bar [:i 9]}
ā s/keys
conforms all key/values if there are specs for them, not just the keys listed in :req
and :opt
.#2017-04-1818:00seancorfield^ @mobileink#2017-04-1818:02seancorfieldYou want s/merge
here to combine key-specs: boot.user=> (s/def ::baz2 (s/merge (s/keys :opt [::foo]) (s/keys :opt [::bar])))
:boot.user/baz2
boot.user=> (s/explain ::baz2 {::foo 1 ::bar 9})
Success!
nil
#2017-04-1818:06mobileinkdāoh! thanks, he mumbled sheepishly.#2017-04-1908:25mike_ananevHi! I would like to add some :doc to my spec? Is there any way to do that? I need some text comments for my spec#2017-04-1908:26mike_ananevfor data, not for fn's#2017-04-1911:35Alex Miller (Clojure team)Not currently but maybe eventually#2017-04-1914:11robert-stuttafordis there anything out there that talks about how to write custom generators for s/multi-spec?#2017-04-1914:12robert-stuttafordif we write a generator that returns something sufficient for the dispatch fn to use, will the generator code know to keep generating data for the specs returned by the defmethods?#2017-04-1914:13robert-stuttafordin our case, we dispatch in two different ways based on the value of :type, so weāre thinking weād gen a map with :type with a valid value from one of the two sets of :type values we keep#2017-04-1915:19Alex Miller (Clojure team)Well that's where retag comes in. However I was looking at something the other day that made me question whether multi-spec was working properly in gen for non-keyword retag cases#2017-04-1915:20Alex Miller (Clojure team)I guess I would turn your question around and ask whether it's doing what you expect#2017-04-1916:48mobileinkhi. I have a map with a type key (:oic.core/rt, for resource type), whose value is a vector of type keys. how can I tell spec to form the conjunction of those types so that valid maps satisfy them all? Something like (apply s/and [::foo ::bar]), which doesnāt work.#2017-04-1916:52ghadiNo need. spec will automatically check all namespaced keys for which there is a spec registered#2017-04-1916:53mobileinkright, but different types will have different specs; for example, :oic.r/temperature requires ::p/temperature#2017-04-1916:54mobileinkso i need map-level validation. a map might look like {::p/temperature 72
:oic.scale/temperature āF"
:oic.core/rt [:oic.r/temperature :oic.r/sensor oic.r/humidity]
:oic.core/if [:oic.if/baseline] }
#2017-04-1916:54mobileinkthis should fail, because :oic.r/humidity requires ::p/humidity#2017-04-1916:55mobileinkitās easy to use s/get-spec, but how can I then apply the spec?#2017-04-1916:59ghadiLook at s/merge, which merges map specs#2017-04-1917:01mobileinkyep, know about that, but i think thatās static - here we wonāt know what to merge until runtime. at least, iāve tried to use it but no luck so far.#2017-04-1917:20mobileinki think i need something like multi-spec with the ability to dispatch multiple times#2017-04-1917:30ghadiAre you changing the meaning of namespaced keys in different maps?#2017-04-1917:31ghadiOr are you having different sets of namespaced keys#2017-04-1917:45mobileinkthe idea is one schema for each oic.r āresourceā at https://github.com/OpenInterConnect/IoTDataModels, but a resource āinstanceā can have multiple types - listed in :oic.core/rt#2017-04-1917:48mobileinkI think i have a possible solution: I use multi-spec, but my multi-methods recur over the vector of keywords. so given :oic.core/rt [::foo ::bar]
, the dispatch method will dispatch on ::foo, and the defmethod for ::foo will recur with (rest (:oic.core/rt arg))
before calling s/keys. seems to work with a simple example at least. kinda cool imho.#2017-04-1917:48mobileinkamazing what you can do with clojure#2017-04-1917:49mobileinke.g. (defmethod oic-rt :oic.r/temperature [m]
(let [rts (:oic.core/rt m)
nextm (merge m {:oic.core/rt (vec (rest rts))})]
(s/merge
(if (rest rts)
(oic-rt nextm))
(s/keys :req [::p/temperature] :opt [:oic.scale/temperature]))))
#2017-04-1917:54mobileinkbasically a runtime s/merge#2017-04-1919:34robert-stuttafordthanks @alexmiller#2017-04-2022:25donaldballI am just now using multi-spec for the first time and am finding it a little weird that it doesn't conform into a variant tuple like or does#2017-04-2221:52pseudI know there's supposed to be some way by which I can refer to a spec, bar
residing in [my.example.foo :as foo]
by its alias, but (s/conform :foo/bar ...)
doesn't seem to be it. Anyone knows/remembers ?#2017-04-2221:57dergutemoritz@pseud You gotta use two colons like that: ::foo/bar
#2017-04-2310:40urbankis https://github.com/jeaye/orchestra usable with clojurescript? It requires clojure.spec, not cljs.spec, and uses StackTraceElement->vec, which I can't find in the clojurescript api#2017-04-2310:54lsenjovLooks to be straight .clj
, so probably not#2017-04-2310:55lsenjovAlthough it doesn't seem to be using any java interop, so a bit of dependency management and it should work#2017-04-2311:19urbankIt does use (.getStackTrace (Thread/currentThread) ... though I suppose that's not really a necessary function, and it would just be different in clojurescript#2017-04-2311:25lsenjovOh, I missed that one#2017-04-2311:26lsenjovHmmm, it also uses .applyTo
just below it, but that shouldn't be a problem#2017-04-2314:08pseud@dergutemoritz cool, thanks š#2017-04-2412:57hkjelsHow do I ensure that a spec never generates the same value twice?#2017-04-2413:17tbaldridge@hkjels you mean like values in a collection are unique? And this is for the generator for a spec?#2017-04-2413:17hkjelsYes#2017-04-2413:20hkjelsI can probably make a custom generator that checks for each item, but I was hoping that something exists already#2017-04-2413:28hkjelsActually, I think I have a good solution here#2017-04-2413:36tbaldridge@hkjels I think that exists#2017-04-2413:37tbaldridges/coll-of takes an optional parameter of :distinct#2017-04-2414:25hkjels @tbaldridge Thank you!#2017-04-2418:19mobileinkany general-purpose spec libs out there? my googling didn't find any. I'm specking routine stuff like href, uri/url, name, uuid, etc. also more complicated but widely used stuff like phone number, postal address, etc. easy enuff, but a well-maintained lib would be better.#2017-04-2418:36mobileinkmore generally: lots of schema languages and schemas out there, that can be expressed in spec. i'm working on the OCF/OIC schema; another possibility is FOAF (https://en.m.wikipedia.org/wiki/FOAF_(ontology)). it would be nice to settle on conventions, e.g. spec.foo.bar for a spec implementation of schema foo.bar.#2017-04-2500:34souenzzoThere is this specs:
(s/def ::a integer?)
(s/def ::b string?)
(s/def ::m (s/keys :opt [::b]))
When I try
(s/valid? ::m {::b "b" ::a "b"})
I get :clojure.spec/invalid
, explanation: val ["a"] fails
...
Is it a bug?#2017-04-2500:35souenzzoIMHO it should be valid
, once spec just looks for what is defined
, and on ::m
spec's, there is no ::a
#2017-04-2500:50ghadi> When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. Weāll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.
https://clojure.org/guides/spec#2017-04-2500:51ghadi@souenzzo this is actually by design#2017-04-2500:53ghadiSpecs registered under namespaced keys give the keys global meaning thus are checked whenever they are encountered#2017-04-2500:54ghadiThis is different than other systems. (I like it)#2017-04-2509:00mbjarlandhave a clojure.spec question, assume I have a number of specs defined for seq:s of chars, how would I define a string spec which uses them?
i.e given:
(s/def ::my-char-sequence
(s/cat :valid-chars (s/* (s/and char? #{\a \b}))))
=> my-char-sequence
(s/explain ::my-char-sequence [\a \b])
Success!
=> nil
(s/explain ::my-char-sequence [\a \c])
In: [1] val: \c fails spec: my-char-sequence at: [:valid-chars] predicate: #{\a \b}
=> nil
how would I write a string spec using the char seq spec:
(s/def ::my-string
(s/and string?
(??? <something using ::myc-char-sequence and perhaps (seq %) > ???)))
#2017-04-2510:09dergutemoritz@mbjarland Inserting (s/conformer seq)
after the string?
predicate in the ::my-string
spec should do the trick!#2017-04-2510:26mbjarland@dergutemoritz !! ahh...I knew there had to be away to make a transformation before handing it off to spec...that is great!! thanks#2017-04-2510:29dergutemoritz@mbjarland You're welcome! Note that when using that with s/conform
it'll also return a seq of chars. Maybe you want to add something like (s/conform (partial apply str))
at the end of the chain.#2017-04-2510:30mbjarlandseems conformer has a second unform argument function#2017-04-2510:30mbjarlandwould that fix it#2017-04-2510:31dergutemoritzYeah but that's for use with s/unform
for recovering the original value - slightly different approach with different pros and cons but works, too š#2017-04-2510:32dergutemoritzAnother option would be to use string regexen instead of spec regexen#2017-04-2510:33mbjarlandyeah, I was hoping not to have to do string regex#2017-04-2510:33mbjarlandI like the composability of specs#2017-04-2510:35mbjarlanderrr...I'm assuming "string regexen" refers to actual regular expression #" "
matching?#2017-04-2510:45Adam MunozHi everyone. How do you spec :args of any type? i.e. takes x that can be anything and returns for example boolean.#2017-04-2510:55Adam MunozOk, any?
š#2017-04-2510:55Adam Munozthanks anyway#2017-04-2510:58mbjarland@adammunoz, I'm a beginner with spec but perhaps something like:
(defn some-fn [x]
true)
(s/fdef some-fn
:args (fn [_] true)
:ret boolean? )
->
(doc some-fn)
-------------------------
string-layout.core/some-fn
([x])
Spec
args: (fn [_] true)
ret: boolean?
#2017-04-2510:58Adam Munoz@mbjarland Thanks. No I meant that predicate that I want is maybe any?
.#2017-04-2510:59Adam Munoz(any? 1) => true
#2017-04-2510:59mbjarland: ) yes looks like it#2017-04-2510:59Adam Munoz(any? "whatever") =>true
#2017-04-2510:59mbjarlandI figured any would be collection based, but you are right#2017-04-2510:59Adam Munozš#2017-04-2512:10dergutemoritz@mbjarland Yeah that's what I meant by string regexen š#2017-04-2512:11dergutemoritzNote that there are also various libraries which provide a structured representation of regexen similar to that of spec which can be composed just as well and compiled down to java.util.regex.Pattern
#2017-04-2512:20mbjarland@dergutemoritz have any clj ones to share?#2017-04-2512:33dergutemoritz@mbjarland I seem to remember having seen multiple ones actually, but I fail to find them again!#2017-04-2512:33mbjarland@dergutemoritz : ) no worries (edit: was on touch screen...they suck)#2017-04-2512:33dergutemoritzI have used Scheme Regular Expressions with great pleasure in the past, though. Perhaps I'm mixing it up.#2017-04-2512:34mbjarlandand as a side question, how do you organize specs for functions? Right after the function itself? in another namespace? somewhere else?#2017-04-2512:35mbjarlandkeeing them close feels good in a sense but also pollutes the code with a lot of "spec noise"...for good or bad#2017-04-2512:36dergutemoritzPersonally, I like to put them before the function definition (if you mean fdef
s) but I don't have enough confidence in it to recommend it as a good idea or anything š#2017-04-2512:36mbjarlandok#2017-04-2512:36dergutemoritzAlso, I'm not specing each and every function just for the sake of it#2017-04-2512:40mbjarlandI got another one (starting to feel like a spambot here), do you see any immediate reason why this:
(s/def ::align-specifier
(s/and char?
#{\L \C \R \l \c \r}))
(s/def ::bracket-expr
(s/cat :left-bracket #{\[}
:align ::align-specifier
:right-bracket #{\]}))
(s/def ::between-bracket-expr
(s/cat :between-char
(s/* (s/and char? (complement #{\[ \]})))))
(s/def ::layout-exprs
(s/cat :le
(s/* (s/alt :br ::bracket-expr :be ::between-bracket-expr))))
would result in:
(s/explain ::layout-exprs [\a \b])
StackOverflowError clojure.core/complement/fn--6675 (core.clj:1438)
#2017-04-2512:41mbjarlandimprovements to the specs also welcome, I can for example see that I could remove the (s/and char?
in the first one#2017-04-2512:42mbjarlandall of ::align-specifier
, ::bracket-expr
and ::between-bracket-expr
work in isolation#2017-04-2512:42mbjarlandas in I can run explain and valid? on them and they behave as I would expect#2017-04-2512:42mbjarlandah...complement...maybe I need to be explicit there instead...#2017-04-2512:43mbjarlandI did feel a bit woozy writing (complement #{\[ \}})
: )#2017-04-2512:51dergutemoritz@mbjarland Looks like you're running into an infinite left recursion because ::between-bracket-expr
may not consume any input#2017-04-2512:58mbjarland@dergutemoritz ok, I will have to marinate on that a bit. Thanks for the pointer#2017-04-2513:00dergutemoritz@mbjarland https://en.wikipedia.org/wiki/Left_recursion#Removing_left_recursion might help#2017-04-2513:02dergutemoritzOr perhaps not š#2017-04-2513:03mbjarlandfood for marination#2017-04-2513:05dergutemoritzBon appĆ©tit!#2017-04-2513:06mbjarlandthanks!#2017-04-2513:17mbjarlandand yes, changing (s/*
to (/s+
in :between-bracket-expr fixes the problem. It probably should have been + to begin with#2017-04-2513:55mbjarlandand so I'm lost in the woods again...and not for the first time with spec : )
If I write standalone specs I can run explain/valid?/exercise etc on them. What if I spec a function, do I have to actually instrument the function to test my spec or is there a way to get hold of the function spec and run the normal spec-exercising-functions on it?#2017-04-2513:56mbjarlandI guess I can do a s/def
on say the args spec and then exercise that separately...#2017-04-2514:19Alex Miller (Clojure team)yes, you can call s/get-spec
on the fully-qualified symbol to get the function spec, and the function spec supports keyword lookups for itās parts (`:args`, etc).#2017-04-2514:21mbjarland@alexmiller ok, nice and data centric as usual. thank you#2017-04-2514:22Alex Miller (Clojure team)(s/fdef user/foo :args (s/cat :a int?))
(s/valid? (:args (s/get-spec 'user/foo)) [100])
#2017-04-2514:22Alex Miller (Clojure team)or you can go the other way and define the args spec as a standalone spec and then assemble the fdef spec from it (Iāve done this in some cases)#2017-04-2514:23mbjarlandthat's what I ended up with, but still good to know it's not a black hole and you have access to the function spec#2017-04-2514:26mbjarlandI find my biggest hurdle with spec so far is not understanding the language but understanding its place in the process and best practice use. Is there for example still a place for using specs as preconditions in functions? I guess it would be nice to see an example of a larger production system using spec (or a similarily reality checked code base) and what kind of usage patterns they ended up with. Any references to presentations or repos with such code base much appreciated....#2017-04-2516:33sparkofreasonIs there any way to use coll-of
or every
(or some other existing capability in spec short of writing custom predicates and generators) to spec non-Clojure collections, like java.util.HashSet
? I feel pretty certain the answer would be "no", but just want to double-check before reinventing any wheels.#2017-04-2517:02dergutemoritz@dave.dixon Since most (all?) Java collections are seqable, you should be able to make it work by and
ing in a seq
conformer#2017-04-2517:10sparkofreason@dergutemoritz thanks. Looks like even every
works if you specify :kind
to be a custom predicate, but still doesn't generate. And just verified every-kv
doesn't seem to work with :kind
, and keys
doesn't give any hooks for custom collections, as far as I can tell. Which seems sensible - can't support everything possible in the JVM with a single API.#2017-04-2517:12dergutemoritz@dave.dixon Oh, right, generation isn't covered by my suggestion either. Note what the doc string says about a custom :kind
predicate, though:
> :kind - a pred/spec that the collection type must satisfy, e.g. vector? (default nil) Note that if :kind is specified and :into is not, this pred must generate in order for every to generate.#2017-04-2517:14dergutemoritzHm but I doubt that passing something like :into (HashSet.)
will just work either š#2017-04-2517:14sparkofreason@dergutemoritz not documented, but the collection must also support the Clojure interface for supporting into. #2017-04-2517:15dergutemoritzYeah, seems reasonable#2017-04-2517:15dergutemoritzI doubt that spec is of much use for mutable collection types#2017-04-2517:36sparkofreasonActually, I want to use it for bifurcan collections. #2017-04-2518:56Alex Miller (Clojure team)covering Java colls was not really a goal of the spec coll preds#2017-04-2518:58Alex Miller (Clojure team)I canāt say weāve really talked about it either way though. doesnāt seem like thereās any technical reason such a thing couldnāt exist. not sure if there are performance concerns in growing/shrinking non-persistent colls in generators.#2017-04-2612:39mbjarlanda followup question to one I posted yesterday, if I have a conformer
from string to seq of chars:
(s/def ::layout-string
(s/and string?
not-empty
(s/conformer seq)
is there any way within spec to have the conformed values turned back into strings?#2017-04-2612:41mbjarlandI take that back, turns out @dergutemoritz already answered this one#2017-04-2612:47gfredericks#2017-04-2615:33Alex Miller (Clojure team)important info re 1.9/spec - https://groups.google.com/d/msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ#2017-04-2616:00bronsa@alexmiller https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L1 wrong ns name here#2017-04-2616:01Alex Miller (Clojure team)thanks :)#2017-04-2615:37dpsuttonspec will be versioned then?#2017-04-2615:38Alex Miller (Clojure team)yes#2017-04-2615:51Alex Miller (Clojure team)versions will be ala core.async: 0.1.<git-change-derived>#2017-04-2615:53gfredericksthe specs for clojure.core
functions/macros will stay with clojure?#2017-04-2615:54bronsano, that's what core.specs.alpha
is for AFAICT#2017-04-2615:54dpsuttonit'll be interesting to see how this impacts CIDER's support for spec. Now the tooling will have to make sure to match the version of spec that the application uses#2017-04-2615:54gfredericks@bronsa that's a namespace in the spec artifact?#2017-04-2615:54bronsait's a separate contrib lib#2017-04-2615:54gfredericksoh so you need three artifacts if you want to use clojure with the core specs?#2017-04-2615:55bronsaI'm assuming clojure will depend on core.spec.alpha but dunno#2017-04-2615:55gfredericksI'm just trying to figure out if/how the specs can drift from the implementations#2017-04-2615:55gfredericksif the specs stay with clojure then that's not an issue#2017-04-2615:59gfredericks> Additionally, this is a first step towards increased support for leveraging dependencies within Clojure.
I don't know what that means#2017-04-2616:02Alex Miller (Clojure team)@bronsa yes, clojure will depend on the two new libs#2017-04-2616:03Alex Miller (Clojure team)but you can also specify a newer version and override the one its using#2017-04-2616:03Alex Miller (Clojure team)@gfredericks towards demonolithing clojure#2017-04-2616:03bronsaFWIW i have a similar dependency tree with t.a/t.a.jvm and I found that it's easier to just depend on the leaf dependency (t.a.jvm or in spec's case core.spec.alpha) and release concurrently when the root dependency needs a change#2017-04-2616:13mobileinkjust curious: why not put the āalphaā bit in the version string instead of the ns?#2017-04-2616:15tbaldridge(note: I'm speaking as a Clojure user here, not as someone who has authority on the subject)#2017-04-2616:15tbaldridgeIt's answered somewhat in Rich's talk on spec: https://www.youtube.com/watch?v=oyLBGkS5ICk#2017-04-2616:16tbaldridgeBut the gist is you could have alpha, beta, v1, v2, v* code all in the same artifact if you wanted. That just because spec moves to "beta" or "RC" doesn't mean the alpha API has to go away.#2017-04-2616:22Alex Miller (Clojure team)at some point there will be namespaces that are not alpha. the alpha stuff will continue to work.#2017-04-2616:23mobileinkah, so this is just until we get to v1?#2017-04-2616:24Alex Miller (Clojure team)well, same idea may also apply to any breaking changes post v1#2017-04-2616:24mobileinkok, thanks#2017-04-2616:24Alex Miller (Clojure team)whether thatās done at the fn level or ns level is a future decision point#2017-04-2616:36mikerodThe only problem I see with making libraries like spec (not part of lang) are ājar hellā dep issues#2017-04-2616:36mikerodin JVM land libraries using deps always leads to trouble because you end up using 2 libs with 2 different version of the same dep and they are incompatible#2017-04-2616:36mikerod(eventually)#2017-04-2616:37mikerodHowever, I know Richās talk at the last Clojure/conj was about never making a breaking change essentially. So maybe clojure.spec will be robust enough to avoid the problem and you can always just ātake the newest versionā of the dep when multiple lib deps diagree on version.#2017-04-2616:38mikerodI just know that I ran into issue with say, Plumatic schema when several libraries were using it and I was using those libraries and then schema came out with a set of breaking api changes#2017-04-2616:39mikerodThis libraries having dependencies conflict problem has been one of the greatest pains of mine on the JVM it seems. I donāt think Iām alone there though. It can get difficult to deal with. I think it often encourages libraries to not use any dependencies. Which I feel is unfortunate for code-reuse or using useful utilities for documentation like spec/schema.#2017-04-2616:40mikerodOr in Java-land sometimes people do build-time repackaging of classes to some āinternalā package name. I think it is harder in clj to confidently add a prefix to a set of namespaces though in a way that doesnāt miss anything.#2017-04-2616:57seancorfieldMy experience ā six years of production Clojure with a code base of now 50K lines ā is that API version conflicts are actually pretty rare in Clojure libraries in the wild.#2017-04-2616:57seancorfieldThey are more common with Java libraries, true, but Clojure people seem to be better about API changes.#2017-04-2616:57seancorfield(says the man whoās guided clojure.java.jdbc
through several breaking API versions, however)#2017-04-2617:00bronsaseancorfield: c.j.j is different than most libs in that it's used much more by application code than by other libs so it's easier to avoid getting into dependency issues IMO #2017-04-2617:03seancorfieldYes, and thatās in line with why I think the utility libraries tend to be the worst offenders here. Itās why I have serious concerns about relying on Timbre, Carmine, Faraday, etc ā even thoā Peter seems really careful about not breaking the API across versions.#2017-04-2616:59seancorfieldI think the danger comes mainly from Clojure libraries that themselves have a large number of transient dependencies where they rely on Swiss Army Knife utilities libraries for just one or two ācool functionsā ā which is an indictment of the questionable validity of such libraries, IMO. They should be broken up into much smaller libraries so you donāt have to pull in the whole thing for just one or two useful functions.#2017-04-2617:00seancorfieldBut this is a case of ādo one thing and do it wellā which such libraries inherently donāt follow š#2017-04-2617:13tbaldridgeRight, part of the issues with "jar hell" are really problems with non-additive changes to APIs.#2017-04-2617:19mikerodYeah, thatās sort of what I was thinking was the answer to this concern here. Itād be great to see that workout.#2017-04-2617:19mikerodItās the āsecret weaponā š#2017-04-2617:46Alex Miller (Clojure team)the whole idea is to make all versions of a library compatible#2017-04-2617:47Alex Miller (Clojure team)but the important thing here is that alpha is a time before this applies and breaking stuff is ok (you as a user have to track the changes)#2017-04-2618:34seancorfield@alexmiller I asked on the list but Iāll ask here too: will the new org.clojure/spec.alpha
artifact be available for a few days before Clojure master SNAPSHOT actually removes the namespaces?#2017-04-2618:35Alex Miller (Clojure team)itās available now#2017-04-2618:35seancorfieldOh, cool!!#2017-04-2618:35Alex Miller (Clojure team)org.clojure/spec.alpha 0.1.94#2017-04-2618:40seancorfield@alexmiller This hasnāt hit Maven Central yet, right?#2017-04-2618:40Alex Miller (Clojure team)it has#2017-04-2618:40Alex Miller (Clojure team)I just downloaded it from there#2017-04-2618:41Alex Miller (Clojure team)https://repo1.maven.org/maven2/org/clojure/spec.alpha/#2017-04-2618:57seancorfieldAh, guess it just hasnāt been indexed yet? It doesnāt show up on https://search.maven.org yet. OK, got it, thanks!#2017-04-2618:35Alex Miller (Clojure team)as of just a few moments ago#2017-04-2618:35seancorfieldThank you! Much appreciated!!#2017-04-2618:35Alex Miller (Clojure team)but we are moving (slowly) towards an updated alpha release of clojure that uses it today#2017-04-2618:36seancorfieldIāll get our codebase switched over today thenā¦#2017-04-2620:01dominicm@alexmiller I'm a little curious about this shift. If you depend on "1.9.0", is it part of clojure's api to expose it's transitive dependency on spec? Or should I always explicitly depend on spec if I'm using it in my lib?#2017-04-2620:03seancorfieldI would expect core 1.9 will conditionally use spec at this pointā¦ So if you have spec loaded, youāll get better macro error messages?#2017-04-2620:03Alex Miller (Clojure team)@dominicm clojure will have a compile-time dependency on spec.alpha so you can always rely on it being available. So you donāt need to explicitly list the new library as a dependency. You may do so though in order to specify a newer version than Clojure is depending on.#2017-04-2620:05Alex Miller (Clojure team)@seancorfield it is not currently conditional in any way#2017-04-2620:05pdlugIs there a mechanism in spec for namespacing keys as part of a conform? For example, a map parsed JSON will not have namespaced keys so I'm applying some functions that namespace them for me but this feels like a pretty common use case which must have a better solution#2017-04-2620:05Alex Miller (Clojure team)s/keys
has :req-un
and :opt-un
options for this#2017-04-2620:06seancorfieldSo Clojure 1.9 will transitively bring in the spec library automatically? But you can bring in a different version yourself? How will that work if the spec library breaks the API?#2017-04-2620:06Alex Miller (Clojure team)@pdlug see the guide for an example https://clojure.org/guides/spec#2017-04-2620:06Alex Miller (Clojure team)@seancorfield yes. yes. then youāre broken (see āalpha = things may changeā).#2017-04-2620:07Alex Miller (Clojure team)sorry, have to jump in the car but can pick up later#2017-04-2620:08pdlug@alexmiller Any pointers as to where? I've been through that a few times, I see the examples on spec'ing unqualified keywords but not coercing to namespaced keywords as part of a conform#2017-04-2620:09seancorfieldAh, so for clojure.spec
to break Clojure itself, youād already have to have a new public release of Clojure available (even if it is 1.10 alpha perhaps at this point)?#2017-04-2620:09seancorfieldIām hoping you wouldnāt make a new release of clojure.spec
that then required users of it to rely on SNAPSHOT builds of Clojure ā youād at least have a new alpha of Clojure available by that point?#2017-04-2620:10seancorfield(as an active user of clojure.spec
in production, this is all a bit frustrating and a little worrying!)#2017-04-2620:57seancorfield@pdlug I think maybe Alex misread what you were trying to do? There's no automated way to produce qualified keys from unqualified ones. #2017-04-2620:58pdlug@seancorfield Thanks, that's exactly what I was asking, that's unfortunate because this seems like a really common use case (JSON APIs, JSONB in PostgreSQL, etc.)#2017-04-2621:00seancorfieldSince any namespace qualifier you would be adding is arbitrary, I don't see how spec plays into this: you control that arbitrary key qualification. #2017-04-2621:02seancorfieldSomewhere you're converting string data to a Clojure representation of JSON. Why not introduce the qualifier at that point -- then spec with qualified keys directly?#2017-04-2621:03seancorfieldThat's why clojure.java.jdbc
provides a :qualifier
option, for example. #2017-04-2621:03mobileinki do it by typing (on the keyboard).#2017-04-2621:04seancorfieldPardon @mobileink ?#2017-04-2621:04mobileinkfor a json schema i'm working with i can come up with namespaces for everything, but automating it seems like a lot more work than just typing it in.#2017-04-2621:09Alex Miller (Clojure team)@seancorfield if you were asking about spec breaking Clojure, I don't see how we would do that. The integration points are pretty small. #2017-04-2621:10seancorfield@alexmiller I was considering a situation where you made a breaking API change in clojure.spec.alpha
that could affect Clojure itself (since it depends on the earlier, non-breaking API)#2017-04-2621:11mobileinke.g.json schema puts "di" in schema "foo.bar". in spec: :foo.bar/di.#2017-04-2621:13pdlugI think this would be easier if there were more functions for manipulating maps to add namespaces to keywords. I keep copying the same keywordize-keys-ns
function in every project to coerce incoming JSON into keywords w/ namespaces. Would be nice to also have a rename-keys
that could do unqualified to qualified mappings en masse.#2017-04-2621:13seancorfieldBut if the integration surface is small, I guess that breakage is unlikely ā and what I meant was that youād already have to have a version of Clojure somewhere that used the new (changed) API in order to test/use it anyway. I just wanted to check that if you ever made such a breaking change to clojure.spec.*
, youād ensure there was a *non-SNAPSHOT* version of Clojure made available at the āsame timeā.#2017-04-2621:14bronsa@seancorfield the only thing clojure depends on from spec atm is s macroexpand-check
, I find it really hard to imagine a situation where that could be broken#2017-04-2621:15Alex Miller (Clojure team)I would consider that broken :)#2017-04-2621:16seancorfieldWell, the World Singlesā main codebase now uses clojure.spec.alpha
with an explicit version dependency so Iām āhappyā š#2017-04-2621:18thheller@alexmiller any particular reason why the core.specs.alpha
is a separate package?#2017-04-2621:18Alex Miller (Clojure team)Vs?#2017-04-2621:18thhellerkeeping it in clojure.spec.alpha or clojure.core I guess#2017-04-2621:19thhellerjust wondering if there are extended plans for it#2017-04-2621:20Alex Miller (Clojure team)You mean why in a different ns or why in a different project/artifact?#2017-04-2621:20thhellerdifferent artifact#2017-04-2621:21Alex Miller (Clojure team)It's logically separate and can evolve at its own rate#2017-04-2621:22thhellerok I'm still confused how the whole fdef
/ macroexpand-check
integration is going to work, guess I'll have to wait and see#2017-04-2621:22Alex Miller (Clojure team)We made the decision long enough ago that I've forgotten the details tbh#2017-04-2621:22Alex Miller (Clojure team)@thheller it's exactly the same?#2017-04-2621:23Alex Miller (Clojure team)The code is just in a different artifact#2017-04-2621:25thhellerok, I keep thinking there is a circular dependency somewhere#2017-04-2621:26Alex Miller (Clojure team)There is#2017-04-2621:26Alex Miller (Clojure team)spec.alpha depends on Clojure#2017-04-2621:26Alex Miller (Clojure team)Clojure depends on spec.alpha#2017-04-2621:27Alex Miller (Clojure team)There is a bootstrap aspect to it#2017-04-2621:29thhellerah right, forgot the current code already does a var lookup.#2017-04-2621:30Alex Miller (Clojure team)I find it's best not to think about it too hard :)#2017-04-2621:30thhellerhehe yeah I'm thinking more about CLJS at the moment#2017-04-2621:30thhellerI was using the core.specs for CLJS but I guess that is just as easy with them in a lib#2017-04-2621:30Alex Miller (Clojure team)It's easier#2017-04-2621:37Alex Miller (Clojure team)The specs are not the same though for things like ns#2017-04-2621:37thhellerwell, everything besides ns
is though#2017-04-2621:38thhellerbut still trying to come up with a good way to intergrate them with CLJS properly. might need to copy them anyways for self-hosted.#2017-04-2621:59Alex Miller (Clojure team)Open to helpful ideas#2017-04-2622:01thhellerstill trying to solve the "helpful errors" issue, currently the spec explain for something wrong in a let
doesn't look much more helpful than the actual compiler error#2017-04-2622:16Alex Miller (Clojure team)Well more to come on that#2017-04-2622:17seancorfield@thheller I think the benefit will come from familiarity ā all spec-powered errors will have a similar output format and folks will quickly get used to them (both from let
and other core constructs, as well as from general libraries that use spec). And any improvements to how explain
works will improve all errors from code that uses spec.#2017-04-2622:18seancorfieldI mean, yeah, itās a lot of output, but itās very regular and structured.#2017-04-2622:24thhellernot too sure about that#2017-04-2622:24thhellerCLJS error in demo/errors.cljs at 3:1
In: [1] val: 1 fails spec: :clojure.core.specs/arg-list at: [:args :bs :arity-1 :args] predicate: vector?
In: [1] val: 1 fails spec: :clojure.core.specs/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))
:clojure.spec/args (x 1)
#2017-04-2622:25thhellerthe most helpful part is demo/errors.cljs at 3:1
š#2017-04-2622:27thhellerthe problem in this case is that 1
is a number which doesn't have reader metadata, so you cannot narrow the cause down further#2017-04-2622:28thheller(defn x 1)
is the actual source so 3:1
is the start of the (defn
not the 1
which would be more accurate#2017-04-2622:28hiredmanI don't follow#2017-04-2622:29hiredmanthat seems pretty clear#2017-04-2622:29hiredman1 isn't a vector, 1 isn't that other big long thing#2017-04-2622:30hiredmaneach case gives the value that failed, and the predicate it failed#2017-04-2622:30hiredmanand which spec the predicate came from#2017-04-2622:30thheller@hiredman the predicate can get cryptic as seen in the second message#2017-04-2622:31hiredmanthe grammar is cryptic š#2017-04-2622:31thhellerI think it is alright as well, a beginner might not be so inclined#2017-04-2622:32thhellerbut my issue for not is more about more accurate source locations#2017-04-2622:32thhellerfor the most part they are accurate enough though#2017-04-2622:38Alex Miller (Clojure team)The lack of reader metadata is annoying but not an endpoint necessarily#2017-04-2622:42thhelleryeah coupled with the spec errors you can usually figure out whats wrong pretty quickly#2017-04-2622:44thhellerit kinda breaks apart if the printed val
gets too large though#2017-04-2622:48seancorfieldI read that first error as āarg-list is not vector?ā which is pretty clear. Even the second one is pretty straightforward: āargs+bodyā is not a sequence of āargs followed by bodyā ā and if you (source defn)
that should triangulate to the thing after the symbol being defn
d is not either of those.#2017-04-2622:53thheller... please don't get hung up on my toy example. This particular example is "good enough" with spec.#2017-04-2622:53thhellerParameter declaration "1" should be a vector
is the default error message without spec ...#2017-04-2622:54thhellerwhich IMHO is better than the spec error (but much less accurate)#2017-04-2623:10thheller@alexmiller will the new spec projects/artifacts get their own JIRA projects or should issues still go to CLJ?#2017-04-2623:33Alex Miller (Clojure team)CLJ #2017-04-2702:52danielcomptonI'm not even sure exactly what this would mean, but is it possible to 'render' a spec for display in for example a Swagger UI? I'm wanting to build a web UI for displaying the specs of commands that could be sent to the system, but I'm not sure if this is even possible, as Spec is so flexible#2017-04-2705:07ikitommi@danielcompton we have been working on the api-docs thing, just finished on the spec -> json schema and will do the -> openapi next. also have some tooling for the runtime conforming needed to support different wire-formats. But like Schema (and ring-swagger), spec is more powerful than the openapi spec, so some information will be lost. Might be a place to do full-clojure thing, with own āspec-uiā.#2017-04-2705:09ikitommiany schedule for cljs-port of the spec-alpha
?#2017-04-2705:09danielcompton@ikitommi yeah, I don't specifically want Swagger, it was just the closest thing I could think of, a spec-ui would be what I was after#2017-04-2705:16ikitommiI think the spec-ui could be done easily, as the spec forms can be serialized. Also, something like the https://github.com/metosin/schema-viz but for spec. Both on todo-list, but with million other things.#2017-04-2706:21thheller@alexmiller I would like to add the var to the ex-data in clojure.core/macroexpand-check
. so v
in https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L686-L698#2017-04-2706:22thhellerso either the v
itself or the (->sym v)
that is currently used in the message for the ex-data
call#2017-04-2706:23thhellerI'm experimenting with some different error presentations and currently the var isn't accessible so can't show the root spec that failed (other than "parsing" the (.getMessage ex)
)#2017-04-2706:34thhellercould either do this in the macroexpand-check
or in the catch
that wraps the error in Compiler.java
#2017-04-2708:29jindrichmHi -- How do you make a spec for a particular value? (partial = :value)
predicate doesn't seem very idiomatic.#2017-04-2709:09dergutemoritz@jindrichm #{:value}
#2017-04-2709:09dergutemoritzUnless your value is nil
or false
š#2017-04-2709:09dergutemoritzBut then you have specific preds#2017-04-2709:11jindrichmThanks! This should work fine and looks more idiomatic than partial
.#2017-04-2709:12vandr0iyHi clojurians!
How does one make up a spec for a huge nested map that may have different values for the same key but in different places?
like this:
{:a [{:foo "foo1" :bar 1}, {:foo "foo2" :bar 8}]
:b {:baz {:foo 3 :bar "quuz"}
:quux {:foo [1 2 3] :baz 10}}
How do I say that :foo
is usually a string, but inside a certain context (b/baz in this case) it's a number, and in certain others (b/quux) it's a vector?#2017-04-2709:22dergutemoritz@vandr0iy If you're asking how you can make s/keys
behave like that, the short answer is: it's not what it was designed for but rather it relies on namespaced keywords. However, you could define keywords of the same name in different namespaces representing your various contexts and piece the specs together accordingly with :req-un
and :opt-un
.#2017-04-2710:31mbjarland@vandr0iy spec-tools data specs might be worth a look? (https://github.com/metosin/spec-tools#data-specs)#2017-04-2710:34mbjarland@vandr0iy errr....ok, they might help, but don't think they solve your different values for the same key problem, @dergutemoritz answer seems to be it#2017-04-2711:33ikitommi@vandr0iy @U4VDXB2TU: data-spec can do that, but I would use the normal s/keys
if you need to reuse the attribute specs elsewhere. If itās just a ~one-time thing, then something like this:
(require '[clojure.spec :as s])
(require '[spec-tools.core :as st])
(def spec
(st/data-spec
::foo
{:a [{:foo string? :bar int?}]
:b {:baz {:foo int? :bar string?}
:quux {:foo [int?] :baz int?}}}))
(def data
{:a [{:foo "foo1" :bar 1}, {:foo "foo2" :bar 8}]
:b {:baz {:foo 3 :bar "quuz"}
:quux {:foo [1 2 3] :baz 10}}})
(s/valid? spec data)
; true
#2017-04-2711:48vandr0iyin the end I just ended up doing specs for every single element that might be recognized as a pattern in its own namespace with the same name as it's called in the data structure I want to specify - but in its own namespace. For instance:
(s/def :type1/stuff string?)
(s/def :type1/foo (s/keys :req-un [:type1/stuff]))
(s/def :type2/stuff number?)
(s/def :type2/element (s/keys :req-un [:type2/stuff]))
(s/def :type2/foo (s/coll-of :type2/element))
this way stuff
is a string in a type1
element and a number in type2
one. This data would match:
(s/valid? :type1/foo {:stuff "foo"})
=> true
(s/valid? :type1/foo {:stuff 2})
=> false
(s/valid? :type2/foo [{:stuff 2} {:stuff 4}])
=> true
(s/valid? :type2/foo [{:stuff 2} {:stuff "arst"}])
=> false
#2017-04-2711:47Alex Miller (Clojure team)@ikitommi re cljs, will happen soon, after clj is done#2017-04-2711:51Alex Miller (Clojure team)@thheller does https://dev.clojure.org/jira/browse/CLJ-2085 help at all?#2017-04-2711:54thheller@alexmiller not sure, will try it. thx#2017-04-2711:55ikitommithanks @alexmiller, looking forward to it.#2017-04-2711:59thheller@alexmiller yep should do, I can get the name of the spec via the meta
:clojure.spec/name
#2017-04-2711:59thhellerwhich corresponds to the var#2017-04-2712:16Alex Miller (Clojure team)I guess I'm not sure why you want the var vs the name#2017-04-2713:33thhellerfor now just the name, maybe the var later. thinking that the var might contain more metadata, will see if I need that#2017-04-2713:34thhellerprobably not though#2017-04-2812:05christianromneyhmm bear with me as i try to understand this. if we have a couple of invariants like: 1. we don't break APIs in non-alpha software and 2. if we do break an API, we pick a new name and 3. clojure.core (non-alpha) will depend on clojure.spec.alpha, then doesn't this imply that for a version of clojure to depend on a clojure.spec (and NOT depend on clojure.spec.alpha), it will have to change its namespace, say to something like clojure2.core? or have i missed something obvious?#2017-04-2812:29Alex Miller (Clojure team)clojure.core doesn't depend on clojure.spec.alpha#2017-04-2813:03christianromneythanks @alexmiller although I'm more confused now than before. i misinterpreted your comment above about the circular dependency. perhaps i should just go look at the artifacts š#2017-04-2813:18christianromneyhmm, @alexmiller could you elaborate a bit more? I'm just not understanding how to reconcile that with your previous explanation that "clojure will have a compile-time dependency on spec.alpha so you can always rely on it being available. So you donāt need to explicitly list the new library as a dependency." sorry if i'm just being a dunderhead...#2017-04-2813:29Alex Miller (Clojure team)my original statements are about artifacts. yours seemed to be about namespaces, so I answered in those terms. can you clarify which level youāre talking about?#2017-04-2813:30Alex Miller (Clojure team)Clojure 1.9 will ship with an alpha version of spec available - use it if you like (but beware that things in spec may change)#2017-04-2813:30Alex Miller (Clojure team)thatās the whole story#2017-04-2813:35mhuebertIs there a natural way to have the generator for a regular expression spec return a vector instead of a sequence? #2017-04-2813:36Alex Miller (Clojure team)currently, no#2017-04-2813:37Alex Miller (Clojure team)but it is something many people (including myself) have felt the need for and something I have discussed a couple times with Rich#2017-04-2813:37Alex Miller (Clojure team)and I think ultimately something will plug that gap#2017-04-2813:46mhuebertOk, thanks. (The two use cases I have in mind are hiccup forms and function argslists)#2017-04-2813:47Alex Miller (Clojure team)yes, Iāve run into it in specāing the latter#2017-04-2814:04christianromney@alexmiller oh i see so you'll declare a dep on the artifact but never require it is that right?#2017-04-2814:05Alex Miller (Clojure team)mostly - there are a couple things in Clojure that use spec#2017-04-2814:05Alex Miller (Clojure team)one is the args check for fdef macro specs during macroexpansion (so in the compiler)#2017-04-2814:06Alex Miller (Clojure team)the other is the lookup and inclusion of specs in printed docs from clojure.repl/doc#2017-04-2814:06Alex Miller (Clojure team)both of those are kind of implicit rather than being part of the API though#2017-04-2814:08christianromneyok thanks, that gives me much more to chew on. š i don't see this necessarily impacting me, i just really want to grok the engineering you're doing š#2017-04-2814:27mhuebertSo in the meantime, something like this should be ok (to return vectors from a regular expression): https://gist.github.com/mhuebert/8fdeedae57bf797778054dcf8f33ab8b#2017-04-2814:34Alex Miller (Clojure team)gen/fmap with vec would be simpler, no need for bind here#2017-04-2814:35Alex Miller (Clojure team)#(gen/fmap vec (s/gen expr))
#2017-04-2906:47fossifoo@ikitommi not sure, where to put this on slack, but i currently try to use spec-swagger from cljs/node and it seems like spec-tools depends on a lot of internals from clojure.spec and doesn't currently build on cljs. i get No such var: s/def-imp
. can you (or anybody else here) confirm that?#2017-04-2906:51fossifooeh, there's a cljs.spec.cljc and a cljs.spec.cljs? how does that work?#2017-04-2911:40Alex Miller (Clojure team)Platform-specific is used first, then falls back to cljc if not found#2017-04-3005:50lincpaMake readability better spec will expressed in the form of SQL DDL?#2017-04-3011:14fossifoohmmm. since ther e is a def-impl in cljs.spec, i guess this should work then#2017-04-3011:15fossifooand actually, i had to rename the imports to cljs.spec from clojure.spec although i think the compiler should find it with either name#2017-04-3011:15fossifooso maybe something else is worng#2017-05-0103:00Oliver GeorgeJust putting it out there: I love clojure.spec#2017-05-0115:47souenzzoolivergeorge: clojure.spec.alpha š#2017-05-0210:27Oliver GeorgeFair call. I had just finished watching Rich's spec-ulation talk before it happened. Makes total sense in that light.#2017-05-0109:37thhellerhttps://gist.github.com/thheller/738698dfff45280f4e004df1c46af4ba#2017-05-0109:38thhellerI think we should maybe add a #(even? (count bindings))
predicate to the clojure.core/let
and other specs#2017-05-0109:39thhellerI certainly couldn't make sense of that error for a while š#2017-05-0113:08gfrederickstest.check and clojure.spec have made it onto neural network garbage twitter https://mobile.twitter.com/BobEbooks/status/859025622689087488#2017-05-0117:55ikitommi@fossifoo spec-tools tests are run on travis with node too, so it should work, tested with cljs-version 1.9.518
. But to enable specs to be created a runtime, data-specs uses the functional internal of clojure.spec (the ^:skip-wiki
fns :() => just extracted āem into separate namespace - donāt have to use those. Spec-swagger needs a lot of attention, next on the todo-list, will start by converting to vanilla specs.#2017-05-0117:57ikitommiIf I remember right, there will be public *functions* to create specs some time in the future.#2017-05-0118:03ikitommiis there a way to unform a conformed value of s/map-of
?#2017-05-0118:05ikitommi(s/def ::a (s/or :int int?))
(s/conform ::a 1)
; [:int 1]
(s/unform ::a (s/conform ::a 1))
; 1
(s/def ::map-of (s/map-of ::a ::a :conform-keys true))
(s/conform ::map-of {1 1})
; {[:int 1] [:int 1]}
(s/unform ::map-of (s/conform ::map-of {1 1}))
; {[:int 1] [:int 1]}
#2017-05-0118:13Alex Miller (Clojure team)known bug https://dev.clojure.org/jira/browse/CLJ-2076#2017-05-0118:29ikitommithanks. is it still a plan to create a public layer of functions to create the specs (besides the macros)?#2017-05-0118:30ikitommialso, would be interested in having a functional way to register specs. the def-impl
looks private as it has the :skip-wiki
meta on it.#2017-05-0118:30ikitommi(and āDo not call this directlyāā¦)#2017-05-0118:43fossifoo@ikitommi okay, just wanted to check the status. i want to go the other way around (use swagger json to generate testdata) and think i might go via spec. a generic spec will also be very helpful for parsing. changing it to standard spec would indeed be better for me. it's hard enough to understand one DSL#2017-05-0119:02Alex Miller (Clojure team)@ikitommi still tbd. As I've said in the past, you can get same benefits of functional API by unforming with spec specs#2017-05-0122:55danielcompton@thheller what was the form that gave that error? Here's what I got, I must be misunderstanding what you meant:
(let [a 1
b 2
c 3
d]
a)
clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs/bindings at: [:args :bindings :init-expr] predicate: any?, Insufficient input
:clojure.spec/args ([a 1 b 2 c 3 d] a)
#2017-05-0123:05mobileinkd needs a value.#2017-05-0123:17danielcomptonyep, I know, I'm trying to reproduce the error @thheller got#2017-05-0200:35jatkinsHi all! Is anyone else getting errors like Call to clojure.core/ns did not conform to spec
? I'm working on a clj/s project and I want to use spec with reframe. I've tried the latest version of clojure (`1.9.0-alpha16`) and I get the aforementioned error. However, when I move the dependency to 1.9.0-alpha10
the error does not occur. Any idea for the change? BTW This is my first attempt at using a clojure alpha version, and I have many dependencies that may be incompatible (I feel that this is unlikely in this case, though correct me if I'm wrong). If needed I can provide more information.#2017-05-0200:41Alex Miller (Clojure team)The namespaces for spec just changed in alpha16 and cljs and other deps are not compatible with it#2017-05-0200:42Alex Miller (Clojure team)You might want to fall back to alpha15#2017-05-0200:43jatkinsI actually tried every alpha from 16 to 10, and 10 was the first one that worked.#2017-05-0200:47Alex Miller (Clojure team)What's the actual error you're getting?#2017-05-0200:48Alex Miller (Clojure team)Many libs had bad ns deps - most of those have been fixed and newer versions exist#2017-05-0200:52jatkinsI put a gist up: https://gist.github.com/JJ-Atkinson/a1d19e48d9440c2ca9f0bed219c8201f. How can I tell where the error is originating from? I see that it likely is from (onelog/core.clj:1:1)
, but I don't know where that is, (probably in timbre though, I'll see if I can bump that version up).#2017-05-0201:05jatkinsOnelog is part of ring apparently part of ring-middleware-logger
. I'm currently using the latest of both. So, is the issue not with clojure itself, but with onelog
? If that's the case, I'll see about a pull request/issue to them.#2017-05-0201:27jatkinsSo, after crawling through the error message more thoroughly, it appears that onelog
is using the syntax (ns ... (:import (com.something SomeClass)))
. Spec changed something about ns with -alpha11
, and it was already reported to them https://github.com/pjlegato/onelog/issues/3. Thanks for the help!#2017-05-0202:40Drew VerleeApologies as this has probably been asked a dozen times, but does clojurescript current support generators?#2017-05-0203:38tbaldridge@drewverlee not that I know of, but generators tend to encourage a impure code. Most of what you would need generators for can be done with more functionally pure constructs like map
, filter
and for
.#2017-05-0206:34curlyfry@alexmiller So if I want to use alpha16 and have cljs in my project I should wait for a compatibility patch or something similar?#2017-05-0206:53dergutemoritz@tbaldridge I think @drewverlee is referring to the test.check kind of generators, given that he's asking in here š @drewverlee test.check supports cljs at least, not sure if clojure.spec's wrapping of it works with cljs, though.#2017-05-0206:58thheller@danielcompton this was the source https://gist.github.com/thheller/738698dfff45280f4e004df1c46af4ba#file-problem-clj#2017-05-0211:07lwhortonWith the announcement of spec moving to its own lib, I see that spec.alpha is under org.clojure, but nothing like org.clojure/clojurescript.spec.alpha? Has it not moved yet?#2017-05-0212:08Alex Miller (Clojure team)Not yet#2017-05-0212:43lwhortonalrighty, thanks#2017-05-0221:55weiwhatās a good example of adding validation on a macro?#2017-05-0222:07mobileinkwhat do you want to validate?#2017-05-0222:22weiactually, might need to rethink this#2017-05-0222:25weiI have a wrap-error macro that takes a map of context data and a body. (defmacro wrap-error [args & body]
(let [e (gensym 'e)]
`(try
I want to make sure that the macro user doesnāt forget to pass in the context map, because if so, the first line of the body wouldnāt get executed. but Iām not sure itās possible to do that at compile time#2017-05-0222:39mobileinkwhat context map? #2017-05-0222:39mobileinkyou mean you want to validate args?#2017-05-0222:41mobileinkno way to do this without a macro?#2017-05-0222:45mobileinkyou want to use spec to validate args? add a clause to your let? #2017-05-0223:07weisorry let me try a better explanation. I want people to use this macro like this: (wrap-error {:some :data :a 1 :b 2} ...body...)
where the first argument is some metadata to associate with the error, if there is one. the macro has been used mistakenly like this: (wrap-error ...body...)
without the first arg, resulting in incorrect behavior because the first element of body is interpreted as the metadata and not executed. was wondering if there was a way to use spec to prevent this at read-time. less about validating the metadata itself so much as the presence of it.#2017-05-0223:09weimight be hard to distinguish at read-time though. i wonder if this has to be a run-time check (`(s/valid? map? args)`)#2017-05-0300:01mobileinkwhy do you need a macro? you're just wrapping try/catch, no?
#2017-05-0300:02mobileinki'm not an expert, but the usual advice is to avoid macros.#2017-05-0300:05mobileink"wrap-error" strikes me as misleading. you're really just wrapping a fn, which may not have an error, no?#2017-05-0301:57cflemingThe doc seems to imply that s/alt
is ordered choice, i.e. the alternatives will be tested in order and the first match returned. Is that correct?#2017-05-0302:00tbaldridge@cfleming yes from the doc string: "Returns a regex op that returns a map entry containing the key of the
first matching pred and the corresponding value. "#2017-05-0302:00cflemingGreat, thanks.#2017-05-0308:27mishagreetings!
I am getting
(s/exercise :dsc/patch 1)
=> ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
is there a way to narrow error location down? (:dsc/patch is a compound spec, ~30 lines long)#2017-05-0309:24slipsetI really wished spec was spec'ed. Just lost som time on why s/keys was valid on my invalid maps. Reason? I used :un-req instead of :req-un#2017-05-0311:04mishais there something built in for what I'am trying to achieve (generator for next-id, which never returns a duplicate)?#2017-05-0311:55mishahow does one combine two map specs? (union)#2017-05-0311:58mishahere I'd like to avoid custom reduce-through-hashmap predicate, and combine s/keys
and s/map-of
specs instead#2017-05-0312:35mishathat one fails because s/and
results in empty set of valid keys. s/merge
will result in an empty set too.
it seems like I need a custom predicate after all:
(s/def :datascript/map-with-current-tx (s/keys :req [:db/current-tx]))
(s/def :datascript/map-with-tempids-lookup (s/map-of :datascript/temp-id :datascript/actual-id))
(def gen-tempids
(tgen/let [ids (s/gen :datascript/map-with-tempids-lookup)
ctx (s/gen :datascript/map-with-current-tx)]
(merge ids ctx)))
(s/def :datascript/tempids
(s/with-gen
(fn [m]
(and
(s/valid? :datascript/map-with-current-tx m)
(s/valid? :datascript/map-with-tempids-lookup (dissoc m :db/current-tx))))
(constantly gen-tempids)))
#2017-05-0312:04souenzzo(s/merge ::a ::b (s/keys :req[::c]) (s/map-of ,,,))
#2017-05-0312:06misha@souenzzo tried that right now. it also gives me
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
#2017-05-0312:07souenzzoI've never used generators. =/
But I use tons of s/merge
#2017-05-0312:14lwhortonhas anyone been able to use generators with cljs? Iām not really sure how (I donāt even see gen
defined in the cljs.spec.impl.gen
ns, and my google foo is failing#2017-05-0312:26bhagany@lwhorton yes, but I don't remember off the top of my head what the differences with clj were...#2017-05-0312:26bhaganyhttps://github.com/bhagany/snowth/blob/master/src/snowth/satellites.cljs#2017-05-0312:26bhaganythere's an example, and there's more in that project if you'd like to click around#2017-05-0312:27bhaganytl;dr - looks like I import the clojure.test.check.generators
ns, and use that in conjunction with clojure.spec/gen
#2017-05-0312:29lwhortonthanks @bhagany ill take a look#2017-05-0312:29bhagany@lwhorton: sure, np. perhaps better examples here: https://github.com/bhagany/snowth/blob/master/src/snowth/astro.cljs#2017-05-0315:46chilleniousHi. Trying out spec and running into a problem right away. Simply adding [clojure.spec.alpha :as s] to a :require section in my clojure file, and then trying to load that file in the repl (Leiningen) via use gives me this error:
java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn
When I use require instead of use, it seems fine. Anything I'm doing wrong/ known bug?#2017-05-0316:09chilleniousactually, scrap that... I get this error with either use or require whenever I pass in the :reload-all flag#2017-05-0316:24seancorfieldYes, thereās an AOT-related issue at present with the version of spec.alpha
that Clojure 1.9.0 Alpha 16 pulls in. If you explicitly depend on org.clojure/spec.alpha "0.1.108"
the problem should go away.#2017-05-0316:24seancorfieldIt specifically seems to affect REPL workflows and anything that reloads code.#2017-05-0316:28chilleniousthanks!#2017-05-0319:08slipsetIām writing specs for my om components, which basically are vararg functions of which I normally just care about the first arg.#2017-05-0319:08slipsetSo I end up with something like#2017-05-0319:08slipset(s/fdef content-area :args (s/cat :args ::grid :rest (s/* any?)))
#2017-05-0319:09slipsetthe :rest (s/* any?)
bit seems a bit silly.#2017-05-0401:03Alex Miller (Clojure team)Why?#2017-05-0407:22slipsetSilly might be wrong, but tedious maybe?#2017-05-0407:25slipsetwhat Iād like to be able to state is āI only care about the first argument and it should be a grid ā#2017-05-0407:54dergutemoritz@slipset That's what you're stating with that spec, though, isn't it? š If you feel it's too noisy and/or you're going to use it in many places, you could define a macro that hides that noise.#2017-05-0407:56slipset@dergutemoritz true#2017-05-0407:57slipsetapart from that, it seems like spec is giving me exactly what I want: the possibility to sprinkle ātypesā over my program when itās ādoneā.#2017-05-0421:41joshjonesNamespace myproj/myns.clj
contains a function, which requires specs defined in myproj/specs.clj
. I have the function fdef
'd and it currently sits next to the function itself in myns
. However, I'd like to move the function spec away from the function itself. So myns
depends on specs
, but the fdef'd function depends on knowing about the function itself in myns
, so I can't move it to specs
. Circular dependency. Where are you putting your spec'd functions (or in general, any of your specs) in your project?#2017-05-0422:07ghadithe specced namespace IMHO should not require the specs#2017-05-0422:08ghadiit was explicit design of clojure.spec that you can spec things you don't own#2017-05-0422:08misha(s/fdef myns/my-fn ...)
without explicitly importing myns
in specs
, @joshjones#2017-05-0422:18mobileinkmisha: is that missing a colon?#2017-05-0422:21mishano, its a fully qualified fn name, like clojure.string/blank?
#2017-05-0422:24mobileinkmisha: so you have to require that ns in your spec file? or otherwise define it there? that seems odd, but i'm a relative noob at spec.#2017-05-0422:27mishano, require
ing needed. afair, spec just uses fn symbol to register spec, and does not call actual fn, so the actual require'ing is not needed#2017-05-0422:08ghadi^#2017-05-0422:12joshjonesmany thanks to you both -- @ghadi , by "the specced namespace" you mean myns
or specs
in my example above?#2017-05-0422:12ghadimyns#2017-05-0422:12ghadiyourproj.specs should depend on yourproj.myns, not the other way around, like @misha 's example#2017-05-0422:12mishayeah, myns
might not need specs imported unless it uses one for explicit validation as a workflow step#2017-05-0422:13joshjonesit does -- (s/assert ::specs/myspec some-data)
#2017-05-0422:14joshjonesMany examples of using valid?
, conform
, etc also#2017-05-0422:15mishathen either keep fdef's closer to functions, or use fully qualified fn names in fdefs w/o explicit imports#2017-05-0422:16mobileinkyou only need to require if you want to use an alias, is that correct?#2017-05-0422:25mishamobileink: I'd say: only if you need to call something from that ns, or import for side effects. alias
works w/o explicit import (at least it worked for me just now)#2017-05-0422:28mobileinkhmm, need to experiment more. i could swear i've used :foo.bar/baz without require or import, but theres a goid chance i misunderstood what i was doing. ;)#2017-05-0422:30misha2 types of specs: for data ā keyword (:foo/bar), for functions ā symbol (foo/baz)#2017-05-0422:31mobileinki didn't think you could alias an ns without "making" it whether by require or sth else. i.e. just a symbol won't work. but i'm away from my machine, will try later.#2017-05-0422:31mobileinkaha, my bad - have not yet worked my way up to fn specs. sorry!#2017-05-0422:32mishaI am trying now, and I see no spec ns import required either: I can define specs for functions in foo.specs
w/o importing foo.fns
, and can use specs for assertions in foo.fns
w/o importing foo.specs
#2017-05-0422:34mobileinktime for me to take another sip from the firehose. not so easy when your lips have already been ripped off!#2017-05-0422:34mishatwo namespaces, neither imports the other. instrumentation and validation work, because specs get registered in global registry#2017-05-0422:35mishacc @joshjones#2017-05-0422:36mobileinki think i assumed reg keys are always kws. bad programmer!#2017-05-0422:37mishaI have spec guide open at all times while doing spec development, for quick reference https://clojure.org/guides/spec#2017-05-0422:39mobileinkme too - just haven't needed to deal with fns yet. it's hard enough to figure out how to do what i need with maps!#2017-05-0422:40mishathe only require
use case I can think of now ā is google closure compiler advanced optimization, where specs might get dead-code-eliminated. Haven't tried that yet though#2017-05-0422:41mobileinkfwiw i just got my clj map specs working in cljs, using cljc code. works like a charm - but haven't done advanced opt yet. shivers#2017-05-0422:45joshjonesthanks @misha, very helpful#2017-05-0510:27slipsetI seem to remember that the fdef
:ret
is not enforced/checked under instrument
. Is that correct, and if so, how does one go about checking that the return value of a fn conforms to the spec?#2017-05-0512:52slipsetJust to answer myself: https://groups.google.com/forum/#!topic/clojure-dev/4W4PO5Di9WU#2017-05-0513:18slipsetThe reason I was wondering (and wanting this) is that in cljs, I run my dev-setup with (stest/instrument)
which is super nice, but it would be nice to get the :ret
bit as well while running in dev-mode#2017-05-0514:15joshjones@slipset The answer you found requires an additional binding in many cases, and will generally mess up the flow of the function. I'm in the same boat as you, looking for a way to automatically check the return, with the ability to disable it for production. Not having the ability to test :ret
except through stest makes a function spec much less useful. something like this works at least on a small example:
(defn foo [x y]
{:pre [(s/assert (:args (s/get-spec `foo)) [x y])]
:post [(s/assert (:ret (s/get-spec `foo)) %)]}
(+ x y))
#2017-05-0514:18joshjonesit can be turned on/off with s/check-asserts
, so it may be a decent workaround to this limitation (it may be by design, but it is still a limitation)#2017-05-0518:18onetomas a https://github.com/Yuppiechef/datomic-schema user, who wrote a few DSLs in Rebol,
I was inspired by the data model example in this article:
http://blog.cognitect.com/blog/2017/4/6/developing-the-language-of-the-domain
since there was no example implementation which could parse this:
(attr :university/full-name string non-blank unique
"Fully-expanded name of the university for public display")
I set out to implement it with clojure.spec
, so I can do:
(->> '[[:person/email one unique str "Email"]
[:person/org many ref "Orgs"]]
(mapv (partial s/conform ::attr))
clojure.pprint/pprint)
#2017-05-0518:19onetomhere is my implementation:
(defn with-ns [ns kw]
(keyword ns (name kw)))
(defn conform-with-ns [ns]
(s/conformer (partial with-ns ns)
(comp symbol name)))
(defn type-aliases [t]
('{str string} t t))
(s/def ::attr
(s/coll-of
(s/or :db/doc string?
:db/cardinality (s/and '#{one many}
(conform-with-ns "db.cardinality"))
:db/valueType (s/and '#{str string int ref}
(s/conformer type-aliases)
(conform-with-ns "db.valueType"))
:db/unique (s/and #{'unique}
(conform-with-ns "db.unique"))
:db/ident keyword?)
:into {}))
#2017-05-0518:20onetomplease share your thoughts on it#2017-05-0518:58dnolenI just changed cljs.spec
to cljs.spec.alpha
now would be a good time to try master - would like to cut a release for this#2017-05-0602:10jimmydnolen: yay š#2017-05-0619:23reefersleepWhen using spec/explain
on a spec which contains spec/coll-of
in cljs and passing in an invalid datastructure, I don't seem to be getting any information about the offending element of the coll, only that something is wrong within the coll. Example:
(spec/explain (spec/coll-of string? []) ["hey" "yo" 9])
val: ["hey" "yo" 9] fails predicate: (coll-checker string?)
For contrast, here is the ouput for a Schema validation similar to the spec explain:
(schema/validate [schema/Str] ["hej" "yo" 9])
#error {:message "Value does not match schema: [nil nil (not (cljs$core$string? 9))]", :data {:type :schema.core/error, :schema [
Can I do something to make Spec give me more detailed output, similar to that of Schema?
In my actual use case, the spec/coll-of
contains more complex maps with nested specs, and if one thing goes wrong somewhere in the coll-of
, I'm left in the dark, fumbling for the specifics.#2017-05-0619:23reefersleepWhen using spec/explain
on a spec which contains spec/coll-of
in cljs and passing in an invalid datastructure, I don't seem to be getting any information about the offending element of the coll, only that something is wrong within the coll. Example:
(spec/explain (spec/coll-of string? []) ["hey" "yo" 9])
val: ["hey" "yo" 9] fails predicate: (coll-checker string?)
For contrast, here is the ouput for a Schema validation similar to the spec explain:
(schema/validate [schema/Str] ["hej" "yo" 9])
#error {:message "Value does not match schema: [nil nil (not (cljs$core$string? 9))]", :data {:type :schema.core/error, :schema [
Can I do something to make Spec give me more detailed output, similar to that of Schema?
In my actual use case, the spec/coll-of
contains more complex maps with nested specs, and if one thing goes wrong somewhere in the coll-of
, I'm left in the dark, fumbling for the specifics.#2017-05-0720:40mobileinkreefersleep: it tells you precisely that your val contains a non-string.#2017-05-0720:42mobileinkreefersleep: do you have an example of being left in the dark?#2017-05-0814:07reefersleep@U0LGCREMU: here's a more elaborate example of what I mean in regards to Schema providing more information than Spec, leaving me desiring more from Spec.
Spec:
(spec/def ::id int?)
(spec/def ::name string?)
(spec/def ::person (s/keys :req-un [::id
::name]))
(spec/explain (spec/coll-of ::person []) [{:id 1 :name "john"} {:id 2 :name :heather}])
val: [{:id 1, :name "john"} {:id 2, :name :heather}] fails predicate: (coll-checker :sleepydo.db/person)
;; No information about _which_ element of the coll failed to validate, nor details about how it failed to validate
Schema:
(def Id schema/Int)
(def Name schema/Str)
(def Person {:id Id
:name Name})
(schema/validate [Person] [{:id 1 :name "john"} {:id 2 :name :heather}])
#error {:message "Value does not match schema: [nil {:name (not (cljs$core$string? :heather))}]", :data {:type :schema.core/error, :schema [{:id
#2017-05-0620:21xiongtxDoes s/with-gen
take a thunk that returns a generator for laziness? Is that way it doesnāt just take a generator directly?#2017-05-0719:08reefersleepIs there an easy way to refer to a registered spec under a different name in a map spec?#2017-05-0723:09taylor@U0AQ3HP9U if I understand your question, I don't think there is and I also think that's by design#2017-05-0723:11Alex Miller (Clojure team)no, other than by using :req-un
for a spec with the same name but different namespace in s/keys
#2017-05-0813:43reefersleepThank you @U3DAE8HMG and @U064X3EF3 for your succinct answers!#2017-05-0722:12caiois there an easy way of defining spec for something inside a ref?#2017-05-0722:12caioI'm trying to write a spec for funcool cats' either monad, but no luck#2017-05-0722:15caioI was able to write something to conform/unform, but got stuck when trying to write a proper generator#2017-05-0723:11Alex Miller (Clojure team)@caio not yet#2017-05-0723:12Alex Miller (Clojure team)possibly will be added in the future#2017-05-0801:11caiowhat about a generator for records? can that be done?#2017-05-0801:38caiohttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L91 one could use this to write the generator for refs, right? (gen/fmap deref (s/gen internal-spec))
#2017-05-0801:42caioalso, I think my problem is a bit more generic than writing specs for refs. I was trying to write a spec for a type defined in cats. after seeing gen/fmap
, looks doable#2017-05-0801:50gfredericksyes technically that will work#2017-05-0801:51gfredericksnormal test.check usage is aimed at writing generators for data, not stateful objects#2017-05-0801:51gfredericksI'm not sure what will happen if spec starts having specs for reference types#2017-05-0801:55bhagany@caio I made a generator for reified protocols, which are similar enough to records https://github.com/bhagany/snowth/blob/master/src/snowth/satellites.cljs#L215#2017-05-0801:58bhaganyI hadn't learned about gen/let
when I wrote this, fyi. I'd probably use it now#2017-05-0801:59caioyeah, I did something like that (though my record was way simpler) https://github.com/funcool/cats/pull/201#2017-05-0801:59caio@gfredericks this is not stateful. IDeref was implemented as a syntatic sugar#2017-05-0802:00gfredericks@caio okay that's not bad then#2017-05-0802:00gfredericks@bhagany btw the bind+return thing there can be simplified to fmap#2017-05-0802:00bhagany@gfredericks thanks š#2017-05-0807:22mpenetany reason why (s/assert x) compiles to x instead of nothing (like core/assert) when s/*compile-asserts*
is false?#2017-05-0808:12NiclasIs there any way of specing async channels and the data that may be sent over them?#2017-05-0808:30misha@looveh wrap >!
with validating fn? you can put anything on a channel, so any limitation would need to be enforced by client code, I think.#2017-05-0808:32Niclas@misha Thatās one way to do it, but it would also be useful to be able to spec channels as function arguments#2017-05-0808:32NiclasNot just the actual get/put on channels#2017-05-0808:32mishawrapping >!
would bail right away. wrapping <!
would bail sometime later/never. Whichever would suite you best.#2017-05-0808:34mishawrap channel in a record, spec record, spec protocols it implements?#2017-05-0808:35misha*with multispec.#2017-05-0808:35NiclasYeah that would work, feels a bit hacky though imo since channels are part of a core lib#2017-05-0808:36mishaI'd imagine you need to add something to the channel-thing itself to be able to use different specs for different channels.#2017-05-0808:37mishaso either spec entry/exit points, or wrap, and spec/pass around/expect wrapped channels.#2017-05-0808:42mishaĀÆ\(ć)/ĀÆ#2017-05-0808:44NiclasHahah thatās amazing!#2017-05-0813:36tbaldridge@misha @looveh specs could be put on a filter
transducer on the channel, or a call to map that throws. On an exception the exception would be handed to the channels' exception handler.#2017-05-0814:06misha@tbaldridge I think @looveh's intention was to "spec a function, so that wrong kind of channel passed as an argument would throw", not "make sure channel would not accept random stuff".
I like "spec as a filter" much more than macro above, thank you.#2017-05-0814:07misha@tbaldridge can that channel's filter be used as a dispatch value once channel is constructed? is it accessible from outside?#2017-05-0814:09tbaldridge@reefersleep what does explain-data show for the spec code?#2017-05-0814:11reefersleep@tbaldridge: Nothing more.
(spec/explain-data (spec/coll-of ::person []) [{:id 1 :name "john"} {:id 2 :name :heather}])
{:cljs.spec/problems {[] {:pred (coll-checker :sleepydo.db/person), :val [{:id 1, :name "john"} {:id 2, :name :heather}], :via [], :in []}}}
#2017-05-0814:12reefersleepNote that I'm talking about cljs, not clj. š#2017-05-0814:16misha@reefersleep why is there []
in (spec/coll-of ::person [])
?#2017-05-0814:17mishafor vector it is (spec/coll-of ::person :kind vector?)
, isn't it?#2017-05-0814:20mishaclojure gives me
(s/explain-data (s/coll-of string? []) ["a" "b" 1])
CompilerException java.lang.IllegalArgumentException: No value supplied for key: []
and in cljs (js) I can imagine it fails silently somewhere, but produces compromised output anyway#2017-05-0815:46reefersleepI thought I should use :kind vector?
as well, until I tried it.
(spec/explain-data (spec/coll-of ::person :kind vector?) [{:id 1 :name "john"} {:id 2 :name :heather}])
---- Could not Analyze <cljs form> line:1 column:20 ----
Wrong number of args (3) passed to: spec/coll-of
1 (spec/explain-data (spec/coll-of ::person :kind vector?) [{:id 1 :name "john"} {:id 2 :name :heather}])
^---
---- Analysis Error ----
Then I tried without any :kind
-indication:
(spec/explain-data (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
---- Could not Analyze <cljs form> line:1 column:20 ----
Wrong number of args (1) passed to: spec/coll-of
1 (spec/explain-data (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
^---
---- Analysis Error ----
So I looked up the documentation:
(cljs.repl/doc spec/coll-of)
-------------------------
cljs.spec/coll-of
([pred init-coll])
Macro
Returns a spec for a collection of items satisfying pred. The generator will fill an empty init-coll.
#2017-05-0814:26Alex Miller (Clojure team)if you remove that invalid trailing [], clojure says:
user=> (spec/explain (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
In: [1 :name] val: :heather fails spec: :user/name at: [:name] predicate: string?
#2017-05-0815:50reefersleepalexmiller: So I guess what I'm seeing is a feature of clj spec missing (as of now) from the cljs port. I kind of suspected this, as I thought I'd seen more useful output in guides/tutorials at the time spec came out.#2017-05-0814:28Alex Miller (Clojure team)which seems better than the schema message to me#2017-05-0814:50andrewboltachevHello. When I have keys #{:a :b :c}
and a spec ::foo
, how do I get
{:a (gen/genereate (s/gen ::foo))
:b (gen/genereate (s/gen ::foo))
:c (gen/genereate (s/gen ::foo))}
generated for me? (edited)
Now with (s/map-of #{:a :b :c} ::foo)
I've got only some keys, not all three used#2017-05-0814:53mishas/keys#2017-05-0814:53andrewboltachevhuh#2017-05-0814:53andrewboltachevso simple š#2017-05-0814:55andrewboltachev@misha but I think s/keys
takes both value type and key names from it's arguments#2017-05-0814:55andrewboltachev(and I have these different)#2017-05-0814:56andrewboltachevi.e. N keys and single value type#2017-05-0814:56andrewboltachev@misha thanks for answering btw š#2017-05-0814:56misha(s/def :foo/bar integer?)
(s/def :my/a :foo/bar)
(s/def :my/b :foo/bar)
(s/def :my/c :foo/bar)
(s/def :my/map (s/keys :req-un [:my/a :my/b :my/c]))
=> :my/map
(s/exercise :my/map 2)
=>
([{:a -1, :b 0, :c 0} {:a -1, :b 0, :c 0}]
[{:a -1, :b -1, :c -1} {:a -1, :b -1, :c -1}])
#2017-05-0814:57andrewboltachevsure, but I'll need to repeat for 4 or 5 different types of ::foo
#2017-05-0814:58mishaafaik, you have to alias desired keys to their desired values' specs#2017-05-0814:59mishaor provide your own generator, which would make sure all the keys are present. (I'd just alias things, custom generators might screw up s/explain-data
error address within the structure)#2017-05-0815:00andrewboltachevyep the custom generators are obviously "approach from the opposite side"#2017-05-0815:01andrewboltachevand well, honestly also having N keys as I do might mean semantically the sequence, not a map#2017-05-0815:02mishaplenty of options there as well: s/coll-of
, s/tuple
#2017-05-0815:02andrewboltachevyep, would take a look at every
function#2017-05-0815:55Alex Miller (Clojure team)I think you must be using an older version of ClojureScript - thatās not what I see with the latest#2017-05-0815:55Alex Miller (Clojure team)afaik, ClojureScript is basically at parity with the Clojure version wrt spec#2017-05-0815:56Alex Miller (Clojure team)cljs.user=> (s/explain-data (s/coll-of ::person :kind vector?) '({:id 1 :name "John"}))
{:cljs.spec/problems [{:path [], :pred vector?, :val ({:name "John", :id 1}), :via [], :in []}]}
cljs.user=> (s/conform (s/coll-of ::person :kind vector?) [{:id 1 :name "John"}])
[{:name "John", :id 1}]
#2017-05-0815:57Alex Miller (Clojure team)Iām using ClojureScript 1.9.293 btw#2017-05-0816:05reefersleepI thought I was up to date, but apparently, I misread the version number... 1.9.93
. š#2017-05-0816:07reefersleepI'll try updating the version, I'm sure that'll make things better for me!#2017-05-0816:07reefersleepThanks all!#2017-05-0816:29Alex Miller (Clojure team)whatās 200 commits between friends? :)#2017-05-0816:37reefersleepNow I'm having a different problem, not spec-related - I'm trying to destruct with (let [{:keys [a b] :as full} (fn-call something...)] ...
#2017-05-0816:41reefersleepI've not changed the code for that destructuring or what goes on underneath between cljs version changes. However, the output of (fn-call ...)
, which is a map {:a "a's value" :b "b's value"}
, - I can tell by printing at the end of my fn-call
- turns into {[:a "a's-value] [:b "b's value"}
upon destruction (I can tell by printing full
), so I cannot destruct`a` or b
.#2017-05-0816:41reefersleepSo weird.#2017-05-0816:51reefersleep(asking in #clojurescript , too)#2017-05-0816:53caioabout specing a deftype, this is what I came up with in the end: https://gist.github.com/caioaao/d15260076bb8ee6f48908507ae73155b#2017-05-0816:55caiowould be nice if someone could validate if this is safe. I had to go through spec's code for doing this, and some stuff were not really clear to me#2017-05-0816:59caioalso saw something in spec-tools
that they do some checking on conform
that made me think it may not be thread-safe. idk if that's the case#2017-05-0817:00caiohttps://github.com/metosin/spec-tools/blob/master/src/spec_tools/core.cljc#L160 this#2017-05-0817:03caiooh, nvm. looks like they do this because they call conform*
from inside explain*
#2017-05-0817:13Alex Miller (Clojure team)What' is either supposed to do?#2017-05-0817:14Alex Miller (Clojure team)Is this just a non conforming or of two choices?#2017-05-0817:18caioit's the either monad, from funcool cats#2017-05-0817:19caiohttps://funcool.github.io/cats/latest/#either#2017-05-0817:27ikitommi@caio about spec-tools - if the conform doesnāt use threads, I should be ok. But not optimal. would be nice to use just 3rd parameter in conform (http://dev.clojure.org/jira/browse/CLJ-2116)#2017-05-0817:27caioYou can say it's a non conforming or between two choices If you want to push it (and probably piss someone š )#2017-05-0817:29ikitommigreat article, with the āSafe Dynamic Scopeā chapter: https://stuartsierra.com/2013/03/29/perils-of-dynamic-scope.#2017-05-0818:00mobileinkIām getting a strange error that seems related to the switch to spec.alpha. In my code this works: (clojure.core/require [pagespace-sym] :reload)
. But if I change it to :reload-all
I get the following error: java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn
clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn, compiling:(clojure/tools/logging.clj:1:1)
#2017-05-0819:48Alex Miller (Clojure team)This is a known issue with spec.alpha not being aot compiled#2017-05-0819:49Alex Miller (Clojure team)It's been fixed in a new build of spec.alpha - 0.1.108. We haven't released a new Clojure alpha yet that depends on it but you can specify that to override#2017-05-0820:03mobileink@alexmiller thanks!#2017-05-0908:42mpenetWould it make sense to have a ^:fdef
(or ^:spec
) meta on fn
to do the :pre/:post automatically from a fdef ? Before you mention instrument, it's not the same (instrument triggers gen).#2017-05-0908:45mpenetthat would avoid some boilerplate :
(s/fdef ::foo
:args (s/cat :a ::a
:b ::b
:c ::c))
(defn foo
[a b c]
{:pre [(s/assert (:args (s/get-spec ::foo))
[a b c])]
:post [(s/assert (:ret (s/get-spec ::foo))
%)]}
42)
vs
(s/fdef ::foo
:args (s/cat :a ::a
:b ::b
:c ::c))
(defn foo ^:fdef ::foo
[a b c]
42)
#2017-05-0908:51mpenetthe spec and where it's applied is still decoupled, the latter would basically expand into the former (almost, omited the potential :fn part of the spec)#2017-05-0908:55Alex Miller (Clojure team)no, not interested in that. first, this re-couples vars and function specs which are intentionally not coupled (and can be created in either order). second, we donāt generally want to enable pre/post checking of specs for performance reasons.#2017-05-0908:55mpenet(it doesn't have to use :pre/:post, would open doors to auto conforming via metadata hint etc)#2017-05-0908:56Alex Miller (Clojure team)I donāt think Rich has any interest in that#2017-05-0908:58mpenetalright, not surprised, I guess this might end up being done as part of a lib#2017-05-0908:59mpenetpre/post is toggleable (s/assert as well), but again this would/should be done without these 2 anyway#2017-05-0909:04mpenet@alexmiller about ordering I am not sure I follow, if that expands to s/assert calls it wouldn't matter/cause any issue#2017-05-0909:10Alex Miller (Clojure team)pre/post effects what ends up in the compiled code (even if toggled off) and thus has a runtime cost#2017-05-0909:10Alex Miller (Clojure team)s/assert can actually be omitted from compilation entirely using s/*compile-asserts*
etc#2017-05-0909:11mpenetok, so lets say it's implemented without pre/post š, just with s/assert#2017-05-0909:11Alex Miller (Clojure team)re ordering, right now you create the specs before the functions or after the functions#2017-05-0909:11Alex Miller (Clojure team)by having defn rely on the spec, youāve changed that#2017-05-0909:11Alex Miller (Clojure team)if youāre compiling#2017-05-0909:12mpenet(s/fdef ::foo-like ...)
(defn foo [x y]
(s/assert ::foo-like [x y])
...
)
#2017-05-0909:13mpenetit would be the same as the above , you can just def ::foo-like after as well#2017-05-0909:13mpenetor I am missing something#2017-05-0909:13Alex Miller (Clojure team)I donāt know, maybe that would work#2017-05-0909:13Alex Miller (Clojure team)aside: the ::foo-like there would be `foo#2017-05-0909:14mpenetI don't follow#2017-05-0909:14Alex Miller (Clojure team)function specs are named by symbol, not keyword#2017-05-0909:14mpenetright!#2017-05-0909:14mpenetmissed that one#2017-05-0909:15Alex Miller (Clojure team)generally, we donāt think you should be doing this though#2017-05-0909:17mpenetthat would be handy to be able to reuse specs defined for instrument on live (staging/test) systems without paying the gen cost when that matters#2017-05-0909:18Alex Miller (Clojure team)there is no āgen costā?#2017-05-0909:18mpenethmm if I recall instrument does trigger gen in some cases#2017-05-0909:19Alex Miller (Clojure team)the only case like this is with fspec#2017-05-0909:20Alex Miller (Clojure team)if thatās a problem for you, then donāt use fspec#2017-05-0909:20Alex Miller (Clojure team)and just spec as ifn?
#2017-05-0909:21mpenetwell ... first class functions are quite common in clj code, but yes that's an option#2017-05-0911:23lambderhello all
I got the following problem:
clojure
(s/def ::s string?)
(s/def ::g
(s/cat :string ::s
:g (s/* ::g)))
(s/conform ::g '["abc" "abc"]) ; --> ok;
(s/def ::g2
(s/cat :string (s/? ::s)
:g (s/* ::g2)))
(s/conform ::g2 '["abc" "abc"]) ; --> java.lang.StackOverflowError
any ideas?#2017-05-0911:27dergutemoritz@lambder That recursion will never terminate because (s/? ::s)
may match nothing so once the two "abc"
strings are matched, it will recur forever on the remaining sequence []
#2017-05-0911:32lambder@dergutemoritz how do I much the following grammar then?
S -> T R | R
T -> :tag
R -> :foo | :foo R
#2017-05-0911:41dergutemoritz@lambder That |
in there is probably s/or
#2017-05-0911:41dergutemoritzErr sorry, it's s/alt
in regex#2017-05-0911:42dergutemoritzEverything else is s/cat
#2017-05-0911:42dergutemoritzJust an educated guess, of course, not sure what grammar notation you're using exactly there#2017-05-0911:43lambderBNF#2017-05-0911:43dergutemoritzOK then yeah, that should work##2017-05-0911:43dergutemoritzHm well it's not really BNF though, that doesn't use ->
at least#2017-05-0911:44lambder@dergutemoritz if you wish, please answer it on https://stackoverflow.com/questions/43868064/clojure-spec-conform-throws-stack-overflow-exception youāll get my upvote#2017-05-0911:44dergutemoritzDon't have an SO account#2017-05-0911:45lambderok#2017-05-0911:46lambder@dergutemoritz many thanks#2017-05-0911:46dergutemoritzYW, hope it works out!#2017-05-0912:05vandr0iysuppose I got a spec, (s/def ::foo (s/keys :req-un [::a ::b] :opt-un [::c ::d]))
and a data structure, (def bar {:a 1 :c 3 :d 4 :e 5})
. How do I remove from the bar
all the key-value pairs not explicitly mentioned in the spec and fill in with nils the data that's missing (so, :b nil
and remove :e
)?#2017-05-0912:24Alex Miller (Clojure team)spec doesn't do that#2017-05-0912:46lambderis it possible to preserve attached meta to structures which are spec conformed?#2017-05-0914:02lambderIāve done this:
(in-ns 'clojure.spec)
(defn conform [spec x]
(let [m (clojure.core/or (meta x) {})]
(let [obj (conform* (specize spec) x)]
(cond
(instance? clojure.lang.MapEntry obj) (with-meta (into [] obj) m)
(instance? clojure.lang.IObj obj) (with-meta obj m)
:else obj))))
#2017-05-0914:03lambderitās a bit hacky#2017-05-0915:49lambder@dergutemoritz I got better answer by @flowthing if you are interested.
https://stackoverflow.com/a/43869208/135892
thanks to @alexmiller#2017-05-0915:51lambderIs it possible to ātapā into a protocol method. Iād like to have āaroundā wrapper for all such method calls.
e.g there is a protocol clojure.spec.Spec
which has conform*
method defined.
Some of the conform*
implementations call conform*
recursively. Iād like to wrap my logic around each such call.#2017-05-0915:51lambderI want to inject some code allowing to preserve meta of the parsed data-structures#2017-05-0915:52lambdermy monkey patch is :
(in-ns 'clojure.spec)
(defn conform [spec x]
(let [m (clojure.core/or (meta x) {})]
(let [obj (conform* (specize spec) x)]
(cond
(instance? clojure.lang.MapEntry obj) (with-meta (into [] obj) m)
(instance? clojure.lang.IObj obj) (with-meta obj m)
:else obj))))
#2017-05-0915:53lambderbut the call to conform
is sometimes omitted as conform*
is called instead#2017-05-0916:01Alex Miller (Clojure team)in short, no and anything touching these impl details stands a good chance of being broken later#2017-05-0916:02lambder@alexmiller yes, i think so too. Any idea why conform*
implementations arenāt calling conform
? performance?#2017-05-0916:42dergutemoritz@lambder Err I just realized that the grammar you pasted is not equivalent to what you were trying to express in spec - according to that, :tag
is only allowed as the first element of the whole sequence and then only :foo
s are allowed (because R
recurs on itself, not on S
). Is that what you intended?#2017-05-0916:43lambdernot really. the grammar was simplification of what i needed#2017-05-0916:44lambderbut thanks to https://stackoverflow.com/a/43869208/135892 now I know how to express this kind of grammers in spec#2017-05-0916:48dergutemoritz@lambder AFAIUI that's more complicated than necessary. Doesn't something like (s/def ::g (s/+ (s/cat :tag (s/? ::tag) :val (s/alt :number ::n :string ::s))))
capture what you want to do much neater + give you a more convenient result structure to work with?#2017-05-0916:49lambderpossibly#2017-05-0916:49dergutemoritzMight be misinterpreting your use case, of course. Anyhow, gotta run. Cheers#2017-05-0916:59lambdercheers#2017-05-0921:18danielcomptonIs there a way to define newtype's in spec? e.g. specialise a Long to an AccountId or a ProductId. I'm pretty sure there's not, but thought I'd ask in case.#2017-05-0922:16ghadisay wha?#2017-05-0922:17ghadi@danielcompton seems like the spec name itself is that information#2017-05-0922:19danielcomptonLet's say I have a function that gets a product (get-product product-id)
. If the product-id and account-id are both Longs, nothing stops me from passing it (get-product account-id)
#2017-05-0922:52caioI solve this by creating :account/minimal
as (s/keys :req [:account/id] :opt [ all the other stuff])
when really necessary#2017-05-0922:53danielcomptonbut if you ever need to extract and pass around :account/id
by itself, then you can't guarantee that that's what the functions are taking?#2017-05-0923:40caioI don't think so. specs are about contracts, not strict types#2017-05-0923:42caioif you want to be this strict, define :account/id
as (s/tuple [#(= % :id/account) int?])
or something like that (maybe multi-spec?) and adapt inside the fns that use the id#2017-05-1003:50Alex Miller (Clojure team)in short, no you canāt do this#2017-05-1005:54Oliver George@gfredericks came across your defn+spec gist (dated 11m ago). Did it ever evolve? I know I'm not meant to want it but seems like a handy way to get type info into my code.
https://gist.github.com/gfredericks/e4a7eafe5dcf1f4feb21ebbc04b6f302#2017-05-1013:32gfredericksI don't think so; but if there's nothing weird about it I could release it in schpec if that would help#2017-05-1013:40Oliver GeorgeThanks. I'm interested for use with CLJS.#2017-05-1016:32gfredericks@olivergeorge k I'll let you know when I get that pushed out#2017-05-1019:23johnIs there a good example of a project with good "test coverage" that leverages spec?#2017-05-1019:29nwjsmith@john might want to try digging through these results https://github.com/search?l=Clojure&q=stest%2Fcheck&type=Code#2017-05-1019:31johnThat's the search-foo I was looking for. Thank you sir.#2017-05-1019:40johnThis looks pretty thorough: https://github.com/mike706574/milo#2017-05-1019:47johnThis looks good too: https://github.com/eauc/tasks-client#2017-05-1107:31rmuslimovCan you please point me what Iām doing wrong in example below:
(defn command [m] 1)
(s/fdef command
:args (s/cat :n int?)
:ret string?)
(command "ff") ;; => 1
It works and is not failing with exception#2017-05-1107:33xiongtxrmuslimov: Are you instrumenting your function? https://clojure.org/guides/spec#_instrumentation_and_testing#2017-05-1107:36rmuslimovah, youāre right. Sorry for dumb question#2017-05-1114:18slipsetalso, remember that the return value is not checked even if you instrument it.#2017-05-1119:22Alex Miller (Clojure team)The :ret spec is checked only by stest/check
#2017-05-1122:31danielcomptonYou can also use Orchestra to instrument your spec return values: https://github.com/jeaye/orchestra/#2017-05-1120:07rmuslimovOne more newbie question about spec. here as example
{:email1 "
Here is spec I wrote, anyway to make it shorter?
(s/def ::email1 string?)
(s/def ::email2 string?)
(s/def ::email3 string?)
(s/def ::email4 string?)
(s/def ::email5 string?)
(s/def ::data int?)
(s/def ::item
(s/keys :req-un [::email1 ::email2 ::email3 ::email4 ::email5 ::data]))
#2017-05-1120:07rmuslimovOne more newbie question about spec. here as example
{:email1 "
Here is spec I wrote, anyway to make it shorter?
(s/def ::email1 string?)
(s/def ::email2 string?)
(s/def ::email3 string?)
(s/def ::email4 string?)
(s/def ::email5 string?)
(s/def ::data int?)
(s/def ::item
(s/keys :req-un [::email1 ::email2 ::email3 ::email4 ::email5 ::data]))
#2017-05-1120:19xiongtxrmuslimov: Seems like youāre missing a few colons in s/keys
.#2017-05-1120:21rmuslimovyep, fixed. but question remain the same. Why spec is longer than data? is it supposed to be like that? Or itās my un-optimized solution?#2017-05-1121:13Alex Miller (Clojure team)itās supposed to be like that.#2017-05-1121:27johnWould it be dangerous for a user to create a macro for that? like (sdef-each seq-of-keys string?)
#2017-05-1121:30Alex Miller (Clojure team)no, macro as you like#2017-05-1121:32johnrgr. so, @U0AR40HJ6, it's possible, bit it's definitely not a newbie/beginner level answer, if you're new to the language in general.#2017-05-1121:33rmuslimov@U050PJ2EU I can do a macro, was just wondering if there exist any proper way to that. thanks!#2017-05-1121:34johnNo prob. Might want to consider using the long form though, from a user/documentation perspective.#2017-05-1121:36johnIf core comes out with such a function in the future, then you can rely on your downstream users to know what you're talking about (if that's a factor for you)#2017-05-1121:36Alex Miller (Clojure team)not going to add that to core#2017-05-1121:37johnright, so if you are making a library you expect to be consumed by others, remember that spec is also intended to be used for documentation.#2017-05-1121:38Alex Miller (Clojure team)if youāre just wrapping calls to s/def
, I donāt see how that matters#2017-05-1121:39Alex Miller (Clojure team)there will be no difference between doing it explicitly or doing it in a macro?#2017-05-1121:39Alex Miller (Clojure team)the macro would presumably just unroll to the same set of calls#2017-05-1121:40john@U064X3EF3 so any documentation functions that operate on sources will probably operate post macro-expansion?#2017-05-1121:40john(Or whatever documentation features are expected to work with spec in the future)#2017-05-1121:40Alex Miller (Clojure team)doc functions donāt use the sources for specs#2017-05-1121:40johnrgr#2017-05-1121:40Alex Miller (Clojure team)docs look up the specs in the registry#2017-05-1121:40johnmacro away then š#2017-05-1121:47caioWhy not a vec ::emails with 5 string elements? š¤ #2017-05-1121:08mobileinknot that i know of. you have an idea?#2017-05-1121:09mobileinke.g. (s/def* string? [email1 email2 ... ]),#2017-05-1200:48tbaldridge@rmuslimov perhaps structure it as a list of strings? That approach is simpler for the programmer, and easier to maintain#2017-05-1200:52bbrinckIf my code accepts JS values at the boundary of the system, whatās the preferred way to use spec to validate it? AIUI, s/keys
only works with keywords. Is it common practice to convert all string keys to keyword keys when converting to CLJS in order to use spec? e.g. (js->clj input :keywordize-keys true)
#2017-05-1200:59rmuslimov@tbaldridge yes it is dead simple, however spec is longer that data it describes. I was wondering if Iām missing some nice shortcut macro for that#2017-05-1203:04tbaldridge@rmuslimov no, I'm saying that this is the better approach:
(s/def ::email string?)
(s/def ::emails (s/coll-of ::email))
(s/def ::item (s/keys :req-un [::emails ::data]))
#2017-05-1203:06rmuslimov@tbaldridge it wasnāt real world example, I just created the most trivial example of the problem. Real world example would be something like:
(s/def ::original_currency ::currency-type)
(s/def ::currency ::currency-type)
(s/def ::original_total (s/and double? pos?))
(s/def ::original_taxes (s/and double? pos?))
(s/def ::taxes (s/and double? pos?))
(s/def ::total (s/and double? pos?))
(s/def ::original_total (s/and double? pos?))
(s/def ::rate (s/keys :req-un [::rate_key ::original_currency ::currency ::original_total
::original_taxes ::taxes ::total ::original_total]))
#2017-05-1203:07tbaldridgeah, I see#2017-05-1203:09rmuslimovso, I thought why not something like:
(s/def
::rate {::taxes ::money
::original_taxes ::money
::total ::money
::original_total ::money
...etc})
#2017-05-1203:27Oliver George@gfredericks Thinking more about defn+spec made me explore why i wanted it and alternative ways to get the the same result.#2017-05-1203:28Oliver GeorgeOne key thing spec gives me is protection from "garbage in garbage out" errors while I'm hacking up new features. Colocating the spec with the defn is helpful during this phase because I'm refactoring heavily as I go - if the s/fdef specs were in another file they're more likely to get stale.#2017-05-1203:28Oliver GeorgeI can do this by adding s/assert statements in my defn.#2017-05-1203:28Oliver GeorgeLater when code stabilises I see the logic of specs living in a different namespace so they are out of the way but still available to reference.#2017-05-1203:28Oliver GeorgeAs an aside, IDE support to allow hiding of spec statements would make that less important. #cursive#2017-05-1203:29Oliver GeorgeBut I appreciate the idea that separate namespaces for specs also allow for different specs to be applied to the same code depending on need.#2017-05-1203:30Oliver George@gfredericks defn+spec would benefit from the s/assert flag so that it can have no impact on prod performance.#2017-05-1203:33tbaldridge@rmuslimov that's covered in some of the spec docs#2017-05-1203:33tbaldridge@rmuslimov https://clojure.org/guides/spec#_entity_maps#2017-05-1204:47rmuslimovI did actually, as far as I can see: only s/keys
is given for building spec for maps. I cannot do something close to way I proposed. Developer should always define explicitly keys like (s/def ::total ::money)
and then build spec for map with s/keys
.#2017-05-1212:52Alex Miller (Clojure team)It's quite intentional that information maps are built as sets of attributes. This is a key design principle in spec - the semantics belong to the attribute, not to the map.#2017-05-1212:52Alex Miller (Clojure team)This is covered a bit in https://clojure.org/about/spec#2017-05-1214:18mishais there any context-independent benefit/downside of defining intermediate predicate fn over inline spec definition?
(defn nsless-keyword? [k]
(and (keyword? k) (nil? (namespace k))))
(defn ::nsless-keyword nsless-keyword?)
vs:
(defn ::nsless-keyword (s/and keyword? (complement namespace)))
#2017-05-1217:22Alex Miller (Clojure team)Well it affects automatic generator capability#2017-05-1217:22Alex Miller (Clojure team)But you should really use simple-keyword? rather than either of those#2017-05-1217:22Alex Miller (Clojure team)Which has a built in generator#2017-05-1218:37creeseI would like to be able to attach a docstring to a spec. Has there been an progress on https://dev.clojure.org/jira/browse/CLJ-1965?#2017-05-1218:39Alex Miller (Clojure team)no#2017-05-1302:37danielcomptonIn https://clojure.org/about/spec#_minimize_intrusion it says "Donāt require that people e.g. define their functions differently."#2017-05-1302:39danielcomptonWill there also be a way to provide a spec along with a function? e.g. (s/defn ...)
or (defn my-fn [args] {:args (s/cat ...)} ...
or something similar?#2017-05-1302:39danielcomptonIf I do want to define my functions with my spec?#2017-05-1302:44danielcompton@rmuslimov I think spec-tools has https://clojurians.slack.com/archives/C1B1BB2Q3/p1494558543543341 as data-specs: https://github.com/metosin/spec-tools/#data-specs#2017-05-1302:47lincpaI think proactive data standardization work is better than passive data validation. Industrial assembly line recommended using standardized materials.#2017-05-1302:51lincpaStandardize input and output data at the beginning and end of data processing#2017-05-1302:57lincpaProactively standardize non-standard data, and if fail do exception handling.#2017-05-1305:00Alex Miller (Clojure team)@danielcompton no, there will not be a way to combine function definition with spec#2017-05-1309:26mishaare there any builtin predicates for queues in clj/cljs?#2017-05-1315:02john@misha What do you mean? like qeueu?
Don't believe so. If you mean query for type, you could use (instance? cljs.core/PersistentQueue #queue [])
#2017-05-1315:32Alex Miller (Clojure team)@misha no#2017-05-1315:32misha@john yes, sort of, with a generator included, please#2017-05-1315:34Alex Miller (Clojure team)You might be able to spec it with coll-of and some of the kind and into options, but I haven't tried#2017-05-1315:35misha@alexmiller, what is a correct way to define spec for dynamic function (like reducing function)?
I end up with following, is that legal?
(s/def ::reducing-fn
(s/fspec :args (s/cat ...))
#2017-05-1315:35Alex Miller (Clojure team)Sure?#2017-05-1315:35Alex Miller (Clojure team)What is dynamic about that?#2017-05-1315:38mishaAFAIR, s/fdef
accepts qualified fn symbol, with intention to use it on a function with that exact name.
There is an f(g)
where g
- is a function. I need to: 1) spec g
as an argument, 2) re-use that spec.
(s/def :foo (s/fspec ...))
seems to work, but I expect it to bite me in some way later.#2017-05-1316:18misha"how to get named spec for anonymous fn", there.#2017-05-1318:31Alex Miller (Clojure team)That should work#2017-05-1409:24thedavidmeisteranyone else seen java.lang.RuntimeException: Unable to resolve symbol: qualified-keyword? in this context, compiling:(clojure/spec/gen/alpha.clj:131:4)
with clojure version 1.9.0-alpha16
?#2017-05-1413:57weavejesterWhatās the value of *clojure-version*
, @thedavidmeister?#2017-05-1418:41Alex Miller (Clojure team)@thedavidmeister no but sounds like you are using spec.alpha with Clojure < 1.9#2017-05-1423:09thedavidmeister@alexmiller @weavejester yeah that was it, boot had its own version of clojure messing with things#2017-05-1423:09thedavidmeisterthanks#2017-05-1509:26danielnealI'm trying to take a reagent vector, conform it, make a modification and unform it back.
(s/def :reagent/vector
(s/cat :element any? :props (s/? map?) :children (s/* any?)))
This conforms fine:
(s/conform :reagent/vector [:div {:a 3} [:div "1207"] [:div "w13"]])
;; => {:element :div, :props {:a 3}, :children [[:div "1207"] [:div "w13"]]}
but unforms back to a lazyseq - is there a spec I can use that will make unform turn it back to a vector?#2017-05-1509:28danielnealI tried prefixing with s/and vector
but I think that was wrong#2017-05-1509:35dergutemoritz@danieleneal Wrapping it in (s/and ... (s/conformer identity vec))
should do the trick#2017-05-1509:36dergutemoritzOh or maybe you have to put it before the s/cat
really#2017-05-1509:36dergutemoritzInteresting ...#2017-05-1509:40danielnealyeah (s/and (s/conformer identity vec) ...
works, thanks @dergutemoritz#2017-05-1509:40danielnealis that appropriate/idiomatic do you know?#2017-05-1509:41dergutemoritzI guess that's an innocent enough use of s/conformer
, yeah š#2017-05-1509:41dergutemoritzMaybe we should call this kind of use "vegan" as opposed to the "meat grinder" pattern#2017-05-1509:44danielnealš#2017-05-1509:45dergutemoritz"No animals were harmed in this use of s/conformer
"#2017-05-1514:08Alex Miller (Clojure team)the need for regex ops in a vector is a known issue - itās pretty challenging right now to get conforming, validation, unforming, generation, and describing to work properly together#2017-05-1521:57zaneWhat's the right way to attach a generator to a value produced by s/conformer
?#2017-05-1521:58Alex Miller (Clojure team)s/with-gen
#2017-05-1522:00zaneRight, but how?#2017-05-1522:00zaneDoes it need to be attached to a keyword?#2017-05-1522:00zanee.g.
(def keyword-conformer
(s/with-gen
(s/conformer
(fn [x]
(cond (keyword? x) x
(string? x) (keyword x)
:else :clojure.spec/invalid)))
#(s/gen keyword?)))
#2017-05-1522:00zaneShould that work? āļø:skin-tone-2:#2017-05-1522:01zanei.e. Should I be able to (gen/sample (s/gen keyword-conformer))
on that?#2017-05-1522:03Alex Miller (Clojure team)yes#2017-05-1522:03Alex Miller (Clojure team)works for me#2017-05-1522:07zaneMight be my version of clj, then.#2017-05-1522:08zaneI have several other specs like:
(s/def ::example (s/and keyword-conformer #{:some :specific :values}))
#2017-05-1522:08zaneOf course, calling (gen/sample ::example)
fails because it runs out of tries.#2017-05-1522:09zaneI've been working around that with
(s/def ::example
(s/with-gen
(s/and keyword-conformer
#{:some :specific :values})
#(s/gen #{:some :specific :values})))
but that feels gross. Is there a better way?#2017-05-1523:56Alex Miller (Clojure team)(s/def ::example #{:some :specific :values "some" "specific" " "values"}) would work :)#2017-05-1523:57Alex Miller (Clojure team)What do you actually need to spec?#2017-05-1600:30zaneThat the incoming values will be in that set when conformed, and that strings will be conformed to keywords.#2017-05-1600:31zaneI'm trying to make my example more DRY.#2017-05-1600:45Alex Miller (Clojure team)(def vs #{:some :specific :values})
(s/def ::example (into vs (map name) vs))#2017-05-1600:48Alex Miller (Clojure team)That doesn't conform to keywords but transformation is not really the point of spec#2017-05-1606:33odinodindoes anyone know of any efforts that present clojure spec validation errors in a nice way? Iām looking for ideas and inspiration for a web based error message component Iām working on#2017-05-1607:05rmuslimov@odinodin just spent some time looking into the same issue, found this https://www.slideshare.net/alexanderkiel/form-validation-with-clojure-spec and can recommend this approach#2017-05-1607:07odinodin@rmuslimov thanks. I was thinking more about how to present raw clojure.spec validation errors in a nicer way during development#2017-05-1607:09rmuslimovah ok, interesting. Not sure how standard can be improved, may be you already have any ideas - how it may look like?#2017-05-1607:16odinodinJust started digging, not sure where it will lead yet#2017-05-1611:10urbankwhy has cljs.spec been renamed to cljs.spec.alpha?#2017-05-1611:25curlyfryurbank: https://groups.google.com/forum/?__s=f7szr7fg4jw7kzeciy43#!msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ#2017-05-1611:26urbank@curlyfry Thanks!#2017-05-1613:14curlyfryAny news on https://dev.clojure.org/jira/browse/CLJ-2123 or something similar? Having some issues right now with name clashes on specs (that we still want in the same file). We're specing a re-frame db and want to keep the specs for a specific page in the same file. Often parts of specs for input field and similar end up having the same names as other parts of the state. The solution right now is either to create an artificial and verbose namespaced keyword (:my-page.input.domain/id) or to add a new file just for the clashing specs.#2017-05-1613:57stathissideris@curlyfry in a similar vein: when you refactor clashing keywords to have different namespaces, does the rename mean that you get rippling changes throughout your code?#2017-05-1614:23pseudI've got a map where all entries should follow some template (e.g. (s/map-of string? number?)
except one optional key, let's call it :foo
.
I tried something like s/merge
, but it seems like it was designed with multiple s/keys
specs in mind.
(s/merge (s/map-of string? number?) (s/keys :opt-un [::foo]))
.
I can of course write my own predicate function to split the map in two and check against each spec with s/valid?
, but I'll lose a lot of contextual information in case of failures.
Is there no better way ?#2017-05-1615:43Alex Miller (Clojure team)@curlyfry no update#2017-05-1615:43Alex Miller (Clojure team)@pseud you can use the āhybrid mapā technique that I describe at http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-05-1615:44Alex Miller (Clojure team)yours is a bit simpler than the one described there#2017-05-1616:59pseud@alexmiller cool, a bit too tired to digest, but bookmarked.#2017-05-1618:35griersonHey, does (s/exercise) work with CLJS?#2017-05-1618:54Alex Miller (Clojure team)afaik#2017-05-1619:06curlyfry@grierson Yup!#2017-05-1619:19griersondo I need to import anything else apart from cljs.spec?#2017-05-1619:28griersonFixed it. Need to import [clojure.test.check.generators]
. I was only using Spacemacs inline eval I wasnāt looking at the REPL.#2017-05-1714:29chilleniousis there a way to in-line this:
(s/def ::a integer?)
(s/keys :req-un [::a])
so that I can state I expect ::a in the map that should be checked against integer? without having to specify ::a first?#2017-05-1714:30luxbock@chillenious no, you could write a macro to do something like (my-keys :req-un [(::a integer?)])
that expands to it#2017-05-1714:31chilleniousok, thx#2017-05-1714:31moxaj@chillenious that's by design, see https://clojure.org/about/spec - Map specs should be of keysets only
#2017-05-1714:33chilleniousYeah, I was afraid you were going to say that š I can't be the first one who thinks that that's a PITA though. I very often have the situation where I'd like to use a different key from the 'type'.#2017-05-1715:21ikitommichillenious: something like this? https://github.com/metosin/spec-tools/blob/master/README.md#data-specs#2017-05-1715:22chilleniousYes! Thanks!#2017-05-1715:23chilleniousThat looks great, will try it out later this week.#2017-05-1715:27wilkerlucio@chillenious I encourage you to avoid this, there are many reasons for spec to encourage doing things in the way it does, to encourage you to have value definitions and just make compositions out of it, if you start thinking on your data as records you gonna lose a lot of the benefits on the long run, I recommend watching this presentation from Rich, where he talks a lot about the grand ideas and why things are the way they are: https://vimeo.com/195711510#2017-05-1715:28chilleniousI'll look at it, thanks. I'm using this as part of test code where I want some validation and documentation as to what data structures can/ should be used. So it's as important to me that it is easy to read and terse, and doesn't have to be a perfect water tight solution for all things spec š#2017-05-1715:19wilkerluciohello š#2017-05-1715:20wilkerlucioI noticed when a spec is created with (s/with-gen)
it loses track of the original form:#2017-05-1715:20wilkerlucio(s/def ::some-spec (s/with-gen int? #(s/gen int?)))
=> :user/some-spec
(s/form ::some-spec)
=> :clojure.spec.alpha/unknown
#2017-05-1715:21wilkerluciobut doing it using the (s/def (s/spec int? :gen #(s/gen int?)))
works fine#2017-05-1715:22wilkerlucio(s/def ::other-spec (s/spec int? :gen #(s/gen int?)))
=> :user/other-spec
(s/form ::other-spec)
=> clojure.core/int?
#2017-05-1715:22wilkerluciois this expected?#2017-05-1717:37Alex Miller (Clojure team)thereās a patch pending to fix this#2017-05-1717:40Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2068#2017-05-1719:07gphilippThere was a question last year about speccing a binary tree: (def BinaryTree
(maybe ;; any empty binary tree is represented by nil
{:value long
:left (recursive #āBinaryTree)
:right (recursive #āBinaryTree)}))
#2017-05-1719:08gphilipp@alexmiller: you told Andrey Grin :āYes, you can create recursive definitions by registering a spec that refers to itself via registered name (a namespaced keyword).ā. I canāt find a way to do it, could please post a snippet of code ?#2017-05-1719:08gphilippRelevant message: https://groups.google.com/forum/#!topic/clojure/5sZdCPQgZz4%5B1-25%5D#2017-05-1719:22misha@gphilipp
(s/def :bt/value number?)
(s/def :bt/left (s/nilable :a/bt)) ;;<- recursive
(s/def :bt/right (s/nilable :a/bt)) ;;<- recursive
(s/def :a/bt
(s/nilable
(s/keys
:req-un [:bt/value]
:opt-un [:bt/left :bt/right])))
(s/explain :a/bt
{:value 1
:left {:value 21 :right {:value 31}}
:right {:value 22 :left {:value 32 :right {:value 4} :left {:value 33}}}})
Success!
=> nil
#2017-05-1719:24mishas/exercise
, however, stackoverflows at different points, so might need to tinker with that.#2017-05-1719:29misha(binding [s/*recursion-limit* 2]
(map first (s/exercise :a/bt 3)))
=>
({:value -1.0, :right {:value -1}, :left nil}
{:value -1,
:left {:value -3.0, :left {:value -1, :right {:value 0.75}}},
:right {:value 0}}
{:value 0})
#2017-05-1719:32wilkerlucio@alexmiller I was having this idea yesterday, about spec hierarchy tracking, currently when you define a spec from another (eg: (s/def ::something ::other-thing)
) the resolving happens on definition time, I understand that is preferable for performance reasons, but doing that makes us lose track of the hierarchy. I was implementing a coerce engine on top of spec, and I miss having this hierarchy information, so I was thinking that we could have Clojure hierarchy for the specs, so when defining those we do a derive on those keywords, so we would have this information aside available for use, do you think this is a reasonable feature to have on spec?#2017-05-1719:36Alex Miller (Clojure team)I donāt know about all that. there are some open tickets about the resolution aspects that I expect will be addressed.#2017-05-1719:36Alex Miller (Clojure team)it is not the intent that you lose that tracking now#2017-05-1719:37wilkerluciofor example, given the following specs:#2017-05-1719:37wilkerlucio(s/def ::a int?)
(s/def ::b ::a)
(s/def ::c ::b)
#2017-05-1719:37wilkerluciois there a way, starting from ::c
to know that it is derived from ::b
and ::a
?#2017-05-1719:38Alex Miller (Clojure team)(s/get-spec ::c)#2017-05-1719:38Alex Miller (Clojure team)should tell you ::b#2017-05-1719:39Alex Miller (Clojure team)but things like https://dev.clojure.org/jira/browse/CLJ-2067 and https://dev.clojure.org/jira/browse/CLJ-2079 make me think that not everything is exactly right in this area#2017-05-1719:41wilkerluciointeresting, I didn't knew that get-spec
would do that, and testing with the previous example works#2017-05-1719:42wilkerlucioglad to hear that my assumptions were wrong on this#2017-05-1719:42wilkerluciothanks Alex#2017-05-1722:16lvhhow do I make a spec refer to a spec in another ns? I did (s/fdef ::name ::storage/name) but I just get No value supplied for key: ::storage/name#2017-05-1722:23lvhnever mind, Iām an idiot and meant s/def#2017-05-1722:38gphilippThx @misha. Would it be possible to have a fn to create that kind of spec with the 'number?' predicate being passed as a parameter to this fn ?#2017-05-1807:19misha@gphilipp I'd think macro is doable, if you really need to have several BTs. However I'd start with (s/or ...)
with all the variants you want in it, and see if that's enough#2017-05-1814:14wilkerlucioHello, I'm trying to use the overrides
argument on a generator, but I'm not sure how I supposed to use it#2017-05-1814:14wilkerlucioI'm trying like this:#2017-05-1814:15wilkerlucio(s/exercise
(s/keys :req [:some/data
:some/more-data])
5
{:some/data #(gen'/for [i (s/gen (s/int-in 1 1000000))]
(-> i bigdec (/ 100.0M) (* -1)))})
#2017-05-1814:15wilkerluciobut that doesn't work, does someone have an example on how to use it?#2017-05-1814:25Alex Miller (Clojure team)that looks roughly correct, maybe share more of the example?#2017-05-1818:07wilkerlucio@alexmiller a more complete example here:#2017-05-1818:08wilkerluciosorry the noise now, I was looking at the wrong number, A is always even here, saying it's correct, I'll check my other example again#2017-05-1818:12wilkerlucio@alexmiller ok, I think I might have found a bug#2017-05-1818:13wilkerlucioit works fine when the spec is not defined in terms of other spec#2017-05-1818:13wilkerluciohere is an example where the override doesn't work#2017-05-1818:13wilkerlucio(s/def ::common-etc int?)
(s/def ::a ::common-etc)
(s/def ::b int?)
(s/exercise (s/keys :req [::a ::b])
30
{::a #(gen/fmap (fn [x] (* x 2)) (s/gen int?))})
#2017-05-1818:15wilkerlucioif ::a
was defined as int?
it works, but when making it reference other spec, the override version is not used#2017-05-1821:41Alex Miller (Clojure team)Oh, yes there is a ticket for this #2017-05-1900:45cflemingIām playing around with the specs for spec from CLJ-2112, and Iām having some problems with it.#2017-05-1900:46cfleming(s/conform ::spec (s/form 'clojure.core/let))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword clojure.lang.RT.seqFrom (RT.java:547)
#2017-05-1900:46cflemingWhen I try to investigate further, things get strange:#2017-05-1900:47cflemingAm I missing something obvious?#2017-05-1900:49cflemingI understand that the specs there are half-baked, but I didnāt expect to get exceptions back.#2017-05-1900:50cflemingThis is with alpha16, in case that matters.#2017-05-1903:01cflemingHmm, clearly I should be quoting the form Iām passing in, but then I just get back to the IllegalArgumentException#2017-05-1903:12cflemingActually, this might be CLJ-2152#2017-05-1904:11Alex Miller (Clojure team)at the beginning when you do this: (s/form 'clojure.core/let)
- that returns the fspec for let, not sure if you wanted that or the args spec which would be (s/form (:args (s/get-spec 'clojure.core/let)))
#2017-05-1904:13Alex Miller (Clojure team)when you do (s/conform ::spec (clojure.spec.alpha/cat :name ... ))
, youāll need to quote there as the spec is expecting a sequential thing#2017-05-1904:13cflemingRight, but when I quote then I get the exception.#2017-05-1904:14cflemingOne sec, I got sidetracked, starting my REPL againā¦#2017-05-1904:14Alex Miller (Clojure team)Iām actually heading to bed but could load it up tomorrow and look at it#2017-05-1904:15cfleming(s/conform ::spec (s/form (:args (s/get-spec 'clojure.core/let))))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword clojure.lang.RT.seqFrom (RT.java:547)
#2017-05-1904:15Alex Miller (Clojure team)can you (pst *e) ?#2017-05-1904:17cflemingjava.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Keyword
at clojure.lang.RT.seqFrom(RT.java:547)
at clojure.lang.RT.seq(RT.java:527)
at clojure.lang.RT.first(RT.java:683)
at clojure.core$first__6389.invokeStatic(core.clj:55)
at clojure.core$first__6389.invoke(core.clj:55)
at clojure.spec.alpha$multi_spec_impl$predx__912.invoke(alpha.clj:904)
at clojure.spec.alpha$multi_spec_impl$reify__919.conform_STAR_(alpha.clj:916)
at clojure.spec.alpha$or_spec_impl$fn__958.invoke(alpha.clj:1035)
at clojure.spec.alpha$or_spec_impl$reify__963.conform_STAR_(alpha.clj:1057)
at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
at clojure.spec.alpha$conform.invoke(alpha.clj:146)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:748)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:744)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1475)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1483)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1486)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1483)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$re_conform.invokeStatic(alpha.clj:1610)
at clojure.spec.alpha$re_conform.invoke(alpha.clj:1601)
at clojure.spec.alpha$regex_spec_impl$reify__1340.conform_STAR_(alpha.clj:1651)
at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
at clojure.spec.alpha$conform.invoke(alpha.clj:146)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:748)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:744)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$multi_spec_impl$reify__919.conform_STAR_(alpha.clj:917)
at clojure.spec.alpha$or_spec_impl$fn__958.invoke(alpha.clj:1035)
at clojure.spec.alpha$or_spec_impl$reify__963.conform_STAR_(alpha.clj:1057)
at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
at clojure.spec.alpha$conform.invoke(alpha.clj:146)
at clojure.spec.specs$eval3133.invokeStatic(form-init2127301951087080660.clj:1)
at clojure.spec.specs$eval3133.invoke(form-init2127301951087080660.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6977)
#2017-05-1904:19cflemingI suspect this is related to CLJ-2152, Metosin in a blog post said that their spec walker/visitor didnāt work with s/&
or s/keys*
because of it.#2017-05-1904:19cflemingBut I may be off there.#2017-05-1904:19cflemingFrom your test cases in that patch, I get this:
(s/conform ::spec (s/form (s/keys* :req [::foo])))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword clojure.lang.RT.seqFrom (RT.java:547)
#2017-05-1904:20Alex Miller (Clojure team)that code had a number of broken cases due to form - weāve fixed most of them since but there are a few known problems still#2017-05-1904:20Alex Miller (Clojure team)s/&, s/keys* in particular#2017-05-1904:21Alex Miller (Clojure team)I guess some of the nested specs in let bindings could be getting tripped up#2017-05-1904:21cflemingAnd I guess s/cat
uses s/keys*
under the hood?#2017-05-1904:22cflemingIāll check that.#2017-05-1904:22Alex Miller (Clojure team)nah#2017-05-1904:23Alex Miller (Clojure team)but the specs for s/cat could use it#2017-05-1904:23Alex Miller (Clojure team)anyhow, that ticket is incomplete because itās a wip and there are many things that donāt work with it yet so it doesnāt surprise me that things donāt work#2017-05-1904:24Alex Miller (Clojure team)rather than just having it sit on my hard drive I figured it was better to be a wip on a ticket#2017-05-1904:24cflemingSure, is there a workaround I could use? It doesnāt seem like a problem with the specs themselves, but more likely with form
#2017-05-1904:25Alex Miller (Clojure team)I donāt think it is - it looks like (s/form (s/keys* :req [::foo]))
returns the right thing#2017-05-1904:25Alex Miller (Clojure team)sorry, grabbed wrong code there#2017-05-1904:25Alex Miller (Clojure team)(s/form (:args (s/get-spec 'clojure.core/let)))
#2017-05-1904:25cflemingI was going to say - pretty sure an exception isnāt the right thing š#2017-05-1904:26Alex Miller (Clojure team)I get (clojure.spec.alpha/cat :bindings :clojure.core.specs.alpha/bindings :body (clojure.spec.alpha/* clojure.core/any?))
for that, which seems good#2017-05-1904:26Alex Miller (Clojure team)the spec specs should only need to have working s/cat and s/* for that to conform#2017-05-1904:27cflemingI get the same, but the conform
fails with that exception.#2017-05-1904:28Alex Miller (Clojure team)well, Iām too wiped to take it any further atm#2017-05-1904:28Alex Miller (Clojure team)will take a glance tomorrow#2017-05-1904:28cflemingIt seems like s/&
is the root cause - if I macroexpand (s/keys* :req [::foo])
I get:
(let* [mspec# (s/keys :req [:clojure.spec.specs/foo])]
(s/with-gen (s/& (s/* (s/cat :clojure.spec.alpha/k keyword? :clojure.spec.alpha/v any?))
:clojure.spec.alpha/kvs->map
mspec#) (fn [] (clojure.spec.gen.alpha/fmap (fn [m#] (apply concat m#)) (s/gen mspec#)))))
#2017-05-1904:28cflemingAnd itās failing on the :clojure.spec.alpha/kvs->map
#2017-05-1904:28cflemingOk, no problem - thanks for the help.#2017-05-1904:29cflemingIām using the code from the patch, only updated to use the new alpha ns.#2017-05-1904:29Alex Miller (Clojure team)the error message looks like the ::spec multi-spec is using the dispatch function first on a keyword#2017-05-1904:30Alex Miller (Clojure team)so maybe itās just that ::spec needs to be expanded to include keywords as valid specs#2017-05-1904:31cflemingOk, Iāll see if I can get my head around that.#2017-05-1904:31Alex Miller (Clojure team)(s/def ::spec
(s/or :set set?
:pred symbol?
:registered-spec keyword?
:form (s/multi-spec spec-form (fn [val tag] val))))
#2017-05-1904:31Alex Miller (Clojure team)I probably just didnāt test that case#2017-05-1904:32Alex Miller (Clojure team)I was spending most of my time fixing s/form bugs :)#2017-05-1904:32Alex Miller (Clojure team)anyhow, good nightā¦#2017-05-1904:32Alex Miller (Clojure team)for real!#2017-05-1904:32cflemingLooks good - thanks!#2017-05-1907:49seantempestaProbably a dumb question, but when I use s/describe
Iām getting back a list in the form of (keys :req [:person/first-name :person/last-name] :opt [:person/contact])
. Is there an easy way to convert this to a map? Is it hacky to use nāth to access the required and optional fields?#2017-05-1912:37pseud@seantempesta I really wouldn't worry about that. If you write a single function to handle the extraction of what you want into a data structure suitable to your needs you only have a single function to refactor should the world change...#2017-05-1912:37Alex Miller (Clojure team)Well the prior conversation here is about specs for specs which allow you to conform a spec form into a map#2017-05-1912:37Alex Miller (Clojure team)But still a wip#2017-05-1919:06arohnerthis looks like a bug to me:
(s/conform (s/keys :req-un [:bogus/bogus]) {:bogus "foo"}))
#2017-05-1919:07arohner1) define an s/keys, without defining :bogus/bogus. Thereās no warning/error that :bogus/bogus is undefined, and the spec conforms#2017-05-1920:41Alex Miller (Clojure team)agreed, file a jira#2017-05-1920:42Alex Miller (Clojure team)although as I try other things, this is also the behavior with :req#2017-05-1920:42mishahow is this bug, when key specs are opt in?#2017-05-1920:43mishaand s/keys
only specifies keys set. why would it complain?#2017-05-1920:43Alex Miller (Clojure team)āThe validator will ensure the :req keys are present.ā#2017-05-1920:44Alex Miller (Clojure team)from the s/keys docstring#2017-05-1920:44mishabut :bogus
is present#2017-05-1920:44Alex Miller (Clojure team)Iām agreeing with you#2017-05-1920:45Alex Miller (Clojure team)ākeys ā¦ are required, and will be validated and generated by specs (if they exist)ā#2017-05-1920:45Alex Miller (Clojure team)so, Iāve changed my mind - I think itās doing what it says will do#2017-05-1920:45misha#2017-05-1920:48mishafor a minute I was questioning 2/3 of specs I wrote#2017-05-1923:34jfntnWhat would be a good way of and-ing multiple specs without the conforming behavior of s/and
?#2017-05-1923:58seantempestaGood point. Thanks!#2017-05-2019:05borkdudeIs it possible to express the following in spec: f accepts a function g from string? to int? and it is also possible to generate random instances of g?#2017-05-2019:07borkdudeI wondered about this after playing with Quickcheck in Haskell, which can do this#2017-05-2019:25luxbock@borkdude (s/fdef f :args (s/cat :g (s/fspec :args (s/cat :str string?) :ret int?)))
#2017-05-2019:47borkdude@luxbock Looks good. How about the generational aspect?#2017-05-2020:40Alex Miller (Clojure team)Gen of fspec is a function that uses its ret generator#2017-05-2020:46stathissiderisHow would I register (and then use) specs that were generated dynamically? I think Iām at the stage where I need to start property testing spec-provider. To do this I need to generate random data (possible), infer specs for them (thatās what the lib does) and then validate that all the random data are valid for those inferred specs.#2017-05-2022:22Alex Miller (Clojure team)Well you can either call s/def and register your specs or you can just call conform on the specs directly without registering, which works for a lot of stuff#2017-05-2108:47stathissideris@alexmiller thanks. But Iām generating the forms of the specs, so I guess Iād have to eval
them in any case, right?#2017-05-2112:49Alex Miller (Clojure team)Yes#2017-05-2118:44stathissideris@alexmiller great, makes sense, thanks#2017-05-2217:34nullptrtomorrow is the 1y anniversary of the clojure.spec announcement!
https://clojure.org/news/2016/05/23/introducing-clojure-spec#2017-05-2219:07mobileinki have a bunch of map specs, each respresenting a "resource type". i want to present them on a web page. is s/form the way to go? is there a library that makes it easy to get and send a json/edn respresentation of a map spec from the registry?#2017-05-2309:24ikitommimobileink: spec visitor should help: https://github.com/metosin/spec-tools/blob/master/README.md#spec-visitors#2017-05-2221:30Alex Miller (Clojure team)s/form will be fully resolved, s/describe will be more terse#2017-05-2221:30Alex Miller (Clojure team)Stuff in there should be edn afaik?#2017-05-2304:16danielcomptonWhat is the right way to spec functions that can throw exceptions? It seems like I need to allow :ret
to be nil
to handle exceptions here#2017-05-2316:11Alex Miller (Clojure team)If there is an exception there is no return value#2017-05-2316:12Alex Miller (Clojure team)Exceptions are not covered by spec#2017-05-2316:12Alex Miller (Clojure team)Because they are ... exceptional cases#2017-05-2317:45deplecthai all, how do we handle ordering with spec?#2017-05-2317:57Alex Miller (Clojure team)youāll need to explain more, not sure what you mean#2017-05-2318:06caio(s/and sorted? ...)
?#2017-05-2318:07caio(not the clojure.core/sorted?
)#2017-05-2318:26andrewmcveighIs this expected? (let [n 2
t int?]
(s/conform (s/coll-of t :into [] :kind vector? :count n) [1 2]))
=> [1 2]
(let [n nil
t int?]
(s/conform (s/coll-of t :into [] :kind vector? :count n) [1 2]))
=> :clojure.spec.alpha/invalid
(let [t int?]
(s/conform (s/coll-of t :into [] :kind vector? :count nil) [1 2]))
=> [1 2]
#2017-05-2318:33andrewmcveighI guess this line is responsible. https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L541#2017-05-2318:34andrewmcveighI just wonder if this is a bug, or if the s/every
macro is meant to check the 'n
symbol rather than what it evals to.#2017-05-2319:19Alex Miller (Clojure team)I think Iād call it a bug, given the docstring states that nil is the default which seems to imply you could pass it#2017-05-2319:19Alex Miller (Clojure team)feel free to file a jira#2017-05-2319:53andrewmcveighOk, will do. Thanks#2017-05-2415:28weavejesterShould instrument
report the name of the function that failed in the ex-data
?#2017-05-2415:29weavejesterIt reports the function name in the exception message, but itās missing from the exceptionās data.#2017-05-2415:37Alex Miller (Clojure team)most likely yes#2017-05-2415:37Alex Miller (Clojure team)example would help, but feel free to file something about itk#2017-05-2415:43Alex Miller (Clojure team)@weavejester just looked at an example - definitely yes#2017-05-2415:43weavejesterAh, I was just about to post one š#2017-05-2415:43weavejesterShall I add a JIRA report, @alexmiller ?#2017-05-2415:47Alex Miller (Clojure team)that would be great#2017-05-2415:47Alex Miller (Clojure team)if you patch it, I will screen it too :)#2017-05-2415:48Alex Miller (Clojure team)btw, enjoyed your defn podcast episode#2017-05-2415:49weavejesterAh thanks š#2017-05-2416:04weavejester@alexmiller I created https://dev.clojure.org/jira/browse/CLJ-2166#2017-05-2416:06Alex Miller (Clojure team)I assume this is obvious, but patch will be against the spec.alpha lib, not core clojure repo#2017-05-2416:18weavejesterSorry, I couldnāt find the spec.alpha lib in the project list when I created the issue.#2017-05-2416:39stathissiderisis there any support for string keys in s/keys
? or any plan to support them later?#2017-05-2417:44ghadino @stathissideris -- s/keys
needs things that are globally registered (keywords/specs)#2017-05-2417:52stathissideris@ghadi thanks#2017-05-2418:05Alex Miller (Clojure team)@weavejester you filed it in the right place - spec issues are being managed in CLJ, just meant that the patch would be on a different repo#2017-05-2418:06weavejesterAh, cool.#2017-05-2418:07jfntnWhatās an idiomatic way to spec args for a multi-arity function where the longer arities only add new arguments to the shorter ones? Say the full args vector is [a b c ...]
but you have aritites with defaults that only take [a]
, or [a b]
etc?#2017-05-2418:12stathissideris@jfntn I think you either do (s/or :arity1 (s/cat ..) :arity2 (s/cat ...) :arity3 (s/cat))
or (s/cat x y (s/? w z))
#2017-05-2418:12stathissiderisnot sure which is more idiomatic#2017-05-2418:12stathissiderisactually itās not s/?
#2017-05-2418:14jfntnAh that works: (s/valid? (s/cat :a string? :b string? :c (s/? string?)) ["a" "b"])
#2017-05-2418:14stathissiderisyeah, but I think s/?
is for a single predicate#2017-05-2418:14jfntnWas looking to remove the duplication from s/or
so I think thatās what Iām after#2017-05-2418:15jfntnSeems to do what I expect: (s/explain (s/cat :a string? :b string? :c (s/? string?) :d (s/? string?)) ["a" "b" "c" :d])
#2017-05-2418:18stathissideris@jfntn ok, but if for some reason you had arities 2 and 4, youād say#2017-05-2418:18stathissideris(s/cat :a string? :b string? :rest (s/? (s/cat :c string? :d string?)))
#2017-05-2418:19stathissideriswhich is valid for ["a" "b"]
and ["a" "b" "c" "d"]
but not for 3 strings#2017-05-2418:19jfntnnice#2017-05-2418:19stathissideristhis works because the nested s/cat
does not imply a nested sequence#2017-05-2418:20stathissiderisif you want nesting, you need to wrap it with a s/spec
:#2017-05-2418:20stathissideris(s/valid? (s/cat :a string? :b string? :rest (s/? (s/spec (s/cat :c string? :d string?)))) ["a" "b"])
#2017-05-2418:20stathissideris(s/valid? (s/cat :a string? :b string? :rest (s/? (s/spec (s/cat :c string? :d string?)))) ["a" "b" ["c" "d"]])
#2017-05-2418:22jfntnthanks @stathissideris#2017-05-2418:27stathissiderisnp š#2017-05-2419:36stephenmhopperHi everybody, whatās the best way to use clojure.spec with Clojure 1.8?#2017-05-2419:36stephenmhopperIs there a clojar for that?#2017-05-2419:38seancorfieldhttps://github.com/tonsky/clojure-future-spec#2017-05-2419:38stephenmhopper@seancorfield thank you!#2017-05-2420:53mattlyis there any way to dynamically provide arguments from a collection to spec/or
? It's a macro so I can't use apply
#2017-05-2420:56donaldballmattly: I have used eval
for similar purposes:
(doseq [[spec-kw field-spec desc] fields]
(eval `(s/def ~spec-kw ~(build-internal-spec field-spec))))
#2017-05-2420:57mattlyyeah, the individual specs I haven't had problems with, what I need is to validate that a value in a map conforms to one of possibly 100 specs#2017-05-2420:59donaldballWell, a way you could do that is to use eval
to build the s/or
form you want#2017-05-2421:00mattlyhm, thanks#2017-05-2421:11arohneris there a way to get the default generator, so I can update it?#2017-05-2421:12arohner(s/with-gen :foo #(gen/fmap update (s/gen :foo)))#2017-05-2421:23arohnerI guess I can (s/def :foo* ā¦) (s/def :foo (s/with-gen :foo* ...))
#2017-05-2421:27Alex Miller (Clojure team)Yeah, having a shorthand for this case is something I think would be useful#2017-05-2421:27Alex Miller (Clojure team)It comes up regularly #2017-05-2522:21jfntnAre recursive specs not supported in Clojurescript? I have (s/def ::parent ::self) (s/def ::self ...)
failing at the first line.#2017-05-2523:53donaldballHas anyone written a macro to combine defn
and s/fdef
yet?#2017-05-2600:15Alex Miller (Clojure team)@donaldball I think maybe @gfredericks did?#2017-05-2600:15Alex Miller (Clojure team)@jfntn not sure#2017-05-2600:16jfntn@alexmiller the workaround was to move the self spec before the parent#2017-05-2600:17Alex Miller (Clojure team)hmm, fails on Clojure too - I think this is related to a couple other existing tickets where s/def chases the spec too early#2017-05-2600:43gfredericksI wrote a defn macro that dispatches on specs matching; it didn't make an fdef#2017-05-2616:04wilkerluciohello people, I wanna share with you a snippet that I wrote here to find bad generators on your project, it's helping me a lot (specially when you have a s/keys
filled with items and don't know which of then is not generating)#2017-05-2616:04wilkerlucio(let [keys (->> (s/registry)
keys
(filter #(and (qualified-keyword? %)
(not (contains? #{"clojure.core.specs.alpha"
"clojure.spec.alpha"} (namespace %))))))]
(doseq [k keys]
(try
(doall (s/exercise k))
(catch Exception e
(println "Bad generator for" (pr-str k))))))
#2017-05-2616:04wilkerluciothis will print Bad generator for :some/key
for the generators that are not being able to find a value after 100 tries#2017-05-2616:05wilkerlucioI hope it may help you as it is helping me š#2017-05-2617:33bbsscool!#2017-05-2617:36bbssI'm having an issue generating date-ranges, getting clojure.test.check.rose_tree.RoseTree cannot be cast to
clojure.test.check.rose_tree.RoseTree
#2017-05-2617:36ghadilooks like an AOT compilation problem ^#2017-05-2617:36bbssit's in cljc#2017-05-2617:36bbssis that something that could be causing it?#2017-05-2617:37ghadino -- it has to do with whether you have superfluous *.class
files present, and whether you've require/reloaded
stuff#2017-05-2617:37bbssokay, then it might be because I hotloaded#2017-05-2617:37ghadicalling @hiredman batsignal#2017-05-2617:37bbsslet me check that, thanks#2017-05-2617:38ghadireloading things will redefine some classes. Those classes will have the same name as existing classes, but will be !=
from the JVM's perspective#2017-05-2617:38bbssmakes sense#2017-05-2617:39ghadithus instances of RoseTree(1) are not instances of RoseTree(2)#2017-05-2617:41hiredmanhmmm?#2017-05-2617:43hiredmanaot drowns kittens in a sack in the river (the kittens are a metaphor for your hopes and deams)#2017-05-2617:47bbsshmm, now I'm running into issues with clj-time.format not being found.#2017-05-2617:48bbssWhich the .coerce namespace depends on, which is weird because when I used the function before I restarted it worked.#2017-05-2617:49bbssIt's really the only kitten that sometimes needs to claw itself to fresh air once in a while.#2017-05-2617:49bbss(metaphor for the only thing that sometimes gets to me with cljx)#2017-05-2618:00ghadiyou're using cljx?#2017-05-2618:00ghadi@bbss#2017-05-2618:01bbss@ghadi is that not short for clojure(script) ?#2017-05-2618:01bbssOh right, it's the pre-conditional reader library#2017-05-2618:01ghadithere was an earlier library called cljx that did something similar to cljc#2017-05-2618:01bbsssorry, it's cljc#2017-05-2618:02bbssI meant to say, the only thing that I find really annoying at times is dependency issues like these. It takes by far the most of my time out of all time-taking things that aren't normal debugging/coding.#2017-05-2618:42bbsslein clean
and lein deps
solved it#2017-05-2618:42bbssthanks for saving my kittens š#2017-05-2622:22seancorfieldFor folks doing programmatic decoding of explain-data
to produce nice messages, it seems that required keys now (in 0.1.123) produce a :pred
of (clojure.core/fn [%] (contains? % :the-key))
whereas they used to produce (contains? % :the-key)
ā this broke a lot of our decoding logic so I figured Iād mention it here in case it helps other debug similar breakageā¦#2017-05-2622:34arohnergiven an (s/keys :opt-un [::foo])
, what is the best way to generate a map with foo always in it? I occasionally see test failures in CI Couldn't satisfy such-that predicate after 10 tries
, when I do (gen/such-that :foo (s/gen ::the-map))
#2017-05-2700:37athosHappy to hear a new version of spec.alpha has been released!! I've been waiting for CLJ-2059 and CLJ-2085 to be merged š#2017-05-2700:43athosHmm, the version in CHANGES.md looks wrong. 0.1.123 is correct, right? https://github.com/clojure/spec.alpha/blob/master/CHANGES.md#version-01109-on-may-26-2017#2017-05-2701:04seancorfieldYes, 0.1.123 is the correct version.#2017-05-2701:04seancorfieldAlex explained there was a build snafu and they had to re-release 0.1.109 under a new version number.#2017-05-2701:05seancorfieldSo itās technically both 0.1.109 and 0.1.123 š#2017-05-2701:14athosI see š¤#2017-05-2701:27Alex Miller (Clojure team)Oh, I will update the changes file - I forgot to do that #2017-05-2701:30Alex Miller (Clojure team)Fixed!#2017-05-2702:02gfredericks@arohner what about generating it from a spec where that key is required?#2017-05-2702:22arohner@gfredericks thatās not a terrible idea. I was hesitant because āin the real worldā, the key is optional, and I only needed it required for a specific test case#2017-05-2702:23arohnerI guess thatās fine. (s/def ::foo...)
(s/def ::foo-with-bar...)
#2017-05-2702:23gfredericks@arohner these are unit tests? #2017-05-2702:23arohneryes#2017-05-2702:23arohner::foo is used in production, ::foo-with-bar would only be used in tests#2017-05-2814:44bbssI'm developing web stuff in cljc with spec and loving it. It's so great to start out with some specs and to generate sample data and build out your front-end from there.#2017-05-2814:45bbssthey can double as react-props with {pre [(valid? spec state)]}
#2017-05-2814:48bbssone thing I'm hitting now is I'd like to generate ids so I can give those as unique react keys, which is necessary when react displays lists. Since I will eventually replace the sample data ids with :db/id
from datascript, I figured I can add uuid as spec pred and have that generate unique id's. But that doesn't match what datascript generates for ids.#2017-05-2814:49bbssthose are just ints. Is there a way to generate unique ints?#2017-05-2815:14bbssI think I'll have to write my own generator for that#2017-05-2815:15bbssI've tried with s/or
, which would work for the validation but it also generates random (non unique) ints. As it should of course š#2017-05-2912:38tianshuwhy spec become spec.alpha?#2017-05-2913:18robert-stuttaford@doglooksgood thereās a post about it in the google group#2017-05-2914:13tianshubut is there any problems with current version of clojure.spec? any feature is missing or critical bug?#2017-05-2914:15mpenetit depends#2017-05-2914:17mpenetI don't think there are "critical" bugs, but there are a few bugs left to fix. missing features: certainly, but that's quite subjective, depends on what you need#2017-05-2914:17mpenetcheck https://dev.clojure.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CLJ+AND+resolution+%3D+Unresolved+AND+fixVersion+%3D+%22Release+1.9%22+ORDER+BY+priority+DESC&mode=hide#2017-05-2916:37bbssI don't understand why generating this gives stack overflows:
(s/def ::thing string?)
(s/def ::test (s/+ (s/cat :thing (s/* ::thing))))
#2017-05-2916:37bbssmy use case is something like this:
(s/def ::thing string?)
(s/def ::other-thing string?)
(s/def ::test (s/+ (s/cat :thing (s/* ::thing)
:other (s/* ::other-thing))))
#2017-05-2916:45wilkerlucio@bbss when using a s/cat
inside of another you must wrap it with s/spec
#2017-05-2916:45wilkerluciotry this:#2017-05-2916:45wilkerlucio(s/def ::thing string?)
(s/def ::other-thing string?)
(s/def ::test (s/+ (s/spec (s/cat :thing (s/* ::thing)
:other (s/* ::other-thing)))))
(s/exercise ::test)
#2017-05-2917:07bbss@wilkerlucio thank you, missed that from the docs!#2017-05-3012:18gardnervickersI have a (s/cat ...)
spec form that I am using for parsing a seq with s/conform
. I want to make this extensible through a multi-spec
. Is there a regex op that can help me here? Perhaps using the dispatch key on the multi-spec
to tag conformed values?#2017-05-3013:27mishais it ok to actually rely on order of s/or
sub-specs?#2017-05-3016:19don.dwoske@stuartsierra I would love to see the clojure.spec definitions used in your recent article : http://blog.cognitect.com/blog/2017/4/6/developing-the-language-of-the-domain We are going through a very similar process now. Your clojure spec and tools, web app to visually explore the models, et. al. are things I wish I had right now. The business analysts are trying to use Avro specifications to describe the domain model, and are immediately hitting limitations of what can be expressed. I am developing a clojure DSL similar to the one in your article to describe our domain, which I can propose as an alternative. Getting my hands on anything you can share would give me a great jumpstart.#2017-05-3017:34stuartsierra@don.dwoske I'm sorry, I do not have permission to share any of that work beyond the examples in the article.#2017-05-3017:36don.dwoske@stuartsierra Sorry to hear that. I enjoyed the article nonetheless, it gave me some good ideas.#2017-05-3017:38stuartsierrathanks, glad it helped#2017-05-3109:27flyboarderhttps://medium.com/degree9/data-validation-schema-spec-5547e33596bd#2017-05-3113:43mishais there a way to highjack a "bottom" component of composed spec to avoid manually writing 2 sets of specs?
use case would be: a) spec for datomic entity with both temp and actual ids in refs; b) and the "same" spec, but with actual ids only?
(s/def :datomic/temp-id neg-int?)
(s/def :datomic/actual-id pos-int?)
(s/def :datomic/id (s/or
:actual-id :datomic/actual-id
:temp-id :datomic/temp-id))
(s/def :datomic/actual-ref (s/map-of #{:db/id} :datomic/actual-id))
(s/def :datomic/ref (s/map-of #{:db/id} :datomic/id))
(s/def :foo/bar :datomic/actual-ref) ;; actual goal is to replace `actual-ref` with `ref` here, and still be able to use both `:entity/foo` specs in different situations
(s/def :entity/foo (s/keys :req [:foo/bar]))
;; is there a way to re-use :entity/foo, and avoid writing following:
(s/def :foo/bar* :datomic/ref)
;^^^ I don't even know how to keep attribute name, but change underlying spec
(s/def :entity/tepm-foo (s/keys :req [:foo/bar*]))
#2017-05-3114:00danielnealMaybe you could use macros to generate both sorts of specs according to a macros convention#2017-05-3114:05misha@danieleneal the fact, that you can't have same spec-name (:foo/bar above) with a 2 different specs under it at the same time, makes me think I need to somehow hot-swap the bottom (:datomic/ref <--> :datomic/actual-ref) spec definition on-demand. Which, given global nature of registry, might not behave/scale well in e.g. treaded env.#2017-05-3114:07danielnealah so you don't want :foo/bar* to exist ever, even if it is autogenerated#2017-05-3114:10mishaI'd want at least something like
(let [actual (with :entity/temp-foo {:datomic/ref :datomic/actual-ref})] ;; swaping `ref` with `actual-ref` inside :entity/temp-foo
(s/valid? actual {...}))
#2017-05-3114:14danielnealah ok, I don't know how to do that#2017-05-3114:15misha@danieleneal I don't know yet. But at least one of the problem with duplicating entire "tree" of :entity/tepm-foo
here, is you can't have 2 different specs for the same attribute at the same time (AFAIK), so you are forced to do s/or
, and then, in app code, go through conform data and see which or
branch attribute conformed to.
So you can't just use 2 simple (s/valid? :entity/maybe-temp-foo entity)
and (s/valid? :entity/only-actual-foo entity)
calls in different places.
You need to conform, and search for those or
branches with custom walkers.#2017-05-3114:16misha+ this is a trivial example, spec tree might be tens of levels deep (imagine nested datomic pull result, where some nodes are pulled, and some are just {:db/id 9999})#2017-05-3114:19mishabut maybe (re-)defining specs on the fly on-demand is ok. Is it, @alexmiller?#2017-05-3114:35Alex Miller (Clojure team)I don't think it's a good idea#2017-05-3114:36Alex Miller (Clojure team)If you want non-conforming s/or, then wrap s/nonconforming around it#2017-05-3114:36Alex Miller (Clojure team)Or maybe it's s/non-conforming, can't remember#2017-05-3114:37Alex Miller (Clojure team)The best way to write a spec is to make true statements about your data#2017-05-3115:16misha@alexmiller how would you go about speccing same entity for validating against these at the same time:
a) being actual entity (containing no datomic temp ids), and
b) being to-be-transacted entity (maybe containing some temp ids)?#2017-05-3119:31seancorfield@misha This feels to me like you might be over-specifying your data. If the difference between actual-ref and ref doesnāt matter in most cases, then donāt specify it at all. When and where it actually matters, have a separate spec that can validate you have the one you want.#2017-05-3119:31seancorfieldDeeply nested specs feel like an anti-pattern to me (based on my usage of spec so far in production).#2017-05-3119:34seancorfieldWhen we first got started with spec, I found myself trying to declare every aspect of every known key in a data structure and soon realized that got in the way of writing generic functions across variants of that data. It took a while to settle into a flow of only specifying what was important for the places where specs were being used to help drive the system.#2017-05-3119:38dpsuttonlike unit tests: don't shoot for 100% coverage, shoot for important constraints?#2017-05-3121:21misha@seancorfield @dpsutton yeah, I am sure such questions are a part of learning and calibration process, on the other hand, there might have been a "function for that" after all#2017-05-3121:25mishaMany of existing things still surprise me, and I'd like to avoid prematurely limit myself in approaches/tools ā we are in lisp world after all! :)#2017-05-3121:37jjttjjif I have a set of possible usernames ["fred" "tom" "mary"]
and a set of possible mail host domains ["" ""]
, how do I combine them into a spec generator to make strings like <mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>#2017-05-3121:45misha@jjttjj clojure.test.check.generators/let
https://clojure.github.io/test.check/clojure.test.check.generators.html#var-let#2017-05-3123:59Alex Miller (Clojure team)@jjttjj Use generators of s/tuple to combine other generators into random combinations, then use s/fmap to apply a function to the combinations#2017-06-0100:01Alex Miller (Clojure team)like
(gen/sample
(gen/fmap (fn [[name host]] (format "%
#2017-06-0100:03Alex Miller (Clojure team)@misha Iād echo Seanās advice -either donāt spec the ids, or spec them with an s/or (validation and gen will work better then), or combine an entity spec with an additional id spec if needed in some places, etc.#2017-06-0104:17jjttjjthanks all#2017-06-0106:28stbgzhey all I was wondering if there is a tool around that would take a swagger spec and transform it to a clojure spec#2017-06-0106:29stbgzI am interested in generating test for api specified in swagger#2017-06-0108:34misha@alexmiller is your f/map
approach superior in some way to the following? Or is it just a matter of taste?
;;clojure.test.check.generators/let
(gen/sample
(tgen/let [name (s/gen #{"fred" "tom" "mary"})
host (s/gen #{"" ""})]
(format "%
#2017-06-0109:44griersonHow do I define a spec with a generator? e.g. (spec/def ::letter (spec/and (gen/char-alpha) #(Character/isUpperCase %)))#2017-06-0109:46misha@grierson s/with-gen
#2017-06-0109:49griersonWhen I try to exercise the def, I get an error#2017-06-0109:50griersonWhen I evaluate the def it works.#2017-06-0109:51andrewmcveighI think the spec/gen
in your custom generator is the problem#2017-06-0109:51andrewmcveighIs (gen/char-alpha)
not already a generator?#2017-06-0109:51grierson@andrewmcveigh https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#characters--strings--things#2017-06-0109:52griersonIt works when I (gen/sample gen/char-alpha)#2017-06-0109:52misha(spec/def ::letter (spec/with-gen
(spec/and char? #(Character/isUpperCase %))
#(gen/char-alpha)))
(spec/exercise ::letter)
=>
([\S \S] ...)
#2017-06-0109:53grierson@misha ā¤ļø I didn't need the (spec/gen)#2017-06-0109:54griersonI was trying to generate a generator :@#2017-06-0109:54andrewmcveighš#2017-06-0109:54andrewmcveighwhy not?#2017-06-0113:15ikitommi@stbgz have been waiting for someone to do the json schema -> spec converter too. Would also enable things like server code-gen from swagger-spec to a clojure(.spec) web server. Might help the enterprise adoption.#2017-06-0113:16ikitommithere are spec-tools & speculate which do the spec -> json schema. and spec-swagger for spec -> swagger (only 80% done)#2017-06-0116:11danielnealis there a recursive version s/describe
which returns definitions of any specs referenced in the top level spec? Kinda like a macroexpand-all?#2017-06-0118:49ikitommi@danieleneal - you could use the spec-visitor from spec-tools: https://github.com/metosin/spec-tools#spec-visitors#2017-06-0117:58royalaidSo I am experimenting with setting up specs for http://aleph.io/manifold/deferreds.html#2017-06-0117:59royalaidBecause I want to try and add specs for aleph responses#2017-06-0118:00royalaidPart of the problem is because deferred are used encapsulate async code and unrealized values the only way I can think of to check the spec is to use a wrapper function that unwraps the value and checks the spec#2017-06-0119:33wilkerlucio@grierson recommendation: use s
as the alias instead of spec
, I say that because it's a standard name for everybody, going away from well defined conventions is not a good ideia#2017-06-0119:37grierson@wilkerlucio I was following this guideline http://tonsky.me/blog/readable-clojure/.#2017-06-0119:38wilkerlucio@grierson yeah, this article is a bit contrived, I recommend you check this one from Stuart Sierra: https://stuartsierra.com/2016/clojure-how-to-ns.html#2017-06-0119:43grierson@wilkerlucio Thank you, I will check it out.#2017-06-0119:46wilkerlucioyou'r welcome š#2017-06-0119:48bfabry@grierson that article has the right idea but we make exceptions for very-widely-used libraries. in those cases there's often a shorthand convention rather than following the rule#2017-06-0119:50Alex Miller (Clojure team)@misha one benefit of my approach is that I use only generators built from clojure.spec.gen.alpha, which uses dynamic loading, which means that you donāt need test.check at runtime to load the namespace. other than that, personal preference.#2017-06-0119:52Alex Miller (Clojure team)@danieleneal no, there is not a deep-describe, but itās something weāve talked about adding. I think having spec specs (CLJ-2112) would be a big help for this to generically walk specs to find sub specs, so Iāve kind of filed it behind that one. Another alternative would be to actually support this as some kind of recursive op built into the protocol.#2017-06-0207:42danielneal@alexmiller great! Yeah spec'd specs would be exactly the thing for that - would make it much easier to write a deep describe#2017-06-0222:23xiongtxIs spec supposed to treat undefined keys as āanyā?
(s/def ::foo
(s/keys :req-un [::x ::y ::z])) ;; x, y, z undefined
(s/valid? ::foo {:x 1 :y "abc" :z clj-uuid/+null+}) ;; => true
#2017-06-0223:31english@xiongtx for s/keys
, yes. Presence of the key will always be checked when using req
and req-un
, but conformance is checked only if specs are registered with the same keys#2017-06-0223:50xiongtxIs that a design decision or just an implementation detail?#2017-06-0223:51xiongtxB/c that seems like it opens the possibility of forgetting to define a key spec and s/valid?
still passing in tests#2017-06-0307:41misha@xiongtx if you s/exercise
it will complain about "no spec for key".#2017-06-0307:43mishaIt is design decision, so you could opt in to writing detailed spec, instead of being forced to.#2017-06-0320:50Alex Miller (Clojure team)@tclamb seems like a bug, but would be curious to see the actual value causing the error#2017-06-0320:57tclamb@alexmiller s/exercise
is generating maps with clojure.lang.LazySeq
keys, like {(lazy-seq '(:r)) :r}
#2017-06-0320:58tclambthat passes s/valid?
and s/conform
#2017-06-0320:58tclambI didnāt expect this to be true as well: (identical? (s/conform spec x) x)
#2017-06-0320:59tclambbut I think thatās an artifact of the map value spec being keyword?
instead of e.g. s/cat
#2017-06-0321:04tclambI ran into this speccing a map-of point to tile for the board in a game like scrabble#2017-06-0321:55stbgzhey all I am trying to wirite a spec that would satisfy some json data in which the keys have a shape but are not defined eg
{
"x-identity": "2"
"x-name": " aaa"
"x-[some other string]": "bbb",
}
#2017-06-0321:57stbgzCan I use something like
(s/def ::x-object (s/keys ???)
where I can plugin a spec that describes the shape of the keys namely %(string/starts-with? % "x-")
#2017-06-0321:59stbgzI though about using s/cat
but that implies order of the keys#2017-06-0322:19tclambhow about (s/map-of (s/and string? #(re-matches #"x-.+" %)) string?)
?#2017-06-0322:21stbgz@tclamb yeah just figured that out#2017-06-0322:21stbgzthanks#2017-06-0322:23stbgznow what if I have a json with a mix of well know keys and patterned keys like the ones above eg
{
"name": .. ,
"age":...,
"x-internal"....
"x-created"...
}
#2017-06-0322:25stbgzI am modeling the well-know
part of the spec with s/keys
and the other with s/map-of
however s/merge
doesnāt seem to be working given that the āoptional x...ā could be 0, does merge understand optional?#2017-06-0322:28tclambthe docs sound like s/merge
only works with s/keys
, not s/map-of
#2017-06-0322:28tclambhttps://clojure.org/guides/spec#_entity_maps#2017-06-0322:36stbgzhmm this is harder than I though#2017-06-0322:37stbgzI think I have to drop-down to everkv
#2017-06-0413:10Alex Miller (Clojure team)merge works with any map spec#2017-06-0413:11Alex Miller (Clojure team)But you do probably need to use every-kv to spec tuples if you're not keywordizing the map keys#2017-06-0413:13Alex Miller (Clojure team)@tclamb seems like a bug if you want to file a jira. The identical thing is expected - spec tries to avoid modification if possible for perf.#2017-06-0519:13wilkerlucio@alexmiller hey Alex, can you tell if there is any progress going on about the namespace aliasing for inexistent namespaces? I remember I read something around this in past, to facilitate having alias used just for namespace resolution of keywords#2017-06-0519:13wilkerluciocurrently I have a bunch of those on my code:#2017-06-0519:13wilkerlucio(create-ns 'some.deep.nested.keyword.person)
(alias 'person 'some.deep.nested.keyword.person)
#2017-06-0519:13wilkerluciothis is ok on Clojure, but doesn't work for CLJS#2017-06-0519:14wilkerlucioI was wondering if would be a good idea to add something into the ns
form, to use like this:#2017-06-0519:14wilkerlucio(ns my.ns
(:alias [some.deep.nested.keyword.person :as person]))
#2017-06-0519:14wilkerluciothat would work same way as require
, but without actually requiring that namespace to exist#2017-06-0519:15wilkerluciogiven the reason for that namespace is just organization sake#2017-06-0520:57Alex Miller (Clojure team)No update right now, sorry#2017-06-0520:58Alex Miller (Clojure team)But I believe Rich has some idea that he has not shared with me#2017-06-0520:58Alex Miller (Clojure team)In general I think there is a strong preference not to make ns any more complicated#2017-06-0600:20mfikesI was surprised by the interaction of fspec
and instrument
. I was expecting instrument
to only check for spec'd function arg conformance, but it seems to additionally result in "exercising" passed functions. A gist with a concrete minimal example of what caught me off guard, where, in the last two forms evaluated in the REPL session, you can see arg conformance checking, followed by the function I'm passing being exercised: https://gist.github.com/mfikes/3ac5ca668b299e104490059ad0ccfcca#2017-06-0600:24mfikesPerhaps ::number-sink
should not be defined using fspec
, but instead using ifn?
, as in
(s/def ::number-sink ifn?)
#2017-06-0601:24Alex Miller (Clojure team)@mfikes the idea here is that if you spec a function arg, the only way to determine that the function you actually passed conforms is to exercise it. I have certainly used ifn?
as an alternate when that is not appropriate.#2017-06-0601:55mfikesMakes complete sense, especially if the functions passed are pure. In my use case they were side-effecting, so ifn?
it is. I did like the fact that fspec
acts as good documentation, declaring the required shape of the passed functions precisely.
Here is my use case and context where this came up, if anyone is interested, which might be typical: https://github.com/mfikes/planck/blob/a7116251a62b05d6ecbeeaff1da809d123ff2b4e/planck-cljs/src/planck/socket/alpha.cljs#L11-L12#2017-06-0608:46Alex Miller (Clojure team)itās possible to do BOTH too by declaring the fspec with the detailed spec, then providing a simpler override spec when you call instrument#2017-06-0611:03degIt seems that (sort (gen/sample (s/gen (s/inst-in #inst "1800-01-01" #inst "2199-12-31")) 100))
has a very strong bias to generate dates within milliseconds of Jan 1, 1970.#2017-06-0613:08wilkerluciothanks for the clarifications Alex#2017-06-0613:33gfredericks@deg would you want a uniform distribution between the two dates you gave?#2017-06-0613:48deg@gfredericks Yes. (Definitely for date instances. Probably also for numbers where I've specified a range. OTOH, for unbounded numbers, I agree that clustering around zero is probably best)#2017-06-0614:01gfredericksIt's tricky because test.check wants to be able to generate "simpler" instances when asked#2017-06-0614:02gfredericksSo you're describing a generator that doesn't attempt to do that#2017-06-0614:03gfredericksBackground: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2017-06-0614:33degI hear ya. But, I don't believe that Jan 1, 1970 is an appropriate simplification nucleus for dates. The fact that it happens to have an internal rep of zero is not interesting for nearly all uses or tests of dates. Clustering around now is much more likely to catch interesting simple cases.#2017-06-0614:34danielnealis there something like instrument
that also checks the :ret
value of functions? Just for catching the bad functions I write as close as possible to the problem#2017-06-0614:35degAnd, even for numbers, if I've specified a range, than interesting simple cases are likely to be the two end-points of the range and the mid-point.#2017-06-0614:39zaneWriting your own generators that have that behavior ought to be pretty straightforward.#2017-06-0614:42degSure, but I think this should also be the default behavior.#2017-06-0615:20gfredericks@deg agreed; the unfortunate thing about "now" is it makes the generator nondeterministic#2017-06-0615:48deg@gfredericks But, aren't generators already non-deterministic? I already can't assume that I'll get the same values each time. This change just means that I can't assume the same probability distribution either.#2017-06-0616:00bronsaI might be wrong but I think test.check uses a PNRG for its generators#2017-06-0616:00bronsawhich is deterministic#2017-06-0616:01bronsathat's how you get reproducibility through the test seed#2017-06-0616:15degAh, ok. Then I see the problem.
In any event, the number range case can still be improved. And, for dates, it might make sense to center around 2015 or 2020, rather than 1970.
To pick one (admittedly contrived) reason: it would be crazed for a test-generator tool written today to only test dates before the y2k problem!!!#2017-06-0616:23degDifferent question.... I just realized that s/tuple
needs a vector, and won't validate a list. Why? And, what is an idiomatic way to test for, say, a list of precisely two ints?#2017-06-0616:25degActually, let me rephrase: what is an idiomatic way to test for a list of one int followed by one string?#2017-06-0616:28gfredericks@bronsa correct#2017-06-0617:26Alex Miller (Clojure team)@deg a tuple is inherently positional (indexed) whereas a list is not (it can only be traversed from the beginning).#2017-06-0617:27Alex Miller (Clojure team)but you can use (s/cat :i int? :s string?)
#2017-06-0617:27Alex Miller (Clojure team)regex op specs can be matched against sequential stuff (lists, vectors, seqs, etc)#2017-06-0617:30Alex Miller (Clojure team)@deg there is no set of random dates that will be sensical for anyone. if you want to influence it, you should provide a generator that does so (via s/inst-in
). inst-in admittedly has a poor generator due to its non-uniform distribution. That is a bug (my fault) and should be fixed.#2017-06-0617:33Alex Miller (Clojure team)looks like I never logged that, so I just did https://dev.clojure.org/jira/browse/CLJ-2179#2017-06-0617:49deg@alexmiller Understood. That Jira case nicely sums up all my concerns about inst-in.
I think s/int-in should also be generated uniformly.#2017-06-0617:49Alex Miller (Clojure team)in my head I thought it did! but it does not.#2017-06-0617:49Alex Miller (Clojure team)will fix that as well#2017-06-0617:49degthx#2017-06-0617:50Alex Miller (Clojure team)and I wrote it!#2017-06-0617:50degI'm just starting with spec (looked at it a few months ago, but only seriously these past couple of hours). So far, I like the direction a lot. Great work!#2017-06-0618:02Alex Miller (Clojure team)ok, patch attached that fixes both, hopefully will get a look at some point#2017-06-0623:14Drew Verleecan a spec be-used as a replacement for de-structuring a function? I know can spec/conform inside your function, but it would be nice if could refer to the symbols directly rather then the map. I suppose even with desctructring you need to refer to the keysā¦#2017-06-0700:07Alex Miller (Clojure team)yes, although due to the performance, I would only recommend doing this in macros (where it happens at compile time)#2017-06-0713:22gfrederickshttps://mobile.twitter.com/gfredericks_/status/872436432459231232#2017-06-0802:03bbrinckIs it expected that maps will match cat
specs?
(s/def ::kv (s/cat :k keyword? :v any?))
(s/explain ::kv {"foo" "bar"})
;; In: [0] val: ["foo" "bar"] fails spec: :radiator-react-native.error-messages-test/kv at: [:k] predicate: keyword?
#2017-06-0802:04bbrinckI find it a little confusing because my original data doesnāt contain ["foo" "bar"]
anywhere#2017-06-0802:25Alex Miller (Clojure team)regex ops match seqable things. maps are seqable and return 2-tuples of k and v#2017-06-0802:25Alex Miller (Clojure team)user=> (first {"foo" "bar"})
["foo" "bar"]
#2017-06-0802:25Alex Miller (Clojure team)^ your data does contain [āfooā ābarā] when viewed as a seq#2017-06-0802:29bbrinckCorrect me if Iām wrong, but maps are intended to be unordered, right? And regexp ops are about specifying an order? When might I use a regex to validate a map?#2017-06-0802:30bbrinckI suppose if I have a map with zero or one pair, it is ordered and I could use a regex sequence to describe it.#2017-06-0802:39bbrinckFor context, Iām trying to write a pretty-printer that uses explain-data. One part of the pretty printer is a way to highlight where the invalid data ālivesā in a larger tree of data. This is tricky in this case (and maybe others) because the :val
and :in
no longer make sense in the context of the original data, but itās not clear to me how to unwind this transformation or even figure out it occurred.#2017-06-0802:39bbrinckhttps://gist.github.com/bhb/c0009583ead29e1b0fef80581481cab9#2017-06-0804:24Alex Miller (Clojure team)Iām not saying you should use a regex op to spec a map, just explaining how spec interprets that :)#2017-06-0804:27Alex Miller (Clojure team)In: [0] says itās the 0th element of the collection that is the problem. in a map the elements are kv pairs and its this kv pair itself that is indicating the problem (not the particular key or value). what other āinā value would help find it better?#2017-06-0804:29Alex Miller (Clojure team)I guess you could constrain the type of the collection that works with regex op specs more to only be sequential?. Then you would get a predicate error at the map level.#2017-06-0804:30Alex Miller (Clojure team)Something like: In: [] val {āfooā ābarā} fails spec :ā¦ at [] predicate: sequential?#2017-06-0804:33Alex Miller (Clojure team)oh, the other thing is that in the latest version of Clojure spec (not sure if cljs is synced up but I think so), you get an extra key :value in the explain-data that is the original root value#2017-06-0804:34Alex Miller (Clojure team)^^ @bbrinck#2017-06-0809:31pithyless(spec-tools/explain-data (data-spec/spec ::foo {:foo string?}) {:foo 42})
Unhandled java.lang.IllegalArgumentException
Don't know how to create ISeq from: clojure.lang.Keyword
#2017-06-0809:32pithyless^ spec-tools
seems really useful, but the error messages are useless. Am I missing something? Was this a regression introduced in alpha16?#2017-06-0809:35pithyless^ maybe @ikitommi can offer some advice?#2017-06-0809:48pithylessok, so it works if I use the keyword directly instead.#2017-06-0809:49pithyless(def foo (data-spec/spec ::foo {:foo string?}))
(spec-tools/explain-data foo {:foo 42}) ;; exception
(spec-tools/explain-data :sample.core$foo/foo {:foo 42}) ;; explain-data works
#2017-06-0814:42bbrinck@alexmiller Thanks for the info!
> I guess you could constrain the type of the collection that works with regex op specs more to only be sequential?
Thatās a promising idea. In my own specs, Iāve tried your suggestion and wrapped my cat
specs with (s/and sequential? ,,,)
and that solves the issue.#2017-06-0814:42bbrinckI wonder what the tradeoffs would be for enforcing this at the spec level? AIUI, we already do enforce predicates in some cases namely:
(s/explain (s/keys) :foo) ; val: :foo fails predicate: map?
There may very well be downsides to requiring the value be sequential?
, but a few benefits I can see are:
1. IMHO, the error about not satisfying sequential is clearer to me than an error about a key/value structure.
2. You can get into the case where two maps are equal, but one matches a spec while another does not.
https://gist.github.com/bhb/b617354b22cd0508d14ccbab4e6f5585
Admittedly, this is a carefully constructed spec designed to exploit this issue. Iām not sure itās a practical concern. But the property of āequal values are equally valid for a given specā would seem like a nice invariant to maintain.#2017-06-0814:42bbrinckIām happy to open a JIRA for this, if it is helpful š#2017-06-0814:43Alex Miller (Clojure team)@bbrinck I was suggesting we could build the sequential? check into the regex op spec itself#2017-06-0814:43bbrinckAh, agreed š#2017-06-0814:43Alex Miller (Clojure team)jira is fine#2017-06-0814:43bbrinckOK, will do#2017-06-0814:44Alex Miller (Clojure team)maps (and sets) are unordered and thus at odds with the value prop of the regex op specs which are about sequentially ordered collections#2017-06-0814:44Alex Miller (Clojure team)so it seems like a good idea to catch that up front rather than accidentally allow something that creates a weird error#2017-06-0814:45Alex Miller (Clojure team)if you really wanted it, you could always seq
a map or set before using it#2017-06-0815:04bbrinck@alexmiller Do you find it helpful for me to link to this conversation in the JIRA ticket?#2017-06-0815:08bbrinckIāve created https://dev.clojure.org/jira/browse/CLJ-2183. If Iāve missed anything or created the bug improperly, please let me know#2017-06-0815:55seancorfield"Do you find it helpful for me to link to this conversation in the JIRA ticket?" -- this conversation will be gone in a few days due to the 10,000 message limit in Slack's free tier so I'm not sure you can link to it directly. #2017-06-0816:39bbrinckOh, good point.#2017-06-0816:40bbrinckOK, well hopefully I captured the relevant bits from the conversation in the JIRA ticket.#2017-06-0818:35bmabeyDoes anyone know of existing specs for Avro schemas? I haven't found any on github yet but I figured someone must have done this already.#2017-06-0818:37bmabeyI'm starting to create some now and it is pretty straight forward but I'm not sure how to make a spec conditional on other values in a map. In this case the ::default
spec for an avro schema is dependent on the ::type
. https://avro.apache.org/docs/1.8.1/spec.html#2017-06-0819:10dergutemoritz@bmabey Check s/multi-spec
#2017-06-0911:51lambderis there a way to have :gen-max
with s/+
?#2017-06-0911:52lambdere.g. to limit the size of the collection generated by s/+
spec#2017-06-0911:59Alex Miller (Clojure team)currently no, although I think that would be a good enhancement#2017-06-0912:29lambderthanks @alexmiller#2017-06-0912:32Alex Miller (Clojure team)@linuss line 7 - should be s/and ?#2017-06-0912:33linussHahahaha, wow, thanks!#2017-06-0912:34linussthat's a waste of a morning...#2017-06-0912:34Alex Miller (Clojure team)and not a function?#2017-06-0912:35linussyeah, I must have dozed off while writing that#2017-06-0915:17stbgzHey all, I haven been having problems trying to write a spec as follows.
I have a map that looks like this
{
:key1 "val1"
:key2 "
In essence the map is a mix of well know keys(:key1, :key2, :key3) and and optional
set of patterned keys (:x-ā¦). I can easily model either part: (s/keys for the first part and s/map-of for the second part) but I am having a hard time expressing both at the same time. Any ideas on how I can do it?#2017-06-0916:37Alex Miller (Clojure team)this is sometimes called a āhybrid mapā#2017-06-0916:37Alex Miller (Clojure team)you canāt use s/map-of#2017-06-0916:38Alex Miller (Clojure team)but you can use an s/merge of s/keys and an s/every-kv that specs the map entry tuples#2017-06-0916:38Alex Miller (Clojure team)there is a slightly more complicated example outlined in this blog: http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-06-0916:39Alex Miller (Clojure team)search for āhybridā in there to find that part#2017-06-0917:22ikitommi@pithyless hi, and thanks for the report - there was a bug in explain, should be fixed in [metosin/spec-tools "0.2.1-SNAPSHOT"]
(which now uses the latest clj, cljs & spec):
(require '[spec-tools.core :as st])
(require '[spec-tools.data-spec :as ds])
(st/explain-data (ds/spec ::foo {:foo string?}) {:foo 42})
; ::s{:problems ({:path [:foo]
; :pred clojure.core/string?
; :val 42
; :via [:user$foo/foo]
; :in [:foo]})
; :spec #Spec{:form (clojure.spec.alpha/keys :req-un [:user$foo/foo])
; :type :map
; :keys #{:foo}}
; :value {:foo 42}}
#2017-06-0918:15pithyless@ikitommi - Perfect! I can confirm it fixes my issue. Thanks for the lightning-fast response and apologies for not going through GH issues. I figured I was just doing something wrong š#2017-06-0919:13don.dwoskeI'd like to pass parameters into a predicate function along with the value being checked ... a simple example might be a predicate to validate a integer with a min and max value... i.e. validate x is between 1-10. What are common design patterns for this? First thought is to write a function that returns a closure ... any real-world, slick examples of such a thing?#2017-06-0919:26joshkhis it possible to have something like (s/coll-of :some.other/spec)
?#2017-06-0919:29ghadi@don.dwoske there's already some builtins like int-in-range?
#2017-06-0919:30ghadierr rather int-in
#2017-06-0919:44don.dwoske@ghadi - thanks for the example, that's what I was thinking - write macros or functions which return specs or predicates. https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L1623#2017-06-0920:03ghadi@don.dwoske yup. fyi all of clojure.spec is being tracked in a different repository now.#2017-06-0920:03ghadihttps://github.com/clojure/spec.alpha/#2017-06-0920:12don.dwoskeAh - of course. In my defense, whenever I search for docs, I end up here : https://clojure.github.io/clojure/branch-master/clojure.spec-api.html and when clicking on "source" for a function.. I end up where I linked to.#2017-06-0920:31misha@joshkh what do you mean?
(s/def :some/spec int?)
(s/def :another/spec (s/coll-of :some/spec :kind vector?))
(s/exercise :another/spec)
=>
([[0 0 -1 0] [0 0 -1 0]]
[[-1 0 -1 -1] [-1 0 -1 -1]] ...
#2017-06-1014:21stbgzthanks for the help @alexmiller#2017-06-1016:32mikecarterif i have a person
comprising of an id (int) and a name (string), how can i write a spec for a collection of people to ensure there are no duplicate IDs?#2017-06-1016:36potetm@mikecarter distinct?
?#2017-06-1016:38mikecarter@potetm would that catch [{:id 1 :name "Alex"} {:id 1 :name "Frank"}]
?#2017-06-1016:48potetmSomething like (apply distinct? (map :id people))
would I think.#2017-06-1016:48potetm(apply distinct? (map :id
[{:id 1 :name "Alex"}
{:id 1 :name "Frank"}]))
=> false
(apply distinct? (map :id
[{:id 1 :name "Alex"}
{:id 2 :name "Frank"}]))
=> true
#2017-06-1017:13mikecarter@potetm thanks! iāll give it a go#2017-06-1213:26lambderhi All
say I have a ns in which I want to define 2 specs , each for map
for example:
(s/def ::a (s/keys :req-un [::value]))
(s/def ::b (s/keys :req-un [::value]))
the problem is that both maps require to have key :value
but there should be different specs of ::value
for each ::a
and ::b
how do I do it?#2017-06-1213:56misha@lambder
(s/def :foo/value integer?)
(s/def :bar/value string?)
(s/def ::a (s/keys :req-un [:foo/value]))
(s/def ::b (s/keys :req-un [:bar/value]))
#2017-06-1213:56lambder@misha cool, thanks#2017-06-1214:01misha@lambder a
and b
in :a/value are not related to ::a in any implicit way, in case you might wonder#2017-06-1214:02lambderok#2017-06-1214:04mishathere, updated it to avoid any confusion#2017-06-1214:53mpenet@alexmiller should we expect a big reveal for your talk in euroclojure, given the recent mentions of "dependency" related work happening? š#2017-06-1214:55Alex Miller (Clojure team)no one expects the spanish inquisition#2017-06-1215:01Alex Miller (Clojure team)there are a few projects I have worked on over the last year in this area and the abstract is sufficiently vague to cover whatever I can talk about by the time I do the talk#2017-06-1215:06misha@alexmiller
multispec assocs dispatch value back to the {} during s/exercise
, right?
is there a way to re-use same enrich {} with a dispatch value, but during s/conform
?#2017-06-1215:13mishaoh, I can just call multimethod on an actual {} to get the dispatch value
... assuming defmethod's return keyword, not an inline spec-from#2017-06-1219:18souenzzoI have a query (all-enums-of :my/attr)
that return's all my enums. There some how to dynamic gen (s/def :my/attr #{})
?#2017-06-1219:30Alex Miller (Clojure team)not sure I understand what the last sentence means - looks like you are trying to spec the empty set, which doesnāt make much sense?#2017-06-1219:33souenzzoI want to dynamic gen the spec of :my/attr
. In case, it's a enum so I describe it as a set of keys.... Maybe (s/def :my/attr (all-enums-of db :my/attr))
Should express better what I mean...#2017-06-1219:36souenzzoself reply
- Yep. I can do a macro, It's a good way? It will be declared after connect with the db. It's a problem?``#2017-06-1219:44Alex Miller (Clojure team)or eval#2017-06-1219:53souenzzoIs it a reasonable solution? "Should I use in production"?#2017-06-1219:54Alex Miller (Clojure team)yes and yes#2017-06-1220:01Alex Miller (Clojure team)(defn load-enum-specs [db]
(let [enums (all-enums-of db :my/attr)]
(eval `(s/def :my/attr ~enums))))
#2017-06-1220:10ikitommiWould be nice to have:
(defn load-enum-specs [db]
(let [enums (all-enums-of db :my/attr)]
(s/register :my/attr enums)))
#2017-06-1220:12Alex Miller (Clojure team)well youāre just a macro away from that :)#2017-06-1220:12mpenetWould be doable with unform with spec of specs too #2017-06-1220:15Alex Miller (Clojure team)(defmacro register [name spec]
(let [s (eval spec)]
`(s/def ~name ~s)))
#2017-06-1220:16Alex Miller (Clojure team)this of course has the significant downside that it only works in a few cases like literal sets#2017-06-1220:19Alex Miller (Clojure team)or maybe wider than just those but not for things like preds, anonymous functions, etc#2017-06-1311:46carocadhey guys, quick question: what is the clojure/spec.alpha
project in github? is it an old version of clojure.spec
?#2017-06-1311:46gfredericksI think it's the new version#2017-06-1311:47mpenetyes, the fact that the readme mentions nothing about what it is doesn't help tbh (or how to use it; you have to dig into the google group announcement instead)#2017-06-1311:49carocadwait what!
so if I update my version of Clojure then it would break š ?#2017-06-1311:49carocadis it also like that for Cljs?#2017-06-1311:50mpenetyes, you need to update your require calls and potentially clj.spec/invalid kw#2017-06-1311:50mpenetdunno about cljs, but I guess so#2017-06-1311:57hkjelsThis problem had me spinning for close to an hour#2017-06-1311:58mpenetwasted quite some time on it too, even tho I knew what to do#2017-06-1314:37joshkhis it possible to use spec to ensure that one spec'ed key in a map containing a number is larger than the value another spec'ed key?#2017-06-1314:38joshkhstarting with something like this:
(s/def :my.app/start int?)
(s/def :my.app/end int?)
(s/def :my.app/timer
(s/keys :req [:my.app/start :my.app/end]))
#2017-06-1314:42andrewmcveighYes, you can do something like (s/def :my.app/timer
(s/and (s/keys :req [:my.app/start :my.app/end])
#(<= (:my.app/start %) (:my.app/end %))))
#2017-06-1314:42joshkhthanks. that makes perfect sense.#2017-06-1314:43joshkhs/def takes a keyword and a predicate (or composite of them)#2017-06-1314:46andrewmcveighYeah, the 2nd arg is a spec, which can be specs from the spec namespace, sets, or really any function that returns truthy/falsey, or any combination.#2017-06-1314:47andrewmcveighThe docs say spec, spec-name, predicate or regex-op
#2017-06-1314:49joshkhi got as far as Usage: (def k spec-form)
and didn't bother to read down š#2017-06-1314:55joshkhthis is probably a dumb question, but when exercising a spec with a custom function (such as the <= above), is clojure generating ints at random and then only filtering out results that pass the spec? or is it truly generating valid specs on the first try?#2017-06-1314:55joshkhin other words, how hard is it working to exercise the spec?#2017-06-1315:32andrewmcveighs/and
uses the first spec as a generator, and the following specs as filters#2017-06-1315:32andrewmcveighas far as I remember#2017-06-1315:33andrewmcveighIf you want something more intelligent, you'd have to provide the generator yourself#2017-06-1315:40joshkhgotcha, thanks again#2017-06-1315:40andrewmcveighnp#2017-06-1316:12mattlyis the plan to move clojure.spec.alpha
back to just clojure.spec
after it's released?#2017-06-1316:13mattlyI might just park at the version of clojure 1.9 I'm using until it goes back#2017-06-1316:23royalaidhttps://groups.google.com/forum/#!msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ <- @mattly, seems that the plan is to move back to a non-alpha namespace after#2017-06-1316:36Alex Miller (Clojure team)It is likely that 1.9 will release before spec is finalized (that is, 1.9 final will depend on clojure.spec.alpha). Being able to do so is the main reason we did the split.#2017-06-1316:37mattlyok, thanks#2017-06-1316:41joshkhcan someone help me understand why this is producing a vector of vectors rather than a single collection of numbers?
(s/def :test/coll (s/coll-of number?))
(s/exercise :test/coll)
=> ([[-2.0 -1 0 -2.0 -2.0 1.0 0 0.5 0 0 -0.5 -2.0 0.5 -0.5] [-2.0 -1 0 -2.0 -2.0 1.0 0 0.5 0 0 -0.5 -2.0 0.5 -0.5]]
[[0 0 -3.0 -1 0 -1 -1 -0.75 -1 -1 0 -1 -2.0]...]
#2017-06-1316:42joshkhi would have expected just a list of collections#2017-06-1316:43gfredericksme too#2017-06-1316:43joshkhi also get some NaNs#2017-06-1316:43joshkh[[0 -4.0 0 26 1 -1.25 NaN] [0 -4.0 0 26 1 -1.25 NaN]]
#2017-06-1316:44gfredericksdoes NaN pass number?
#2017-06-1420:35waffletoweryou probably already answered that for yourselfā¦
(defn nan? [x]
(if (number? x)
(not (= (double x) (double x)))
true))
#2017-06-1420:36waffletower(fact "NaNs are true and numbers are false"
(nan? 0) => false
(nan? 1) => false
(nan? 3.4) => false
(nan? -7.0) => false
(nan? "yoplait") => true
(nan? "") => true
(nan? Float/NaN) => true
(nan? Double/NaN) => true
(= (nan? Float/NaN) (number? Float/NaN)) => true ;; this highlights that (nan?) is not a precise complement of (number?)
(= (nan? Double/NaN) (number? Double/NaN)) => true
)
#2017-06-1422:46gfredericksI didn't; NaN is a pretty thorny case for generators, primarily because they don't equal themselves#2017-06-1422:46gfredericksotherwise I think it's appropriate for a number?
generator to generate them#2017-06-1316:45joshkhi don't even know how you'd represent NaN in clj...#2017-06-1420:47waffletowerjoshkh: Float/NaN
and Double/NaN
are distinct floating point literals available in clojure. Sorry if I am late#2017-06-1421:01joshkhgood to know! since you seem savvy, how does one use the Infinity literal?#2017-07-2619:03waffletowerOops missed your reply sorry š¦
Double/POSITIVE_INFINITY
& Double/NEGATIVE_INFINITY
#2017-07-2621:00joshkhit's all good! your guidance steered me true. i just had to adjust for javascript which... against all odds... supports positive infinity š#2017-06-1316:45gfredericksit's an instance of Double
#2017-06-1316:45joshkhbut i'm still more confused about the deeply nested collections#2017-06-1316:46gfredericksah#2017-06-1316:46gfredericksread the docstring for s/exercise
#2017-06-1316:47gfredericksit's not a coincidence that you have pairs of identical collections#2017-06-1316:47joshkhpairs of generated and conformed values for a spec
ahhh#2017-06-1316:47dpsuttonit's generating ten collections of things satisfying number?
, right?#2017-06-1316:48Alex Miller (Clojure team)@joshkh exercise generates values, and conforms them, then returns a collection of those pairs#2017-06-1316:48Alex Miller (Clojure team)you can also (gen/sample :test/coll) to just get the samples#2017-06-1316:48joshkhthat's the second time i've stared right past the docs today. time for a coffee. thanks guys.#2017-06-1316:48Alex Miller (Clojure team)or (map first (s/exercise :test/coll))
#2017-06-1316:49Alex Miller (Clojure team)@mpenet I updated the spec.alpha readme a little#2017-06-1316:53mpenetIt's not pushed yet, right?#2017-06-1316:53Alex Miller (Clojure team)it is pushed#2017-06-1316:54Alex Miller (Clojure team)https://github.com/clojure/spec.alpha/blob/master/README.md#2017-06-1316:57mpenetI was looking at core.specs.alpha :D mixed the two. I guess it might get a readme update as well there#2017-06-1316:57Alex Miller (Clojure team)oh, yeah. will do#2017-06-1317:03mpenetA lot better, thanks#2017-06-1317:05Alex Miller (Clojure team)certainly better than nothing! I meant to come back to those, just forgot to do so after the release.#2017-06-1403:30souenzzo(s/def ::bar (s/cat :boo string?))
(s/def ::foo (s/cat :foo symbol? :bar ::bar))
This ::foo
matches with (my-sym "my-string")
. But I'm trying to match with (my-sym ("my-string"))
. How to describe?#2017-06-1403:54madstap@souenzzo (s/def ::foo (s/cat :foo symbol? :bar (s/spec ::bar)))
#2017-06-1404:02seancorfield@souenzzo By way of explanation, s/cat
-- as one of the "regex specs" -- combines with other regex specs by concatenation rather than nesting.#2017-06-1404:24souenzzoseancorfield: I was not remembering/understanding the spec
usage. š#2017-06-1414:49odinodinHaven't pushed it yet, more of a proof of concept right now. #2017-06-1414:53richiardiandreaI would be interested in checking, the last missing piece in tooling is something like that#2017-06-1415:24odinodinSure, I will push it some time this week#2017-06-1418:25royalaid@odinodin Looks great! I would recommend looking at other kinds of error reporting that happens in other langs, I hear elm-lang has great error reporting, and I trying to get inspiration from something like that because I donāt think there is a real effort around making something like what you have.#2017-06-1418:37odinodinThanks for the tip, will do#2017-06-1508:48thheller@odinodin thats awesome, any chance of a non-html version? š#2017-06-1508:51odinodinthat would be great, but I wonāt make it. However, anyone can do it, itās not hard š#2017-06-1508:53thhellerdamn š ā¦ will that be standalone or something integrated in another library?#2017-06-1508:53thhellerprobably tied to reagent?#2017-06-1508:56odinodinWhat Iām working on is tied to reagent, but it is really not that hard to make.#2017-06-1508:57thhellerI tried and failed a couple times ā¦ Iāll check out your implementation ... maybe I just missed something#2017-06-1508:59thhellerprobably shouldnāt have started with extreme errors like this one#2017-06-1508:59thhellerhttps://gist.github.com/thheller/738698dfff45280f4e004df1c46af4ba#file-spec-errors-need-some-work-txt#2017-06-1509:07athosI'm also working on a similar project (https://github.com/athos/Pinpointer), though it's now broken because of the recent radical rewrite.#2017-06-1509:11athosFor me, it's not so easy work, considering some corner cases.#2017-06-1509:14thheller@athos thats awesome and text as well#2017-06-1509:20athos@thheller thank you, and yeah, it's intended to be available from cider.#2017-06-1509:21thhellerIāll definitely check it out#2017-06-1509:26thhellerI added pretty warnings to shadow-cljs recently which works nicely for cljs analyzer errors#2017-06-1509:26thhellerhttps://user-images.githubusercontent.com/144930/27010397-e0adacbc-4ea3-11e7-89be-7b512cf01c53.png#2017-06-1509:27thhellerbut shadow-cljs uses spec to parse ns forms and stuff#2017-06-1509:27thhellerneed something to make those errors prettier as well#2017-06-1509:27thhellerhttps://gist.github.com/thheller/c49f97183405343b93ed71749e5ccca5#2017-06-1509:27thhellercurrently the errors are basically useless#2017-06-1516:42carocad@alexmiller thanks for updating the spec alpha readme. helps a lot š#2017-06-1614:23tap(defn foo [a] a)
(stest/instrument `foo)
(s/def ::m int?)
(s/def ::a (s/* (s/keys :req-un [::m])))
(s/fdef foo
:args (s/cat :a ::a))
#2017-06-1614:24tapWhy (s/valid? ::a [{:m 1}])
returns true, but an error is raised for (foo [{:m 1}])
?#2017-06-1614:25nwjsmithI think that you want the spec for ::a
to be (s/every (s/keys :req-un [::m]))
#2017-06-1614:28nwjsmiths/*
is a regexp operator, which are used for specifying sequences of data.#2017-06-1614:29nwjsmithIn your example above, an error would be raised for (foo [{:m 1}])
, but not (foo {:m 1})
or (foo {:m 1} {:m 2})
#2017-06-1614:29nwjsmith(I think)#2017-06-1614:32nwjsmithAh, looks like you'll want to used coll-of
instead of every
#2017-06-1614:40tapAhh, ok. Thanks @nwjsmith#2017-06-1706:44colinkahnIf I have some data like this [:a :b :c :b :a]
where the structure is mirrored, but those could be any values, like [:x :y :z :y :x]
, what would I use from spec to write that?#2017-06-1715:09weavejester@colinkahn: You could have a predicate like #(= % (reverse %))
#2017-06-1715:09weavejester(s/and sequential? #(= % (reverse %)))
#2017-06-1715:15colinkahn@weavejester thanks, I realized you could do something similar but my solution wasn't as concise š#2017-06-1715:23colinkahnis there a more declarative way to define it? I was searching around and found recursive regular expressions with this solution: (\w)(?:(?R)|\w?)\1
But that requires you to be able to reference the group, which I don't see a way to do that with clojure.spec. Maybe using tags?#2017-06-1715:27colinkahnthat's from this article: http://www.rexegg.com/regex-recursion.html
I know it's for palindromes which wasn't what I originally posted, but curious if you could solve problems in that way using clojure.spec#2017-06-1717:55flyboarderHello everyone, I have a clojure macro that is in a .clj
file, I use this macro within .cljs
files however it seems I cannot spec this macro???#2017-06-1718:41matanCan someone kindly explain the point of what is called in spec docs "instrumentation"?#2017-06-1718:41matanI am not sure I get what it is that it actually does, or why it is called that way#2017-06-1718:42matanAren't args checked anyway, a la :pre
? if not, what is the rationale?!#2017-06-1718:42matanThanks in advance for clarifying..#2017-06-1718:57weavejester@matan Instrumentation is a little like :pre
, except that you can turn it on and off selectively.#2017-06-1718:57weavejesterSo during development you might have error messages that you do without in production.#2017-06-1720:39mobileinkanybody else working on spec-based transformation? use case is representing html head/meta stuff as a clojure map. the task is to transform such a map into html <link>, <meta>, etc. my xform code mimics spec : register a transform for each specked kw, then crawl the validated structure to xform it. this seems like a natural offshoot of spec. i can't be the only person who wants it. what else is out there?#2017-06-1721:19zaneAny idea why cljs.spec.test.alpha/check
would return a 0
-element vector?#2017-06-1721:42zaneSeems like cljs.spec.test.alpha/check
is #{}
. Trying to figure out why that is.#2017-06-1722:29zaneWhat's the recommended way to integrate testing the :ret
part of function specs in an automated fashion via clojure.test
?#2017-06-1722:30zaneHere's one answer: https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite
Is that the current recommended path?#2017-06-1722:32zaneThere's also this: http://spootnik.org/entries/2017/01/09/an-adventure-with-clocks-component-and-spec/index.html#2017-06-1802:02flyboarderHow can I use a spec on a macro to test for a cljs custom type?#2017-06-1802:18gfredericksprobably rewrite as a macro + function and put the spec on the function#2017-06-1802:18gfredericksmacros can't recognize runtime types#2017-06-1802:49matan@weavejester thanks, so which parts of clojure.spec are ignored when code runs uninstrumented, and which parts always execute?#2017-06-1802:50weavejester@matan All parts are ignored. If you write a spec for a function, it doesnāt affect how the function runs unless you instrument it.#2017-06-1803:35matanmmmm now it all makes sense. thanks a lot @weavejester š#2017-06-1803:37matannice stuff, spec, IMO should have been there from the beginning of clojure, as a way of taking/extending the good parts of OO rather than dismissing OO benefits so dichotomously#2017-06-1804:01Alex Miller (Clojure team)I think Clojure has always embraced some parts of OO (like interfaces and polymorphism) while letting some parts go (encapsulation, concrete inheritance)#2017-06-1804:28seancorfield@matan I'm curious: how do you think spec relates to the "good parts of OO"?#2017-06-1804:37seancorfield(and I guess that would also prompt me to ask "What do you think the good parts of OO are?" -- and, full disclosure, I was on the ANSI C++ Standards Committee from '92 to '99, and got started with Java in '97, so with that background I'm often very interested when I hear folks talk on this topic)#2017-06-1906:39matanI think the good part, as it relates to what spec is, is that structure is kept validated (in OO, only in terms of types and composite types, which is what most objects in a program are) at most points. This helps 50% of the programmer population keeping programs in check, although, recently, unit-tests have culturally evolved into another aid in that regard.#2017-06-1906:39matan@U04V70XH6 obviously my thoughts aren't that interesting, I've not been on any committee.#2017-06-1922:09seancorfieldThe problem is that with Java, and several other OO languages, you have no way to separate type and structure ā they are conflated. You canāt talk about data structures without giving them class/type names and your only form of āvalidationā really is the type system. Itās both ānot enoughā and ātoo muchā. So I donāt find that to be a good part of OO ā I find it to be a failing. The same with encapsulation: it makes it harder to treat data generically and so you end up with a stack of design patterns to workaround the restrictions of the OO type system ā and Iāve just gradually found the downsides outweigh the benefits. Mind you, youāre right about the ā50% of the programmer populationā: Java is very effective at allowing large numbers of average developers to work together to build large, complex systems. Java makes developers fungibleā¦ But I donāt think developers should be fungible š#2017-06-2010:54matanI concur on all points made. Just not sure how encapsulation makes it hard to generically treat data#2017-06-2010:57matanIt might seem valid to encapsulate, especially when delivering a library (to make the codebase have a restricted API facade or making it explicit what are the internals of a module vs how to use it)#2017-06-2010:57matanBut as said, I probably need help seeing the negative implications. I am sincerely curious about them#2017-06-2016:13seancorfieldIf you encapsulate your data in a specific type, then you need a function on that specific type to manipulate it. If you want to use a generic function, you have to use the underlying implementation data, ābreakingā encapsulation, and then put all the data back afterward. In other words, in Java, if you have an object that contains an array of other data, you canāt use generic array functions without exposing the array directly, i.e., unencapsulating it. Clojureās approach is that the āAPIā is just data ā immutable data ā so if the implementation is a vector (array), then you can use any generic sequence function on it.#2017-06-1820:20richiardiandreaNewbie question, why does s/or
have keyword branches for predicates and s/and
does not? I have never noticed that until now š I was kind of assuming the same API but go burned there.#2017-06-1820:39gfredericksmy guess is because it's useful for s/or
to know which branch matched#2017-06-1820:40gfredericksand not useful for s/and
, since you already know that all of them matched (though I suppose you could argue it would be useful to know which one didn't match in the case of an error#2017-06-1820:40gfredericks)#2017-06-1820:47richiardiandreaYep that is true, it would be great to know which one didn't and also to have uniformity in the API I guess#2017-06-1820:56gfredericksthe difference is that one is used for conforming and the other isn't#2017-06-1820:56gfrederickswhether that's a good reason for the API to be different is a different question#2017-06-1901:45flyboarder@gfredericks thanks, I guess thats the difference between runtime specs for cljs#2017-06-1901:46gfredericks@flyboarder the same applies to clj-jvm#2017-06-1901:47flyboardermakes sense#2017-06-1906:40matan@alexmiller @seancorfield what's wrong with encapsulation, from the colujarians' world view? š#2017-06-1906:40matanI don't recall a very convincing blog post, or have just forgotten if I saw one#2017-06-1906:41matanOops should have moved it to #off-topic#2017-06-1906:42matanAnyway I am very happy about spec, it is really a "missing piece" for robust programs that teams can collaborate on#2017-06-1912:15Alex Miller (Clojure team)@matan in OO (particularly thinking of Java / C++ here), encapsulation is used to hide or protect the data inside an object and access it via methods. Because this data is mutable, the accessor/mutator methods are also a place where you can apply a locking strategy to protect that mutable data in multi-threaded usage.#2017-06-1912:16Alex Miller (Clojure team)Clojure turns this inside out and says, data is actually the most important thing here - make it visible, introspectable, and directly manipulable via a generic data interface, rather than via a custom set of API methods that is different for every class.#2017-06-1919:34mobileinkalexmiller: #off-topic an alternative perspective, just for fun: "immutable datum" is just another way of saying "pure function". we could discard the notion of "data" altogether.#2017-06-1912:17Alex Miller (Clojure team)Immutable data structures allow you to do this while also being automatically thread-safe and avoiding the need for locking around access and manipulation of mutable data completely.#2017-06-1912:24gfrederickseven more than thread-safety, you get independence of usage -- you can pass the same object to different pieces of code and not worry about what things they will each "change" that might affect the other
which may or may not be what alex's last sentence meant#2017-06-2011:10matangfredericks: sorry for being late to return to the party š
I am not sure why I would not worry about different pieces collaborating on the raw data. Or how this is different than the claim of concurrency/thread-safety. #2017-06-2011:26gfredericksthey're not collaborating, they're doing completely independent things#2017-06-1912:27leonoel@alexmiller to make data explicit and visible is undoubtedly a good thing, but there is definitely situations where encapsulation is valuable
https://github.com/clojure/clojure/blob/clojure-1.9.0-alpha14/src/clj/clojure/core.clj#L4939#2017-06-1912:29leonoelcore.async channels and go blocks are another good example of encapsulation#2017-06-1912:33Alex Miller (Clojure team)Go blocks are about process, not data. Channels are about conveyance of values, not encapsulation #2017-06-1912:34leonoelsure, but they both have opaque internal state#2017-06-1912:35leonoelwhich doesn't make them bad primitives, btw#2017-06-1912:36leonoelI get your point, but "encapsulation is bad" goes a bit too far imo#2017-06-1912:37Alex Miller (Clojure team)Agreed, but I didn't say that#2017-06-1912:41Alex Miller (Clojure team)Most encapsulation in OO is incidental, not useful#2017-06-1912:44gfredericksthis is not quite the same topic but it just occurred to me that one of my biggest frustrations with legacy OO codebases that I haven't heard articulated before is that the classes I encounter have unclear lifecycles#2017-06-1912:45gfredericksthe thing I want to understand about a class to help me understand the whole codebase is how/why the objects get instantiated, what you do with them while they're alive, how they get discarded, how that relates to the lifecycle of the program as a whole; and I've never seen an OO codebase where that's remotely clear#2017-06-1912:46gfredericksperhaps there's a similar question about how functions are used in a functional codebase, but it seems like less crucial of a question for some reason#2017-06-1912:55Alex Miller (Clojure team)ālifecycleā implies some degree of statefulness to me. mutable state requires coordination. all of that is often a mess in OO.#2017-06-1912:56Alex Miller (Clojure team)although Iād say that aspect is a distant second problem for me after the notion of not having a generic data interface and/or clear equality semantics in OO#2017-06-1916:02carocadalexmiller: that is an interesting thought. I am wondering, is there a consensus about āthis is not generic enoughā. Currently I am working on a program were I need to squeeze every inch of performance that I can get so I ended up defining some custom interfaces to access the data. Nevertheless this puts the clear equality semantics
and generic data interface
a bit to the limit since the interface is no longer Clojure-like but rather case-specific.#2017-06-1916:02carocadBtw I used some of the current benchmarks as inspiration š
https://github.com/jafingerhut/clojure-benchmarks/blob/master/binarytrees/binarytrees.clojure-rh.clojure#L18#2017-06-1919:37Alex Miller (Clojure team)I have no problem with breaking every rule in the book in very narrow areas where you are seeking the ultimate performance :) hopefully that is about 0.1% of the code#2017-06-1920:20carocadWell that depends on how you define 0.1% of the code. The loc that use those custom interfaces are very short but the ones that consume it are probably very long and very different. Which is what got me thinking.
I think it is a similar situation to what Clojure does i.e defining algorithms and interfaces in java yet using them in Clojure.#2017-06-1912:58gfredericksthe "no generic data interface" complaint also applies to type-oriented functional programming (haskell etc.), right?#2017-06-1912:58Alex Miller (Clojure team)I havenāt done enough of it to say#2017-06-1912:59gfredericksthat feels like the biggest difference to me between clojure and most of the rest of FP, even more than static vs dynamic type system#2017-06-1912:59gfredericksthough I'm sure the static vs dynamic dimension is highly correlated with generic vs typey#2017-06-1918:07spiedenyeah not having equality, robust hashing and string representations built into your data structures is terrible#2017-06-1918:09gfredericksI had a giant dump of ruby data the other day and would have appreciated being able to read it into a repl and analyze it, but I just had to decide to not want to do that#2017-06-1919:02mobileinkgfredericks: that must'v hurt like hell.#2017-06-1919:04gfredericksso much #<...>
#2017-06-1919:10mobileinkgfredericks: i hate sharps in my dumps.#2017-06-1918:13spiedenone of my complaints with clojure is actually that the default string representation isnāt EDN ā this decision confuses me. e.g. (println {:foo "bar"})
{:foo bar}
=> nil
#2017-06-1918:14spiedeni use prn-str for everything, so itās not a big deal#2017-06-1918:16spiedeni guess not all clojure data can be represented as EDN because of nested Java objects, but why not quote āfooā above?#2017-06-1918:28gfredericksbecause then (println "foo")
would be surprising#2017-06-1918:29gfredericksyou'd need some other mechanism for printing a raw string#2017-06-1918:31spiedenah yeah, that makes sense#2017-06-1918:32spiedeni guess you could special case strings inside of collections#2017-06-1918:33gfredericksthat would bother me even more#2017-06-1918:33gfredericksbut that might be a personality thing; some people like things that are less complicated to describe, others like things that are more likely to do what you wanted#2017-06-1918:34spiedenwell said. i lean towards the former too#2017-06-1918:35gfredericksironically, now that I think about it, I feel like gripes about clojure are pretty evenly split between people wanting the former and getting the latter, and vice versa#2017-06-1918:41spiedena good knifeās edge to balance on =)#2017-06-1918:43gfredericksI suppose there are also gripes about situations where neither ideal is achieved#2017-06-1920:27shaun-mahood@alexmiller: Is there a place on the http://clojure.org site that would make sense for compiling links to blog posts, videos, and other external resources related to spec? There's a bunch of good ones on the Cognitect blog and elsewhere that I wouldn't know how to find if I were looking at it for the first time.#2017-06-1921:39Alex Miller (Clojure team)Depends what they are #2017-06-1921:41Alex Miller (Clojure team)The community/resources page is one place but may not be best for everything #2017-06-1921:41potetm@spieden Just to make sure it's said: pr
does exactly what you want.#2017-06-1921:42potetmThe docs actually say:
> By default, pr and prn print in a way that objects can be read by the reader#2017-06-1921:45Alex Miller (Clojure team)@shaun-mahood give me examples and I can help find answers#2017-06-1922:05shaun-mahood@alexmiller: Here's the list just gleaned from the Cognitect blog
http://blog.cognitect.com/blog/2017/3/24/3xeif9bxaom78qyzwssgwz1leuorh4
http://blog.cognitect.com/blog/2017/1/3/spec-destructuring
http://blog.cognitect.com/blog/2016/12/9/works-on-my-machine-self-healing-code-with-clojurespec-1
http://blog.cognitect.com/blog/2016/10/5/interactive-development-with-clojurespec
http://blog.cognitect.com/blog/2016/9/29/agility-robustness-clojure-spec
http://blog.cognitect.com/blog/2016/9/14/focus-on-spec-combining-specs-with-sor
http://blog.cognitect.com/blog/2016/8/24/combining-specs-with-and
https://swannodette.github.io/2016/06/03/tools-for-thought
A bunch from searching Medium
https://product.gomore.com/end-to-end-integration-tests-with-clojure-spec-d4a48cbf92b5
https://medium.com/@rplevy/temporal-higher-order-contracts-with-clojure-spec-e92e795665d
https://medium.com/degree9/data-validation-schema-spec-5547e33596bd
There are a bunch of conference videos as well, plus the spec screencasts on ClojureTV
One other thing that I haven't seen anywhere are the notes from your spec workshop, though I don't know if that's meant to be publicly available.
If having any of this stuff (or type of stuff) is desirable, I'm happy to work on a PR for it. I know I've seen other interesting and exciting stuff on Spec elsewhere too.#2017-06-1922:10shaun-mahoodI just noticed your previous reply - the main motivation for my question was realizing that if I only look at http://clojure.org as it stands right now, none of the really interesting or exciting use cases jump out at me. Might not be a goal though, so I didn't want to put any work into a PR without figuring out what kind of thing might be useful for others.#2017-06-1922:33naomariki could appreciate such a compilation#2017-06-2005:26Alex Miller (Clojure team)@shaun-mahood broadly, I think itās hard to give a list of blog entries the right context to be useful and survive well over time. It just doesnāt make sense to try to track every spec blog (most of which are now out of date). Whatās useful on the site is curated reference or tutorial content that is kept up to date. So I guess largely, I would say these should not be added to http://clojure.org. One thing that I have been (very slowly) working on is a reference page for spec which I think is a good complement in between the rationale, the guide, and the api docs.#2017-06-2005:30Alex Miller (Clojure team)and the spec workshop notes are not intended to be public - they are training materials copyright Cognitect and represent probably 100 hours of work over the last year.#2017-06-2005:38shaun-mahoodThat sounds reasonable to me. I love the idea of a reference page - I'm still trying to come to grips with the boundaries of the new clojure and clojurescript site. #2017-06-2005:41shaun-mahoodI hope the workshop notes at least made the book easier to write.#2017-06-2005:43shaun-mahoodIt was definitely worthwhile to take at last years Conj, I would consider taking it again to pick up the new stuff and the bits I missed. #2017-06-1921:50spieden@potetm yes thanks#2017-06-2011:17matanSorry but after all the discussion about encapsulation, I still find that hiding (encapsulating) internal implementation of "complex" mechanisms (not silly field values like in Java) is a reasonable thing for helping others/self use of components without shooting one's own foot or scanning dozens of fn
s to figure what is meant to be used from the outside#2017-06-2014:17Alex Miller (Clojure team)matan: well, youāre arguing against something I didnāt say. I didnāt say that all encapsulation was bad, I said needless encapsulation of data is bad and OO as typically practiced encapsulates all data.#2017-06-2014:30donaldballWhen I have a complex side-effecting machine with a simple use interface, I reach for a protocol. You can also convey intentions about var usage with namespace design.#2017-06-2014:47colinkahn@U04V4HWQ4 whatās an example of using namespace design? I might be overthinking the meaning, but I havenāt heard that term before.#2017-06-2014:50donaldballMaybe better examples but one that jumps to mind is core.async, where there is a public api (clojure.core.async) and a bunch of impl namespaces (e.g. clojure.core.async.impl.buffers)#2017-06-2011:18matan(and that's why we have namespace private
declarations I think)#2017-06-2014:12Alex Miller (Clojure team)matan: note that even private vars are still accessible - anyone can obtain and use them still. The private meta is really more documentation than encapsulation.#2017-06-2015:21matanRight! #2017-06-2013:47bbqbaronthatās a good insight; i think the clojure difference is probably avoiding 1. colocating state with logic in memory 2. requiring encapsulated fn A to bring B, C, D, E, F and all their dependencies with it#2017-06-2014:05joshjones@matan this belongs in #off-topic since this is not about clojure.spec (or even clojure itself) FYI#2017-06-2019:34chilleniousHow would I rewrite this to avoid duplication?
(s/def ::PrimaryContact (s/keys :req-un [::FirstName ::LastName ::Email]
:opt-un [::Title ::Company ::Type ::Phone ::Fax]))
(s/def ::SecondaryContact (s/keys :req-un [::FirstName ::LastName ::Email]
:opt-un [::Title ::Company ::Type ::Phone ::Fax]))#2017-06-2019:43Alex Miller (Clojure team)(s/def ::Contact (s/keys :req-un [::FirstName ::LastName ::Email]
:opt-un [::Title ::Company ::Type ::Phone ::Fax])
(s/def ::PrimaryContact ::Contact)
(s/def ::SecondaryContact ::Contact)
#2017-06-2019:45chilleniousah, well that's easy... thanks!#2017-06-2102:32bbrinckIs multi-spec
intended to work with the :default
implementation of the multimethod?#2017-06-2102:35bbrincki.e. is it intended that I could use the :default
implementation to provide a default spec? https://gist.github.com/bhb/7eefe3034a0622b6499a5b5dd2f7e52a#2017-06-2104:39Alex Miller (Clojure team)yes, this should work (it does in Clojure)#2017-06-2104:41Alex Miller (Clojure team)so, that's a bug in cljs spec#2017-06-2104:45Alex Miller (Clojure team)actually, that's already been fixed if you update your cljs version#2017-06-2115:44bbrinck@alexmiller Ah, shoot, I should have checked JIRA. Thanks for the info!!!#2017-06-2118:58royalaidIs there a currently recommended way to integrate clojure.spec
s generative testing with clojure.test
?#2017-06-2209:17carocadroyalaid: I wrote a similar macro for fdef
which is basically a simplification of that one. Here it is:
(defmacro defspec-test
([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil))
([name sym-or-syms opts]
`(t/deftest ~name
(let [check-results# (clojure.spec.test/check ~sym-or-syms ~opts)]
(doseq [result# check-results#]
(t/is (nil? (:failure result#)) (str (clojure.spec.test/abbrev-result result#)))
(when (nil? (:failure result#))
(println "[OK] - " (clojure.spec.test/abbrev-result result#))))))))
#2017-06-2209:17carocadyou would use it like this: https://github.com/n7a235/data.hypobus/blob/dev/test/hypobus/basic_test.clj#L27#2017-06-2118:59royalaidI have found this https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite#2017-06-2118:59royalaidBut it seems a bit hacky and the output on failure isn't very helpful#2017-06-2119:03mattly@royalaid I can't speak to clojure.spec, but with test.check I use test.chuck, which has a macro checking
that behaves similar to testing
#2017-06-2119:03mattlyI'd link you to it except slack's electron app decided I can't use my clipboard anymore#2017-06-2119:10sneakypeetHi all. started using spec in cljs today. I am running into a weird issue
(ns tenandsix.app.flow
(:require-macros [cljs.spec :as s])
(:require [cljs.spec :as spec]))
(s/def :flow/action-type keyword?) => works
(s/def :flow/action (s/tuple keyword? keyword?)) => No such namespace: s, could not locate s.cljs, s.cljc, or Closure namespace ""
#2017-06-2119:11dpsutton:flow
vs :flo
possibly?#2017-06-2119:12sneakypeetno that was actually me testing if the keyword made a difference š#2017-06-2119:12sneakypeetbut no that does not fix it#2017-06-2119:13sneakypeetany thoughts? I mean this is as basic as it gets#2017-06-2119:13dpsuttonthoughts, perhaps s/tuple
is a macro and the import macro statement is malformed#2017-06-2119:14dpsuttonbut only when actually trying to macroexpand#2017-06-2119:14dpsuttonhttps://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/alpha.cljc#L395#2017-06-2119:15dpsuttonlooks like tuple is a macro#2017-06-2119:15sneakypeetit is yes.#2017-06-2119:16sneakypeetso actually. it works, but my repl gives the error, when I use the spec it works#2017-06-2119:18dpsuttoni don't understand your statement#2017-06-2119:18sneakypeetI'll try again#2017-06-2119:18dpsuttonif you're on a recent cljs, it looks like you no longer need a require-macros part#2017-06-2119:19dpsuttontry just requiring spec#2017-06-2119:19dpsuttonhttps://groups.google.com/forum/#!topic/clojurescript/FoiqNV5nunQ#2017-06-2119:19sneakypeet1. evaluating that actually creates the spec#2017-06-2119:19sneakypeet2. I can use the spec#2017-06-2119:19sneakypeet3. It gives the error when I create the spec#2017-06-2119:20sneakypeetchecking the link#2017-06-2119:22sneakypeet@dpsutton thanks your link helped#2017-06-2119:22sneakypeetthats a pretty big improvement#2017-06-2121:27adamfreywhat's the cleanest way to make a generator that constantly returns the same value? Similar to constantly
in core?#2017-06-2121:30adamfreyI came up with this, which works. I just wanted to make sure I wasn't missing an easier way
#(gen/fmap (constantly 1)
(gen/int))
#2017-06-2121:33gfredericks@adamfrey (gen/return 1)
#2017-06-2121:33adamfreythanks!#2017-06-2121:33gfredericksprobably with an extra #
in a spec context#2017-06-2123:24Alex Miller (Clojure team)@adamfrey I would say (s/gen #{1})#2017-06-2201:27madstapIn Rich's speculation keynote he mentioned that there are ways to check if backwards compatibility is broken in both set theory and regex theory.
The set one would presumably be something like:
(defn breaking-change? [old-required-keys new-required-keys]
(not (set/subset? new-required-keys old-required-keys)))
Are there any links to info about the regex part? Or am I misunderstanding something?#2017-06-2202:05Alex Miller (Clojure team)madstap: the regex part is based on Matt Mightās papers with regex derivatives and I believe Rich has spent enough time with it to believe that the math there makes this computable. Iām having a strong sense of deja vu - have you asked this before?#2017-06-2202:13madstapI'm pretty sure I haven't. Thanks for the pointers š#2017-06-2203:14Alex Miller (Clojure team)Must have been someone else, or my addled brain#2017-06-2201:35sooheonHey is there function to coerce all keys in a map to namespace-qualified keys, other than the #::{}
syntax? Iām receiving an unqualified map from an API call, and would like to spec the result.#2017-06-2201:37jjttjj@sooheon if you just need to spec un-namespaced keys you can just do (s/keys :req-un [::my-key1 ::my-key2])
to require those keys un-namespaced, ie {:my-key1 "x" :my-key2 "y"}
#2017-06-2201:39sooheonThanks, I did see that in the docstring, and it did work#2017-06-2201:39sooheonBut now when I pass around this map and assoc my own keys to it, it will have a mix of qualified and un- keywordsā¦#2017-06-2201:41jjttjjs/keys can take vectors for both :req-un and :req if you have both namespace qualified and unqualified keys#2017-06-2201:41sooheonYeah, itās just a bit of overhead for me#2017-06-2201:41sooheonwould have preferred to just go all namespaced, you know?#2017-06-2201:41sooheonThanks anyways :)#2017-06-2202:02seancorfield@sooheon You could just use reduce-kv
and produce a new map from the old one with namespace-qualified versions of the keys (assuming you want the same qualifier on all of them).#2017-06-2202:03seancorfieldSomething like (reduce-kv (fn [m k v] (assoc m (keyword "qualifier" (name k)) v)) {} api-result)
#2017-06-2202:04sooheonah cool! hadnāt thought of that. Was just hoping something like (into #::{})
would magically work#2017-06-2202:04sooheonis this something that is recommended, btw? to use qualified keywords for everything?#2017-06-2202:05seancorfieldI think itās a reasonable approach for handling domain data within your system, yes.#2017-06-2202:06seancorfieldBear in mind thoā that you probably want to keep those API result keys separate from your actual domain keys in order to avoid conflicts.#2017-06-2202:06seancorfieldSo you might have :external.api/name
and :my.domain/name
with different specs.#2017-06-2202:08seancorfieldThis is one of the reasons that the latest java.jdbc
lets you specify :qualifier
in the options on almost every call so you can get namespace-qualified keys in your SQL result hash maps.#2017-06-2202:08sooheonI see. I think itāll just take some experience (just like knowing how to split up and organize nsāes in the first place)#2017-06-2202:09sooheonthanks!#2017-06-2202:18seancorfieldBear in mind that the namespace-qualifiers on keywords do not need to correspond to actual code namespaces so you have a lot of flexibility.#2017-06-2202:19seancorfieldFor example, we use a qualifier of wsbilling
for all entities that relate to our (World Singles) billing system but we donāt have a namespace called wsbilling
.#2017-06-2202:21seancorfieldWhen we accept JSON/Clojure data from outside (APIs, databases), we can normalize it to whatever qualifier we want to distinguish it, within our system, from our billing data.#2017-06-2202:22seancorfield(if this was a public Clojure API weād use a reverse-domain-name prefix, like com.worldsingles.billing
, I expect)#2017-06-2202:23seancorfieldYou can choose how to keep unique groups of entity names separate from each other.#2017-06-2202:23joshjones@sooheon @seancorfield I made a function a while back to namespace-qualify a map. here's what i have, if it's useful to anyone
(defn map->nsmap
"Creates a namespaced-map from a standard one and a namespace (obj or string)"
[m n]
(reduce-kv (fn [acc k v]
(let [new-kw (if (and (keyword? k)
(not (qualified-keyword? k)))
(keyword (str n) (name k))
k) ]
(assoc acc new-kw v)))
{} m))
#2017-06-2202:23sooheonAh great, thatās very useful#2017-06-2202:25sooheon@seancorfield Iāve still got to experience first hand how ns qualifying keys works with stuff like destructuring, and making sense of how I want to organize things, but thanks for the pointers#2017-06-2202:25seancorfieldThatās good for preserving existing name-qualification. Sometimes you need to override that. And of course my simplistic example only works for keys that you can call name
on (and assumes you want all-keyword keys back out)!#2017-06-2202:25seancorfield@sooheon Yeah, it takes some getting used to, after so many years of unqualified keywords!!#2017-06-2202:27seancorfieldIn terms of destructuring: user=> (let [{:foo/keys [a b c]} {:a 1 :foo/b 2 :c 3 :foo/c 4}] (println a b c))
nil 2 4
#2017-06-2202:27joshjonesyes, i wrote it to behave the way the #:
reader macro behaves, namely, it does not disturb non-qualified keys#2017-06-2202:27joshjones#:foo{:a 1 :bar/b 1}
=> {:foo/a 1, :bar/b 1}
#2017-06-2202:27sooheonyou mean already qualified ;)#2017-06-2202:28joshjonesyes, sorry#2017-06-2202:28joshjonesand non-keyword keys#2017-06-2202:28joshjones#:foo{:a 1 :bar/b 1 42 100}
=> {:foo/a 1, :bar/b 1, 42 100}
#2017-06-2202:28seancorfieldalso user=> (let [{:keys [a b foo/c]} {:a 1 :foo/b 2 :c 3 :foo/c 4}] (println a b c))
1 nil 4
So you can put qualifiers in the key vector ā and you get unqualified symbols back out!#2017-06-2202:29sooheonhuh#2017-06-2202:29sooheonhow would you use the :: sugar w/in destructuring#2017-06-2202:30seancorfieldLike this: user=> (let [{::keys [a b c]} {:a 1 :user/b 2 :c 3 :user/c 4}] (println a b c))
nil 2 4
Note that weāre in the user
namespace so ::keys
means :user/keys
#2017-06-2202:31seancorfieldYou can also do this (which Iām surprised is allowed): user=> (let [{:keys [a ::b ::c]} {:a 1 :user/b 2 :c 3 :user/c 4}] (println a b c))
1 2 4
#2017-06-2202:32joshjonesactually it seems to work by putting the ::
in the vector of keys too, though i'm not sure it's a good idea#2017-06-2202:32joshjones(let [{:keys [a b c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 3
=> nil
(let [{:keys [a b ::c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 4
#2017-06-2202:32sooheonso the last example would be the way to mix and match ::b
:foo.bar/c
and :d
#2017-06-2202:32seancorfieldI expected that to be an error but apparently you can use keywords in the key vector? @alexmiller Is that supposed to be valid?#2017-06-2202:33sooheonYou canāt have multiple :keys
::keys
and :foo/keys
deconstructions separately at once, right? Youād do {:keys [a ::b foo/c]}
#2017-06-2202:55seancorfieldThatās my understanding, yes.#2017-06-2203:10Alex Miller (Clojure team)@seancorfield yes, we support keywords there so that you can use autoresolved keywords #2017-06-2203:11seancorfieldIs that documented? (I guess the docs could have been updated since I last looked at themā¦ š )#2017-06-2203:11Alex Miller (Clojure team)And you can have multiple keys destructurings at the same time for different namespaces#2017-06-2203:12Alex Miller (Clojure team)It was in the changelog when it went in but that was a while ago#2017-06-2203:12seancorfieldYup, the destructuring guide contains examples.#2017-06-2203:13seancorfieldNot of ::k directly but of ::p/name#2017-06-2203:13Alex Miller (Clojure team)Rich and I worked on some updated docs for destructuring but I can't remember if those actually got all the way done#2017-06-2203:16seancorfieldItās fairly comprehensive, TBH. Thank you. And Rich.#2017-06-2215:48hlship@alexmiller From my issue in the tweet, here's the tail end of a very long stack trace:
clojure.spec.alpha/regex-spec-impl/reify/gen* alpha.clj: 1672
clojure.spec.alpha/re-gen alpha.clj: 1601
clojure.core/every? core.clj: 2572
clojure.core/next core.clj: 64
...
clojure.core/map/fn core.clj: 2657
clojure.spec.alpha/re-gen/ggens/gen alpha.clj: 1582
clojure.spec.alpha/re-gen alpha.clj: 1597
clojure.spec.alpha/gensub alpha.clj: 275
clojure.core/ex-info core.clj: 4617
clojure.lang.ExceptionInfo: Unable to construct gen at: [:exception] for: (instance? java.lang.Throwable %)
clojure.spec.alpha/failure: :no-gen
clojure.spec.alpha/form: (clojure.core/fn [%] (clojure.core/instance? java.lang.Throwable %))
clojure.spec.alpha/path: [:exception]
#2017-06-2215:49hlshipWhat I don't get is why it is trying to create a generator. It just supposed to be validating inputs into our schema/compile function.#2017-06-2215:49hlshipBasically, the function I'm passing in doesn't have a fspec
, so I'm going to add one and see if that helps.#2017-06-2215:53hlshipNope. Bad guess. Providing my own fspec didn't work.#2017-06-2216:25Alex Miller (Clojure team)Right, so that was my guess#2017-06-2216:25Alex Miller (Clojure team)the way that fspec args are validated in instrumentation is by using the fspecās generator to generate values and invoke the function#2017-06-2216:26Alex Miller (Clojure team)and check the ret and fn spec to verify the function with random inputs is valid according to the fspec#2017-06-2216:28Alex Miller (Clojure team)thus all fspecās should have an args spec that generates#2017-06-2216:28Alex Miller (Clojure team)the surprising bit here is that they need that not just for check, but also for instrument#2017-06-2216:29Alex Miller (Clojure team)same is also true of fdef for check, but not for instrument#2017-06-2216:29Alex Miller (Clojure team)(because checking an fdef involves generating using its args generator)#2017-06-2216:36hlship(in a meeting, will digest and get back to you)#2017-06-2216:41mpenetJust wondering, why not just wrapping args/ret of fspec fns with s/valid? when instrumented #2017-06-2216:42mpenetIt would fail "later" (invoke time) but might be more intuitive (and faster)#2017-06-2216:46mpenetCould be an option#2017-06-2217:05Alex Miller (Clojure team)not sure if Rich considered that or not, havenāt talked to him about it#2017-06-2312:35mpenetalexmiller: just wondering: wouldn't it be better to keep https://dev.clojure.org/jira/browse/CLJ-1936 open then?#2017-06-2217:31mpenetNot the first time this comes up: clj-1936#2017-06-2217:44hlshipSo I'm a bit lost .... is there no way I can simply declare what my callback should look like without coming up with a generator for the callback?#2017-06-2219:41Alex Miller (Clojure team)hlship: in short no, if you want to also validate the fdef. other options are to override the fdef args generator or to use a generator override on instrument#2017-06-2217:44hlshiphttps://dev.clojure.org/jira/browse/CLJ-1936#2017-06-2218:17mpenetYou can use fn? (meh)#2017-06-2219:27hlshipmpenet: Yes, that's where I'm headed.#2017-06-2219:40Alex Miller (Clojure team)actually, ifn?
is better if you go that route#2017-06-2218:36hlshipI almost don't care about validation as much as documentation of my Argos and result. #2017-06-2219:18arohner(gen/sample (s/gen (s/spec ::clojure.core.specs.alpha/map-bindings)))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
#2017-06-2219:20arohner@gfredericks in core.specs.alpha#2017-06-2219:20arohnerthe clojure.core specs that got split out into a separate lib when spec got split out#2017-06-2219:20arohner@alexmiller Iām confused by the definition of ::core.specs/map-bindings
. What does it mean for (s/every ... :into {})
I donāt see a guarantee that the every coll has an even number of elements, nor a requirement that the input coll is a map#2017-06-2219:20arohner(this is coming up because Iām trying to run spectrum inference on clojure.core)#2017-06-2219:22gfredericksif you're wondering where the such-that
error is coming from, I suspect https://github.com/clojure/core.specs.alpha/blob/c33689f75dbe3188ee00b32bb798bcd0cfd6cacc/src/main/clojure/clojure/core/specs/alpha.clj#L36#2017-06-2219:22gfredericks@arohner every branch of map-bindings seems to be a pair, does that explain your confusion?#2017-06-2219:23arohnerpartially#2017-06-2219:28arohner(s/conform ::clojure.core.specs.alpha/map-bindings '[[foo bar]]) => [[foo bar]]
#2017-06-2219:28arohnerit seems very weird to me that :into is used for generating, but the resulting conform doesnāt change#2017-06-2219:29arohnerthe real source of my trouble is just below, when I have to handle (s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))
#2017-06-2219:29arohnerwhere s/merge takes maps#2017-06-2219:43Alex Miller (Clojure team)sorry, just reading this now. where is your question at now?#2017-06-2219:44Alex Miller (Clojure team)note that s/merge doesnāt flow conformed values so only the last map spec in the merge will matter for the conformed value#2017-06-2219:44arohner::map-binding-forms
takes two maps, because of s/merge
, but map-bindings
is not guaranteed to return maps, because it uses :into
and not :kind
#2017-06-2219:49Alex Miller (Clojure team)I donāt agree with the last clause there?#2017-06-2219:50Alex Miller (Clojure team)both :into
and :kind
affect s/every gen#2017-06-2219:50arohner(s/conform ::clojure.core.specs.alpha/map-bindings '[[foo bar]])
is that expected?#2017-06-2219:51Alex Miller (Clojure team)Iād say thatās not a supported map binding form#2017-06-2219:51Alex Miller (Clojure team)I guess we could use :kind to narrow that#2017-06-2219:51arohnerright, spectrum is currently picky about that, and Iām trying to understand it / make it work#2017-06-2219:52arohnerspectrum says map-binding isnāt necessarily a map, so the merge isnāt guaranteed to work in all cases#2017-06-2219:58Alex Miller (Clojure team)yeah, that seems good. if you want to file a jira, we can try to get that in next release#2017-06-2220:01arohnersure. in CLJ, or does spec have itās own project now?#2017-06-2220:01Alex Miller (Clojure team)CLJ#2017-06-2220:01Alex Miller (Clojure team)if you want to make a patch, that will be faster than me making it (as I can screen it)#2017-06-2220:02Alex Miller (Clojure team)patch against core.specs.alpha of course#2017-06-2220:08arohnersure#2017-06-2221:40peejaWhat's the best way to define a function :args
spec for a function which takes a single argument?#2017-06-2221:46joshjones@peeja - same way as you do for any :args
spec
(s/fdef your-func
:args (s/cat :single-arg some-pred?)
...)
#2017-06-2221:46peejaSo, still use s/cat
then?#2017-06-2221:46joshjonesyes, as it's still a vector of one element#2017-06-2222:06Alex Miller (Clojure team)agreed and to take that a step further, itās useful to use a regex op (s/cat) here because the conformed value will be a map with the arg name as the key, and thatās useful in the :fn spec or in explanations#2017-06-2312:02odinodingiven a spec (obtained from ex-data of an exception thrown by inspect, i.e the :cljs.spec.alpha/spec), how can one obtain the key of that spec in the registry?#2017-06-2312:02odinodinI want to go from spec -> the name of the fdefāed function that failed#2017-06-2312:28odinodinMy current solution is to parse it out of the exception message, which contains āCall to #āname-of-var did not conform to specā¦ā. It would be nice if instrument included the key in the ex-data, but I might be misunderstanding something here.#2017-06-2313:13wilkerlucio@odinodin can you give please show the data you are trying to go from -> to?#2017-06-2313:13wilkerlucioI think you might want to use (s/form)
or (s/get-spec)
, but I'm not sure yet what you are trying to get#2017-06-2313:16odinodin@wilkerlucio it is a spec record object, as provided by the exception thrown by instrument when a fdefāed function is called with non-conforming input.#2017-06-2313:18odinodinit is the spec object that (s/get-spec 'the-fdefed-var)
would return as far as I can tell#2017-06-2313:19wilkerluciothat's a long chain, hehe, I'm not sure what kind of data you are looking at, can you paste what your (ex-data)
result looks like?#2017-06-2313:24odinodinThis is what (ex-data) returns: {:cljs.spec.alpha/problems ({:some-description-of-the-problem "..."})
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha56508],
:cljs.spec.alpha/value ({:foo "bar"}),
:cljs.spec.alpha/args ({:foo "bar"})
:cljs.spec.alpha/failure :instrument}
#2017-06-2313:25odinodinI want to know what the :cljs.spec.alpha/spec
object is referring to#2017-06-2313:26odinodinor it would probably be more correct to say, what key points to this spec in the registry#2017-06-2313:28wilkerlucio@odinodin gotcha, you can try calling (s/form)
and (s/get-spec)
on it, but I'm not sure if will work#2017-06-2313:28odinodinIāll try that#2017-06-2313:29odinodinthanks#2017-06-2313:39odinodin@wilkerlucio (s/form)
only returns the spec as defined on the (s/fdef)
var. Iāll just parse the var out from the error message instead. Anyway, thanks for your suggestions š#2017-06-2313:39wilkerluciono problem, I'm glad you can at least work around it, maybe other people will have a better solution š#2017-06-2314:50Alex Miller (Clojure team)@odinodin there is a ticket filed for this at https://dev.clojure.org/jira/browse/CLJ-2166 - patches welcome#2017-06-2314:50Alex Miller (Clojure team)I just havenāt had time to look at it yet#2017-06-2314:54odinodin@alexmiller nice! Just what I was looking for :)#2017-06-2316:12samueldevhey all#2017-06-2316:13samueldevdoes anyone know of a library to convert clojure.spec problem explanations to more human-readable output?#2017-06-2322:02bbrinckhttps://github.com/bhauman/strictly-specking seems to have some work in this direction, but I donāt think itās a drop-in solution right now#2017-06-2322:02bbrinckIām also working on something now, but itās early days and not yet ready for release#2017-06-2322:03bbrinck(inspired by the work in strictly-specking and elm errors)#2017-06-2322:03bbrinckIām a bit confused by the value of āinā when using āmap-ofā https://gist.github.com/bhb/6f06dd07bcf5b275a7d4faf3167bfc85#2017-06-2322:04bbrinckI searched JIRA but could not find an issue, so it may be a misunderstanding on my part#2017-06-2322:05bbrinck@samueldev ^-- (sorry, forgot to mention you, see my comment above)#2017-06-2322:11bbrinck@samueldev I think @ericnormand is also maybe working on something for more human-readable error messages?#2017-06-2322:12samueldevthanks @bbrinck ! I'll do some digging with @ericnormand's work and bruces!#2017-06-2323:46richiardiandreaI think also the above odinodin is working on something for re-frame
#2017-06-2417:46Drew Verleeany idea why evaling (clojure.spec.alpha/valid? (clojure.spec.alpha/coll-of int? :kind vector?) [1 2 3])
would cause a stack over flow error?#2017-06-2417:47joshjonesworks fine over here -- can you give a context for how you're using it?#2017-06-2417:49Drew Verlee@joshjones from within the cider repl#2017-06-2417:52Drew Verleehmm#2017-06-2417:52Drew Verleei restarted my repl and it works fine. I think i know the interaction thats causing it, if not whyā¦#2017-06-2417:54Drew Verleenope, i was wrong, i have no idea what was causing it. š#2017-06-2417:55joshjoneshmm strange#2017-06-2516:34jjttjjI know spec is maybe too new for idioms to develop but is there a clear answer to having :x.specs.user/email
, :x.specs.user/password
and then :x.specs.user/user
for the whole user entity or is :x.specs/user
a better name for the "entity", what's everyone's preferences here?#2017-06-2519:24jjttjj(ns x.specs.user
(:require [clojure.spec.alpha :as s]))
(s/def ::username string?)
(s/def ::password string?)
(s/def ::user ;;or is :x.specs/user better?
(s/keys :req [::username ::password]))
#2017-06-2520:16jjttjjI'm personally convinced x.specs/user
is clearly better than ::user which expands to :x.specs.user/user
, just trying to get a second opinion before imposing my will on others.#2017-06-2520:18luchiniWe have been doing something similar to :x.specs.user/user
in your example. It seemed like a natural choice at first and it quickly became a shared language across the team.#2017-06-2520:20luchiniSame pattern you suggested: always the last s/def
of the file and one ns
per entity.#2017-06-2520:21luchiniIām also not sure if this will evolve into an idiom but surely has made the communication in our team much easier.#2017-06-2520:22jjttjjthanks for the input!#2017-06-2520:26luchiniThereās an interesting side-effect thatās been great for the specifics of our project. We receive entity names from a remote system in a simple string. We can then convert the string into :x.specs.<string>/<string>
and boom: we have our spec š#2017-06-2520:34jjttjj@lucascs awesome yeah that's basically what I'm going for with the overall project set up but is there a reason you go with :x.specs.<string>/<string>
instead of :x.specs/<string>
for the composite entity at the end of the file? Because what you're saying could be done with either of those options just as easily right? (Sorry to painfully over analyze small details here)#2017-06-2520:55luchiniIndeed it could. In practice the reasoning behind our choice was purely aesthetics. Kind of āeverything related to the user entity is namespaced with :x.specs.user
.ā Otherwise we would need to always have the entity itself as a bit of an exception (defining it with :x.specs/user
instead of autoexpanding with ::user
).#2017-06-2520:55luchiniPretty much a personal taste thing#2017-06-2520:56luchiniFrom a communication perspective, itās one less cognitive level.#2017-06-2520:57jjttjjcool thanks again š#2017-06-2609:30vikeriI have trouble running stest/check
in a repl. It just returns an empty vector immediately, just as if the symbol was undefined. I have required the symbol and I can evaluate it in the REPL but it seems the check
canāt find the symbol. How can I make sure that stest/check
finds the function that I gave to it?#2017-06-2611:14degI'm very excited about the goal of clojure.spec leading to improved error messages, but there is still a ways to go.
I just typo'd :require
in
(ns raven.intro
(:requre [clj-http.client :as http]
[raven.secrets :as secrets]))
and got the following user-friendly error:
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec:
In: [1] val: ((:requre [clj-http.client :as http] [raven.secrets :as secrets])) fails at: [:args] predicate: (cat :docstring (? string?) :attr-map (? map?) :clauses :clojure.core.specs.alpha/ns-clauses), Extra input
:clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x1331d7d5 "
#2017-06-2611:16mpenetthere's room for improvement for sure, there isn't a lib that does human readable translation yet?#2017-06-2611:18degIt takes a lot less than that. Just a spec that had a list of valid keywords for the ns form could trivially pump out ":requre is not one of [:import :require :use ...]"#2017-06-2615:48Alex Miller (Clojure team)deg: Itās actually a lot trickier than this from a spec perspective due to the āformā structure of the dsl. If we were to design the ns api now, it is highly unlikely we would do it this way (would probably be a map). This turns out (due to the high fanout and nested differences) to be a particularly challenging case for spec to generically give a good error message.#2017-06-2611:19mpenetI guess it goes against the choice of making specs maps open to extension tho (which is arguably good or bad)#2017-06-2611:20degThe problem is that there are very few people in the community who are simultaneously likely to (1) hit this kind of error; (2) take more than a few seconds to spot the problem and feel the pain; and (3) have the comfort level to dive into the source of clojure core.#2017-06-2611:20mpeneteven tho it's not a map here#2017-06-2611:20mpenetsure I agree, personally I wish we could specify strict sets for some kind of specs#2017-06-2611:20mpenetnot sure how it's implemented for ns tho#2017-06-2611:21degAnd you are right too, of course, that spec's philosophy is generally against closed lists of keywords. But, enough of us have written that for our own purposes. And, I'm sure that very few people would be against tight checking for the sake of error checking of special forms or canonical macros.#2017-06-2615:49Alex Miller (Clojure team)deg: just because itās hard for spec to produce a good generic error here does not mean that the ns macro canāt take matters into its own hands instead to provide a customized response. not a done deal.#2017-06-2611:21mpenetyeah but here it actually is strict, my bad š#2017-06-2611:22mpenetit's just the error that s cryptic#2017-06-2611:22degYup.#2017-06-2611:23degAnd, relatively easy to fix any one error. The challenge is creating a framework where, anytime a newbie is bitten by one of these messages and reports it, it triggers a process that makes that message be forever better for the next user.#2017-06-2611:23luxbockif you specify the legal keywords for the NS form as a set, then that's still very much open to extension, no?#2017-06-2611:24luxbockjust add more keywords to the set if the form ever gains more functionality#2017-06-2613:34wilkerlucio@jjttjj one other thing to consider is: what is user
? because in my experience that can vary wildly inside of your system, a login user might require login
and password
while a registering user might require much more, think about this, and you end up having more specific entities (like: login-user
and new-user
) or you might drop the entities are all (that might vary a lot depending on much re-use you can give to those entities)#2017-06-2614:09bbrinckIām a bit confused by the value of āinā when using āmap-ofā. When there is something wrong a value in the map, the in
path doesnāt actually seem to point to that value. For example:
https://gist.github.com/bhb/6f06dd07bcf5b275a7d4faf3167bfc85#2017-06-2614:11joshjones@vikeri Are you sure you have fdef
'd the function?
(defn foo [x] x)
=> #'sandbox.spec/foo
(s/fdef foo :args (s/cat :x int?))
=> sandbox.spec/foo
(stest/check `foo)
=>
({:spec ...,
:clojure.spec.test.check/ret {:result true,
:num-tests 1000,
:seed 1498486054141},
:sym sandbox.spec/foo})
#2017-06-2614:15joshjones@vikeri If you fail to either (1) fdef
a defined function, or (2) give check
a symbol which does not resolve to a function, it will exhibit the behavior you describe:
(defn bar [x] x)
=> #'sandbox.spec/bar
(stest/check `bar)
=> ()
(stest/check `not-a-function)
=> ()
#2017-06-2614:17stathissiderisis there any way to provide a custom message in the output of s/explain
?#2017-06-2615:50Alex Miller (Clojure team)stathissideris: no#2017-06-2615:52stathissideristhanks. Are there any plans for it, or is it out of scope/a bad idea?#2017-06-2615:56Alex Miller (Clojure team)there are no plans for it right now. the idea is that specs should be able to give you generically consistent errors. The explain-data error you get has enough info that you should be able to build custom user errors from that if desired - I believe Sean Corfield is someone doing a lot of this right now.#2017-06-2616:17stathissiderisI get it. I guess I have a slightly more complex case where I use s/&
to further validate a conformed value and s/explain
reports the whole code of the anonymous function as the failing predicate, which is useful, but it would be even better to have a human readable message to say what was expected#2017-06-2616:17mpenetsomewhat related: https://dev.clojure.org/jira/browse/CLJ-2115#2017-06-2616:18stathissiderisIt looks like that: val: ... fails spec: :monitor.settings/aliases-header predicate: (fn [{:keys [aliases]}] (apply distinct? (map :alias aliases)))
#2017-06-2616:19mpenetyou can create custom Spec impl but it's risky#2017-06-2616:19stathissideris@U050SC7SV I donāt want it that bad, itās more of a ānice to haveā for me š#2017-06-2616:36Alex Miller (Clojure team)s/&
is indeed a special case and we are likely going to have to add support for custom forms there regardless (to support things like s/keys*
which use it for implementation), but unlikely we would have a custom explain there beyond that#2017-06-2616:53stathissideris@alexmiller ok, thanks for the explanation and thanks for your efforts in general!#2017-06-2709:50stathissiderisOne way to make this a bit more self documenting would be for the second argument of s/&
to be named function with a descriptive name instead of an in-place anonymous function#2017-06-2614:17vikeri@joshjones Hmm, I required the ns where the fn was defined but maybe that didnāt define the fdef. That is probably the issue.#2017-06-2614:18vikeriThanks for the pointer#2017-06-2614:30bbrinckA simpler example: https://gist.github.com/bhb/c8d01c455494921a3698a9cf951272ff of how :in
works with map-of
. I think Iām beginning to understand how the path works. [:hi 0]
means something like āconstruct a key/value pair from the map and the key :hi
, then navigate to the 0th element of that k/v pairā#2017-06-2615:51Alex Miller (Clojure team)correct - maps are conformed as a sequence of map entries (specāed as a tuple of k and v)#2017-06-2615:53Alex Miller (Clojure team)however, that path wonāt get you to the right place so I think that 0 is actually going to get you to the wrong place.#2017-06-2615:57Alex Miller (Clojure team)and that seems like a bug#2017-06-2615:58Alex Miller (Clojure team)and would be happy to see a jira about that. it is related to https://dev.clojure.org/jira/browse/CLJ-2080 but not addressed by that ticket#2017-06-2616:08joshjonesI've been looking at it for a few minutes now @alexmiller -- and I was going to say it looked like a bug too (but was not confident enough that my understanding about it was correct). the second element of the :in
in this case is which element of the tuple spec caused the error. But the first element (`"hi"`) is coming from the every-impl
spec, and this part particularly does not seem correct#2017-06-2616:09Alex Miller (Clojure team)It's tricky #2017-06-2616:09joshjonesyes, it seems so#2017-06-2616:10joshjonesputting some println's in the spec code:
(clojure.spec.alpha/explain-data :foo/user-map {"hi" "foo"})
EVERY IMPL, mapping
i: 0
v: [hi foo]
TUPLE IMPL, mapping
i: 0
form: clojure.core/string?
pred: #object[clojure.core$string_QMARK___6415 0x674c583e
#2017-06-2616:11Alex Miller (Clojure team)I would consider this on top of the patch for 2080, which improves things in the explain case, whereas this is the conform case which doesn't currently use the kfn#2017-06-2616:12Alex Miller (Clojure team)Rich and I have an ongoing discussion/argument about what happens here :)#2017-06-2616:13joshjonesi can see why, as it's not necessarily a clear cut answer as to what should be there. from one of your spec slides, I have that :in
represents: "vector of specs from root spec to failing spec"#2017-06-2616:15joshjones@bbrinck it seems your confusion was justified š#2017-06-2616:20bbrinck@alexmiller @joshjones I appreciate the info. I will look at the tickets and patches mentioned above and file a follow up bug. I agree itās tricky - especially in the case where the key is wrong. Given a nested data structure, how do I provide a path to a map key? AIUI, it doesnāt work if I consider a āpathā to equal āa vector of keysā that would work with get-in
.#2017-06-2616:21bbrinckWhat Iāve started to do is try to construct a function (maybe this already exists?) that takes some data + an :in
path and retrieves the value.#2017-06-2616:22bbrinckThatās really what is driving this: Iād like to be able to take some (invalid) data + a problem (which contains the :in
) and be able to use the unique :in
value to get the problematic value, which is likely deep inside the original data#2017-06-2616:23bbrinckI had originally (naively) thought that I could use get-in
for that function, but I think itās more subtle than that, so Iām writing my own to use these special :in
paths.#2017-06-2616:30Alex Miller (Clojure team)the idea is that you should be able to do that#2017-06-2616:32bronsaisn't that impossible if you also want to be able to reach keys?#2017-06-2616:34Alex Miller (Clojure team)itās not possible in all cases#2017-06-2616:34Alex Miller (Clojure team)but I donāt know that itās useful to specify [<key> 0] either#2017-06-2616:38bbrinckYeah, tricky. For my cases, Iām considering changing my code to accommodate a āspec pathā that allows me to reach keys i.e. writing new functions that work like get-in
or update-in
with this new type of path#2017-06-2617:14stathissiderisjust a thought, not a question: I think Iāll write a few āreadableā and size constrained generators for strings, keywords and symbols so that my eyes donāt bleed whenever I try to read the output of s/exercise
(and I know itās a good thing that it produces āchallengingā output)#2017-06-2711:44abhiragHi, kinda new to clojure as well as clojure.spec#2017-06-2711:45abhiragWas trying out specifying the above function and stest/check hangs forever#2017-06-2711:45abhiragam I doing anything wrong here?#2017-06-2713:44Alex Miller (Clojure team)@abhirag how long did you wait? (presumably not āforeverā :) it works for me, but takes a while.#2017-06-2713:44Alex Miller (Clojure team)āa whileā being a couple minutes#2017-06-2713:48gfredericksš³ why should that take a couple minutes?#2017-06-2713:49Alex Miller (Clojure team)a fair question :)#2017-06-2713:50abhiragI did try and restrict the count to 10 to reduce time, my dev machine might be slower than yours Alex, I will try again and try and not get afraid of the cooling fans revving up :)#2017-06-2713:50abhiragI also tried to restrict the :test-nums to 10#2017-06-2713:51abhiragBut still the running time wasn't getting shorter#2017-06-2713:51gfredericksthat definitely sounds like a problem to me#2017-06-2713:51Alex Miller (Clojure team)yeah, that seems weird#2017-06-2713:52Alex Miller (Clojure team)s/coll-of with a :count parameter will create a gen/vector with a size#2017-06-2713:53abhiragYeah I did learn that from your guide :)#2017-06-2713:54abhiragAnd also if I just specify the function using fdef and don't give an implementation#2017-06-2713:54abhiragAnd then try check#2017-06-2713:54abhiragIt just gives a null pointer exception#2017-06-2713:55abhiragNo error message regarding the fact that no implementation was provided :P#2017-06-2713:56abhiragThought I should mention that too#2017-06-2713:56abhirag:)#2017-06-2713:56Alex Miller (Clojure team)@gfredericks (g/sample (g/vector g/large-integer 10) 1000)
seems like it should be close to what Iād expect and thatās fast#2017-06-2713:59gfredericksalexmiller: yeah that's what I was imagining#2017-06-2714:01Alex Miller (Clojure team)although itās also going to put those in lists, then put that in a vector for the args, then invoke the function, then check the return is a list, reverse that list, and compare it to the input list#2017-06-2714:03Alex Miller (Clojure team)but even so#2017-06-2714:43Alex Miller (Clojure team)this is indeed pretty jacked and is already logged https://dev.clojure.org/jira/browse/CLJ-2103#2017-06-2714:43Alex Miller (Clojure team)but I did not fully understand this issue before#2017-06-2715:01gfredericksI don't understand it at a glance; let me know if it'd be helpful for me to look closer#2017-06-2713:57Alex Miller (Clojure team)@abhirag there is a ticket for that error case, which I think went into alpha17, not sure which youāre using#2017-06-2713:58abhiragThe version mentioned in the guide#2017-06-2713:58abhiragLet me check#2017-06-2713:58Alex Miller (Clojure team)I think the guide says alpha16 right now#2017-06-2713:59abhiragAlright I will upgrade, thanks for your help :)#2017-06-2714:00Alex Miller (Clojure team)maybe Iām confusing that with something else. if you still see, feel free to log a jira for it#2017-06-2714:02abhiragWill do :)#2017-06-2714:29abhirag@alexmiller I am still getting this error#2017-06-2714:30abhiragI haven't ever logged a defect before in clojure, but if you feel that this deserves a ticket I'll log one š#2017-06-2714:31Alex Miller (Clojure team)yeah, I think so. you can create an account at https://dev.clojure.org/jira/secure/Signup!default.jspa and then log a defect at https://dev.clojure.org/jira/browse/CLJ#2017-06-2714:32Alex Miller (Clojure team)Iām still looking at your gen stuff too. It shouldnāt be that slow. I believe itās the :kind list?
that is causing the slow-down.#2017-06-2714:34abhiragyeah I had tried property testing in other languages before, that snippet was an overkill, my main motive with that was that if we already have a function tested and need to rewrite it for optimization etc. we could just use the old function to test it#2017-06-2714:35abhiragnow I realize that with spec all I really need is that the rewritten function have the same spec#2017-06-2714:37wilkerlucio@abhirag I noticed on your REPL example that you didn't defined the my-reverse
function#2017-06-2714:38abhiragyeah I realize that š that was just to get that null pointer exception, I was gonna log a ticket to make that error message better#2017-06-2714:40wilkerluciocool, just wondered if was on purpose :)#2017-06-2714:43Alex Miller (Clojure team)@abhirag the slow gen is actually already logged https://dev.clojure.org/jira/browse/CLJ-2103 and is definitely in need of some work#2017-06-2714:45abhiragone more quick question (stest/check `my-reverse {:num-tests 10})#2017-06-2714:45abhiragis this the correct way to reduce the number of tests?#2017-06-2714:50Alex Miller (Clojure team)no :)#2017-06-2714:51abhiragthat actually didn't throw any exception, but as I was not getting any output, there wasn't any other way to know š#2017-06-2714:51Alex Miller (Clojure team)I think
(stest/check `my-reverse {:clojure.spec.test.check/opts {:num-tests 10}})
#2017-06-2714:52abhiragalright got it, thanks š#2017-06-2714:52abhiragI'll try and use :onto vector#2017-06-2714:53abhiragas described in the issue description#2017-06-2714:53abhiragto get away from the slow running time for now#2017-06-2714:58Alex Miller (Clojure team)I think using :into ()
in addition to :kind list?
would help#2017-06-2714:59abhiragalright will give that a try š#2017-06-2816:43lwhortonwhats the proper way to define a spec referring to another namespace?
(s/def ::foo :bar.alice/bob)
(s/def ::my-spec (s/keys :req-un [::foo]))
#2017-06-2816:44lwhortonfor some reason i get complaints about unable to resolve spec :bar.alice/bob
using the above method#2017-06-2816:44lwhortonmight i be required to be explicit in :require [bar.alice :as bar.alice]
so load-order is taken care of properly?#2017-06-2816:50seancorfieldThe ns containing that spec (`:bar.alice/bob`) must be loaded before you can use it in another spec. Otherwise the s/def
will not have been executed.#2017-06-2817:16stathissiderisis there a way to override the default generator of s/keys
but only for a single key? (without renaming the key)#2017-06-2817:19bbrinckWe have some JSON data coming in over the wire that uses strings as keys e.g. {"city" "Denver" "state" "CO"}
.
Weāve been converting all strings keys to keywords in order to spec them with s/keys
, but of course, doing the mapping takes a call to clojure.walk/keywordize-keys
and then, if there is a problem that we want to report to the dev, we might need to convert back to strings to make a sensible error about the data.
In this case, do people do the conversion like this? It would seem like being about to have string keys would be useful e.g.
(s/keys :req-str [:location/city :location/state])
where :req-str
is like :req
and :req-str-un
would be like :req-un
, but for string keys#2017-06-2819:27Alex Miller (Clojure team)bbrinck: would be reasonable to file a jira enhancement for htis#2017-06-2819:28bbrinck@U064X3EF3 Can do!#2017-06-2817:40grzmI had some slow checks that I was trying to performance tune. One approach I wanted to try was swapping out the fdef spec during the stest/check
run, similar to how you can swap out generators using the :gen
option. I tried instrumenting the function and using the :spec
option prior to the stest/check
run, but that didn't have an effect. Does this seem like a reasonable thing to do? Is there a way to do it?#2017-06-2819:28Alex Miller (Clojure team)grzm: check takes a generator override map - did you try that?#2017-06-2819:32grzmIt's not clear to me how providing a generator would replace the function spec: I'm not trying to change the values that are supplied to the function: I'm trying to change the spec that's applied to the function as a whole for the scope of the check. Or am I misunderstanding?#2017-06-2819:32grzmWhat would the generator be generating in this case?#2017-06-2819:33Alex Miller (Clojure team)oh, then instrument with a replacement spec should be what you want#2017-06-2819:34Alex Miller (Clojure team)however, calling instrument correctly is sometimes hard#2017-06-2819:40Alex Miller (Clojure team)in particular, anything that is being changed (including the thing whose spec is changing) needs to be in the list of instrumented vars. You should see that var in the return value from instrument as well - if you donāt, it wasnāt changed.#2017-06-2819:41grzmI'm working up a short example that describes what I'm trying.#2017-06-2819:43Alex Miller (Clojure team)hereās a spec replacement example:
(defn a [x] (if (zero? x) "a" (inc x))) ;; special behavior on 0
(defn b [y] (if (zero? y) 0 (+ (a y) 10))) ;; but b guards this
;; so use simpler spec when testing b
(stest/instrument `a
{:spec {`a (s/fspec :args (s/cat :a int?) :ret int?)}})
(stest/check `b)
#2017-06-2819:48grzm(defn a [x])
(s/fdef a
:args (s/cat :x int?)
:fn (fn [_] true))
(s/fdef b
:args (s/cat :x int?)
:fn (fn [_] false))
;; should pass
(stest/check `a)
(stest/instrument `a {:spec {`a `b}})
;; should fail
(stest/check `a)
#2017-06-2819:49grzmWhat I've been trying is more direct. In your example, you're checking b
which calls a
. I'd like to check a
directly.#2017-06-2819:52Alex Miller (Clojure team)Iām not sure the val of the :spec map will actually resolve that via the registry - did you try passing the actual spec there?#2017-06-2819:52Alex Miller (Clojure team)(stest/instrument `a {:spec {`a (s/get-spec `b)}})
#2017-06-2819:54Alex Miller (Clojure team)doesnāt seem like that works either#2017-06-2819:55grzmI don't see it working, either. You're a faster typer than I am š#2017-06-2819:56grzmTaking a step back, does this seem like a reasonable thing to do?#2017-06-2819:56Alex Miller (Clojure team)yes#2017-06-2819:57grzm(stest/instrument `a {:spec {`a (s/fspec :args (s/cat :x int?) :fn (fn [_] false))}})
#2017-06-2819:57grzmThat doesn't work either, btw (for completeness)#2017-06-2819:58grzmI'm considering taking your spec course at the Conj. How far into the weeds are you going to get? Do you think you'll have a syllabus available?#2017-06-2820:00Alex Miller (Clojure team)the example I gave you above is from the spec course#2017-06-2820:01Alex Miller (Clojure team)so about that far :)#2017-06-2820:03grzmGotcha. š#2017-06-2820:05grzmWould you like me to open a ticket for this?#2017-06-2820:07Alex Miller (Clojure team)sorry, I was looking at the code. I understand why it doesnāt work, still thinking about what that means though.#2017-06-2820:07Alex Miller (Clojure team)the spec override is used when building the wrapped var in instrument#2017-06-2820:08Alex Miller (Clojure team)but check still uses the version in the spec for checking ret and fn, not the overridden spec#2017-06-2820:09Alex Miller (Clojure team)and indeed check canāt even see that - itās just part of the check-fn on the instrumented var#2017-06-2820:09Alex Miller (Clojure team)seems like it would be totally reasonable to want to do something like this in the context of check though#2017-06-2820:10Alex Miller (Clojure team)instrumented vars will only catch invalid invocations, not invalid results.#2017-06-2820:11Alex Miller (Clojure team)so I think a ticket to add the ability for check to take spec overrides etc would be reasonable. we have talked about making things like generator overrides, spec overrides etc a consistent api used across exercise, exercise-fn, instrument, check, etc which would be in this area as well#2017-06-2820:12grzmOkay. I'll open one. (No need to apologize, by the way. I suspected as much, and even if you were doing something else, I'm grateful for the time and attention you're providing.)#2017-06-2820:17grzmI was similarly surprised about generator overrides with instrument and check. I expected generator overrides to apply globally. From what I can tell, they only apply to the checked function, not the functions called by the checked function.#2017-06-2820:17Alex Miller (Clojure team)itās a little more subtle than that#2017-06-2820:18Alex Miller (Clojure team)they do apply globally, but there are a couple cases where they donāt get picked up#2017-06-2820:18Alex Miller (Clojure team)there are some tickets on this#2017-06-2820:19grzmHappen to have pointers to those tickets handy to see if they cover my use case?#2017-06-2820:19Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2079#2017-06-2820:20grzmCheers.#2017-06-2820:21Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2095 is another, although Iām not sure that this is something we actually can or will change#2017-06-2820:28grzmHere's the ticket I opened. https://dev.clojure.org/jira/browse/CLJ-2193#2017-06-2820:29grzmAre you guys happy with Jira? Is it worthwhile for someone to make a Clojure syntax highlighter plugin?#2017-06-2823:51grzmHere's a gist showing the instrument
generator override behavior I mentioned above. https://gist.github.com/grzm/0ada176caee5bcdab39d820577ed3823 I'll bring this up in #clojure-spec as well and continue discussion there unless you bring in back here. Thanks again for your help today.#2017-06-2817:54lwhorton@bbrinck whatās your concern, performance?#2017-06-2817:57bbrinckPerformance and the fact the validated data doesn't match the original, so error messages either need to be converted back or are somewhat confusing. #2017-06-2817:58lwhortonas always with performance i would test and verify that itās going to be an issue before worrying about doing 2x conversions#2017-06-2818:00grzm@bbrinck with respect to the validated data doesn't match the original, you're already doing a conversion (I suspect) from JSON objects to Clojure maps. The similarity between the two (including syntax) obscures this a bit.#2017-06-2818:01bbrinck@lwhorton. Fair point about perf. My greater concern is complexity of implementation when converting to validate and then back to report for what seems like a common case of string keys (when working with JSON). #2017-06-2818:03bbrinckFrankly it's not a huge cost. Just wondering if this workaround is how others are tackling this case. #2017-06-2818:03lwhortonmaybe im looking at it wrong, but i dont see complexity around (keywordize-keys json)
and (string-keys edn)
#2017-06-2818:04lwhortonif you convert to edn, run a spec/validate and it fails, then convert it back to json and make a POSTā¦ that all seems fairly straightforward?#2017-06-2818:06bbrinckI'd ideally like to give error messages that are relevant to the original data. Imagine an API that accepts JSON. If I want to give an errr message about non conformance, I want this to be about strings not keywords. #2017-06-2818:07bbrinckI suppose I can explain-data and then walk each problem and convert back to strings. The n format the data?#2017-06-2818:08lwhortonor hold a reference to the original json, right?#2017-06-2818:08bbrinckBut that's just the whole thing right? That's not the specific section that has the problem #2017-06-2818:08lwhortoni suppose if you wanted to get into the exact location where a conform failed, yes you might have to do explain-data#2017-06-2818:09bbrinckYou'd need to walk each problem and reconvert, then reformat. But you are correct, it's not a huge amount of work#2017-06-2818:09lwhortonyou could also write your spec to be some form of s/cat :value string? ...
etc.#2017-06-2818:10bbrinckAnd I reformat I just mean reimplement 'explain-str'#2017-06-2818:10lwhortonbut that might be way more complex than what is already offered by (s/keys)
#2017-06-2818:10lwhorton(essentially spec out the JSON implementation)#2017-06-2818:11lwhortoni would also look around for some libs that might already exist for this sort of thing#2017-06-2818:11bbrinckRight. Thanks for the ideas! Good to know I'm not missing an obviously simpler way :)#2017-06-2818:12lwhortonwith plumatic.schema there was coersion, but iām not sure if spec went down that path as well#2017-06-2818:16bbrinckAIUI, spec is avoiding coercion but I could be mistaken. I might just end up implanting a version of 's/keys' that accepts strings (and produces a generator that uses string keys). Could be useful for JSON validation #2017-06-2818:17bbrinckNote I agree the specs themselves should be keywords. I'm just thinking that they could be included in maps as strings. #2017-06-2818:17lwhortonIāve got a question for you š ā¦ have you ever used branching in a spec to define a spec? Say you have a key :foo and :bar, but :bar can pull from set A or B of valid values. Is there a way to make :bar read :foo and decide which is a valid spec?#2017-06-2818:19grzm@lwhorton Does multi-spec do what you want? Or are you getting at something different?#2017-06-2818:22lwhortonhah, that looks exactly like what Iām talking about#2017-06-2821:32dealyspec newbie here, please excuse my ignorance. I have a data structure which I've defined a spec for. I've just needed to add a channel as one of the items. How do I update my spec for this type of a field?#2017-06-2821:44seancorfieldPretty sure thereās no way to spec a channel (same as thereās no way to spec atom/delay/etc) so you can just use any?
(or not provide an actual spec for that key).#2017-06-2821:46seancorfieldSpecāing something to be a channel wouldnāt be very useful: you couldnāt generate values for it (since it has type channel).#2017-06-2821:47dealyoh, ok, there's a lot of spec I'm still pretty confused about. So, spec'ing data structures is primarily for building up things from basic types (ints doubles bools and functions) as opposed to interfaces and complex objects?#2017-06-2821:49seancorfieldHmm, not reallyā¦#2017-06-2821:50seancorfieldYou can spec domain-level entities ā so you can give things meaning ā but you need to think about what youāre trying to achieve with spec. Why are you specāing a particular structure? What are you going to use the spec for?#2017-06-2821:50seancorfieldThereās not much point in specāing everything and I think itās a common misunderstanding to try to treat spec like a type system and āspec everythingā.#2017-06-2821:52seancorfieldYou also need to consider whether you want specs to be āgeneratableā (you probably mostly do) so you need to avoid specs with types that canāt be generated (since theyāre not very useful, in general).#2017-06-2821:54dealyyea, that makes sense. I haven't gotten into that part yet. Still only using it as a way of just making sure my datastructure remains in the format I expect/need.#2017-06-2821:57seancorfieldAre you using conform
or valid?
or some other way of checking against the spec?#2017-06-2821:58dealyvalid?#2017-06-2821:58dealythis is all part of my first GUI app using re-frame so its validating the main app-db as the UI changes#2017-06-2821:58seancorfieldSince spec checks are all runtime, if you really need to āassertā your data structure has a specific key, you can always spec it as any?
and if you use something as a channel and it isnāt, itās going to blow up at runtime anyway.#2017-06-2821:59dealyits a lot to learn all at once#2017-06-2821:59seancorfieldI would be a bit suspicious of data including a channel thoāā¦#2017-06-2822:00dealynormally yes, but it is an artifact of a timer that needs to be cleaned up later#2017-06-2822:00seancorfieldafter all, thatās not data you can serialize and pass around so itās not really dataā¦#2017-06-2822:00seancorfield(I may be taking a bit of a purist view here ā not sure how others feel?)#2017-06-2822:11lwhortoncan someone point out my (probable) misuse of multi-spec? what iām trying to go for, in plain english, is to specify a map where some keyās spec depends on the value of another key. If key a is one of ācatā or ādogā, key b should be specād differently in either case.
(defmulti b-depends-on-a :a)
(defmethod b-depends-on-a "cat" [_] (s/or :cat-only-thing ::cat-thing
:cat-or-dog-thing ::shared-things))
(defmethod b-depends-on-a "dog" [_] (s/or :dog-only-thing ::dog-thing
:cat-or-dog-thing ::shared-things))
(s/def ::a #{"cat" "dog"})
(s/def ::b (s/multi-spec b-depends-on-a :a))
(s/def ::foo (s/keys :req-un [::a
::b]))
#2017-06-2822:14lwhortonI would expect to be able to run something like (s/conform :: foo {:a "cat" :b ...})
or (s/conform :: foo {:a "dog" :b ...})
and have the response [:cat-only-thing ā¦]
or [:dog-only-thing ā¦]
or [:cat-or-dog-thing ...]
depending on the value of b.#2017-06-2822:14lwhortonIs this totally off?#2017-06-2822:24seancorfieldI think your multi-spec needs to be a hash map that has a key :a
#2017-06-2822:24seancorfieldSo ::b
would be spec a hash map ā do ::cat-thing
etc spec hash maps?#2017-06-2822:25seancorfieldSee this page https://clojuredocs.org/clojure.spec/multi-spec#2017-06-2822:26lwhortoniāve been banging my head on that page for quite a while now .. i dontā really understand the ātagā argument to be honest#2017-06-2822:26seancorfieldThis https://clojure.org/guides/spec#_multi_spec also shows the multi-specs as being hash maps.#2017-06-2822:27lwhortonthe call to s/multi-spec event-type
in that example is calling on a defmulti event-type
, unless Iām misunderstanding#2017-06-2822:27seancorfieldThe :event/type
key is common to all the (hash maps) that are valid for that set of specs.#2017-06-2822:29seancorfieldIn your example, you should have a hash map that has a key :a
that is the ātagā and it can have the value "cat"
or "dog"
and the defmethod
s should returns specs for the cat, or dog, hash map that includes the tag :a
.#2017-06-2822:29lwhortonoh you mean the object on which iām invoking the spec? i.e. s/conform ::foo {:a "cat" (or "dog") :b something-else}
?#2017-06-2822:30seancorfieldRightā¦ so your defmethod
s should be (s/keys :req-un [::a :cat/b])
and (s/keys :req-un [::a :dog/b])
#2017-06-2822:31seancorfieldthen the multi-spec
selects which spec to use based on the tag :a
#2017-06-2822:31lwhortonah so i guess I cannot get away with what I was originally trying to do#2017-06-2822:31seancorfieldDepends what ::cat-thing
, ::dog-thing
, and ::shared-things
areā¦#2017-06-2822:32lwhortonwhich is something like āif the multimethod matches on ācatā, return a spec which is an (s/or ā¦)
#2017-06-2822:32lwhortonso you are saying ::cat-thing, ::dog-thing, ::shared-things would have to be maps containing the key :a, correct?#2017-06-2822:34seancorfieldWhat are you trying to do?#2017-06-2822:35seancorfieldIf you explain how you want :b
to vary, I can probably show youā¦#2017-06-2822:35lwhortoni have a piece of data that looks like this {:status "" :operation "" :state ""}
#2017-06-2822:35lwhortonactually let me thin that down even more, i have a map {:status "" :operation ""}
#2017-06-2822:36lwhortonoperation can belong to any member of the set #{:idle :downloading :patching}
#2017-06-2822:37lwhortonbut depending on the value of :operation
, the spec for status should be ālimitedā. that is, given the idle
operation, the available statuses should only be one of #{:a :b :c}
, and given the downloading
operation, statuses should only be one of #{:c :d :e}
(note the overlap)#2017-06-2822:37lwhortoni was trying to use spec to enforce this pattern-matching behavior#2017-06-2822:38lwhortonthe (s/or ..)
comes into play because there is some overlap between :downloading / :patching / :idle, but only 1-2 states#2017-06-2822:39seancorfieldSureā¦ so you need to spec the different types of status values, e.g., (s/def :downloading/status #{:c :d :e})
and (s/def :idle/status #{:a :b :c})
#2017-06-2822:40seancorfieldand then your defmethod
would return (s/keys :req-un [::operation :downloading/status])
for the downloading branch and (s/keys :req-un [::operation :idle/status])
for the idle branch#2017-06-2822:41lwhortonoof, so all this pain and confusion because i was trying to do an s/or
(to union) where I should have just defined the full sets outright, regardless of overlaps?#2017-06-2822:42lwhortonill give that a whirl and get back to you.. regardless of the outcome thanks for your help#2017-06-2822:44seancorfield(s/def ::operation #{:idle :downloading :patching})
(s/def :idle/status #{:a :b :c})
(s/def :downloading/status #{:c :d :e})
(s/def :patching/status #{:a :c :f})
(defmulti operation-spec :operation)
(defmethod operation-spec :idle
[_] (s/keys :req-un [::operation :idle/status]))
(defmethod operation-spec :downloading
[_] (s/keys :req-un [::operation :downloading/status]))
(defmethod operation-spec :patching
[_] (s/keys :req-un [::operation :patching/status]))
(s/def ::machine (s/multi-spec operation-spec :operation))
#2017-06-2822:44seancorfield(s/conform ::machine {:operation :idle :status :a})
succeeds#2017-06-2822:45seancorfield(s/conform ::machine {:operation :idle :status :d})
fails#2017-06-2822:45seancorfield(s/conform ::machine {:operation :downloading :status :a})
succeeds#2017-06-2822:45lwhortonindeed it does, and a gen/sample gives me 10 good values !#2017-06-2822:46lwhortonwell itās good to know now that defmethod has to return āthe original matchā + the spec, not some hob-goblin pseudo spec#2017-06-2822:46lwhortonmany thanks for your guidance#2017-06-2822:46seancorfieldThanks for the question ā I hadnāt played with multi-spec
before and this gave me a good opportunity to try it out.#2017-06-2822:47seancorfieldThe different branches could also have different keys etc.#2017-06-2822:48seancorfieldAnd, indeed, doesnāt have to be a hash map but then the tag function would need to be something other than a simple keyword.#2017-06-2822:49lwhortonso does the tag arg on (multi-spec spec tag)
have to be identical to the fn/kwd in defmulti name fn/kwd
?#2017-06-2822:50lwhortonnot really sure why the tag is there.. unless its a way to generate fields under a different key from the one of the multi-match?#2017-06-2822:53bbloomrandom thought: i kinda wish there was a way to alias keys, just as there is a way to alias attributes in datomic - would help with refactoring and āsubtypingā#2017-06-2822:58seancorfield@lwhorton I believe the tag is needed in multi-spec
so that the generator can figure out what key to work with.#2017-06-2822:58seancorfield(since defmulti
could take a function etc)#2017-06-2822:59lwhortoni see .. if you still have that ^above code in a register, what happens when you try to generate some samples?#2017-06-2822:59seancorfieldIf defmulti
was some complex selector function, the argument passed to multi-spec
would have to kind of reverse that I thinkā¦#2017-06-2822:59lwhortondo you end up with {:data-state "" :data-status {:data-state "" :data-status ""}
?#2017-06-2823:00seancorfield([{:operation :idle, :status :c} {:operation :idle, :status :c}]
[{:operation :patching, :status :c}
{:operation :patching, :status :c}]
[{:operation :downloading, :status :c}
{:operation :downloading, :status :c}]
[{:operation :idle, :status :b} {:operation :idle, :status :b}]
[{:operation :downloading, :status :e}
{:operation :downloading, :status :e}]
[{:operation :patching, :status :f}
{:operation :patching, :status :f}]
[{:operation :idle, :status :c} {:operation :idle, :status :c}]
[{:operation :idle, :status :b} {:operation :idle, :status :b}]
[{:operation :patching, :status :c}
{:operation :patching, :status :c}]
[{:operation :idle, :status :b} {:operation :idle, :status :b}])
from s/exercise
#2017-06-2823:00lwhortoninteresting, i probably typo-d somewhere#2017-06-2823:00lwhortonthis spec gen stuff is so great#2017-06-2823:03seancorfieldYup, I pretty much always try to s/exercise
my specs as a sanity check, and also to see the sort of crazy stuff they can produce. We use regex fairly heavily so we rely on @gfredericks test.chuck
which has a generator for regex string specs ā awesome stuff!#2017-06-2823:16spiedengreat clojure.test integration too =)#2017-06-2910:52harriganIs the following behavior for s/&
expected?
(s/conform (s/* int?) []) ;; => []
(s/conform (s/& (s/* int?)) []) ;; => nil
#2017-06-2910:53harriganAFAIKT, (s/& (s/* ...) ...) [])
always returns nil
no matter what predicates are passed to s/&
.#2017-06-2912:13gmercer@harrigan not sure if it is relevant but there was an earlier discussion regarding the difference between s/&
and s/and
#2017-06-2912:16gmercer(s/conform (s/and (s/* int?)) []) ;; => []
#2017-06-2912:21harriganI donāt think this is due to the difference between s/&
and s/and
. The actual instance that tripped me up was:
(s/conform (s/& (s/+ int?) #(= 3 (count %))) []) ;; => :clojure.spec.alpha/invalid
(s/conform (s/& (s/* int?) #(= 3 (count %))) []) ;; => nil
#2017-06-2912:27harriganI expected s/conform
to return invalid in both cases.#2017-06-2912:31gmercerlooking back, it was your discussion ..#2017-06-2912:43gmercerdang!#2017-06-2912:43gmercer(require '[clojure.spec.alpha :as s])
(s/conform (s/& (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 1))) [1])
(s/conform (s/and (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 1))) [1])
(s/conform (s/and (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 0))) [])
(s/conform (s/& (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 0))) [])
#2017-06-2912:47gmercer@harrigan curiouser and curiouser ... šæ#2017-06-2913:18Alex Miller (Clojure team)This is a bug with s/& where it won't check the extra preds if the empty coll passes #2017-06-2913:18Alex Miller (Clojure team)There is a ticket with a patch waiting to go in#2017-06-2913:20Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2182#2017-06-2913:21Alex Miller (Clojure team)@harrigan ^^ I am bad at notification#2017-06-2913:22gmercerthanks Alex - any idea about the double 'exec test' on the (s/conform (s/& (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 1))) [1])
#2017-06-2913:24Alex Miller (Clojure team)Can happen - there is no guarantee on how many times a predicate may be called#2017-06-2913:25Alex Miller (Clojure team)I'd have to spend more time to explain why here#2017-06-2913:25Alex Miller (Clojure team)To understand why that is#2017-06-2913:25gmercernw - but interesting!#2017-06-2915:12stathissiderishas anyone noticed multi-spec
being slow to generate values? it also sometimes fails randomly#2017-06-2915:14stathissiderisIām doing a gen/sample
with a size of 1, and I get ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.
#2017-06-2915:14stathissiderisbut not always#2017-06-2915:15stathissideristhe spec is a multi-spec
that contains maps with strings, nothing fancy#2017-06-2915:15stathissiderisone of the keys is a multi-spec itself#2017-06-2915:15gfredericksthe such-that
exceptions happen when you have an s/and
somewhere and the generator for the first clause isn't likely to satisfy the remaining clauses#2017-06-2915:19gfredericksin the worst case the problem is solved by writing a custom generator for the (s/and ...)
#2017-06-2915:19gfredericks(the slow generation is probably a different symptom of the same problem)#2017-06-2915:20stathissideristhe only suspect thing in there was this:
(def email-re #"[a-zA-Z0-9._%+-]
#2017-06-2915:20stathissiderisbut I removed it and still got the same behaviour#2017-06-2915:20gfrederickswell that looks like it already has the custom generator#2017-06-2915:20stathissideriscorrect#2017-06-2915:21gfredericksis it possible you're using some higher-level spec that expands to an s/and
?#2017-06-2915:22stathissiderishm, I think s/merge
might#2017-06-2915:24gfredericksš¤ that sounds strange#2017-06-2915:25stathissiderisI think I know what it is: Iāve been using this macro to disallow āunknownā keys (I know itās discouraged):
(defmacro only-keys
[& {:keys [req req-un opt opt-un] :as args}]
`(s/merge (s/keys
#2017-06-2915:26stathissiderisI donāt think it plays well with generation#2017-06-2915:26stathissiderisremoving it fixed my problem#2017-06-2915:26stathissideris(it seems)#2017-06-2915:26gfredericksI would've expected s/and
to be used there instead of s/merge
š#2017-06-2915:27gfredericksif that's equivalent, I bet the s/and
would actually generate just fine#2017-06-2915:27gfrederickssince I don't think s/keys
generates any extra keys#2017-06-2915:37EdI'm having a problem with macro expansion and specs ... sometimes the specs are not checked#2017-06-2915:37Edhas anyone else seen this?#2017-06-2915:37gfredericksspecs on macros?#2017-06-2915:37Edi'm having trouble working out the circumstances where it works or doesn't#2017-06-2915:37Edyes#2017-06-2915:38gfredericksthese are specs about the compile-time forms, or the runtime data?#2017-06-2915:38Edi've narrowed down this test case:#2017-06-2915:39Edwhich fails if i use lein test#2017-06-2915:39Edbut passes if I load the code with C-c C-k in cider#2017-06-2915:39Ed@gfredericks compile-time forms#2017-06-2915:40gfredericks"load the code in cider" means actually running the test, not just compiling it?#2017-06-2915:41Edno ... load the code in cider means running cider-load-buffer#2017-06-2915:41Edand then either running the tests with C-c C-t C-n ... or running the tests at the repl#2017-06-2915:41stathissideris@gfredericks thanks, Iāll give it a try#2017-06-2915:41Edwith run-tests#2017-06-2915:42gfredericks@l0st3d okay, so if you add (is false "dangit")
as a second assertion in your test, it will fail there but not with the first assertion?#2017-06-2915:42Edyup#2017-06-2915:43gfredericksokay, I have no idea then#2017-06-2915:43Edit reports that the test was run, but that succeeded ...#2017-06-2915:44Edok ... @gfredericks ... thanks for your help#2017-06-2915:45Edi'm stuck then#2017-06-2915:50lwhortonspec is my new addiction. how did i ever do front-end development without you, my love?#2017-06-2915:51lwhortonif i need to build a feature but the backend team is busy and wonāt get to an api data endpoint in time, no worries: spec can auto-generate a static api response for me and I can go on my way building the html.#2017-06-2916:02gfredericks@l0st3d if I were you I would try to reproduce the problem without cider; I assume you've tried restarting the cider process?#2017-06-2916:03Edyes ... the 3 files I posted above always fail to check macro expansion when run from the terminal using lein#2017-06-2916:04Edcider seems to fix it ... which is how i didn't notice at first š#2017-06-2916:10gfredericksoooh#2017-06-2916:10gfredericksI think I read the whole thing backwards#2017-06-2916:10gfredericksdue to ambiguity of "fail" and "pass" in this case#2017-06-2916:10gfredericksso it's leiningen that has the problem, not cider#2017-06-2916:11gfredericksin that case I'd try reproducing the problem using just java -cp ... clojure.main <file>
#2017-06-2916:22Edyeah ... started doing that ... what's the minimal classpath that you need to start a repl with 1.9.0-alpha17?#2017-06-2916:23Edi've clearly got something missing#2017-06-2916:28gfredericksyou could try running lein classpath
and using that#2017-06-2916:28gfrederickswhich reminds me that if you have deps in your ~/.lein/profiles.clj
then it can be helpful to turn that stuff off via LEIN_NO_USER_PROFILES=1
#2017-06-2916:28Edfair point#2017-06-2916:29Edthanks#2017-07-0618:09wilkerlucio@peeja check this: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L132-L185#2017-07-0618:50peejaIs there any way to use with-gen
without blowing away generator overrides?#2017-07-0619:01peejaIt seems like it can't, since it doesn't have access to the overrides
like gen*
does#2017-07-0619:01peejaAre we meant to define our own macros a la keys
/`map-spec-impl` etc?#2017-07-0717:15englishIāve been working on a Ruby port of spec: https://github.com/english/speculation
Implementation-wise itās got all the important stuff: data validation, instrumentation and generative testing. Iām currently working on improving the docs and writing up some guides.
If there are any Rubyists here jealous of clojureās spec, Iād been keen to hear your thoughts and feedback!#2017-07-0909:21jimmyenglish: hey this is amazing. It's a good lib for me to train some junior Ruby dev before converting them to Clojure completely š#2017-07-0916:58englishThanks! The api is pretty much the same so transitioning should be straightforward#2017-07-0719:53ghadiWhat is the farthest along library with making nice experiences for spec-based errors?#2017-07-0802:07xiongtxIs the file/line/col of spec definitions stored anywhere, as they are for vars when the latter are read?
Itād be nice to have that information so we can jump to a specās definition.#2017-07-0806:04richiardiandrea@xiongtx there has been some plan for that if I remember correctly, unfortunately I cannot find the reference (I think it was exactly in this channel)#2017-07-0817:44jsselmanHow can I use the result of clojure.spec.test/check
to see if a test failed? The doc seems to hint at checking the value of the :failure
key, but that value is false in my test despite there being a clear check error#2017-07-0817:47gfredericks@jsselman is it false
or [false]
?#2017-07-0817:47jsselmanprintln shows it as false
#2017-07-0817:48gfrederickshmm#2017-07-0817:48jsselmanI should mention this is ClojureScript not Clojure#2017-07-0817:48gfrederickswhat's the full return?#2017-07-0817:51jsselmanthe function call is just (stest/check
``sut/normalize)`#2017-07-0817:51jsselmanarg how do i insert a literal backtick#2017-07-0817:51gfredericksthe value under [:clojure.test.check/ret :result]
should work#2017-07-0817:52gfredericksI have no idea what the failure key is meant to mean#2017-07-0817:52jsselmanthe value there is false, does that indicate the test failed?#2017-07-0817:52gfredericksyes#2017-07-0817:52jsselmani guess i can quickly check that#2017-07-0817:56jsselman@gfredericks that does work, thanks!#2017-07-0817:59jsselmanThe spec guide shows an example using (stest/abbrev-result (first (stest/check `ranged-rand)))
, but here it loses the failure. Guess I can manually check#2017-07-0817:59jsselman(I get {:sym raytrace-weekend.vec3/normalize, :failure false}
)#2017-07-0900:15mathpunkThis didn't seem fancy, but it gave me a stack overflow: (s/coll-of map? :kind seqable?)
#2017-07-0900:16mathpunkwhat am I failing to understand here?#2017-07-0900:20mathpunkIn case it's not clear, I'm attempting to express, "This here data is a sequence of maps"#2017-07-0900:22mathpunkSince another way to express that is, (s/* map?)
, I'm using that. But what a strange error that was. (SOLVED)#2017-07-0900:22gfredericksis seqable?
a valid :kind
?#2017-07-0900:56mathpunk"`:kind` - a predicate or spec that the incoming collection must satisfy, such as vector?
"#2017-07-0900:57mathpunkmind you, I'm getting seqable?
out of clojure.future.spec#2017-07-0900:57mathpunkmaybe i'm holding it wrong somehow#2017-07-0911:56gfredericksI'm planning to release schpec with defn+spec
today https://github.com/gfredericks/schpec/commit/4b0de0bc836111fc9953e37c68ca1756a0d09bb8
/cc @olivergeorge#2017-07-0914:12mpenetCouldn't it be made more efficient by conforming 1 arg at a time instead of creating a s/cat at invoke time? That would also allow to avoid & args?#2017-07-0914:13mpenetAh but you do this to pattern match depending on arg type for same arg length. Nevermind#2017-07-0914:14mpenetCool stuff#2017-07-0923:07Oliver Georgehey wow. that goes beyond typing the defns. interesting stuff.#2017-07-0923:09Oliver GeorgeI've been looking at porting some ocaml code. they rely heavily on functions with match statements which looks a lot like this. (I've been using core.match)#2017-07-0923:10Oliver GeorgeHere's a random example: https://github.com/links-lang/links/blob/master/queryshredding.ml#L1234#2017-07-0923:11Oliver GeorgeThat's pretty much sugar for the match statement#2017-07-0923:11Oliver Georgehttps://github.com/links-lang/links/blob/master/queryshredding.ml#L1153#2017-07-1001:01gfrederickscool#2017-07-0920:31gfredericksokay, I released defn+spec
in case you like that sort of thing
https://github.com/gfredericks/schpec#comgfredericksschpecdefnspecdefn#2017-07-1005:28anticrisisHiya folks, does this look like a valid spec? (s/def ::test (s/cat :k keyword? :rest (s/* any?)))
#2017-07-1005:28anticrisisThe reason I ask is I get different results with s/valid?
and an instrumented function#2017-07-1005:32anticrisisI.e. (s/valid? ::test [:foo]) => true
but when used in s/fdef test-fn :args (s/cat :t ::test)
it fails instrumentation#2017-07-1005:38seancorfield@anticrisis regex-style specs combine unless you wrap them with s/spec
as I recall.#2017-07-1005:39seancorfieldSo (s/fdef test-fn :args (s/cat :t (s/spec ::test)))
#2017-07-1005:39anticrisisoh wow, ok#2017-07-1005:40anticrisisthere's something I have a great deal of trouble wrapping my head around with these regexp specs I think#2017-07-1005:42anticrisisthat does the trick, many thanks -- I will meditate on why, for a few hours.#2017-07-1005:43seancorfieldIf you do (s/exercise (s/cat :t ::test))
you'll see what's going on...#2017-07-1005:44seancorfield...it generates sequences and they conform to {:t {:k first-element :rest [other-elements]}}
#2017-07-1005:47anticrisishmm#2017-07-1005:50seancorfieldPerhaps easier to see with this example: boot.user=> (s/def ::pair (s/cat :a keyword? :b pos-int?))
:boot.user/pair
boot.user=> (s/def ::four (s/cat :ab ::pair :cd ::pair))
:boot.user/four
boot.user=> (s/valid? ::four [:x 1 :y 2])
true
boot.user=> (s/conform ::four [:x 1 :y 2])
{:ab {:a :x, :b 1}, :cd {:a :y, :b 2}}
boot.user=>
#2017-07-1005:53anticrisisthat part makes sense, but I'm not connecting it to the use of (s/spec)
to change the behavior of the s/*
operator when defining a function spec#2017-07-1005:54anticrisisI'm scrutinizing the output of s/exercise
with and without s/spec
right now#2017-07-1005:58anticrisisTo me, the conformed output of (s/exercise (s/cat :t (s/spec ::test)))
and (s/exercise (s/cat :t ::test))
look the same#2017-07-1006:01anticrisisThanks for tip about s/spec
with regexp ops and s/exercise
, they point me in the right direction, even if I don't completely understand it. Many thanks.#2017-07-1006:25anticrisisI still think there's something inconsistent going on, either because I'm misusing this simple s/*
form, or because something's broken. Here's a short walkthrough to demonstrate:#2017-07-1006:25anticrisis(s/def ::test (s/cat :k keyword? :rest (s/* pos-int?)))
(s/conform ::test [:foo 1 2 3 4])
;; => {:k :foo, :rest [1 2 3 4]}
(s/conform (s/spec ::test) [:foo 1 2 3 4])
;; => {:k :foo, :rest [1 2 3 4]}
(s/conform (s/cat :t ::test) [:foo 1 2 3 4])
;; => {:t {:k :foo, :rest [1 2 3 4]}}
(s/conform (s/cat :t (s/spec ::test)) [:foo 1 2 3 4])
;; => :clojure.spec.alpha/invalid
(defn test-fn [test] nil)
(s/fdef test-fn :args (s/cat :t ::test) :ret nil?)
(stest/instrument `test-fn)
(test-fn [:foo 1 2 3 4])
;; => exception
(s/fdef test-fn :args (s/cat :t (s/spec ::test)) :ret nil?)
(stest/instrument `test-fn)
(test-fn [:foo 1 2 3 4])
;; => nil
;; Note that in order to get test-fn to pass instrumentation,
;; its :args must use the same form which conform considers invalid
#2017-07-1006:45seancorfield(s/conform (s/cat :t (s/spec ::test)) [[:foo 1 2 3 4]])
One element, of type ::test
.#2017-07-1006:45seancorfieldsimilarly your function accepts one argument, of type ::test
#2017-07-1006:45seancorfieldIt's consistent @anticrisis#2017-07-1006:46seancorfieldThe :args
in s/fdef
is the sequence of arguments.#2017-07-1006:50seancorfieldFor your first s/fdef
-- :args (s/cat :t ::test)
-- your test-fn
would have to be (defn test-fn [kw & nums] nil)
-- first argument :k
, a keyword, and the :rest
of the arguments are numbers.#2017-07-1006:50seancorfielddoes that help @anticrisis ?#2017-07-1006:52anticrisisI see what you're saying... but I thought :args (s/cat :x ::whatever)
meant "the arguments are a list of one element, labeled 'x', which can be conformed with spec '::whatever'#2017-07-1006:57anticrisisYour example of fdef'ing [kw & nums] is helpful to tell me I've been thinking about this all wrong. I would have fdef'd it like this: :args (s/cat :k keyword? :nums (s/* pos-int?))
-- which you're telling me is the same as :args ::test
-- which it is, but those particular neurons in my brain apparently weren't connected.#2017-07-1007:04anticrisisFor the record, I'd like to point out that nesting data structures is confusing. Is there a version of clojure I can use that doesn't do that? Thanks. š#2017-07-1007:14seancorfieldIf you want actual sequences (and nested sequences) then use coll-of etc, not cat šø #2017-07-1007:39hkjelsHow can I stub a function? Typically Iād like to require that on-click
is a function, but it fails when I later exercise it since theres no generator for it#2017-07-1007:40hkjelsCreating a custom generator would be overkill I think#2017-07-1011:12gfredericks@hkjels would a custom generator of (gen/return (constantly nil))
be overkill?#2017-07-1011:41hkjels@gfredericks ohh, hehe. no#2017-07-1011:41hkjelsš#2017-07-1011:41hkjelsthanks!#2017-07-1013:13rickmoynihanHmmmā¦ Iāve just made an observation about specs which Iām wondering if others could validate for meā¦
Basically I have a layered architecture where we do something like this: (->> input transform-1 transform-2 transform-3)
and Iāve just realised that in layout terms it seems better to organise the input specs alongside the transforms, rather than the :ret
specs.
It also seems to more closely align with things like instrument
checking :args
rather than :ret
s
Has anyone else noticed this?#2017-07-1013:55rickmoynihanWhat is the most idiomatic way to spec something to be a single value?
Iām guessing:
(s/def ::single-value #{:foo})
#2017-07-1013:58mpenetyep, you get free gen with that one#2017-07-1014:51rickmoynihanmpenet: yeah the free gen is a big motivator š#2017-07-1013:58mpenetin theory (constantly :foo) would work for validation but no gen#2017-07-1014:01gfredericks(constantly :foo)
would work? That seems identical to any?
to me#2017-07-1014:01gfredericksit's a predicate that always returns truthy#2017-07-1014:01mpenetright#2017-07-1014:01mpenetbad example#2017-07-1014:01gfredericks#(= % :foo)
I guess?#2017-07-1014:01mpenetyes or #(identical? :foo %)#2017-07-1014:02gfredericksjust for keywords#2017-07-1014:02mpenetyup#2017-07-1014:02mpenetbut anyway, the set solutions is better in most case#2017-07-1014:03gfredericksI can imagine people stumbling on #{nil}
relatively often#2017-07-1014:04Alex Miller (Clojure team)use nil?
then#2017-07-1014:04mpenetwell I guess you want nil? in that case#2017-07-1014:04gfredericksyep#2017-07-1014:04gfredericksI guess that sort of error isn't likely to go unnoticed#2017-07-1017:41ghadihttps://clojurians.slack.com/archives/C1B1BB2Q3/p1499457189858308#2017-07-1017:41ghadiBumping that question -- I was looking for some prior art for custom printing mostly#2017-07-1018:41bbrinck@ghadi Iām working on something now, but itās still pre-alpha#2017-07-1018:41bbrinckhope to have something ready by thursday š#2017-07-1018:43bbrinckThe code is under change right now, but a preview of what Iām trying to achieve is in the tests e.g. https://github.com/bhb/expound/blob/master/test/expound/spec_test.cljs#L120-L137#2017-07-1019:06ghadibbrinck: very cool#2017-07-1018:43bbrinckExcuse the lack of documentation or sane organization#2017-07-1018:46bbrinckHas anyone found a use case for specing a map with coll-of
? I see that you can create a spec like (s/def :foo/map (s/coll-of int? :kind map? :into {}))
but I canāt see why this would be useful since everything but the empty map would always fail.#2017-07-1018:48bbrinckI suppose you could maybe replace int?
in that example with a spec for the key/value pairs, but in that case, map-of
seems superior.#2017-07-1018:48Alex Miller (Clojure team)there are some cases where it is useful to spec a map as a sequence of entry tuples#2017-07-1018:49Alex Miller (Clojure team)āhybridā maps (combination of s/keys and s/map-of styles) is one case#2017-07-1018:49Alex Miller (Clojure team)example here: http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-07-1020:09souenzzoalexmiller: When I look to the result of (s/conform ::binding-form
it remember me about ast
's ... may in the future, can spec/conform be used as analyzer?#2017-07-1020:18Alex Miller (Clojure team)If you use that in the general sense, then yes absolutely - itās a great tool for turning s-expressions into a tree of maps describing a DSL syntax#2017-07-1018:54bbrinck@alexmiller Ah, I see. Youāre using a tuple
as the āinnerā spec. That makes sense. Thanks for the info!#2017-07-1018:54Alex Miller (Clojure team)thatās actually how s/map-of and s/every-kv are implemented#2017-07-1018:57bbrinckInteresting. Your example was instructive. I can see how or
ing together several tuples
would let me define different valid key/value pairs#2017-07-1018:58bbrinckBasically, it lets me create a clear relationship between the spec for certain keys and the specs for their values. But combine them all into one big map spec#2017-07-1018:59bbrinckāvalid key/value pairsā Sorry, I meant āvalid key/value pairs of specsā#2017-07-1020:03lfn3Is fspec
meant to handle cases where the provided function throws? Itās causing spec errors when instrumented.#2017-07-1020:05lfn3Iāve got a relatively minimal repro:
(t/use-fixtures :once #(do (s.test/instrument)
(%1)
(s.test/unstrument)))
(defn takes-fn [f]
(try (f true)
(catch Error e
false)))
(s/fdef takes-fn
:args (s/cat :f (s/fspec :args (s/cat :bool boolean?)
:ret boolean?))
:ret boolean?)
(deftest can-handle-throws
(is (false? (takes-fn #(if %1
(throw (ex-info "scary exception!" {}))
true)))))
#2017-07-1020:06lfn3That yields: In: [0] val: (true) fails at: [:args :f] predicate: (apply fn), scary exception!
when the test is run#2017-07-1020:24Alex Miller (Clojure team)when you have an fdef with an fspec arg, the instrumented var will check the fspec by generating from its args spec and invoking the function to validate it conforms to the spec#2017-07-1020:24Alex Miller (Clojure team)in this case it will generate boolean values and invoke f#2017-07-1020:25Alex Miller (Clojure team)specs cover non-exceptional use (as those are the exceptions :) so I think a more accurate fspec args spec in this case would use false?
, not boolean?
. it would then invoke f with only false and not trigger the exception#2017-07-1021:38lfn3Right I guess I just wasn't expecting the fspec'd arg to be invoked? Wouldn't it be better to use with-gen
so the spec still captures the allowed range of values? #2017-07-1022:28Alex Miller (Clojure team)well the idea is that you said this function takes a function that responds a certain way - the only way to validate that is to invoke it#2017-07-1022:33lfn3Yeah agreed. I donāt think that was really a question, I was just kinda surprised. I would have assumed that the fn would be instrumented and validated when it was invoked rather than when it was passed into another function. Might be something worth mentioning in the docs, if it isnāt already?#2017-07-1022:45Alex Miller (Clojure team)This has been brought up several times but I haven't had a chance to discuss it with Rich#2017-07-1023:14lfn3Ok. Thanks for the clarification and explanation in any case, itās much appreciated.#2017-07-1101:46danielcomptonI'm trying to use spec.test/check to check all my functions, but I'm getting a couldn't satisfy such-that predicate exception. Is there any way to figure out which spec wasn't able to be satisfied?#2017-07-1102:03seancorfieldI don't know of any way to debug that -- other than to s/exercise
your specs as you write them...#2017-07-1102:04seancorfieldYou could call s/registry
and go through all the keys in that (which are specs) I guess...#2017-07-1102:04seancorfield...I've just gotten kinda paranoid about exercising each spec in the REPL as I write it.#2017-07-1102:26bfabryI think there's a request open to include the spec in that message. iterating the registry sounds like a good option in the meantime#2017-07-1107:39danielcompton@bfabry do you know which issue it is? I took a look in JIRA but couldn't see anything#2017-07-1108:39hkjelsHow do I express, many strings and/or many vectors? And I donāt care which one it is, so I donāt need any branching#2017-07-1109:12rickmoynihan@danielcompton: Iāve had that same issueā¦. Supposedly the latest version of test.check exposes more information on that, infact I updated my test.check to the latest version which provides a bit more info, but still doesnāt tell you the spec that failed (as it knows nothing of spec).#2017-07-1109:15danielcomptonš#2017-07-1109:31rickmoynihanfinding them is a bit of a PITA, lucky for you bisecting specs with exercise is logarithmic š#2017-07-1109:31rickmoynihanbut better to exercise them as you write them bottom up like seancorfield suggests#2017-07-1111:12gfredericks@danielcompton @rickmoynihan @bfabry what's needed is to find the spot[s] where clojure.spec calls gen/such-that
and get it to pass that debugging info along; I have no idea how difficult that would be, might be super easy#2017-07-1111:12gfredericksthe test.check change is that you can now pass an option (`:ex-fn`) that lets you customize the exception#2017-07-1115:14rickmoynihanSome of my spec generators seem to grow to pretty big values under the default 1000 generations used by clojure.spec.test.alpha/check
. I donāt know how big theyāre getting in numbers of items, but theyāre OOMing. Is there an easy way to make specs flatten out at a certain size, but keep iterating over random generations?#2017-07-1115:16rickmoynihanobvs I could add an extra constraint e.g. #(> 1000 (count %))
to maps etcā¦ but it feels like itād be better done in the map/sequence generators#2017-07-1115:23gfredericks@rickmoynihan under default configurations your tests should max out their size at 200 trials and cycle back to small at that point;
there should be a way to restrict the size
further than that#2017-07-1115:23gfredericksSee this option to quick-check
that I expect spec provides a mechanism for passing through: https://github.com/clojure/test.check/blob/05a53600aab21576420984a54f3ff75714981bc3/src/main/clojure/clojure/test/check.cljc#L46#2017-07-1115:23rickmoynihanhmmmā¦ not sure why Iām seeing so much heap then#2017-07-1115:24gfredericksthe underlying problem here relates to collection sizing, which we have an open test.check issue for#2017-07-1115:24gfredericksbut the workaround is to explicitly limit sizing via the option I just pointed to#2017-07-1115:25rickmoynihanyeah you can pass those through via (check ,,, {:clojure.spec.test.check {:max-size 10}})
#2017-07-1115:25gfredericksAll I was saying with 200 vs 1000 is that you should probably see the same issues with only 200 trials#2017-07-1115:25gfredericksI'd try 50
for starters#2017-07-1115:27rickmoynihan100
for num-tests seems to work but more than that GC seems to dominate the run time#2017-07-1115:28rickmoynihanwill try reducing max-size#2017-07-1115:29gfredericksnum-tests
can be as high as you want as long as max-size
is low enough#2017-07-1115:29gfredericksmax-size
defaults to 200
, and the sizes used throughout the run are (cycle (range max-size))
#2017-07-1115:29rickmoynihancool#2017-07-1115:30gfredericksso num-tests
can "fix" sizing issues only when num-tests < max-size
#2017-07-1115:31rickmoynihanmakes sense#2017-07-1115:32rickmoynihancool. That max-size trick at 50 seems to work nicely.#2017-07-1115:33rickmoynihan@gfredericks: so does :max-size 50
mean maps have a maximum of 50 pairs, strings have a maximum of 50 chars, vectors have a maximum of 50 items etc?#2017-07-1115:33rickmoynihanor is it a weight/multiplier?#2017-07-1115:33gfredericksthat tends to be true, but there's no true definition of size
; it's an abstract thing that could theoretically be interpreted differently by any given generator#2017-07-1115:34rickmoynihanok thatās what I thought#2017-07-1115:34gfrederickse.g., you can easily defy that by generating vectors with a specified length of 75#2017-07-1115:34rickmoynihanš#2017-07-1115:34rickmoynihanbut is it true of the default generators?#2017-07-1115:35gfredericksyeah#2017-07-1115:35gfrederickspresumably the fix for all this will be that lower-down nested structures will be substantially smaller#2017-07-1115:35gfredericksbut that wouldn't contradict what you just said#2017-07-1115:39rickmoynihanlowering it to 50 seems to work nicelyā¦ it also means I can run more generations in the same time which seems like the right trade off for my data.#2017-07-1116:47lwhortondoes anyone know a way to express āa map of keys of spec A or B, where A key points to spec A0 and B key points to spec B0ā?
(s/def :foo (s/map-of #{:A :B} ...maybe (s/or :a :A0 :B0)
something along those lines?#2017-07-1116:53seancorfield@lwhorton Sounds like a multi-spec to me.#2017-07-1116:54lwhortonheh, back to that olā zinger again#2017-07-1116:56seancorfieldTBH, I'm not sure what you're actually trying to do, based on what you said. Can you give a concrete example?#2017-07-1119:43lwhortonmulti-spec turned out to be the right thing again. thanks!#2017-07-1118:15bbrinck@lwhorton You might also want to use tuples inside a coll-of. Something like (untested):
(s/coll-of (s/or :a (s/tuple ::A ::A0) :b (s/tuple ::B ::B0)) :into {} :kind map?)
#2017-07-1201:04danielcomptonI just realised that if you're using a tools.namespace based workflow, the spec registry isn't cleared when you refresh
. Is there a recommended way to "clean the registry" safely?#2017-07-1201:05danielcomptonI suspect that if I just reset!
the registry I might be missing internal/clojure.core.specs specs?#2017-07-1201:05twashingQuestion. This will break with Clojure telling me that āNo such var: eidā#2017-07-1201:05twashing(s/def ::eid (s/with-gen (s/and number?
pos?
#(instance? java.lang.Long %)
(complement nil?))
gen/int))
(s/fdef foo/bar
:args (s/cat :a any?
:b ::eid)
:ret (s/coll-of ::some/things))
#2017-07-1201:06danielcomptonwhat is es/eid
?#2017-07-1201:06twashing::eid is the first form#2017-07-1201:06twashingā¦ sorry wrong namespace#2017-07-1201:07twashingfixed#2017-07-1201:07twashingWhy canāt I use ::eid
in my function spec :args
?#2017-07-1201:09danielcompton@twashing I can load this into a REPL fine:
(defn bar [])
(s/def ::eid (s/with-gen (s/and number?
pos?
#(instance? java.lang.Long %)
(complement nil?))
gen/int))
(s/fdef bar
:args (s/cat :a any?
:b ::eid)
)
how are you generating the error?#2017-07-1201:12twashingItās the function that generates generative tests, based on the function spec.#2017-07-1201:12twashing... one sec, lemme find it.#2017-07-1201:15twashingclojure.spec.test/check
#2017-07-1203:12souenzzo::stuff
is a map with a fixed key :data
that will never change,.
(s/def :foo/data (s/keys :req [:relevant/key :another.relevant/key]))
(s/def ::stuff (s/keys :req-un [:foo/data]))
There is some how to "omit" :foo/data
definition?#2017-07-1210:10joost-diepenmaatwhatās the recommended way to run spec checks during automated testing? weāre currently using lein test with clojure.test and it would be nice to use something like the solution here https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite#2017-07-1210:11joost-diepenmaatif I have to I can copy & paste and adapt that code but it would be nice to have a supported library#2017-07-1211:50gfredericks@joost-diepenmaat if you paste it into https://github.com/gfredericks/schpec and make a PR, I'll release it
not sure if that counts as "supported" or not#2017-07-1214:23jpmonettashi everyone! I'm trying to understand how to match :clojure.spec.alpha/problems inside :clojure.spec.alpha/value in explain-data result independent of the specs that were applied and I'm having some trouble#2017-07-1214:27jpmonettasso the thing is that seeing a problem in [::k 1] you don't know if it means, it's the value of that key, or it's the element at pos 1 of a seq in the value of that key#2017-07-1214:27jpmonettasdoes it make sense?#2017-07-1214:40bronsasee https://dev.clojure.org/jira/browse/CLJ-2192#2017-07-1214:41bronsayou're right in your analysis, it's been discussed before and it's logged in that ticket#2017-07-1214:43jpmonettasgreat, thx a lot @bronsa#2017-07-1216:58urbankSo I'm wondering, is spec with s/def
+ s/valid?
supposed to replace predicate functions? Otherwise I find it hard to decide what should be a spec and what should just be a function#2017-07-1217:04urbankOh, I suppose that isn't really a good question. More pertinent would be whether new predicates should be written as anon functions in s/def
or should s/def
refer to a named predicate function#2017-07-1310:10rickmoynihandependsā¦ sometimes the predicate function is generic and reusable but the s/def
can give it a more specific name/abstraction.
e.g.
(s/def ::user-id integer?)
(s/def ::post-id integer?)
#2017-07-1310:10rickmoynihanothertimes the predicate is too specific to be reusable, so just keep it anonymous.#2017-07-1310:11rickmoynihan(No idea if this is best-practice btwā¦ but itās what Iāve been doing)#2017-07-1217:30assoc-inI am wondering how I would write a spec for a nested map. I have something like this
{:1 {:user-id 1 :username "bob"}
:2 {:user-id 2 :username "jim"}}
Where I have a (s/def ::user (s/keys :req-un [::user-id ::username])).
I know how to do this if I had a vector of users by using (s/coll-of ::user), but how would I approach this with having the map keys being the ::user-id spec and the values of each map being the ::user?#2017-07-1217:57bfabry@assoc-in (s/map-of keyword? ::user)#2017-07-1217:58assoc-in@bfabry Ah yes I didn't see that looks like just what I needed. Thanks!#2017-07-1217:58bfabrynp#2017-07-1300:14bbrinckSeveral people on this channel have asked about pretty-printing spec failures. Iāve just released Expound, a library that formats clojure.spec errors https://github.com/bhb/expound#2017-07-1307:07richiardiandreabbrinck: great great great thanks!#2017-07-1300:15bbrinckIāve got a lot more features planned, but hopefully this first release is useful to people on this channel#2017-07-1300:15bbrinck^- cc @ghadi#2017-07-1300:17bbrinckcc @samueldev (who also asked about this awhile back)#2017-07-1301:20samueldevAwesome @bbrinck I'm going to try this out tomorrow morning!#2017-07-1306:13naomarikHow can I constrain a multispec to a specific key? With the event example in spec guide Iāve tried this: (gen/sample (s/gen :event/event {:event/type #(s/gen #{:event/search})}))
which doesnāt work#2017-07-1306:28naomarikSolved it using gen/fmap
to constrain the type, wondering if more succinct solution is available though.#2017-07-1310:13rickmoynihan@bbrinck: awesome!! Iāve been wanting something like this for a long timeā¦ Another thing that might be useful is integrating this into prone for rendering spec errors in stack traces over http (in dev):
https://github.com/magnars/prone#2017-07-1312:56bbrinckGood idea!#2017-07-1401:26ghadiNice job @bbrinck https://clojurians.slack.com/archives/C1B1BB2Q3/p1499904883413529#2017-07-1401:34bfabryya looks really nice @bbrinck#2017-07-1403:21bbrinckThanks! Let me know if you run into any problems.#2017-07-1418:04adamfreyis there a way to generate an args seq from an fdef
with an :args
spec?#2017-07-1418:15bbrinck@adamfrey would this work?
(s/fdef ::foo :args (s/cat :i int? :s string?))
(s/exercise (:args (s/spec ::foo)))
#2017-07-1418:16bbrinckor
(require '[clojure.test.check.generators :as gen])
(gen/sample (s/gen (:args (s/spec ::foo))))
#2017-07-1418:26adamfreythat does work, thanks#2017-07-1520:16arohnerhereās a fun crash-the-process bug:
(s/fdef clojure.core/hash-map :args (s/* (s/cat :k any? :v any?)) :ret map?)(clojure.spec.test.alpha/instrument)
#2017-07-1520:25arohnerhrm, (clojure.spec.test.alpha/check 'clojure.core/hash-map)
also throws OOME, on a -Xmx1024m JVM#2017-07-1522:17joshjonesI can replicate the instrument
stack overflow -- however, for the check
, it's just creating huge maps. So, best to run check with some constraints IMO usually:
(stest/check `hash-map {:clojure.spec.test.check/opts {:num-tests 500 :max-size 30}})
#2017-07-1622:27lwhortonwhatās the proper way to spec protocols? iām assuming I can just spec some client fn instead of the protocol? but Iām having a conceptual issue about how to have multiple implementations of the protocol with this approach:
;; protocols.cljs
(defprotocol IMyProto
(-foo [this v]))
;; client.cljs
(defn foo [this v]
(protocols/-foo this v))
;; impl-A.cljs
(defrecord AImpl []
IMyProto
(-foo [this v] ...))
(defn foo [this v] (-foo this v)) ;; this is the fn I would instrument for A types
;; impl-B.cljs
(defrecord BImpl []
IMyProto
(-foo [this v] ...))
(defn foo [this v] (-foo this v)) ;; this is the fn I would instrument for B types
in client.cljs
I want to be able to call protocols/foo
instead of prototols/-foo
, because I can instrument (defn foo...)
. But this breaks the purpose of a protocol - how do I know which implementation to invoke from the client a-impl/foo
or b-impl/foo
?
Whatās the proper way to organize these guys so I can stest/check 'foo
?#2017-07-1622:37lwhortonI suppose I could turn it on its head so that client calls always invoke (protocols/foo this v)
, and each impl points to an internal function IMyProto (foo [this v] (-foo this v))
? But this doesnāt seem friendly, particularly once the records start having fields and if any protocol fn needs to refer back to those fields.#2017-07-1622:46lwhorton*or perhaps I should re-apply s/fdef
to the protocolās foo
when I actually want to run my checks? (s/fdef protocols/foo ::args (s/cat :type ::a-type)
, then re-def with ::b-type
?#2017-07-1623:21seancorfieldProtocols are open for extension -- so you can't write a spec for all possible argument types. I think you could write a multi-spec for foo
-- where the defmulti
discriminator function returns the type
of the first argument?#2017-07-1700:06danielcompton@lwhorton I don't think you can spec protocols, the recommendation I got at the time was to create a regular function in front of the protocol function and spec that#2017-07-1700:08danielcomptonhttps://clojurians-log.clojureverse.org/clojure-spec/2016-06-09.html#inst-2016-06-09T00:06:42.000541Z#2017-07-1700:18lwhortoncongratulations by the way. i hope you have a heck of a time with the kiddo. Iāāll certainly miss The REPL.#2017-07-1702:55danielcomptonI won't be gone for too long š#2017-07-1700:08lwhortonyea .. thatās what Iām rolling with. I probably wasnāt clear, but in my question Iām trying to address the issue with trying to test.check multiple āregular functions in front of the protocol functionā for each possible argument type#2017-07-1700:09lwhortonas @seancorfield mentions, theyāre open for extension so you canāt cover every base. that being said, iām only trying to cover my particular bases.#2017-07-1709:11rickmoynihanWorth remembering also that you can spec that the value your wrapping fn receives satisfies?
the protocol. Youāll need to write a custom generator to generate conforming values for each of the implementations you care about though. You donāt get that for free like you do with multi-spec.#2017-07-1709:15rickmoynihanThis said if your implementations had a common construction interfaceā¦ e.g. they all took a string as a single argument to their constructor, you could easily write a dynamic generator by inspecting the protocol i.e. by using (keys (:impls foo.bar/Protocol))
#2017-07-1709:15rickmoynihanthen mapping the implementation classes to constructors/generators#2017-07-1711:40samueldevdoes anybody have any specs around HTML5 filetypes?#2017-07-1711:40samueldevI have a fn I'd like to spec but am currently omitting spec'ing one particular argument, which I'm expecting to be an HTML5 File/Blob#2017-07-1711:53samueldev@bbrinck hey ben, do I have to do anything special to make expound work in CLJS?#2017-07-1711:59samueldevnvm I didn't realize the transition from cljs.spec -> cljs.spec.alpha#2017-07-1711:59samueldevhad happened#2017-07-1711:59samueldevall good now š#2017-07-1714:29bbrinck@samueldev Iāll add that to the README#2017-07-1714:32bbrinckA user has also reported a issue on CLJS 1.9.671 (I had been testing on CLJS 1.9.542). Just FYI: https://github.com/bhb/expound/issues/3#2017-07-1715:08samueldev@bbrinck using the lib and converted all my spec explains to yours this morning; quite pleased š#2017-07-1715:09samueldevits aim seems to be to output developer-friendly error messages; have you considered the use case of going all the way and having it output non-developer-friendly error messages, that can be presented to an end-user?
for example if I'm doing some simple validation on a POST route in my API to create an entity, and they omit a required key, an output-string provided along the lines of "Missing required key: _____"
#2017-07-1715:09samueldevas it stands now, I'm using regular spec explains and iterating over the problems
and generating these strings#2017-07-1715:10samueldev(with some hard-coded mappings from something like a str?
predicate failing --> a string being generated along the lines of "Value provided for key ____ must be a string."
#2017-07-1715:11samueldevthis is something I've considered lib-ifying for my own sake but could instead PR it into expound perhaps, if you see value in it#2017-07-1716:05bbrinckYes, in the future, I want to have good defaults for a number of contexts: test failures, REPL, end-user, etc#2017-07-1716:06bbrinckThat wonāt replace the current error message, but itāll either be another namespace or perhaps some way to configure expound#2017-07-1716:06bbrinckBut more generally, I want to extract out a set of helper functions that work on clojure.spec.problem
s such that you can trivially build your own custom string representation without dealing with all the details of spec#2017-07-1716:09bbrinckThat way, if you donāt like the default āuser-facingā error message, you can peek underneath and see how its built and build your own.#2017-07-1716:10bbrinck@samueldev If you want to create a GH issue with an example of a spec, a value, and the error string youād like to see for a user-facing error, that would be very helpful!#2017-07-1715:17adamfreyfor a spec that can potentially generate huge values (like giant nested maps with long keywords), what is the most efficient way to get a small values (for playing around with at the repl)#2017-07-1715:21schmeefor some reason, my spec errors are printed without newline, which makes them completely unreadable::error-while-loading kleinheit.datomic.impl
#error {
:cause "Call to clojure.core/refer-clojure did not conform to spec:\nIn: [2 1] val: :as fails at: [:args :exclude :op :quoted-spec :spec] predicate: #{:exclude}\nIn: [2 1] va......
#2017-07-1715:21schmeeany idea what might cause this?#2017-07-1716:15bbrinck@schmee It looks like youāre printing out the entire error record. Is it possible to just get the :cause
and print that?#2017-07-1716:15schmeeI hope so š#2017-07-1716:15schmeeI donāt get why it prints the whole error record in the first place#2017-07-1716:16bbrinckSorry, what is āitā?#2017-07-1716:16schmeelein repl
š#2017-07-1716:17bbrinckAh, I see. Can you perhaps write a wrapper function that calls the function in a try/catch?#2017-07-1716:21bbrinckI have not tested this, but you could#2017-07-1716:21bbrinck@schmee Or, after you see an error like that, could you just do something like (println (:cause (ex-data *e)))
to print out the cause of the last error?#2017-07-1716:22schmeeI suppose so, but I would prefer if I could get the REPL to always do that for me#2017-07-1716:22bbrinck(just put that on the REPL after the error has been caused)#2017-07-1716:24bbrinck@schmee I havenāt tried this, but it looks like lein letās you configure the error handler https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L357-L370. Note that by only printing the cause, youāll presumably be losing other contextual information about the error, which may or may not be useful.#2017-07-1716:25bbrinckIn any case, this isnāt really related to spec, so you may have more luck tracking down an answer in #leiningen . Good luck!#2017-07-1716:25schmeeahh, now thatās what Iām looking for, thanks! š#2017-07-1717:34schmeeis there a ārecursiveā version of describe
, that resolves non-fn specs?#2017-07-1717:39Alex Miller (Clojure team)not currently#2017-07-1717:49schmeewhatās the easiest way to constrain a s/cat
into a vector?#2017-07-1717:50schmee(s/and vector? (s/cat ...))
seem to fail to generate data with exercise
#2017-07-1717:51schmeehereās how Iām doing it now, just curious if there is a better way:
(spec/def ::pred* (spec/cat :k any? :pred any?))
(spec/def ::pred (spec/with-gen (spec/and vector? ::pred*) #(gen/fmap vec (spec/gen ::pred*))))
#2017-07-1718:21bbrinck@schmee My understanding is that vector sequences is a known problem that isnāt yet solved: https://clojurians.slack.com/archives/C1B1BB2Q3/p14988354#2017-07-1718:23bbrinckOr, perhaps I should say āregex specs that are also vectorsā#2017-07-1718:43gfredericksI feel like in some platonic sense the regex specs fit seqs much more than vectors
but that doesn't mean it wouldn't be terribly useful to have regexes for vectors#2017-07-1718:43gfredericksprobably mostly/entirely for macros?#2017-07-1718:53bbrinckIāve also had use cases when using conform/unform on vectors#2017-07-1718:54bbrinckWe have cases where we like the succinctness of using vectors e.g. something like hiccup: [:p {} "hi"]
. When we manipulate it, itās handy to have named parts by using a regex spec, but then we can unform it back to the vector form#2017-07-1718:55bbrinckRight now we have to unform, then convert the list to a vector#2017-07-1718:57schmeeyeah, a catv
would be nice#2017-07-1718:57schmeeis there a JIRA or something for this?#2017-07-1719:00bbrinckhttps://dev.clojure.org/jira/browse/CLJ-2021#2017-07-1719:00schmeecheers :+1:#2017-07-1719:40taylorare there/will there be specs for the return values of explain-data
?#2017-07-1719:56schmeethereās a JIRA for āspecs-for-specā: https://dev.clojure.org/jira/browse/CLJ-2112#2017-07-1721:54lwhorton@schmee I was doing exactly that yesterday with clojure.test.check.generators
and the gen/vector
api.#2017-07-1721:54lwhortonhttps://clojure.github.io/test.check/clojure.test.check.generators.html#var-vector#2017-07-1721:55lwhortonyou can define the spec with (s/with-gen (s/cat ...)
and hand the with-gen something that always returns a vec#2017-07-1813:38stathissiderisis there any way to āmergeā two map generators?#2017-07-1813:40mpenetI guess you can cheat your way into it with (s/gen (s/merge x y))#2017-07-1813:41stathissideriswell, I donāt have specs for the maps I need to merge (so itās not strictly a spec question!)#2017-07-1813:42gfredericks(gen/let {m1 gen-m1, m2 gen-m2} (merge m1 m2))
#2017-07-1813:42stathissiderisoh thereās gen/let!#2017-07-1813:42stathissideristhatās great, thanks @gfredericks!#2017-07-1819:46plinshello everyone, how can i validate a month inside a string?
is the regexp approach the best?
it should be a string containing 2 digits ranging from ā01ā to ā12"#2017-07-1819:49gfredericksit's certainly easy as a regex #"0[1-9]|1[0-2]"
#2017-07-1819:51plinsthx#2017-07-1822:37jcf(into #{} (map #(format "%02d" %)) (range 1 13))
You could build a set of the strings as the count is small too. š#2017-07-1822:39jcfThen with clojure.spec you can do something like (s/def ::month-number-str (into #{} (map #(format "%02d" %)) (range 1 13)))
.
And/or just write it out like so:
(s/def month-number-str
#{"01" "02" "03" ...}
#2017-07-1823:08creeseI'd like to generate a spec dynamically, where the name of the spec is returned from a function. Is there any prior art on this?#2017-07-1823:11creese(defn register-spec
[spec-k attr-ks]
(s/def (keyword "foo" spec-k) (s/keys :req-un attr-ks)))
#2017-07-1823:11creeselike this#2017-07-1823:12jcf@creese I'm not sure it's a good idea or that it'd work as expected, but the spec registry is an atom you can swap!
etc. #2017-07-1909:04ikitommi@schmee spec-tools has a spec visitor, which traverses specs recursively, one use case is the recursive describe. Has regression (& progression) tests for all core-specs: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/visitor_all_test.cljc#2017-07-1909:10schmeecool, Iāll try it out later today!#2017-07-1910:47danielnealI'm still new with spec. Is it considered unidiomatic to use conformers to coerce, as in the following. What would be the idiomatic approach
(s/def :conformers/int
(s/conformer (fn [x]
(cond
(integer? x) x
(re-matches #"\d+" x) (edn/read-string x)
:else :cljs.spec.alpha/invalid))))
(s/def :data.subscription/quantity :conformers/int)
#2017-07-1911:12danielnealI've asked a stackoverflow question just so in case someone answers it won't get lost in the mists of slack history#2017-07-1911:12danielnealhttps://stackoverflow.com/questions/45188850/is-use-of-clojure-spec-for-coercion-idiomatic#2017-07-1911:39schmee@danieleneal AFAIK the jury is still out on that one#2017-07-1911:39schmeebut spec-tools mentioned above has a lot of stuff for that#2017-07-1911:44danielnealah cool - so it might be a case that the boundaries around idiomatic use will emerge with time#2017-07-1911:44danielnealmakes sense#2017-07-1912:03ikitommimy 2 cents go to separating conforming from specs. There is CLJ-2116 for it.#2017-07-1913:56jcfhttps://dev.clojure.org/jira/browse/CLJ-2116#2017-07-1914:32danielneal@ikitommi what do you do for the coercing bit#2017-07-2006:03ikitommi@danieleneal spec-tools but I'm biased as co-authoring the lib ;) haven't seen anything else for it, really. Cognitect people have said many times that runtime transformations are not is scope of spec :(#2017-07-1915:28tcouplandGot a bit of a spec design and validation question: We've got a few messages flying around and are working on adding spec's for them now (i know, wrong way round), the messages generally look like this:
{:type "msg1"
:version "1.0.0"
:payload {:msg1-specific-key "val"}}
The trouble is the payload key being reused across all the the different :type fields, making writing a spec for :payload very tricky as it lacks the context of the :type value. I'm thinking we should have gone for a flat structure:
{:type "msg1"
:version "1.0.0"
:msg1-specific-key "val"}
A multi spec would then have something to go on. Just wondering if there's anything I've missed that would let us write a decent :payload spec? Feels like the current format just doesn't work with spec very well at the moment.#2017-07-1915:47jcf@tcoupland if you're using a multimethod with your multi-spec you can implemtent arbitrarily complex dispatch logic.#2017-07-1915:48jcfYou can get at what's in the :payload
, and transform it, use or
to pick from multiple options.#2017-07-1915:49jcfOf course, if your data looks really similar that might be tricky, but then how specific does your spec really need to be? š#2017-07-1915:53tcouplandš#2017-07-1915:55tcouplandyeah, i'm worried that the dispatch function would get a bit nuts, also it wldn't really solve the problem of attaching the type to a specific payload#2017-07-1915:58jcfTime for version "2.0.0" š#2017-07-1916:00nwjsmithI was fiddling with some spec tooling last night, and there is something I'd like to express that is difficult in spec. I'd like to be able to express "this is a collection of As for some comparable type A".#2017-07-1916:03nwjsmithI don't know much about types, but I think in Haskell this can be expressed as Comparable a => List a
. It would be useful in situations like spec-ing the args of clojure.core/sort
. Especially since vectors are comparable#2017-07-1916:03nwjsmithThat way you could express "this is a collection of As or a collection of vectors of As for some comparable scalar A"#2017-07-1916:18andrewmcveighCan't you write: (s/coll-of #(instance? Comparable %))
#2017-07-1916:18andrewmcveigh?#2017-07-1916:19andrewmcveighSeeing how spec is working on values, you don't really need type constraints#2017-07-1916:20nwjsmithYes, totally. It's on the data/test-generation side that this becomes a little awkward.#2017-07-1916:21andrewmcveighAh, OK. Then yes I see your point š#2017-07-1916:21nwjsmithI'd like my cake and I'd like to eat it too š#2017-07-1916:25andrewmcveighI think that (s/coll-of (s/and any? #(instance? Comparable %)))
is closest in meaning to Haskell's Ord a => List a
#2017-07-1916:25andrewmcveighProbably will get super slow though#2017-07-2006:05jimmyhi guys, I'm in a situation that we have two kind of specs:
# "correct" spec, this one is used to do validation, instrument
and # "test" spec, this one is mainly to use in gen test. Since when doing test, sometimes I want a function to handle all kind of input instead of the "correct" one generated by "correct" spec.
How do you guys manage these in terms of code organization ( "test" spec in test namespace, "correct" spec in the same ns of the function def ) or ... ? I would love to hear your idea on this.#2017-07-2006:06seancorfield@nxqd I would have a single spec but override the generator.#2017-07-2006:07seancorfieldYou want the same "correct" spec in both production code and test code -- but it's reasonable to have a generator that produces simpler data. You do not want the other way around.#2017-07-2006:07seancorfieldPerhaps you're not explaining what you're really trying to do?#2017-07-2006:14seancorfieldIf you're explicitly doing validation inside the function, then the arguments it takes are a different spec, by definition, since the function takes a broader range of data -- it's expecting a broader range of data so that it can validate a small range of data and do something different and observable for the "invalid" data.#2017-07-2006:17seancorfieldi.e., (if (s/valid? ::some-spec input-data) (do-good-stuff input-data) (do-something-else input-data))
-- the function is anticipating input-data
that does not conform to ::some-spec
and it is expected, defined, testable behavior that the function does something else -- and the function accepts, as part of its core production spec input data that is beyond just ::some-spec
. Does that make sense @nxqd ?#2017-07-2006:23jimmyyeah you are correct. The approach I'm doing atm is basically the same as you mentioned above. what I have in mind is to design it like this:
code-ns
(s/fdef ... :args (s/cat :args ::args))
(defn ... )
test-ns
(s/fdef ... :args (s/cat :args ::args)) ;; the only different of this ::args is the gen function as you said.
;; register this fdef instead of the one above and run tests.
;; since the real implementation will still use the "correct" spec, there is no problem regarding valid? in function.
Is there any better solution to swap different gen fn in different env, so we don't have to create another test ns.#2017-07-2006:24seancorfieldNo, what I'm saying is the function itself has one spec that is the same for test and production. It may be a different spec from what it checks (validates) inside the function.#2017-07-2006:25seancorfieldA generator for a spec cannot generate data that fails to conform to the spec, so the generator may generate a subset of acceptable data, but not a superset.#2017-07-2006:26jimmyhmm, interesting. I got your point now#2017-07-2006:28seancorfieldConsider an API function that accepts arbitrary strings as arguments but then conforms the arguments to numeric values -- the function argument spec is basically string?
and then it will have a separate spec for conforming numeric strings to numbers.#2017-07-2006:29seancorfieldSince you want the arguments to function to be mostly numeric, you need a custom generator that mostly generates numbers and calls str
on them, but sometimes generates arbitrary strings (which will fail the other spec) and cause the function to do whatever error handling you expect.#2017-07-2006:30jimmyagree. this clears my mind. What is your naming convention for fdef args spec and separate spec?#2017-07-2006:30jimmy(s/def :fdef/arg )
and (s/def :correct/arg ) ?
#2017-07-2006:31seancorfieldIt depends on the business domain. In this case, I'd probably have a :domain/spec
for the conforming spec and a :api/spec
for the function argument spec.#2017-07-2006:32jimmygreat ! Thanks a lot. Have a nice day sir š#2017-07-2006:32seancorfieldGlad to help!#2017-07-2017:39phreedCan I get some help with writing a spec?
I want a vector of of alternating types, similar to a tuple spec but repeating where the types of the first and last are the same. The following is invalid but hopefully suggests what I am looking for.
(s/def ::graph-path (s/tuple-ish ::node ::edge ::node ::edge ... ::node))#2017-07-2018:02phreedIn think I figured it out...
(s/def ::graph-path (s/cat :h ::node :r (s/* (s/cat :e ::edge :n ::node)))#2017-07-2109:25ikitommi@danieleneal @alex - gave a shot at the CLJ-2116: https://github.com/clojure/spec.alpha/pull/1. Idea is to support a optional conforming callback
for conform & explain, allowing specs to be separated from conforming. Would be used like this:
(deftest conforming-callback-test
(let [string->int-conforming
(fn [spec]
(condp = spec
int? (fn [_ x _]
(cond
(int? x) x
(string? x) (try
(Long/parseLong x)
(catch Exception _
::s/invalid))
:else ::s/invalid))
:else nil))]
(testing "no conforming callback"
(is (= 1 (s/conform int? 1)))
(is (= ::s/invalid (s/conform int? "1"))))
(testing "with conforming callback"
(is (= 1 (s/conform int? 1 string->int-conforming)))
(is (= 1 (s/conform int? "1" string->int-conforming))))))
#2017-07-2110:50scaturrAre there any known issues with clojure.test.check and multi arity functions?#2017-07-2110:52scaturrfor some reason when I try to run (st/check make-action)
my repl just hangs forever#2017-07-2110:53scaturrsame thing out of the repl too - lein test
just hangs (running generative tests via clojure.test.check in my normal test suite)#2017-07-2110:54scaturrI did verify that I could generate data for all the (s/def)
expressions individually - that seems to work#2017-07-2111:12gfredericks@scaturr did you try generating (s/cat :type ::type :error? ::error? :payload (s/? ::payload))
as well?#2017-07-2111:13scaturrI did not - I will try that now#2017-07-2111:14scaturryes that worked š#2017-07-2111:14gfredericksI'm trying this out too#2017-07-2111:15gfrederickscan't reproduce; check
ran fine for me#2017-07-2111:15scaturrI changed ::payload
back to map?
for that example#2017-07-2111:15scaturrinstead of string?
- though both produce the error#2017-07-2111:16scaturrwhat version of test.check are you running?#2017-07-2111:16scaturrthank you for your help by the way š#2017-07-2111:17gfredericksprobably 0.10.0-alpha2#2017-07-2111:17gfredericksyeah#2017-07-2111:17scaturrIām on 0.9.0#2017-07-2111:17scaturrIāll try updating#2017-07-2111:18scaturrthat fixes it#2017-07-2111:19scaturrthank you so much#2017-07-2111:20gfrederickswell I'd certainly like to know what the difference is, but I don't think it matters enough to look into š#2017-07-2111:20gfredericksbut if you figure it out, let me know š#2017-07-2111:20scaturrwill do!#2017-07-2113:45Alex Miller (Clojure team)@ikitommi noted. I'm not going to do anything with it until I get a chance to ask Rich about it#2017-07-2116:22wilkerluciohello people, for those looking at how to do coercion with specs, spec-coerce
leverages your spec definitions to coerce value types š https://github.com/wilkerlucio/spec-coerce#2017-07-2201:21danielcomptonLooks neat, I see there is "Coercion overrides map to specify contextual coercions" on the TODOs, will that let you specify different contexts like a JSON payload or form encoding, e.t.c.?#2017-07-2203:04wilkerlucio@danielcompton thanks for the interest š for that I was thinking more like when you do generator overrides, like you do on specs, I think the main use case would be to set a custom parser for a different date/time format depending on the source#2017-07-2205:01thedavidmeisterhey, is there a way to and
additional predicates onto existing keywords when used in spec/keys
for a new def?#2017-07-2205:02thedavidmeistere.g. i have :db/id
which can be any int?
but when i have a :db/id
in a (spec/def :item/new ...)
i also want to ensure that it is the specific int -1
#2017-07-2205:55Alex Miller (Clojure team)No, but you can s/and additional constraints over the s/keys#2017-07-2206:11thedavidmeister@alexmiller like this?#2017-07-2206:11thedavidmeister(spec/def ::item
(spec/and
(spec/keys :req [:db/id])
(comp #(spec/valid? ::id %) :db/id)))
#2017-07-2206:11thedavidmeisterthat's a little different to what i asked#2017-07-2206:11thedavidmeister(spec/def ::id
(spec/or
::id--new #{new-item-id}
::id--existing pos-int?))
#2017-07-2206:14thedavidmeisterbut seems to be what i want when i run exercise
, thanks š#2017-07-2206:22Alex Miller (Clojure team)I would try to avoid the comp valid? ā¦ and instead state it as an additional spec or predicate#2017-07-2206:23Alex Miller (Clojure team)you can also use s/merge instead of s/and to combine two map specs together#2017-07-2206:23Alex Miller (Clojure team)one general, and one more specific#2017-07-2206:24Alex Miller (Clojure team)s/keys already checks that the value for the key conforms to the spec, so the extra check in your ::item is duplicating that#2017-07-2206:28thedavidmeister@alexmiller ah, i'm pretty new to this, could you give me a simple example of that?#2017-07-2206:29thedavidmeister(spec/def :db/id int?)
this is :db/id
#2017-07-2206:29thedavidmeisterso that's any int#2017-07-2206:30thedavidmeisterbut the item id has to be either a positive int or new-item-id (which is -1)#2017-07-2206:30thedavidmeisterit can't be just any negative int#2017-07-2206:30Alex Miller (Clojure team)I think you mostly have it above already#2017-07-2206:30Alex Miller (Clojure team)your ::id
looks good#2017-07-2206:31Alex Miller (Clojure team)and then just (spec/def ::item (spec/keys :req [::id]))
is sufficient#2017-07-2206:31Alex Miller (Clojure team)spec/keys checks that the map has the required keys and that every registered key spec has a value that conforms to that spec#2017-07-2206:32thedavidmeisterexcept that when i do exercise
it generates maps with ::id
instead of :db/id
#2017-07-2206:33thedavidmeisteri can't actually pass it ::id
because the key :db/id
is coming from upstream#2017-07-2206:45thedavidmeisterso, related question#2017-07-2206:45Alex Miller (Clojure team)sorry, it was unclear whether those were the same or different things#2017-07-2206:45thedavidmeisteroh well yeah#2017-07-2206:45thedavidmeisteri've got something related#2017-07-2206:45thedavidmeisteri have a :project/id
#2017-07-2206:45thedavidmeisterand i want to reference it with :item/project
#2017-07-2206:45thedavidmeisterthey have the same predicate so i thought i could do (spec/def :item/project :project/id)
#2017-07-2206:45thedavidmeisterbut it doesn't like that#2017-07-2206:46Alex Miller (Clojure team)should work fine - whatās not working?#2017-07-2206:46thedavidmeisterUnable to resolve spec: :project/id
#2017-07-2206:47Alex Miller (Clojure team)could you give a larger example that produces that?#2017-07-2206:48thedavidmeisterhmm#2017-07-2206:48thedavidmeisterso#2017-07-2206:48thedavidmeister(spec/def :project/id uuid?)
(spec/def :item/project :project/id)
#2017-07-2206:48thedavidmeisterworks#2017-07-2206:48thedavidmeisterbut when :project/id
is in a different ns (that i required) it gives the error#2017-07-2206:48Alex Miller (Clojure team)can you give that example?#2017-07-2206:49Alex Miller (Clojure team)there is a known issue right now with ordering of something like that such that the aliased spec needs to be defined first#2017-07-2206:49Alex Miller (Clojure team)are you running into something like that?#2017-07-2206:50thedavidmeisterpotentially#2017-07-2206:50thedavidmeisteri'm stopping and starting my test runner now#2017-07-2206:50thedavidmeistermaybe it got confused with me changing things#2017-07-2206:50Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2067#2017-07-2206:51thedavidmeisteryeah it seems fine now#2017-07-2206:51thedavidmeisterso i cannot reproduce#2017-07-2206:51thedavidmeisterweird#2017-07-2206:51Alex Miller (Clojure team)probably an ordering issue#2017-07-2206:51Alex Miller (Clojure team)Iām taking off! Laterā¦#2017-07-2206:54thedavidmeisterthanks for the help!#2017-07-2207:14ikitommiok @alexmiller, added the current changes as a patch too if that helps the conversation with Rich.#2017-07-2308:14thedavidmeisteris there something like spec/or
that just takes a list of predicates rather than the k/v list?#2017-07-2308:14thedavidmeisteri don't totally understand why spec/or
and spec/and
are different in this way#2017-07-2308:29thedavidmeisteralso, what's the best way to use spec to see if something only has a certain set of keys?#2017-07-2308:32schmeespec/or
takes a k/v list so that when you conform, you see which of the branches was taken#2017-07-2308:32schmeecheck out the āComposing predicatesā here: https://clojure.org/guides/spec#2017-07-2308:33schmeethis is not needed for s/and
, since there is no branching there (all predicates must be true)#2017-07-2308:35schmeechecking that something only contains a certain set of keys is sort of against specs pricinples, since key sets are supposed to be open#2017-07-2308:35schmeeof course you can do that anyway by using a custom predicate fn#2017-07-2309:35thedavidmeister@schmee i have a map that contains some keys that represent "machine data" like ids/references and some that represent "user data" which is what the user actually typed in#2017-07-2309:35thedavidmeisterwhen an item has only machine data left in it i want to be able to clean it out of my db#2017-07-2309:36thedavidmeisterbecause that item no longer represents anything useful to the user#2017-07-2309:36thedavidmeisterso i want a spec that can tell when only machine keys are left#2017-07-2309:36thedavidmeisterso it is open when adding, but closed when making a decision about whether to "clean up"#2017-07-2309:38schmeethat can be done with a predicate fn spec#2017-07-2309:39schmeebut for instance s/keys
has no way to say āthese keys are not allowedā#2017-07-2309:40thedavidmeisterit's not that they're not allowed#2017-07-2309:40thedavidmeisteri mean they are#2017-07-2309:40thedavidmeisterbut that's not the right way to think about it in context#2017-07-2309:42thedavidmeisteri'm trying to detect something not restrict it#2017-07-2309:49thedavidmeisterit's like#2017-07-2309:49thedavidmeister(if (spec/valid? :item/item--machine-only) (do ...) (do ...))
#2017-07-2309:50thedavidmeister@schmee the extra keys are totally allowed but i do want to treat items that have only machine keys a bit differently#2017-07-2311:14schmeeperhaps something like this would work?
(s/def ::machine-keys (s/keys :req [:a :b :c]]))
(s/def ::human-keys (s/keys :req [:d :e]]))
(s/def ::all-keys (s/merge ::machine-keys ::human-keys))
#2017-07-2311:14schmeethen you could validate thing separately when needed but still have a spec for the combination#2017-07-2312:21thedavidmeister@schmee the problem is that ::machine-keys
is valid when there are human keys too#2017-07-2312:22thedavidmeisterand human keys is valid for machine keys#2017-07-2312:36schmeethen do something like (and (s/valid? ::machine-keys foo) (not (s/valid? ::human-keys foo))
#2017-07-2400:15thedavidmeister@schmee but then :f
would trigger#2017-07-2400:15thedavidmeisteri actually do want my keys to be open#2017-07-2400:16thedavidmeisteri don't want to have to stipulate every human key, and none of them are required, they're all optional, it's just important that there is at least one#2017-07-2310:52thedavidmeister@schmee also, when you're doing spec/or
and using keywords it still seems odd to force k/v#2017-07-2310:53thedavidmeistere.g. (spec/or :foo/bar :foo/bar :a/b :a/b)
#2017-07-2311:11schmeethe first keyword there is just a tag#2017-07-2311:11schmeee.g dev=> (s/def ::test (s/or :foo keyword? :bar int?))
:dev/test
dev=> (s/conform ::test 1)
[:bar 1]
#2017-07-2311:12schmeespec makes you name all the branches so that you can use that in error messages etc.#2017-07-2312:17thedavidmeister@schmee but if the branch is just a keyword that is already registered it already has a tag, so it could use that for error messages#2017-07-2312:17thedavidmeisterhmmm how is it possible that#2017-07-2312:18thedavidmeister(valid? ::foo a)
and (valid? ::bar a)
are both true#2017-07-2312:18thedavidmeisterbut (valid? (spec/and ::foo ::bar) a)
is false?#2017-07-2312:37schmeeit would be weird if there was a different syntax for registered keywords, better to have a uniform interface and accept the extra typing#2017-07-2312:38schmeeyou can make a macro that does what you want#2017-07-2313:10thedavidmeister@schmee mmk, any idea why spec/and
can be false when each of the individual items are true?#2017-07-2313:11schmeewell, in your example you test that a
is valid for ::foo
and b
is valid for ::bar
#2017-07-2313:12schmeethen you test if ::foo
and ::bar
is true for b
#2017-07-2313:12schmeeso youāre testing two different things#2017-07-2313:12schmeethe first doesnāt imply the second#2017-07-2313:22thedavidmeister@schmee typo, it's all supposed to be a
#2017-07-2313:23thedavidmeisterboot.user=> (spec/valid? :item/item--exists {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}})
true
boot.user=> (spec/valid? :item/item--user-input {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}})
true
boot.user=> (spec/valid? (spec/and :item/item--user-input :item/item--exists) {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}})
false
#2017-07-2313:25schmeewhat does item/item--exists
and :item/item--user-input
look like?#2017-07-2313:28thedavidmeister(spec/def :item/item--exists
(spec/and
:item/item
(comp #(spec/valid? :item/id--exists %) :db/id)))
#2017-07-2313:28thedavidmeister(spec/def :item/item--user-input
(spec/and
:item/item
#(not (all-keys-in? % base-keys))))
#2017-07-2313:28thedavidmeister(defn all-keys-in?
[coll allowed-keys]
(= #{}
(clojure.set/difference
(set (keys coll))
allowed-keys)))
#2017-07-2313:31thedavidmeisterbut does it matter?#2017-07-2313:31thedavidmeisterhow can (and true true)
be false?#2017-07-2313:36schmeethat is a good question#2017-07-2313:46thedavidmeisterinterestingly, {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}}
was generated by exercise
for the spec/and
#2017-07-2313:48thedavidmeisterwait, that's a lie š#2017-07-2313:48thedavidmeisteri generated it with slightly different code#2017-07-2313:54thedavidmeisterhmmm, so if i do exercise for :item/item--exists
or :item/item--user-input
independently i can generate :item/list-id
#2017-07-2313:54thedavidmeisterbut together i cannot#2017-07-2419:39devthcontinually bitten by not knowing if instrumentation is on during interactive dev (e.g. turn it on, then redefine a symbol - now it's off) and tests for a given symbol. i think it's on, then find out i had an invalid spec in a lib for weeks because the upstream lib turned on instrumentation and threw an exception. š¤#2017-07-2419:43caiomutable shared state is the root of all evil š#2017-07-2419:40devthi think i want a single jvm flag to turn it on globally#2017-07-2419:47wilkerlucio@devth a flag would be complicated, if you think about how instrumentation works, it actually overrides your var, much like a monkey patch in Ruby if you are familiar with, so it would have to re-trigger on every def situation. a simpler solution would be to find a way to handle it on your editor, make it automatically call s/instrument
on file save or something#2017-07-2419:48devthfamiliar with how it works and monkey patching. that puts the onus of using this thing correctly on users, and the many tools/editors they use#2017-07-2419:50wilkerlucioyup#2017-07-2419:50devthso i'm not satisfied with that solution š#2017-07-2419:52devthi kinda like how schema had their own s/defn
. that would provide the means to flip a global switch.#2017-07-2419:53devthyes i know i'm free to build my own wrapper / macro / lib thing#2017-07-2419:53devthbut first have to answer "should i build my own wrapper marcro lib thing?"#2017-07-2422:11wilkerluciothe main issue on having a global toggle would be that it requires a conditional test overhead, imagine if you have to had this overhead on each function call to check if the parameters should be validated or not#2017-07-2422:12wilkerluciothat can be quite expensive, the way instrumentation works right now adds no overhead at all when you are not using it#2017-07-2422:12wilkerluciocurrently can be annoying to use, but editors and other features on top of it can solve this issue, while having the global check maybe not have a way to solve from the user code base#2017-07-2423:41devthwith a macro it could be zero cost#2017-07-2419:57tetriscodesHello, Iām trying to gen some JSON for the Netflix Eureka API and it has @ās in the keys#2017-07-2419:58tetriscodes(spec/def :port-def/$ int-str-gen)
(def enabled (keyword (symbol "port-def" "enabled")))
(spec/def enabled boolean?)
(spec/def ::port-def (spec/keys :req-un [:port-def/$ :port-def/enabled]))
#2017-07-2420:04tetriscodesproduces :port {:$ "9", :enabled false},
#2017-07-2420:05tetriscodesIāve been trying
(spec/def :port-def/$ int-str-gen)
(def enabled (keyword (symbol "port-def" "@enabled")))
(spec/def enabled boolean?)
(spec/def ::port-def (spec/keys :req-un [:port-def/$ enabled]))
#2017-07-2420:06tetriscodesproduces :port {:$ "9"}
#2017-07-2420:06moxaj@tetriscodes I think you should use a qualified keyword instead of a symbol for the enabled
spec#2017-07-2420:08moxajoh, you can't, the reader throws#2017-07-2420:09tetriscodesI thought :port-def/@enabled
was qualified#2017-07-2420:17moxaj@tetriscodes ugly workaround:
(def enabled (keyword (symbol "port-def" "@enabled")))
(defmacro foo []
`(s/def ~enabled boolean?))
(foo)
#2017-07-2420:18moxajhopefully you don't have too many references to the enabled spec ^^#2017-07-2420:32tetriscodesstill no enabled key#2017-07-2420:32tetriscodesspec/keys takes keys that are registered in the catalog as a spec#2017-07-2420:34avihi all š anyone happen to know why clojure.spec.gen
doesnāt include let
, and if thereās an idiomatic approach to doing what clojure.test.check.generators/let
does?#2017-07-2420:36tetriscodesHow are you using it?#2017-07-2420:37avi(def ^:private gen-datetime
(tc-gen/let [year (gen/large-integer* {:min 2017 :max 2117}) ; we should be so lucky
month (gen/large-integer* {:min 1 :max 12})
day (gen/large-integer* {:min 1 :max 28}) ; damn you February
hour (gen/large-integer* {:min 0 :max 23})
minute (gen/large-integer* {:min 0 :max 59})
second (gen/large-integer* {:min 0 :max 59})]
(str (string/join "/" [year month day])
" "
(string/join ":" [hour minute second]))))
#2017-07-2420:38avihmmm maybe thereās an altogether better approach hereā¦ I should probably just generate a unix timestamp integer within a certain range, then use clj-time
to convert it into the desired string formatā¦#2017-07-2420:39aviIām still curious about let
though#2017-07-2420:42tetriscodesyou could use spec/with-gen
to make your clause a regex and then have your generator create the date formatted as a string and then use clojure.spec.gen/sample
to generate data.#2017-07-2420:44aviThanks! Not sure I 100% follow but Iāll try to parse that (mentally) and eventually report back. Thank you!#2017-07-2420:44tetriscodesLooks like you are trying to generate data#2017-07-2420:45tetriscodesSo writing the generator can create some random data for you, or the library test.chuck has a string-from-regex function that you can use but it may be harder to get the ranges on your numbers.#2017-07-2420:45aviaha, string-from-regex sounds promising, I didnāt know about that!#2017-07-2420:46avithank you!#2017-07-2420:46tetriscodeshttps://github.com/gfredericks/test.chuck#2017-07-2420:46avioh ah I see#2017-07-2420:46avivery cool#2017-07-2420:46aviwill check it out#2017-07-2420:46avithank you!#2017-07-2420:47tetriscodeshttps://clojurians.slack.com/archives/C1B1BB2Q3/p1500927450713826 that still gives me errors because they vector needs to look up the keywords#2017-07-2420:47tetriscodesThat helps get the spec defined but the lookup of the specās keyword return nil#2017-07-2420:48tetriscodesI had heard that in the future spec/keys
may take a function, but I donāt see that in the latest alphas#2017-07-2420:51moxaj@tetriscodes well, anywhere you'd use that spec, you'll have to wrap that form in a macro to bypass the reader#2017-07-2420:51moxajhence my comment#2017-07-2420:51moxajbut i'm sure there are cleaner solutions#2017-07-2420:51tetriscodesBut how does the definition of the s/keys vector find that key?#2017-07-2420:52tetriscodesBecause I still have to do :port-def/@enabled
#2017-07-2420:52tetriscodeswhich isnāt a valid keyword#2017-07-2420:52moxaj(defmacro bar []
`(spec/def ::port-def (spec/keys :req-un [:port-def/$ ~enabled])))
(bar)
#2017-07-2420:53tetriscodesoh#2017-07-2420:53tetriscodesthat worked#2017-07-2420:53moxajmaybe you could do some preprocessing with spec/and
, afaik conformed values flow through#2017-07-2420:53tetriscodesI should use macros everywhere#2017-07-2420:54moxajdefinitely not š this is last resort#2017-07-2420:54tetriscodesThanks. Its in one spot, so Iāll keep it hidden.#2017-07-2420:54tetriscodesYes, iāve been encouraged to avoid them, so Iām not familiar.#2017-07-2420:55moxajwell, macros in general, but particularly in this case, since this is a 'hack'#2017-07-2423:41aviany chance anyone can see whatās wrong with this?
(s/def ::stream-of-string-rows
(s/with-gen
#(instance? InputStream)
(fn [] (gen/fmap (fn [v] (->> (map (partial string/join ",") v)
(string/join "\n")
.getBytes
io/input-stream))
(gen/vector (s/gen ::string-row))))))
I keep getting errors like this when I run stest/check
on the namespace this is in:
java.util.concurrent.ExecutionException: clojure.lang.ArityException: Wrong number of args (1) passed to: events/fn--4703
clojure.lang.ArityException: Wrong number of args (1) passed to: events/fn--4703
Not sure whatās happening, because when I define these specs in the REPL they work just fine with gen/generate
ā¦#2017-07-2423:47aviah crap I figured it outā¦ the spec should have been #(instance? % InputStream)
#2017-07-2423:47avipicard-facepalm#2017-07-2423:48avior, rather (partial instance? InputStream)
#2017-07-2423:48avipicard-facepalm picard-facepalm#2017-07-2502:21jpmonettas@bbrinck I remember yo were working on a lib for formatting clojure.spec errors, I've been also experimenting with the same but with GUIs#2017-07-2502:21jpmonettashttps://github.com/jpmonettas/inspectable#2017-07-2502:21jpmonettasthat's a link to it#2017-07-2502:22bbrinckLooks great!#2017-07-2512:50mishaHow do I spec various length tuples? For example datomic's datoms, which can be
[e a v t op]
[e a v t]
[e a v]
[e a]#2017-07-2512:50mishas/or
multiple s/tuple
s (or s/cat
s)?#2017-07-2512:54mishabtw, what are more suitable use cases for s/cat
as opposed to s/tuple
?#2017-07-2512:58mishaI think s/cat
is suitable when you need to destructure seq into map, for example for clojure.pprint/print-table
#2017-07-2513:17schmee@misha use s/?
for variable length tuples#2017-07-2513:18schmees/cat
requires you to tag the elements which can be useful for conforming while s/tuple
doesnāt#2017-07-2513:19misha@schmee s/?
s/+
expect homogeneous elements as far as I can tell reading docs.
I, on the other hand, need to spec a seq where particular elements are of specific types#2017-07-2513:19schmeenot sure what that means, can you give an example?#2017-07-2513:20misha(defmacro +
"Returns a regex op that matches one or more values matching
pred. Produces a vector of matches"
[pred-form]
#2017-07-2513:22mishait'd be ok to use it to spec, say, seq of keywords.
but I need to specify, seq of
[int keywords type-a type-b bool] or
[int keywords type-a type-b] or
[int keywords type-a]#2017-07-2513:22misha(basically each element can have its own spec)#2017-07-2513:23mishaI end up with or
and tuple
(s/def :fsm/transition
(s/or
:6-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger :fsm/guard :fsm/behavior :fsm/internal-transition?)
:5-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger :fsm/guard :fsm/behavior)
:4-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger :fsm/guard)
:3-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger)))
#2017-07-2513:24schmeesomething like (s/def ::datom (s/cat :e int? :a (s/? keyword?) :v (s/? map?) :t (s/? inst?) :added (s/? boolean?)))
?#2017-07-2513:25mishayeah, I might replace tuple
with cat
if I will require seq-to-map destructuring.#2017-07-2513:26mishabut in your example you need to wrap it in the s/or
and add more "arities", because it'll fail validation on shorter seqs with Insufficient input
#2017-07-2513:26schmeenope: dev=> (s/conform ::datom [1 :type])
{:a :type :e 1}
dev=> (s/conform ::datom [1 :type {:asdf 1}])
{:a :type :e 1 :v {:asdf 1}}
#2017-07-2513:27mishathat's odd. how did I get "Insufficient input" then?#2017-07-2513:27schmeecan you post your spec?#2017-07-2513:28mishaI wiped it out already opieop#2017-07-2513:28mishaah, lol, you wrapped each subspec in (s/?)#2017-07-2513:29mishayour spec probably will accept missing-element-specs too.#2017-07-2513:29mishalike [e a t]#2017-07-2513:29mishawhich are not the droids I am looking for#2017-07-2513:30schmeedang, you got me on that one#2017-07-2513:32mishathe only thing I don't like about or
- those :6-tuple
labels always feel like a hack.
It's like I put effort into designing/naming specs, but those labels? just barf some random name out to make compiler stop complaining.#2017-07-2513:33misha(or rather about me using or
and similar specs)#2017-07-2513:36schmeewell, itās great for some things, like when writing parsers and you want to dispatch on the tags#2017-07-2513:36schmeewell, itās great for some things, like when writing parsers and you want to dispatch on the tags#2017-07-2513:38hmaurercan you give an example of that please? I am interested#2017-07-2513:40mishawhen you conform, and then pass conformed value to any multimethod#2017-07-2513:45schmeeI did a toy macro to create maps which has some special syntax for not including nil values#2017-07-2513:45schmeefirst I wrote regular clojure code to parse it, but then I thought, why not do it with spec?#2017-07-2513:45schmeeso the conformed input looks like this: dev=> (spec/conform ::the-spec '(:a 1 :b 2 (maybe :c) nil [:d (pred-fn)] 123))
[{:k [:any :a] :v 1}
{:k [:any :b] :v 2}
{:k [:maybe {:k :c :maybe maybe}] :v nil}
{:k [:pred {:k :d :pred (pred-fn)}] :v 123}]
#2017-07-2513:46schmeeso you can see the :any
, :maybe
and :pred
tags in there which come from s/cat
specs#2017-07-2513:47schmeewhich I then use to dispatch and parse to the appropriate form#2017-07-2513:48schmeeslack is acting up on me, excuse the double posts#2017-07-2513:51hmaurer@schmee itās acting up on me too; probably a server issue#2017-07-2513:51hmaurerthanks for the explanation š#2017-07-2513:36schmeebut in your case I agree it doesnāt add much value#2017-07-2513:38mishaI just have not had enough experience with spec to be actually using those tags much "later" in the code, so I don't really have naming intuition developed yet.#2017-07-2514:23mishaare there any apparent downsides of reusing spec names as dispatch keys in s/or
/`s/cat`?
(s/def :foo/bar
(s/or
::spec-one ::spec-one
::spec-two ::spec-two))
(apart from verbose error messages because of all the long qualified dispatch kw namespaces)#2017-07-2515:39mishaIs there a preferred way to coerce value during conforming? If it'd become or
's dispatch value ā it's fine.
something like:
(s/def ::ft/internal?
(s/or
false #{false nil :fsm/external}
true #{true :fsm/internal}))
(s/conform ::ft/internal? :fsm/internal)
;; => [true :fsm/internal]
#2017-07-2515:50Alex Miller (Clojure team)there is a currently undocumented s/nonconforming that you could wrap around the s/or for that#2017-07-2516:08mishathe exact problem I have in the s/or
above is "Assert failed: spec/or expects k1 p1 k2 p2..., where ks are keywords", and I used booleans hoping to get this kind of coercion#2017-07-2516:29mishafound this! https://gist.github.com/Deraen/6c686358da62e8ed4b3d19c82f3e786a#2017-07-2516:33Alex Miller (Clojure team)oh sorry, I didnāt even read your spec! the ks need to be keywords. And sets of falsey values wonāt work (as they will return falsey values even when thereās a match)#2017-07-2516:38Alex Miller (Clojure team)hard to suggest an exact rewrite without knowing your goals re conforming#2017-07-2516:38mishaI wrote a conformer for that:
(defn internal-transition? [t]
(case t
nil false
false false
true true
:fsm/external false
:fsm/internal true
#?(:cljs :cljs.spec.alpha/invalid
:clj :clojure.spec.alpha/invalid)))
(s/def ::ft/internal? (s/conformer internal-transition?))
(s/conform ::ft/internal? :fsm/internal) ;=> true
(s/conform ::ft/internal? :fsm/external) ;=> false
(s/conform ::ft/internal? :foo/bar) ;=> :clojure.spec.alpha/invalid
#2017-07-2516:39Alex Miller (Clojure team)bleh#2017-07-2516:39mishaopieop#2017-07-2516:39Alex Miller (Clojure team)you donāt need a conformer#2017-07-2516:40Alex Miller (Clojure team)the problem with conformers is that you are doing a lossy conversion that throws away the original value, and making that decision for all future users of your spec#2017-07-2516:41mishathis is true. I can return [true :fsm/internal]
though, right? and write an unformer for such values#2017-07-2516:42Alex Miller (Clojure team)I would spec it as (s/def ::ft/internal? (s/or :b (s/nilable boolean?) :k #{:fsm/external :fsm/internal}))
#2017-07-2516:42Alex Miller (Clojure team)the tagged result tells you how to coerce the value if needed#2017-07-2516:43mishaI just wanted to avoid dispatching on arbitrary keyword later, instead of doing something like:
(->> huge-map ... ::ft/internal? first)
;;or just
(->> huge-map ... ::ft/internal?)
#2017-07-2516:44Alex Miller (Clojure team)I would at the very least write the spec as I have it above, then write a second spec that applies a conformer to the result of the first if needed#2017-07-2516:44mishayeah, and so I'd need to
(->> huge-map ... ::ft/internal? coerce-internal)
#2017-07-2516:44Alex Miller (Clojure team)yeah#2017-07-2516:46mishajust going through the options I have atm. Baked in conformer is indeed "bleh" comparing to just using coerce-fn there. I just don't really know how many client call sites there will be, don't want to forget calling coercion somewhere.#2017-07-2517:27wilkerlucio@misha also you can check the spec-coerce project: https://github.com/wilkerlucio/spec-coerce#2017-07-2518:36mishathanks @wilkerlucio#2017-07-2520:26aviš Hi all! Iām new to spec and Iāve got a fairly complex fdef
thatās taking a very long time to runā¦ ~30 seconds. Are there any tips, articles, best practices, etc on debugging this sort of thing? Thanks!#2017-07-2520:27avi(I can try to reduce it down to a gist if anyoneās interested in seeing my slow ugly code)#2017-07-2520:45english@aviflax a gist might be useful. also, whatās taking a long time to run? is it stest/check
?#2017-07-2520:47aviAh, yes, it is stest/check
ā thanks!#2017-07-2520:47aviI will try to put together a gist that isolates the problem#2017-07-2601:29Alex Miller (Clojure team)@aviflax there are a couple known issues related to s/coll-of and the other collection specs. if you posted your args spec, I might be able to suggest something#2017-07-2613:59avi@alexmiller great, thank you! Iām working on a gist now ā almost done, but having a snag with tools.deps.alpha. Where/how should I report this snag?#2017-07-2615:49Alex Miller (Clojure team)@aviflax The jira link for tools.deps is on the readme page #2017-07-2615:55aviGot it ā will report there. Thanks!#2017-07-2616:12avi@alexmiller this reproduces and demonstrates my problem and how Iām currently working around it by setting :num-tests
low: https://gist.github.com/aviflax/1a9ba7e73d45157bfc03f6b11c3b9b18#2017-07-2616:47Alex Miller (Clojure team)Two things:
1) there is a known issue when using s/coll-of with :kind but without :into (https://dev.clojure.org/jira/browse/CLJ-2103) - this should be fixed soon, but adding your own :into clause will work around it. This could easily be enough.
2) using s/coll-of with a :gen-max option can also help constrain the size of generated collections#2017-07-2616:48Alex Miller (Clojure team)actually, #2 may be a non-issue for you since youāre using :count#2017-07-2616:48Alex Miller (Clojure team)so Iād recommend in line 20ish, adding :into []
#2017-07-2618:29avi@alexmiller will do ā thanks so much!#2017-07-2619:08gfrederickswould a conj talk on test.check be useful? if so, what angle exactly? usage patterns? implementation details? something else?
I haven't really considered this before because I figured that reid had already talked about it a few years back and it hadn't changed much since then, but maybe that's not true and/or not a good reason#2017-07-2619:30avi@alexmiller I added the :into
but it doesnāt seem to have made a difference ĀÆ\(ć)/ĀÆ#2017-07-2619:35aviadding :gen-max
doesnāt seem to make a difference either š#2017-07-2623:01Alex Miller (Clojure team)Well on the upside, you can rule those out as the problem :)#2017-07-2700:11aviShould I open a JIRA ticket?#2017-07-2700:18aviIāve shrunk down the example just a bit by removing the CSV parsingā¦ itās still slow š#2017-07-2700:18avihttps://gist.github.com/aviflax/1a9ba7e73d45157bfc03f6b11c3b9b18#2017-07-2620:00misha@gfredericks I'd listen about approaching to property based testing, something in between cookbook and best practices, to improve test-design intuition (what ever that means)#2017-07-2620:01gfredericks@misha cool; I think that's more or less what I meant by "usage patterns", maybe#2017-07-2620:01mishamaybe kappa#2017-07-2620:10wilkerlucio@gfredericks a fresher way to go about it might be using spec as reference, how to use and extend generators for specs, and how to write tests using those#2017-07-2620:11gfredericksyeah, intersecting with spec would probably be more relevant. potentially harder for me since I haven't used spec very much#2017-07-2620:12wilkerlucioyou can use that as a reason to get more into spec too š#2017-07-2620:12gfredericksright š#2017-07-2620:32danielcompton@gfredericks the hardest thing I've found with property based testing is figuring out useful properties that can be calculated, without reimplementing the original function#2017-07-2620:32gfredericksyeah I think that's common. that would fit better as a pure test.check patterns talk#2017-07-2622:19dadairis there a way to do nested/consecutive multi-specs? e.g., {:type :a}
, then also expect :subtype :b
, and now multi-spec based off :subtype
?#2017-07-2623:00Alex Miller (Clojure team)You can have a multispec return a spec that is another multispec or you can dispatch on both at the same time using a more complicated dispatch function#2017-07-2623:02Alex Miller (Clojure team)@gfredericks a talk on building more complex generators would be very useful and relevant to both spec and test.check#2017-07-2623:05gfredericks@alexmiller so limiting the scope to generators, enabling a deeper exploration?#2017-07-2623:05Alex Miller (Clojure team)I think there is plenty to talk about#2017-07-2623:06Alex Miller (Clojure team)Property test patterns is good too but less relevant re spec and has somewhat been done#2017-07-2623:08gfredericksokay cool; that happens to also be probably the most enjoyable for me to put together#2017-07-2623:20dadair@alexmiller do you have an example of what a multispec returning a multispec would look like? I've tried that and didn't seem to be able to get it to work#2017-07-2713:22misha@alexmiller @gfredericks I'd listen about "highjacking" specs with generators, and generators strategies in general, e.g. for cases, where you need to define some root rules, and all downstream generators need to inherit it.
Which is, probably, the same issue/situation, as "I wrote a lib, and need to either hardcode some resource to use it implicitly (bad, but easy), or need every function to accept it as an argument (good, but messy verbose)". Think taxi-driver
's browser arg, component
vs. mount
, etc.
One use case would be testing datomic/datascript, where on the one hand you have all the specs about db
/`tx-report` structure, but on the other, when you test actual transaction, ā an extra layer of testing the data flow is required, like "my temp ids appear in tx-report, and transacted data appears in db". Such "flow" specs can be described, and tested generatively, but it is somewhat messy, and is hard to keep de-complected from "structure" specs.#2017-07-2713:24gfredericks@misha is this a situation where you're doing a higher level test and want to model some external system? #2017-07-2713:27mishathe closest to real life I got ā is datascript testing, which I described. It probably is "higher level testing" from some pov, but feels like essential for any lib a larger than few util functions#2017-07-2713:29mishaYou might perceive datascript as being external to the app code, yes. Not sure if I answered your question#2017-07-2713:31gfredericksIs "flow" vs "structure" partially about one call vs multiple calls?#2017-07-2713:33mishayes, though, those multiple calls could be a single fn.#2017-07-2713:33gfredericksRight; okay cool that makes sense#2017-07-2713:39mishaok, how'd you go about testing datscript/datomic's transact!
?
I'd have specs for db, tx-report.
I'd see, if structure conforms the specs.
But then I'd need to test that, all temp-ids are permanent, that mapped temp-ids - are the same and for same entities, as in submitted data.
That extra data in new db - is in fact submitted data.
I feel, like it can be done on "generators" level, and not in just custom test code, which does not leverage, say, spec.
Because after I'm done testing datascript, I want to model it (lol now I understand your question opieop) to enable testing app code build on top of it, and I would not be able to do this with custom test code, which does not leverage generators#2017-07-2713:42gfredericks@misha so when you say "custom test code" you're thinking of example-based tests, not something like vanilla test.check properties?#2017-07-2713:42mishayeah, I think so.#2017-07-2713:44mishaI think the question can be narrowed down to "how to use system-wide seed data, and set of rules, which affect all(most) of the generators"#2017-07-2713:46mishaI don't have an intuition developed in this area much, so I might be talking gibberish here and there. That's why I'd listen to something structured on the topic opieop#2017-07-2713:48gfredericksyeah, it sounds like you need a higher level overview; which is somewhat at odds with alex's more focused talk idea, but it might be that going over certain generator patterns would illuminate other aspects as a side effect#2017-07-2713:59mishaanything helps, and generative testing is so huge, but "under-appreciated" from talks perspective.
Everyone's just "here, you can generate strings, ints, and booleans. Boom! Off you go."#2017-07-2721:45misha@gfredericks check this out, interesting approach to closures, to solve what I was trying to describe earlier today: https://clojurians.slack.com/archives/C03S1KBA2/p1501187926443798#2017-07-2721:46mishamore on this in this thread https://clojurians.slack.com/archives/C03S1KBA2/p1501179951981871?thread_ts=1501070045.404030&cid=C03S1KBA2#2017-07-2721:50misha~alternative to closures and dynamic vars#2017-07-2722:08gfredericks@misha this is related to test.check generators somehow? #2017-07-2722:30misha@gfredericks no, but this might be useful in a highjacking generators with seed data. Seemed relevant to what I wrote earlier today...#2017-07-2722:35gfredericks@misha I'm having trouble figuring out what you mean by hijacking a generator#2017-07-2722:43mishaI am having trouble describing it opieop#2017-07-2722:47mishaI had a problem with my spec usage, where I wanted to have a spec for, say datascript entity. This spec contains datascript id spec, which can specify either id-before-transaction (actual id or temp id), or id-after-transaction (actual id only). So I have this nested spec, at the bottom of which there is an id-spec. For some tests I wanted that id-spec to be id-before-transaction, and for the others ā id-after-transaction.#2017-07-2722:49bfabryok I can't find a gif to demonstrate it. but imo that sounds like a sign that you have two different things that should have different names#2017-07-2722:52misha@bfabry that's sort of true, but I doubt, that redefining entire entity spec with only id spec change ā is THE solution.#2017-07-2722:52seancorfieldWhat about s/merge
?#2017-07-2722:53seancorfieldDefine a datascript-core
spec and then have datascript
, datascript-before-transaction
, and datascript-after-transaction
for the three cases where you 1) don't care, 2 & 3) do care.#2017-07-2722:53bfabry^#2017-07-2722:54seancorfield(or possibly you only need datascript
and transacted-datascript
since the ID in the first one is either anyway)#2017-07-2722:54seancorfieldGiven that you have two ID specs, presumably there are situations where you care which type of ID you have?#2017-07-2722:54seancorfield(if you don't care, you don't need two specs -- you're over-specifying!)#2017-07-2722:56misha@seancorfield in this example, you are might be 100% right. But I just want to illustrate a situation, where you might have a composed spec more than 2 levels deep, and you want to replace a leaf sub-spec or a generator for it.#2017-07-2722:57mishain that case I cared about pre/post transact ids, hence the "story"#2017-07-2722:58mishaall of it is to describe "what I'd like to hear about in test.check conj talk" opieop#2017-07-2817:26bfabry@alexmiller is there anywhere to look to find out which core functions are spec'd / will be spec'd?#2017-07-2817:26bfabryactually there's an obvious answer to that, just grep for fdef in master, doh#2017-07-2818:56jpmonettas@bfabry check out https://github.com/jpmonettas/inspectable/ , specially the browse-spec functionality#2017-07-2820:04nopromptdoes anyone know of existing code which produces fully resolved spec descriptions with respect to the namespace from whence they came?#2017-07-2820:05noprompte.g. (spec/describe some-spec) => (clojure.spec.alpha/coll-of foo.bar/bean-sprout?)
not (`(coll-of bean-sprout?)`)#2017-07-2820:06nopromptiām asking before i embark on the journey.#2017-07-2820:29jpmonettasif I understood correctly (spec/form some-spec) is what you are looking for#2017-07-2820:30jpmonettasalso check https://github.com/jpmonettas/pretty-spec if you want to pretty print it#2017-07-2820:39wilkerlucio@noprompt I did some stuff that does reflection on the specs, I would be interesting to know if you can figure that out. afaik, this can be tricky, because to correctly resolve that you need to know the namespace definitions for the place where the spec was defined (which can be different from the spec namespace itself), so I'm not sure if that is even possible, given that you might not be able to disambiguate the non-qualified names (or even the qualified, considering aliases can change). but anyway, please let me know if you figure it #2017-07-2820:42jpmonettas@noprompt @wilkerlucio but isn't (spec/form ...) just that?#2017-07-2820:47wilkerlucio@jpmonettas that returns the original form as data, but it doesn't give you the context about, so imagine that I have a spec like (s/def ::some-spec my-pred?)
, just by reading the form, how can you tell from where my-pred?
comes from?#2017-07-2820:48wilkerluciomaybe by getting the var
from the function reference, might be possible there, but then I have no idea on how to do that on CLJS#2017-07-2820:51jpmonettas@wilkerlucio sorry still don't get it, s/form returns everything qualified, I think the idea is you can serialize, deserialize specs#2017-07-2820:52wilkerlucio@jpmonettas ha, you are right, hahha, man, I was assuming it didn't because of what noprompt said, but I did the test now and you are right, it returns with the full name š
#2017-07-2820:54noprompt@jpmonettas @wilkerlucio thank you!#2017-07-2917:15stathissiderisdoes stest/check
expect the :gen
option to contain a map of spec-names to generator-returning functions?#2017-07-2923:49stathissiderisfor the record, the answer is yes#2017-07-3000:13camdezApologies because Iām sure this gets asked all the time butā¦if I have nested data which reuses keys, like thisā¦
{:id 214566,
:name "Fake Co",
:phone_number {:id 141683, :phone_number "555 123 4567"}}
ā¦if I donāt want to do something gross like a semantic-defying s/or
, is my only option to make have two phone_number
specs in separate namespaces?#2017-07-3000:30camdezAnd, assuming that is right, any advice on where those additional namespaces live? Do you actually go with a proliferation of tiny files? Or create the namespaces without creating files? And then do you alias those namespaces or just go with something like this?:
(ns foo.specs
(:require [clojure.spec :as s]))
(s/def ::id integer?)
(s/def ::name string?)
(s/def :foo.specs.phone_number/phone_number string?)
(s/def ::phone_number (s/keys :req-un [::id :foo.specs.phone_number/phone_number]))
(s/def ::business (s/keys :req-un [::id ::name ::phone_number]))
#2017-07-3001:13bfabry@camdez you don't need to actually create namespaces to use them, you can just
(alias 'foo.customers 'customers)
(alias 'foo.customers.phone_numbers 'phone_numbers)
and then do
(s/def ::phone_numbers/phone_number string?)
(s/def ::customers/phone_number (s/keys :req-un [::phone_numbers/phone_number))
#2017-07-3001:16bfabrystandard disclaimer that if you can figure a way to make your app use namespaced keywords using :req and :opt rather than :req-un and :opt-un you'll probably have a better time in the long run#2017-07-3002:22camdezThanks, @bfabry.
A couple small notes for posterity after experimenting a bitā¦
1. The argument order for alias
is actually the other way: (alias 'phone_numbers 'foo.customers.phone_numbers)
2. The namespace actually needs to exist before you can alias it (one could (create-ns 'foo.customers.phone_numbers)
), even though the namespaced keywords used for the spec registry donāt necessarily need to exist.
I appreciate the disclaimer but Iām trying to wrap a spec around an API response I donāt control, so it is what it is.#2017-07-3002:24camdezThis all makes me wonder even more how people are actually doing things. Does your spec registry tend to match actual namespaces? Are those in the same namespaces as your code? Or a parallel set of namespaces? Inquiring minds want to know. š#2017-07-3002:27camdezIMHO Spec seems broad enough in its scope to require not just how can I but how should I documentation.#2017-07-3004:35seancorfield@camdez We have a combination of keywords where the namespace matches a code namespace and others where they just use a unique single-segment prefix.#2017-07-3004:42camdez@seancorfield Thanks! Whatās the deciding factor between the former and the latter?#2017-07-3004:42seancorfieldI think the recommended approach for alias
currently, if you want to introduce a new ns is (alias 'alias (create-ns 'the.full.name))
FYI.#2017-07-3004:43seancorfield@camdez It Depends(tm) š#2017-07-3004:46seancorfieldFor specs that are internal to a ns, the obvious choice is just to use ::name
and it'll be in that ns. For specs that are intended to be used across nses, then you need a "unique enough" name for the intended usage. So if that's a subsystem of your own application, you can just use a simple qualifier. For example, we use wsbilling
for World Singles Billing. You could use a prefix that matched a DB table, for example, or a component in your app.#2017-07-3004:47camdez@seancorfield Thanks. Good explanation.#2017-07-3004:47seancorfieldFor specs that are intended to be more global, then the prefix also needs to be more global.#2017-07-3004:52seancorfieldI think it's a bit early yet with clojure.spec
for "best practices" to have truly settled down.#2017-07-3004:57seancorfieldWe've been using it heavily in production for quite a while. We've defined the data model for a few systems with it and we use those for both test generation of data and validation of some inputs. We also have a series of coercing specs defined around one of our REST APIs: we s/conform
the raw parameters and get either ::s/invalid
or the valid parameters conformed to longs, doubles, strings, keywords etc.#2017-07-3005:17camdez@seancorfield I definitely agree RE best practices but it feels like such a buffet that we definitely need to be having the conversations about what might become best practices.#2017-07-3005:18camdezDespite some valiant documentation efforts (largely around whatās possible) I think itās still hard for a spec beginner to know where to start.#2017-07-3006:27seancorfieldYes, definitely hard. I actually submitted a talk for one of the Clojure conferences about our use of spec, but after talking to Alex Miller, I pulled it because I wasn't sure we were doing things a good way (I've since changed my mind again š )#2017-07-3006:28seancorfield(to be fair, I had a few reasons for pulling the talk but that was the main one)#2017-07-3006:29seancorfieldI'm looking forward to hearing other people's spec talks at future Clojure conferences!#2017-07-3008:19matanHi all, are we approaching a release of clojure 1.9 with the finalized spec in it?
I still see in the docs a mention of :clojure.spec.alpha/invalid
, will I have to change my code to use a differently named value later on or is it just out-of-date documentation?
https://clojure.org/guides/spec#2017-07-3008:19matanThanks!#2017-07-3015:10camdez@matan Iām not speaking with any authority here, but I donāt believe the docs are out of date (see latest here: https://github.com/clojure/spec.alpha). I believe you will have to use a different named value later, but if youāre requiring spec with an alias (e.g. (:require [clojure.spec.alpha :as s])
) and using that alias to refer to the keyword (e.g. :s/invalid
) then youāll only need to change the require
statement later.#2017-07-3015:12camdezIām assuming that code living under an alpha
namespace is not subject to the standard guidelines about accretion and breakage (indeed, thatās likely the exact meaning of the alpha
namespace), and thus clojure.spec.alpha
will become clojure.spec
in a non-backwards compatible way.#2017-07-3018:11stathissideristhis is how to generate params for a function based on its spec:
(gen/generate (s/gen (:args (s/get-spec (resolve `my-fn)))))
#2017-07-3018:11stathissideris(1 sample)#2017-07-3101:30camdez@seancorfield Aha! Thank you!#2017-07-3105:51Oliver GeorgeI'd like to check some javascript data based on a regular spec which uses s/keys and s/coll-of.#2017-07-3105:52Oliver GeorgeI know it's not in scope for clojure-spec but perhaps someone has written something which does this?#2017-07-3105:52Oliver George(s/assert-obj ::typical-spec #js {:a [1 2 3]})
#2017-07-3106:31ikitommi@camdez if you are dealing with deeply nested legacy models and only want to validate those in the api layer - after which you only transform them to conform your own application specs - you could check out data-specs (in spec-tools lib). Builds on top of clojure.spec. Itās *not* a good/best practise, but food for thought anyway. Something like:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.data-spec :as ds])
(s/def ::business
(ds/spec
::business
{:id integer?
:name string?
:phone_number {:id integer?
:phone_number string?}}))
(s/valid?
::business
{:id 214566,
:name "Fake Co",
:phone_number {:id 141683, :phone_number "555 123 4567"}})
; true
#2017-07-3106:32ikitommia sample bare-bones http api with it: https://github.com/ikitommi/business, added a :tags
(a set of keywords) to demonstrate the automatic coercion (string->keyword & vector->set here)#2017-07-3112:26camdezThanks! Iāve used Plumatic (nĆ©e Prismatic) Schema quite a bit, so this schema-resembles-data approach is quite familiar. But Iāve been using keyword prefixes divorced from the extant namespaces and itās working pretty well for me, if a tad verbose.#2017-07-3116:20bfabry@olivergeorge simplest thing that comes to mind is to just js->clj
and run spec over the result#2017-07-3117:50misha@bfabry @olivergeorge js->clj
and clj->js
are not symmetric, beware#2017-07-3119:19plinshello everyone, im trying to spec a map where all keys are optional but you should have at least one of them, is this achievable?#2017-07-3119:20bfabry@plins sure, (s/and (s/keys :opt [...]) not-empty)#2017-07-3119:21plinsi wasnt aware i could use not empty as a spec, thx!!!!#2017-07-3119:23bfabryyou can use any predicate function as a spec š though not-empty isn't technically a predicate function but it's good enough#2017-07-3119:24plinsso spec treats nil
as false
and truthy values as true
?#2017-07-3119:25bfabrywell, clojure treats nil and false as false and everything else as true. so yeah spec does too#2017-07-3119:25bfabrycljs.user=> (s/valid? (s/and (s/keys :opt [::foo]) not-empty) {::foo nil})
true
cljs.user=> (s/valid? (s/and (s/keys :opt [::foo]) not-empty) {})
false
cljs.user=> (s/explain (s/and (s/keys :opt [::foo]) not-empty) {})
val: {} fails predicate: not-empty
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha16935]
:cljs.spec.alpha/value {}
nil
#2017-07-3119:56mfikesIt doesnāt appear to be possible to use an s/cat
spec for an argument. Does this instrument failure make sense?
user=> (require '[clojure.spec.alpha :as s] '[clojure.spec.test.alpha :as st])
nil
user=> (s/def ::ingredient (s/cat :quantity number? :unit keyword?))
:user/ingredient
user=> (s/valid? ::ingredient [0 :teaspoon])
true
user=> (defn none? [ingredient] (zero? (first ingredient)))
#'user/none?
user=> (s/fdef none? :args (s/cat :ingredient ::ingredient))
user/none?
user=> (st/instrument)
[user/none?]
user=> (none? [0 :teaspoon])
ExceptionInfo Call to #'user/none? did not conform to spec:
In: [0] val: [0 :teaspoon] fails spec: :user/ingredient at: [:args :ingredient :quantity] predicate: number?
:clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x379e0a93 "
#2017-08-0101:12athosYou should wrap ::ingredient
with s/spec
, like (s/fdef none? :args (s/cat :ingredient (s/spec ::ingredient)))
#2017-07-3119:58plins@bfabry, i think i found a problem
(s/def ::my-spec (s/and not-empty (s/keys :opt-un [::a ::b])))
(s/valid? ::my-spec {} => false
(s/valid? ::my-spec {:A 1 :c 1}) => true
#2017-07-3119:59bfabry@plins well, that's not so much a problem you just didn't fully explain your requirements to me š#2017-07-3119:59plinsim sorry for that#2017-07-3120:00plinslet my try express myself better#2017-07-3120:00bfabryno it's ok I get it now#2017-07-3120:00plinsš#2017-07-3120:01bfabryyou probably want (s/def ::my-spec (s/and (some-fn :a :b) (s/keys :opt-un [::a ::b])))#2017-07-3120:02bfabrythere's various ways to get around that duplication, fwiw#2017-07-3120:04plinsiāll dig deeper into the docs, thx š#2017-07-3120:09plinsive settled with
(s/def ::my-spec (s/and (s/map-of #{:a :b} any?)
(s/keys :opt-un [::a ::b])))
thx!#2017-08-0104:55athos@plins I think (s/keys :req-un [(or ::a ::b)])
would be a smarter solution#2017-08-0110:39misha@athos is this even legal?#2017-08-0110:41mishait is!#2017-08-0110:43athos@misha Of course! s/keys
docstring refers to this feature.#2017-08-0110:44mishanot sure how I missed it then. anyway: wow, nice#2017-08-0111:33mishais "mutual exclusiveness" and/or "orthogonality" of a set of the predicates is even approachable, let alone provable automatically (e.g. during testing with spec)? Does it imply complete exhaustion of the possible input combinations?#2017-08-0111:42mishaContext is:
In Harel's charts (or UML state diagrams), state can have multiple transitions to mutually exclusive states, via the same trigger.
What ensures that system ends up in a valid state, is a combination of guards (predicates) on those transitions, which should guarantee mutual exclusiveness of the guard outcome. I want to try to automatically test whether guards on a set of transitions are mutually exclusive. e.g. :
(def FSM
{:A {:B {:trigger :a :guard pos-int?}
:C {:trigger :a :guard neg-int?}}})
Current state is :A
. Some event triggers transition :a
. Machine has to switch to either :B
or :C
, but not both.
I want to make sure guards (`pos-int?`, and neg-int?
in this case) are mutually exclusive. Automatically. At compile time.#2017-08-0111:44mishaIs it even possible? Or the best I can do is to just issue a "dude, make sure [:A :B :guard] and [:A :C :guard] are mutually-exclusive" warning?#2017-08-0111:44gfredericksthere's no general solution to that, but you could write a test.check property that asserts it#2017-08-0111:44gfredericksif you have a meaningful generator for the state#2017-08-0111:44gfredericksor whatever it is that's getting passed to the guards#2017-08-0111:45misha@gfredericks the complication with this, is the huge arity of the guards: they need to accept app-db, whole-machine, and event trigger with its args harold#2017-08-0111:46mishawhere for machine
I may supply generator, but app-db
and trigger args
structure, contents, and specs are entirely up to user#2017-08-0111:48gfrederickssounds tough š
you can validate at compile time of course#2017-08-0111:48mishaand for even as simple as a single string arg, exhausting 2 arbitrary predicates (e.g. with regexps inside) is doubtful, I think(?)#2017-08-0111:49gfredericksbut for a turing-complete language you can't in general check if two predicates are mutually exclusive#2017-08-0111:49gfredericksregexes might be doable, depending on the details#2017-08-0111:49gfredericksnot worth it though#2017-08-0111:49gfredericksjust a theoretical curiosity#2017-08-0111:49mishaI feel... relief š#2017-08-0111:53mishawhat if the predicate's source code is available? Is there something to, say, reverse-engineer things like (select-keys app-db [:foo :bar])
to reduce relevant input scope? Or at least to deduce, that guards, in fact, are orthogonal (which is useful too, but I forgot why just now)?#2017-08-0111:54gfredericksdefinitely for special cases#2017-08-0111:54gfredericksbut the general case impossibility I mentioned earlier applies to source or compiled or whatever else#2017-08-0111:55mishait'd be useful to cover at least some basic cases, because most of (UI) FSMs would actually use pretty basic guards#2017-08-0111:59misha(There are at least 2 implementation options: 1) pseudo states, where you define a fork in the transition as a separate state, and just ask to for cond/case
guard there. 2) duplicate transitions with separate boolean
guards, which (implicitly) need to be mutually-exclusive.
I am exploring (2) at the moment, to make literal FSM definition less verbose.)#2017-08-0112:32mfikesThe solution to my s/cat
in s/cat
question yesterday was, of course, s/spec
:
(s/fdef none? :args (s/cat :ingredient (s/spec ::ingredient)))
#2017-08-0115:19jpmonettasif someone finds it useful, just released a version of https://github.com/jpmonettas/inspectable with clojure and clojuresript support, also with an enhanced browser that shows the spec together with a sample#2017-08-0211:40NiclasCould someone help me understand why for the given example fn
(defn my-fn [selector]
(condp = selector
1 :b
2 :c
false))
it fails on the :ret
spec when calling the fn with the following spec
(s/fdef my-fn
:args (s/cat :selector any?)
:ret #{false :b :c})
but not for the following spec
(s/fdef my-fn
:args (s/cat :selector any?)
:ret (s/or :a #(= % false)
:b #(= % :b)
:c #(= % :c)))
#2017-08-0211:48mpenetprolly comes down to this: (s/valid? #{false š :c} false) -> false or (#{false} false)
-> false#2017-08-0211:49mpenetsince it will return the value matching in the set (false) this will break#2017-08-0211:52mpenettypical quirk with Sets#2017-08-0211:52scaturrIs there some established/idiomatic way to use spec with core.async? Particularly when reading from a channel?#2017-08-0211:52scaturrOr would it come down to adding a spec to a function that pushes on to the channel you are reading from?#2017-08-0211:53scaturrProbably way too broad of a questionā¦ lol#2017-08-0211:56mpenet@looveh so in short, use contains? (and specify a custom :gen if you need to)#2017-08-0211:56Niclas@mpenet Ah, thatās quirky indeed, thanks!#2017-08-0212:48schmeeunless you have a transient collection, for which contains?
doesnāt work#2017-08-0213:39matan@camdez thanks for last week's answer about clojure.spec.alpha/invalid
in the context of spec's code stability š#2017-08-0213:39camdez@matan Youāre very welcome. I hope it was helpful.#2017-08-0213:46matan@camdez yes, it certainly was!#2017-08-0213:47matan@camdez will I bump into a lot of stuff coming from its "alpha" namespace when using spec today?#2017-08-0213:47camdez@matan I donāt think youāll run into much.#2017-08-0215:04cddrAm I correct in thinking that spec would be the wrong tool to assert relationships between keys in a map.
For example, lets say we're representing a bill split between n
folks, can/should you leverage spec to assert that the sum of amounts under the payees key is greater than or equal to the top-level amount?
If not spec, is there some other library that allows you to organize validations like these in a way that lends support to generating good error messages
{:amount 60
:payees [{:name :andy
:amount 20}
{:name :rich
:amount 20}
{:name :alex
:amount 20}]}
#2017-08-0215:08mpenetseems fine to me, could be a (s/and (s/keys ...) sum-amounts>=total?)#2017-08-0215:13mpenetthen it's just pattern matching on the explain to return something more "flat" for error messages (depending on your usage), which you can do with/without spec (you could build it with a spec errors conformer that maps to your precise error types)#2017-08-0215:15ghadiagree with @mpenet. i don't think it's the wrong tool at all. just a predicate!#2017-08-0215:25cddrDoh! It never occurred to me that (s/keys)
could be anded.#2017-08-0219:45caio@cddr https://github.com/xsc/invariant this looks better for that#2017-08-0219:48ghadion the contrary, I really don't think you need a library to describe a datastructure -- this is literally the rationale for clojure.spec#2017-08-0219:49ghadiwhat cddr wants is a predicate that applies to a collection#2017-08-0219:49ghadiSimilar to how fspec
provides the :fn
spec that relates a function's :args
and :ret
#2017-08-0219:49caioyou're trying to define an invariant about a data structure, not a contract. I like the idea of separating those concepts (thus using a library to define invariants looks like a good idea)#2017-08-0219:50caiobut yeah, that's just being pedantic, so for a simple structure like this, I agree adding a new dependency is probably an overkill#2017-08-0219:51cddrI definitely prefer using the standard lib when it supports what I'm trying to do. Cheers for the suggestion though. Looks interesting š#2017-08-0219:53bfabryI think invariant is just a library for defining data structure predicates. if you have to write enough/complicated enough data structure predicates I think it would be useful to use with spec. similar to how specter is useful if you've got to do enough deeply nested data manipulation#2017-08-0219:53caio@cddr yeah, it is! I'm probably biased towards it because I used it and liked it a lot hahaha. In my case, I had to define invariants on a custom markup language#2017-08-0219:54caioso spec wasn't going to help there#2017-08-0221:16rgdelatois it possible to define a spec for map keys without def
ing a separate spec for each key? I guess what I have in my head is something like:
(s/keys :req-lit {:first-name string?
:last-name string?
:email (s/and string?
#(re-matches email-regex %))})
...where you could spec map keys inline?#2017-08-0221:17gfredericksI think that's an intentionally excluded feature#2017-08-0221:18rgdelatois there anywhere where I could read up on why that's the case?#2017-08-0221:18bbrinckhttps://clojure.org/about/spec - āMap specs should be of keysets onlyā#2017-08-0301:37bbrinckIām trying to make a patch to spec.alpha, but Iām running into problems testing my changes (with provided tests) and building a local jar (so I can include in my own projects for further testing). Iām used to doing everything with lein
, so Iām less familiar with ābareā clojure development. Does anyone know of a good guide? My googling is failing me.#2017-08-0301:40gfredericksmvn test
maybe?#2017-08-0301:47bbrinck@gfredericks yes, that works! thank you so much. Iām realizing I never bothered to learn much about the underlying java foundations before jumping straight into lein
for managing clj projects.#2017-08-0316:27bmaddyDoes anyone see what I'm doing wrong here?
user> (defn foo [x] :bar)
#'user/foo
user> (clojure.spec/fdef foo :args (clojure.spec/coll-of int?) :ret int?)
user/foo
user> (clojure.spec.test/instrument `foo)
[user/foo]
user> (foo :fail)
clojure.lang.ExceptionInfo: Call to #'user/foo did not conform to spec:
In: [0] val: :fail fails at: [:args] predicate: int?
:clojure.spec/args (:fail)
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file "form-init7754260867026619201.clj", :line 230, :var-scope user/eval98211}
user> (foo 1)
:bar
user> *clojure-version*
{:major 1, :minor 9, :incremental 0, :qualifier "alpha12"}
I would expect (foo 1)
to throw an exception with a message about how (int? :bar)
is false.#2017-08-0316:29mpenetyou need to call check
#2017-08-0316:29bfabry@bmaddy instrument does not check :ret specs#2017-08-0316:29mpenetinstrument only checks args (sadly)#2017-08-0316:30mpenetI think everyone hits that one once. Not really intuitive I guess#2017-08-0316:30mpenetarguably it can be considered a naming issue at least#2017-08-0316:31bmaddyInteresting.... Is there a reason for instrument
not looking at :ret
or is it just not implemented?#2017-08-0316:32mpenetit's intentional#2017-08-0316:32mpenetI think it's to allow granularity with gen overrides and such, in theory bundling the 2 in a single function isn't really difficult, it's just not provided#2017-08-0316:35bmaddyOk. Thanks for the help you two!#2017-08-0317:18misha@bmaddy
https://groups.google.com/forum/#!msg/clojure/jcVnjk1MOWY/UwP5bc1oCAAJ
https://groups.google.com/d/msg/clojure/JU6EmjtbRiQ/uND70kAFBgAJ#2017-08-0317:28matanJust went through the spec guide today... and looking forward to heavily using it for better code and better testing workflow. How do you control whether to instrument
, and whether to run test code reliant chiefly on stest.check
, using leiningen?
What would a leiningen workflow look like?#2017-08-0317:29bmaddyInteresting, thanks @misha. Those comments were very helpful.#2017-08-0318:44seancorfield@matan We're mostly using spec for defining and validating data structures, rather than testing. I've suggested to people that they call instrument
either in their test fixtures or directly in each test namespace as appropriate (to instrument functions in the namespace under test).#2017-08-0318:45seancorfieldI think there are lots of ways to go about using clojure.spec in tests -- and no "best practices" have arisen yet because it's still alpha and early days.#2017-08-0318:46seancorfieldAs Rich and others have said, generative testing -- via test.check
and clojure.spec.test/check
-- should probably be viewed as separate from your "unit testing", since it can take a while and you want "unit tests" to run very quickly (for fast feedback).#2017-08-0318:47seancorfieldSo it's probably a good idea to separate those out into "tests" that don't run as a normal part of your "unit" test suite, but can be run via a separate Leiningen/Boot task (so they can still be run automatically as needed).#2017-08-0318:48seancorfieldI will say that with clojure.java.jdbc
, which has optional specs, the "unit tests" run much, much slower with instrumentation in place so even that probably needs to be something you optionally enable for testing.#2017-08-0318:50gfredericksI wish there was a good way of specifying different profiles for test.check properties#2017-08-0318:51gfredericksyou want to do different things with it at different times#2017-08-0318:52seancorfieldFor example, clojure.java.jdbc
tests on Clojure 1.8 take about 30 seconds (user; 12.5s real) whereas tests on Clojure 1.9 with instrumentation of all java.jdbc functions take 1 minute 30 seconds (user; 1 minute real). So that's a huge overhead.#2017-08-0318:56royalaid@seancorfield thanks for clarifying that! I have been trying to add specs to an internal tool at where I work and seems like a massive uphill battle to get them to work in the way that I was expecting. Having the generative tests be a separate thing all together make so much more sense#2017-08-0320:08seancorfieldWe use s/conform
a lot for validating (and coercing) input to our REST API and also for user input validation, where we can pick apart s/explain-data
to generate informative error messages. That stuff's all in production.#2017-08-0320:10seancorfieldWe use test.check
and stest/check
for a small handful of what would otherwise be "unit tests" where they don't introduce a drag on our tests. We use s/exercise
and rand-nth
to get random, conforming input in some of our tests.#2017-08-0320:11seancorfieldSo far we've generally kept instrument
and stest/check
primarily for manual, isolated test runs. We'll probably integrate that based on environment variables or Boot task flags at some point.#2017-08-0320:12seancorfield@royalaid One thing to be careful of: don't try to spec everything! Use spec where it provides the most leverage, at system boundaries, and for key APIs, and/or key arguments to those APIs. One of the really nice aspects of spec (compared to, say, Typed Clojure) is that you really can opt-in one piece at a time, where you need it most.#2017-08-0321:24andrewboltachevHello. Is it possible to specify a complex spec inline (w/o s/def
ing something)?#2017-08-0321:30seancorfield@andrewboltachev You mean directly in a s/conform
or s/valid?
call, for example? Sure, specs are "just" predicates.#2017-08-0321:31andrewboltachev@seancorfield To be specific, I want to validate a map#2017-08-0321:31andrewboltachevbut e.g. (s/keys :req-un [::a])
refers to a symbol defined in (some) ns#2017-08-0321:32seancorfieldFor s/keys
you must s/def
the keys if you want them validated.#2017-08-0321:32andrewboltachevi.e. it takes both name and value from it#2017-08-0321:32seancorfieldIf you read the spec rationale, it explains why you can't specify both key names and value "types" together.#2017-08-0321:32andrewboltachevgot it, thanks!
looks anyway slightly opinionated solution for me š#2017-08-0321:32seancorfieldAt least it's the correct opinion š#2017-08-0406:13matanThere's not really something like that, a "correct opinion" for how to provide an API for humans š#2017-08-0321:35mishad#2017-08-0402:59bbrinckBesides core.specs.alpha
, does anyone know of an open-source project with a fair number of specs?#2017-08-0403:00ghadi@bbrinck https://github.com/ring-clojure/ring-spec/blob/master/src/ring/core/spec.clj#2017-08-0403:01bbrinck@ghadi Perfect#2017-08-0403:01bbrinckThanks!#2017-08-0406:11matanThanks for yesterday's short discussion of spec.check and all of that. I guess I'd use custom leiningen tasks and all those related leiningen features, to fiddle my workflow. As for test duration, a quick glimpse reveals that the number of generated inputs/runs is a parameter, and I'd gladly use slightly longer-running tests while controlling it#2017-08-0414:46yendaIs there a way with spec to validate maps like {:a 1 :b {:c 2}} were a is optional and c is only required when a is not present#2017-08-0417:23Alex Miller (Clojure team)you can s/and a custom predicate to make any check you like#2017-08-0417:23Alex Miller (Clojure team)something like (s/and (s/keys :opt-un [::a ::b]) #(if (:a %) (contains-key? (:b %) :c) true))
(can be improved, but you get the idea)#2017-08-0417:24Alex Miller (Clojure team)I flipped the case on :a but you know#2017-08-0423:53leongrapenthinIs there a way to prevent spec from checking fspeced lambdas with generative testing when instrumentation is enabled? It's not helpful when those functions are Callbacks that do side effects#2017-08-0423:58leongrapenthinI'd really like a knob to disable validation of fspeced lambdas through generative testing#2017-08-0423:58leongrapenthinAugmentation with a wrapper that checks the fspeced lambda during invocation against the fspec would be fine.#2017-08-0423:58leongrapenthinWould be great, even more so#2017-08-0500:26wilkerlucio@leongrapenthin I think you can do that, you have to give the fspec a name (make a s/def with it), use that name on your usages, then use spec overrides during the run of check, as documented here: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L381#2017-08-0500:34leongrapenthin@wilkerlucio Not sure thats what I'm looking for. My case is like this: A global functions :args are speced like this (s/cat ::config (s/keys :req [::on-click-save])), then there is (s/def ::on-click-save (s/fspec :args (s/cat :cust ::customer)))#2017-08-0500:34leongrapenthinNow when I invoke the global function with {::on-click-save (fn [cust] (persist-on-server cust))}, on-click-save will be invoked a hundred times with generated ::customers by specs validation code#2017-08-0500:35wilkerlucioah, true, that's for generators#2017-08-0500:36wilkerluciowell, not sure if there is an API for that, but something you could do is to change the spec registry directly, maybe that's a bad idea, but I guess it could work#2017-08-0500:36leongrapenthinYeah sure#2017-08-0500:36wilkerlucioits just a map in an atom, nothing fancy#2017-08-0512:59leongrapenthin@wilkerlucio it appears that setting s/fspec-iterations is a workaround, but it disables it for all#2017-08-0512:59leongrapenthinsetting it to 0#2017-08-0518:37leongrapenthincreated a ticket for it https://dev.clojure.org/jira/browse/CLJ-2217#2017-08-0522:39souenzzoAnyone using spec to specify strings/routes/path/ns?
I do not know if it was made for this, but it looks promising
(s/def ::path (s/or :path1 (s/cat :start #{\/}
:name (s/* (s/and char?
#(not= % \/))))
:path2 (s/cat :start #{\/}
:name1 (s/* (s/and char?
#(not= % \/)))
:div1 #{\/}
:name2 (s/* (s/and char?
#(not= % \/))))))
=> :user/path
(s/conform ::path (map identity "/foo"))
(s/conform ::path (map identity "/foo/bar"))
=> [:path1 {:start \/, :name [\f \o \o]}]
=> [:path2 {:start \/, :name1 [\f \o \o], :div1 \/, :name2 [\b \a \r]}]
#2017-08-0522:44souenzzoEven generators work: (map (partial s/conform ::path) (gen/sample (s/gen ::path)))
#2017-08-0616:40scaturrHow would s/conform
work in an http context? Like how would I conform JSON payloads for specs defined with namespaced keys? Like if I have a spec for ::entity/id
instead of just :id
?#2017-08-0616:42scaturr(s/conform ::entity {:id some-uuid})
-> invalid#2017-08-0618:22dergutemoritz@scaturr You can use :req-un
instead of :req
to achieve what you want in this case#2017-08-0618:42scaturrwould that lead to a situation where you have specs exist just for json conforming? or can :req
and :req-un
co-exist?#2017-08-0618:42scaturrseems like they could not#2017-08-0618:43scaturrthank you for the suggestion though š#2017-08-0621:04stathissiderisJust released spectacles: lenses for #clojure -- checked at runtime using #spec https://github.com/stathissideris/spectacles#2017-08-0622:11schmee@scaturr have a look at this: https://github.com/metosin/spec-tools#dynamic-conforming#2017-08-0622:11schmeeand the blog post: http://www.metosin.fi/blog/clojure-spec-as-a-runtime-transformation-engine/#2017-08-0719:05souenzzoMy actual spec says (s/def ::my-fns (s/coll-of fn?))
. Can I specify that it's a array of functions with arity 2?#2017-08-0719:11souenzzo(s/def ::my-fns (s/coll-of (s/fspec :args (s/cat :foo integer? :bar integer?)))
#2017-08-0719:14bfabryseems like that should work, does it not?#2017-08-0719:16souenzzoworks š. But i think that it's not on docs. I dig into code...#2017-08-0802:00didibusCan I spec a map in a way where I want it to have at-least one of two keys. So a valid map would be: {:x :y} {:x} or {:y}, but {} would not be valid?#2017-08-0802:24souenzzo@didibus
(s/valid?
(s/or :a (s/keys :req [:foo/bar]
:opt [:bar/foo])
:b (s/keys :req [:bar/foo]
:opt [:foo/bar]))
{}
)
#2017-08-0802:26didibusAh yes, that should work!#2017-08-0802:26didibusthx#2017-08-0803:25joshjones@didibus @souenzzo it's even more concise:
(s/valid?
(s/keys :req-un [(or ::x ::y)])
{:y 42 :x 33})
#2017-08-0813:26souenzzoWhy does it do not generate`{:x 33}`?
(gen/sample (s/gen
(s/keys :req-un [(or ::x ::y)])))
=>
({:x -1, :y 0}
{:x 0, :y -1}
{:x 0, :y -1}
{:x 0, :y 1}
{:x 0, :y -1}
{:x -2, :y 0}
{:x 6, :y -21}
{:x 6, :y 0}
{:x 4, :y -23}
{:x -8, :y 117})
#2017-08-0823:11madstap@souenzzo https://dev.clojure.org/jira/browse/CLJ-2046#2017-08-0814:58mbarbieriI'm playing for the first time with spec. I want to generate a spec from a contract map and test if another map is conforming.
As first step I would like that all keys in the map are present in the contract map.
But i read the contract from a json file, converting to unqualified keywords, so
(s/keys :opt-un (keys contract-map))
gives me Assert failed: all keys must be namespace-qualified keywords
Do I have to convert them to qualified? Or is it a better way to handle it?#2017-08-0815:01misha@mbarbieri :opt-un
expects coll of qualified keys (which might point to their specs), -un
part indicates that data will have no namespaces in keys.#2017-08-0815:04mbarbieri@misha ok data will have no namespaces, but not even the keys I'm passing in the spec will have, hence the failure. If I'm not wrong, that exception happen because my code will evaluate in something like:#2017-08-0815:04mbarbieri(s/keys :opt-un [:a :b :c])
#2017-08-0815:06mishait is ok for data to have no namespaces, but in spec declaration you need to supply namespaces, so later you can assign actual specs to those qualified keys, and s/keys
will check not only keys presence, but will check values against those specs as well#2017-08-0815:08mishayou need to supply qualified keys, yes. how exactly - depends on your use case.
If you are want to generate spec once and put in the source file ā just come up with a namespace and list those manually.
If you need to generate that spec anew dynamically every time ā that'd seem odd to me#2017-08-0815:08mishaanyway. check out this https://github.com/stathissideris/spec-provider#2017-08-0815:09mishamight be useful in any case#2017-08-0815:09stathissiderisIām here in case there are any questions š#2017-08-0815:10mishaI have one! not lib related tho :)
(s/explain-data
(s/map-of qualified-keyword? fn? :min-count 0)
{:foo/bar 9})
;;=>
#:clojure.spec.alpha{:problems ({:path [1],
:pred fn?,
:val 9,
:via [],
:in [:foo/bar 1]})}
(get-in {:foo/bar 9} [:foo/bar 1])
;;=> nil
(replaced string with number to illustrate my confusion with 1
as path inside 9
(?))#2017-08-0815:11mishawhat 1
in :in [:foo/bar 1]
is supposed to point to?#2017-08-0815:11mishahow do I actually use this :in
path? Which data structure do I apply it to?#2017-08-0815:12mbarbieri@misha ok thanks#2017-08-0815:13mishafrom
:in - the key path through a nested data val to the failing value. In this example, the top-level value is the one that is failing so this is essentially an empty path and is omitted.
#2017-08-0815:22misha(s/def :foo/bar (s/tuple int? int? fn?))
(s/explain-data
(s/map-of qualified-keyword? :foo/bar :min-count 0)
{:foo/bar [1 2 3]})
gives :in [:foo/bar 1 2]
, where 2 makes sense as index of val 3
, but 1
still does not, at least while I am thinking in terms of (get-in data in)
#2017-08-0815:28mishahttps://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L488-L494
harold#2017-08-0817:05souenzzoI'm about 5 minutes waiting this repl command
(-> (test/check `spec-utils/atributos-do-pattern)
(test/summarize-results))
There is some way to limit the number of tests that test/check
do?#2017-08-0817:41spinningtopsofdoomI think
(-> (test/check 10`spec-utils/atributos-do-pattern)
(test/summarize-results))
#2017-08-0817:41spinningtopsofdoomis what you want#2017-08-0817:56souenzzoIllegalArgumentException Don't know how to create ISeq from: java.lang.Long clojure.lang.RT.seqFrom (RT.java:542)
#2017-08-0817:57souenzzoAnother question: how to define zero args? (s/fdef foo :args (s/cat))
works, but not sure if it's right...#2017-08-0818:03spinningtopsofdoomOops my mistake I was thinking of the test.check
library
(-> (test/check spec-utils/atributos-do-pattern {:num-tests 10})
(test/summarize-results))
see this link for details (look for ::stc/opts
) https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec.test/check#2017-08-0818:06spinningtopsofdoomI think (s/fdef foo :args (s/cat))
is correct. s/cat
constructs a sequence on 0 - N specs (like it's regex namesake) so no specs given to it should mean zero arguments expected#2017-08-0818:39kennyIs there a best practice you guys are following regarding enabling of instrumentation? I am finding that many times if I don't enable or forget to enable it, my functions will silently fail and I receive an obtuse error message. Normally I would have an assertion at the beginning of the function to produce a clear error message but I figured spec instrumentation would replace this.#2017-08-0818:43souenzzosubscribe#2017-08-1012:48souenzzoBump!#2017-08-1013:44souenzzoCheckout (clojure.spec.test.alpha/instrumentable-syms)
and (stest/instrument)
#2017-08-0820:30souenzzoWhat is the current status of clojure.spec.specs.alpha
? (just curious/studding)#2017-08-0822:06mrkaspais there a library to convert clojure.spec/explain-data to something friendly to respond in a json api? I was using the struct library I want something similar#2017-08-0822:12Oliver GeorgeHas anyone written a spec function coverage tool? e.g. report the percentage of functions with a spec defined by namespace or project.#2017-08-1013:52souenzzoIt's easy to write...
Checkout clojure.spec.test.alpha/instrumentable-syms
and clojure.repl/dir-fn
#2017-08-1023:17Oliver GeorgeThank you I'll try those#2017-08-1101:50Oliver GeorgeThanks for the help. I'm writing clojurescript which isn't quite as self aware but I had success on a few fronts.#2017-08-1101:50Oliver GeorgeSimple case is checking a namespace.
(defn var->sym [var]
(let [{:keys [ns name]} (meta var)]
(symbol ns name)))
(defn interns->fn-syms [interns]
(->> (vals interns)
(filter (comp fn? deref))
(map var->sym)))
(deftest sims-logic-coverage
(let [instrumentable-syms (stest/instrumentable-syms)
instrumentable? #(contains? instrumentable-syms %)
fn-syms (interns->fn-syms (ns-interns 'sims.logic))]
(doseq [fn-sym fn-syms]
(is (instrumentable? fn-sym) "No s/fdef for function"))))
(test/run-tests)
#2017-08-1101:51Oliver GeorgeThe "total coverage" option was possible but messy. I had to use cljs.analyzer.api to know what namespace and interns existed.#2017-08-0901:15Oliver George@mrkaspa clojure.spec provides a basis for that sort of thing but leaves other tools to craft user friendly messages.#2017-08-0901:15Oliver GeorgeExpound is getting a bit of attention: https://github.com/bhb/expound#2017-08-0901:15Oliver GeorgeThere are others.#2017-08-0901:15Oliver GeorgeThe other JSON api consideration is that clojure.spec deals in clojure data structures (maps, sequences...) not javascript objects and arrays.#2017-08-0901:16Oliver GeorgeYou can do something like js->clj to translate the data into a format clojure.spec likes#2017-08-0901:17Oliver GeorgeThe error messages might be a little confusing where they say "this keyword is missing" when a JS consumer might expect "this property is missing"#2017-08-0910:21ikitommi@mrkaspa I'm returning everything (problems + extra info of the cause) to clients, specs can be serialized via s/form
. Not sure how usefull this is thou. Spec-tools has utility for adding human readable :reason
for specs (like struct does).#2017-08-0912:03mishahow can I properly delete the spec? (undo s/def, actually, undo s/fdef)#2017-08-0912:08mishause case is:
first, I get some function symbols from config, resolve them vars, and see if those are actually functions.
next, I want to enforce common function signature on those, so I want to dynamically do (s/def 'some-handler :lib.specs/handler)
.
This is all cool, but I want to clean specs registry from those dynamically set specs on some tear down event (some reload in REPL, or what not).#2017-08-0912:11misha(or at least to know I don't have to do cleaning up at all, because spec registry thing was designed to not worry about it.)#2017-08-0912:12mishaso far I found only this https://dev.clojure.org/jira/browse/CLJ-2060#2017-08-0912:16mishaactually, I control the call site, and can ensure args are ok, but does fspec actually check the arity of a function? or does it check args, and if a function (not args) does not conform to the spec ā it will blow up, but not in the spec-way, but in usual invalid arity one?#2017-08-0912:34mishayup
(s/fdef user/foo :args (s/cat :x int? :y int?) :ret nil?)
;; => user/foo
(defn foo [x])
;; => #'user/foo
(st/instrument)
;; => [user/foo]
(apply foo [1 2])
;; clojure.lang.Compiler$CompilerException: clojure.lang.ArityException:
#2017-08-0912:39mishaso the best UX I can provide, is to make "handler" fspec
available, so user could instrument handlers during development at will.#2017-08-0913:37mishaGenerating queue
, am I doing it right?
(:import
#?(:clj [clojure.lang PersistentQueue])))
(s/exercise
(s/coll-of int?
:kind #(instance? PersistentQueue %)
:into (PersistentQueue/EMPTY))
1)
;([#object[clojure.lang.PersistentQueue 0x66b3bed3 "
#2017-08-0914:01bbrinck@misha Looks OK to me. I would have written :into PersistentQueue/EMPTY
(no parens), but in practice, it doesnāt seem to make a difference#2017-08-0914:41misha@bbrinck you might be right (originally I had a (q)
cljc function call there, edited for brevity). Still need to see how it'd work in cljs.#2017-08-0914:50mishato create spec for keyword with arbitrary namespace, I need to (create-ns ...)
first, right (if there is no file for that ns in a project)?#2017-08-0915:00bbrinck@misha In CLJS you can do :kind #queue []
#2017-08-0915:01bbrinck@misha Iām not sure if this works for your use case, but specs just need to be namespaced, but that namespace doesnāt have to be a clojure namespace. e.g. (s/def :user/name string?)
is totally valid.#2017-08-0915:02bbrinckThe jury is still out on the pros/cons, but Iāve tended to build my specs based on my logical domain, not my Clojure namespaces. I never get to use the ::
syntax, but the plus side is that when I print out my specs (or get spec failures), things are a bit more succinct#2017-08-0915:03bbrinckFor the example above, namespacing further by app or company helps avoid conflicts e.g. :my-app.user/name
#2017-08-0915:52misha@bbrinck ah, I went step further, and it complained about aliased namespace, which I need to create before aliasing.
So if I want to use ::u/name
, I need to (create-ns 'user)
and (alias 'u 'user)
first.#2017-08-0917:43Alex Miller (Clojure team)thatās totally fine (and something Rich has ideas about ways to improve specifically re working with keyword namespace aliases)#2017-08-0917:45misha@alexmiller any news on unregistering specs?#2017-08-0917:45Alex Miller (Clojure team)yeah, thereās a ticket out there, should be in next spec batch#2017-08-0917:45Alex Miller (Clojure team)plan is to have (s/def ::foo nil)
do this#2017-08-0917:46Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2060#2017-08-0917:56didibusQuestion: I'm not sure I understand what fmap is for. When would I want to use it. Its one of the few functions that has no doc.#2017-08-0917:58Alex Miller (Clojure team)all of the gen namespace functions are dynamically loaded and thus donāt have doc, but they are just aliases for the equivalent function in clojure.test.check.generators#2017-08-0917:58Alex Miller (Clojure team)fmap is used to construct a generator based on applying an arbitrary function to some other generator#2017-08-0917:59Alex Miller (Clojure team)there is an example in the guide https://clojure.org/guides/spec#2017-08-0918:00Alex Miller (Clojure team)test.check.generators doc is at https://clojure.github.io/test.check/clojure.test.check.generators.html#2017-08-0918:00Alex Miller (Clojure team)and more examples at https://clojure.github.io/test.check/generator-examples.html#2017-08-0918:00Alex Miller (Clojure team)although fmap is not really covered there#2017-08-0921:41dspiteselfit appears that s/? breaks up the merging of the s/cat#2017-08-0922:21ghadi@dspiteself https://dev.clojure.org/jira/browse/CLJ-2105 & https://dev.clojure.org/jira/browse/CLJ-2003#2017-08-0922:22ghadiYou may want to add that test case if it's substantially different#2017-08-0922:23ghadi(I haven't looked closely, just aware of the bug)#2017-08-0922:46dspiteselfThanks#2017-08-0923:17didibus@alexmiller Thanks. I meant test.check doesn't have a doc for fmap. I get it now though.#2017-08-0923:26didibusQuestion: Is there a way I can specify that a spec can be one of a set of specs? Such as (s/def ::user-info #{::name ::address}). Where (s/def ::name (s/keys [::first ::last])) and (s/def ::address [::street ::city ::country]))#2017-08-0923:57didibusOkay, I think multi-spec is what I want: https://clojure.org/guides/spec#_multi_spec#2017-08-1000:37gfredericks@didibus the alpha versions of test.check have a docstring for fmap#2017-08-1000:43bfabry@didibus I don't think you want multi-spec, that's for multimethods. you just want s/or
I think#2017-08-1000:44didibusHum, what would multi-spec do that s/or wouldn't?#2017-08-1000:47bfabryrequires a defined multimethod by the looks#2017-08-1000:48didibusI guess its just that multi-spec is open for extension#2017-08-1000:49didibusSo maybe useful for libraries?#2017-08-1000:49bfabryseems to require a tag of some sort on your entity#2017-08-1000:49bfabrywhich you don't have in your original example?#2017-08-1000:50didibusOh right, hum. I guess I see. I think multi-spec is more what I'm imagining in my head, since I'm thinking of it as an inheritance of some sort. Like a shape can be a block square or cube.#2017-08-1000:51didibusBut, I guess I don't really care about the type name, what's that called, when you have types defines by there shape and not a name?#2017-08-1000:52bfabryin static typing land I believe it's called "structural typing", but I've been writing in dynamic languages for like 8 years at this point#2017-08-1000:52bfabryso not the person to ask š#2017-08-1000:53didibushaha, anyways, thanks for the alternative, I'll think about which one works best for what I'm trying to do#2017-08-1002:46dspiteself@alexmiller I submitted a patch on https://dev.clojure.org/jira/browse/CLJ-2003 . I tried to find a spec.alpha JIRA project. Am I correct to submit the patch to Clojure?#2017-08-1012:07Alex Miller (Clojure team)Yep!#2017-08-1006:50mpenetquestion of style: when you spec (deeply) nested maps inside a namespace. do you create tons of new deep ns for keys specs corresponding to the map shape ex: ::foo.bar.baz.etc/*
or do you use a more flat naming scheme (like inner classes in java) so ::foo$bar$baz/*
(spec-tools seems to do something like this for the Schema like api I think)?#2017-08-1012:09Alex Miller (Clojure team)I would definitely not do the latter and would potentially not even do the former but instead use the same namespace for many of the levels#2017-08-1012:21mpenethow would you do that when you spec a json api for instance, there often are duplicates in key names#2017-08-1012:21mpenetthing.id foo.otherthing.id that.id etc#2017-08-1012:23mishayeah, like in
(s/def :foo.bar/name one-name-pred?)
(s/def :foo.baz/name different-name-pred?)
(s/def :foo/bar (s/keys :req-un [:foo.bar/name]))
(s/def :foo/baz (s/keys :req-un [:foo.baz/name]))
#2017-08-1012:25mishaI try to keep all specs in a single file, at least in active design/development phase, so I end up with something like above (or your 1st approach, @U050SC7SV )#2017-08-1012:26mpenetyup, involves a lof of create-ns
juggling#2017-08-1012:26mpenetsame, I tend to create foo.clj foo_specs.clj pairs#2017-08-1012:27mishanot, if you are willing to use long qualified keywords, instead of aliases.#2017-08-1012:28mpenethence the create-ns stuff#2017-08-1012:28mishaI still did not calibrate my preference of aliases over long kws. Converted 300 lines of specs yesterday to aliases ā can't understand shit now dafuq#2017-08-1012:29mishae.g.
(alias 'fc 'fsm.compiled)
(alias 'fcs 'fsm.compiled.state)
(alias 'fce 'fsm.compiled.event)
(alias 'fcg 'fsm.compiled.guard)
(alias 'fcb 'fsm.compiled.behavior)
(alias 'fcm 'fsm.compiled.machine)
(alias 'fctran 'fsm.compiled.transition)
(alias 'fctrig 'fsm.compiled.trigger)
#2017-08-1012:29mpenetI don't abuse aliases (I never create them manually)#2017-08-1012:30mishanow quick, whose ids are those?
::fcb/id
and ::fcs/id
kappa#2017-08-1012:30mpenetI just create-ns under the current ns (via a macro) and add dotted childs foo.clj -> :foo/*
:foo.bar/*
foo.bar.baz/*
#2017-08-1012:33mishaI might revert all of the aliases, and settle down with just long qualified kws. Will have to spend ~same amount of time to comprehend same spec, but at least I will not need to jump around the file in process#2017-08-1012:34mpenetif you do it "locally" aliases are not really necessary you can use ::bar.baz.prout within 'foo ns for foo.bar.baz.prout etc#2017-08-1012:34mpenetit's what I have found to be the less horrible to deal with that stuff, and semantically it's kinda clean#2017-08-1012:35mishaon the other hand, those namespaces are not that long.
I'd not like to have 1kloc of specs for, say, ring.middleware, where nss are 100 chars long harold#2017-08-1012:36mishalet me try that. (sorry, for highjacking thread, and driving chances of @alexmiller reading all of it to zero opieop )#2017-08-1012:37misha@U050SC7SV
::foo/bar
Invalid token: ::foo/bar, compiling (/.../src/kek/fsm_spec.cljc:15:1)
#2017-08-1012:38mpenetyou still have to create-ns the ns yep#2017-08-1012:39mpenetno way around this (for now)#2017-08-1012:39mishasame after
(create-ns 'kek.fsm-spec.foo)
=> #object[clojure.lang.Namespace 0x80d220d "kek.fsm-spec.foo"]
#2017-08-1012:40mishabasically, I just need longer alias names#2017-08-1007:19mpenet(I usually go with the former)#2017-08-1015:11mishadoes not really feel "reuse", man harold#2017-08-1017:03misha@souenzzo interesting, but I will still require to copy/paste dispatch keys by hand#2017-08-1019:43didibusQuestion: What versioning scheme does clojure.spec uses. Specifically, what part of the version do indicates non-backward compatibility if it changes? "1.9.0-alpha17". Is it the number following alpha? Is it the minor 0 version, or the 9 or the 1?#2017-08-1019:49souenzzoinside -alphaXX
, any change can break i think#2017-08-1020:15seancorfieldAlso note that Spec is currently clojure.spec.alpha
and has a different version number to Clojure itself, as do the also separate core specs.#2017-08-1020:23didibusOh I see: 0.1.17#2017-08-1020:24didibusSo, the way they have it now, is there a strict notion of breaking change on the version? Like 0.2 would be breaking, but not 0.1.18 ?#2017-08-1020:32bfabryI would say while it's still labelled alpha any release could be breaking, as it's more just a "new build"#2017-08-1021:36seancorfieldI get the impression -- from talking to Rich and Alex and others, and from what they've said in public talks at conferences -- that "The Clojure Way" doesn't really pay much attention to version numbers... After all, "major" releases of Clojure are all 1.x so far, and nearly all the Contrib libraries are 0.x.y but are production-ready...#2017-08-1021:37seancorfieldI suspect we'll see a bigger push -- across the whole community -- for non-breaking future releases while the artifact/namespace name is unchanged.#2017-08-1021:37seancorfieldFor example, Rich's Spec-ulation talk.#2017-08-1021:38seancorfieldI've struggled with this with clojure.java.jdbc
because I've had several releases that have been breaking ones because the API took a long time to drift toward "best practices" as they've evolved over the last 5-6 years I've been maintaining it.#2017-08-1021:39didibusIts true, but all Clojure versions have historically maintained backwards compatibility. Also, while Rich Hickey doesn't seem to like versions, he does like clearly indicating when things break backward compatibility, though he suggest a whole new name instead of a version bump. At least that's what I got from his talk.#2017-08-1021:40seancorfieldAt one point, I pushed the entire API into a "deprecated" namespace, allowing for a simple cross-application edit for users, if they wanted to upgrade without changing all their calls. I think the new way means we'll get "versions" encoded in the namespace names instead, at least until we figure out a better approach.#2017-08-1021:40didibusNot really an aversion for versions, just that we could do much better I guess#2017-08-1021:41seancorfieldAt least with clojure.spec.alpha
, you know it's subject to change and can (and has!) cause(d) code breakage.#2017-08-1021:41seancorfieldOnce it's ready to become stable, it'll get renamed back to clojure.spec
(and folks can continue using clojure.spec.alpha
until they're ready to switch).#2017-08-1021:42didibusYa, I'm assuming this now. I think its the right assumption for spec when in alpha#2017-08-1021:42didibusthanks#2017-08-1021:44seancorfieldWe've just had a long discussion about this with clj-time
because I'd originally pushed to switch its implementation from Joda Time to Java Time. With discussions over about a year, it became clear that wouldn't be possible while retaining the clj-time/clj-time
coordinate due to possible conflicts in versions across multiple libraries that depend on clj-time
-- so it's sort of become the poster child for "don't break code when you only change the version".#2017-08-1021:48didibusHum, ya#2017-08-1021:50didibusWell, in fact, my reason for asking was that at my company, we internally maintain our own package repo. And our version scheme actually goes like this: Any.Number.Affects.Backwards.Compatibility. And we drop all part of a version that doesn't affect backwards compatibility.#2017-08-1021:50bfabrylol, I appreciate you not breaking clj-time sean, that would definitely give me a headache#2017-08-1021:50didibusIn that way, conflicts auto-resolve to the newest version that does not break backwards compatibility. But for that, you have to tell the build tool what numbers in the library versioning scheme is the backward breaking one.#2017-08-1021:52didibusSo, I think its actually more in line with Rick's vision. Just freely upgrade things, but don't break people. And if you need to break people, then and only then, just make a fork as a whole new name. So sure, that could be clj-time_v2, but the version is now only a way to indicate incompatibility, and nothing else.#2017-08-1021:55didibusP.S.: Aren't you stuck using joda-time until clojure drops supports for Java 1.6 and 1.7?#2017-08-1022:56seancorfieldHeh... read the discussion (that started in November 2015!) about that: https://github.com/clj-time/clj-time/issues/196 /cc @U050MP39D#2017-08-1022:57seancorfieldMy solution to this at work was to introduce clojure.java-time
and start switching code over to use that instead (of clj-time
and date-clj
which we were previously using).#2017-08-1022:58seancorfieldGiven the presence of clojure.java-time
, I'm not very inclined to create a "somewhat API-compatible version of clj-time" based on Java Time -- I'd rather support clojure.java-time
instead.#2017-08-1100:11didibusAh, didn't know about clojure.java-time. Looks good. Though what I'd like to see is a common clojure/clojurescript API for date and time.#2017-08-1100:16seancorfieldWell, there's cljs-time which mirrors clj-time, and in the clojure.java-time repo there's an issue proposing a cljs API that matches...#2017-08-1100:18didibusOh, okay ya, I guess that's good enough.#2017-08-1105:51tianshuCan I add spec to a function before I define the function? like @spec in elixir. And is it possible to have a variable T to represent some type like generic types?#2017-08-1105:56seancorfield@doglooksgood Not sure what you mean about "generic types" in the context of Clojure?#2017-08-1105:57seancorfieldas for declaring a spec for a function before its defn
, yes, that definitely possible...#2017-08-1106:09tianshumy mistake, I was thought that I have to write spec for function after that is defined.
for my first question, maybe I shouldn't call it a generic type. I want to know something that can help me writing a spec for functions like map.#2017-08-1106:17seancorfieldmap
is a pretty hard function to write a spec for -- especially since the one argument version returns a transducer š#2017-08-1106:18seancorfieldYou don't need to spec everything. Just spec at the boundaries of your subsystems and/or APIs.#2017-08-1106:20tianshuit seems impossible for writing spec for a function that receive T and returns T?#2017-08-1116:14Alex Miller (Clojure team)correct - specs are not parameterized#2017-08-1117:17didibusI think it doesn't make sense to have parameterized specs. Because they're predicate based, the parameters would be non intuitive. For each one you'd have to pass in a pred, but you wouldn't know how that pred would combine with the spec. And each spec could do it differently. That said, I'm not sure, it's a good idea. I'm trying to think if there's useful real use cases for it, nothing jumps at me right away, can you suggest one?#2017-08-1118:49Alex Miller (Clojure team)well, weāre not going to do it, so Iām not going to invent a reason why :)#2017-08-1106:20seancorfieldI think spec is much more valuable for spec'ing data structures than functions, to be honest, but that might be the domain I'm working in (where I'm using spec mostly to describe what API parameters should be, so s/conform
and s/invalid?
are being called).#2017-08-1106:20seancorfieldSo you want the type of the return to match the type of the first argument?#2017-08-1106:21tianshuyes#2017-08-1106:21tianshuso I should use :fn in fdef?#2017-08-1106:21seancorfieldYou can spec that with :fn
to check that yes.#2017-08-1106:22seancorfieldDo you really want identical types tho'? What sort of function are you talking about?#2017-08-1106:25seancorfieldFor example, map
takes a function and a collection and returns a lazy sequence so the types don't match there -- but I guess you could check non-empty collection/sequence members being the same type? But you can't do much for empty collection/sequence. Would you want to check the whole input/output? What about lazy sequences not being entirely consumed, like (take 5 (map inc (range)))
?#2017-08-1106:26seancorfieldYou need to be careful that the spec doesn't realize the whole sequence.#2017-08-1106:43misha@doglooksgood :fn
will not be checked by instrumentation. you will need you run tests explicitly.#2017-08-1106:55tianshuI got a library for this#2017-08-1106:56tianshuBut I think it will be nice to have some spec inference in editor. but without type variable, it seems to be difficult.#2017-08-1108:31misha@doglooksgood you might like this https://github.com/stathissideris/spec-provider#2017-08-1117:25didibus@doglooksgood I'd give a look at https://github.com/arohner/spectrum It gives everything you spec a type based on the spec, and then uses that to validate types at compile time. Maybe you could use it to infer things in an editor.#2017-08-1119:16arohnerIām slowly working on inference of untyped fns#2017-08-1119:29arohneralso, spectrum isnāt really usable for production use yet#2017-08-1119:59souenzzoFunctions with the signature like +
: (fn [& args])
...
How to declare theirs specs? (s/fdef my-fn :args (s/coll-of integer?))
?#2017-08-1120:02bfabry@souenzzo (s/cat (s/* integer?))
#2017-08-1120:08bfabry@souenzzo actually minus the cat, just (s/* integer?)#2017-08-1120:17bfabryboot.user=> (defn +'' [& args] (apply + args))
#'boot.user/+''
boot.user=> (s/fdef +'' :args (s/* integer?))
boot.user/+''
boot.user=> (st/instrument)
[boot.user/+'']
boot.user=> (+'' 1 2 3)
6
boot.user=> (+'' 1 2 3 'a)
clojure.lang.ExceptionInfo: Call to #'boot.user/+'' did not conform to spec:
In: [3] val: a fails at: [:args] predicate: integer?
#2017-08-1205:28mattlyIām trying to migrate a project from pre clojure.spec.alpha
to post-that, and I get this error:#2017-08-1205:29mattlynowhere in the project is clojure.spec.gen.alpha
referenced directly#2017-08-1205:29mattlyIām on clojure 1.9-alpha17#2017-08-1205:29mattlyany ideas?#2017-08-1205:59mattlyhm, apparently I had a clojure version mismatch somewhere#2017-08-1205:59mattlyits working now I think#2017-08-1208:09danielstocktonIs it possible to remove all conforming on a spec so that it just returns identity
(but still does the validation)?#2017-08-1208:53mpenetWith a conformer that calls unform? (yuk)#2017-08-1208:55mpenetSounds like the wrong solution to the pb. Maybe you need to decompose upstream spec(s) to build this kind of things#2017-08-1209:28danielstocktonYeah, I abandoned that approach. I'm trying to submit a patch for https://dev.clojure.org/jira/browse/CLJ-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel#2017-08-1209:29danielstocktonCame to what I think is a better solution.#2017-08-1211:14misha@danielstockton clojure.spec.alpha/nonconforming
(defn ^:skip-wiki nonconforming
"takes a spec and returns a spec that has the same properties except
'conform' returns the original (not the conformed) value. Note, will specize regex ops."
[spec]
#2017-08-1221:38Alex Miller (Clojure team)As a note here, this was intentionally left out of the docs (the āskip-wikiā meta) because Rich is still mulling whether to include it. Some time has passed since we last talked about it, but at this point I think we should either keep it, or we should have a nonconforming version of s/or (which is the really common case where you want it).#2017-08-1211:18danielstockton@misha thanks!#2017-08-1216:43mishahttps://www.infoq.com/presentations/clojure-spec toot#2017-08-1221:14Alex Miller (Clojure team)hey, itās me! :)#2017-08-1219:18mishaTIL, regular-expression specs work like splicing, so nested reg-ex specs expect flat data structure dafuq#2017-08-1219:19mishaI read guide like 7 times already, and missed it complitely#2017-08-1221:15Alex Miller (Clojure team)tbh, it took me like flubbing this like 5 times before I really knew it#2017-08-1221:16Alex Miller (Clojure team)and I even wrote 2 implementations of it before the current one!#2017-08-1221:28mishathe spec is still alpha
thing makes more sense now troll#2017-08-1309:02mishais there something out of the box for getting conforming subset of a data structure? e.g.
(s/def ::foo (s/keys :req-un [::a ::b]))
(s/valid? ::foo {:a 1 :b 2 :c 3}) ;; => true
(conformed-part ::foo {:a 1 :b 2 :c 3}) ;; => {:a 1 :b 2} (only keys from ::foo spec are kept)
#2017-08-1316:30Alex Miller (Clojure team)in this case, the data structure does conform to the spec#2017-08-1316:31Alex Miller (Clojure team)because s/keys supports open maps#2017-08-1316:31Alex Miller (Clojure team)so, essentially no. there is something here Rich is considering adding but not sure if and when that will happen#2017-08-1406:31mishathe use case is to re-use ui-approved-spec in a select-keys to not leak any sensitive or just extra information about entity, exactly because of s/keys
's support of open maps#2017-08-1408:09ikitommi@misha try spec-tools:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])
(s/def ::a int?)
(s/def ::b int?)
(s/def ::foo (st/spec (s/keys :req-un [::a ::b])))
(s/valid? ::foo {:a 1 :b 2 :c 3}) ;; => true
(st/select-spec ::foo {:a 1 :b 2 :c 3}) ;; => {:b 2, :a 1}
#2017-08-1408:47misha@ikitommi will check it out, thanks#2017-08-1412:15hmaurerHi! I have a quick question. I have a spec for a map, which defines a set of required and a set of optional keys (`(s/keys :req [...] :opt [...])`). I would like to generate a version of that spec suitable for āpartial updatesā. By that I mean that the spec should pass if a required key is missing, but it should not pass if the required key has value nil
.#2017-08-1412:16hmaurerIs that possible?#2017-08-1412:17hmaurerFor a real-world use-case, think of a CRUD scenario. When creating an entity I need to validate the presence of some attributes, but when updating I only need to make sure that an attribute marked as required isnāt set to nil
#2017-08-1412:38Alex Miller (Clojure team)The vals for the required keys (actually all keys) in s/keys will be verified according to the spec for that key. If the spec does not allow nil, it is not valid.#2017-08-1416:17hmaurerOk, thanks!#2017-08-1412:40Alex Miller (Clojure team)user=> (s/def ::i int?)
:user/i
user=> (s/def ::m (s/keys :opt [::i]))
:user/m
user=> (s/valid? ::m {::i nil})
false
user=> (s/valid? ::m {})
true
#2017-08-1412:41Alex Miller (Clojure team)If the key is required, then its absence will cause validation to fail as required keys are ā¦ required. :)#2017-08-1416:17hmaurerHi againā¦ I am getting this error whenever I run the ādocā command (and others) in the REPL: java.lang.NoClassDefFoundError: clojure/spec/alpha$get_spec
. Any ideas?#2017-08-1416:18bronsathere's already a ticket + patch on jira#2017-08-1416:37hmaurerah, thanks @bronsa#2017-08-1501:55bbrinckFor anyone that uses Orchestra to instrument :fn
and :ret
specs, Expound 0.2.0 now supports pretty error messages for Orchestra errors. https://github.com/bhb/expound#using-orchestra#2017-08-1510:36hmaurerHi. I am having an issue with clojure.spec
. This code throws an error: https://gist.github.com/hmaurer/d030f9ff9250482c6dea3e3738737863#2017-08-1510:37hmaurer(Iāve added the error as a comment on the gist)#2017-08-1510:37hmaurerI think this is because s/keys
is a macro, but I am not sure how to get it to do what I want#2017-08-1510:37hmaurer(dynamically build that spec from the map)#2017-08-1510:59moxaj@hmaurer left a comment on the gist#2017-08-1511:01hmaurer@moxaj thank you! So I assume I cannot / should not define specs at runtime? e.g. I canāt define specs based on my database schema after opening a connection?#2017-08-1511:01moxajeval
is another option#2017-08-1511:02hmaurerah, Iāll try that. Is it an outright terrible idea to define specs at runtime though?#2017-08-1513:31Alex Miller (Clojure team)no#2017-08-1511:10urbankDoes anyone here define specs in the same file as the functions? Above the functions like haskell type annotations#2017-08-1511:16souenzzoI just started to write my specs like that.
Not sure if is the best practice or not.#2017-08-1511:17gfredericksI usually do#2017-08-1511:31urbankThe second answer here gives a good arguments#2017-08-1511:31urbankhttps://stackoverflow.com/questions/37942495/where-to-put-specs-for-clojure-spec#2017-08-1513:02mishaI lean towards: data specs in a separate file(s), function specs ā next to actual implementation. (Yet to see how it'll turn out in longer run)#2017-08-1513:08urbank@U051HUZLD Right, that makes sense to. Because functions might share data specs, right?#2017-08-1513:13misha@U489U338R yes, and, functions are more "implementation detail"-y than data, I think. But I'd start with everything in one file, and see what's up later in project.#2017-08-1518:58didibusYa, data spec in their own namespace gives you a clean domain model. And fn specs I put below the functions.#2017-08-1519:13seancorfieldMost of our specs are for data structures and they go in whatever is the appropriate namespace for those data structures -- which is often a separate ns but sometimes is a ns with some "utility" functions specific to the data structure. Where we spec functions we generally put the spec right above the function (like a type annotation).#2017-08-1519:13seancorfieldBut, as noted, if you want your code -- a library -- to be usable by Clojure < 1.9 then you have to put specs in a separate and completely optional namespace.#2017-08-1519:14seancorfieldSee, for example, how clojure.java.jdbc
handles this (specs in a separate namespace on their own -- even for functions).#2017-08-1512:24stathissiderishttps://gist.github.com/stathissideris/8d892c78c76b6d615ced53004ea32840#2017-08-1512:24stathissiderisMake clojure.spec behave more like schema in terms of keys strictness (no unknown keys allowed)#2017-08-1512:31stathissiderisdisclaimer: untested, most likely buggy#2017-08-1512:36gfredericksschpec has excl-keys
: https://github.com/gfredericks/schpec#things-it-has#2017-08-1514:03stathissiderissure, but I wanted to be able to be strict conditionally#2017-08-1514:47gfredericksoh sorry, I didn't look closely#2017-08-1512:41hmaurerRe-iterating my earlier question, does anyone have a big objection to defining specs at runtime (with eval
), based, .e.g, on the appās database schema? I am a beginner so I am not sure if it is āgood taste/practiceā in the clojure world#2017-08-1513:27mishait possible, but keep in mind, you cannot yet retract specs from registry as of now @hmaurer#2017-08-1513:28mishaso if your specs will change during application run time - make sure you migrate those appropriately#2017-08-1513:32Alex Miller (Clojure team)well, you can if you reach in and munge the registry atom :) but thatās coming soon.#2017-08-1513:32mishaI meant in official way opieop#2017-08-1513:33Alex Miller (Clojure team)@hmaurer I think dynamic generation of specs is fine#2017-08-1516:29scaturrI am having a hard time understanding s/keys
. I keep my specs organized in separate namespaces - i.e something like entity.clj
and entity/spec.clj
. I have the following used to spec a generic entity map#2017-08-1516:30scaturrI define ::id
to make importing it elsewhere easier, and then use it in ::entity/id
because its my understanding that is how the :req
field in ::entity
is able to conform that specific key?#2017-08-1516:30scaturris that correct? am I doing this in a weird way?#2017-08-1516:33scaturrmainly so some other āentity specā can merge in ::entity
#2017-08-1516:33scaturrwhich implies todos would need an ::entity/id
key#2017-08-1519:01didibusI'm not sure I follow your logic. It seems to me your overcomplicating it.#2017-08-1519:15didibusIf conceptually the id of entity is supposed to be the same as the id of todo, the you s/def :some-ns/id. And in your spec for entity you say keys req [:some-ns/id]. And do tje same in your todo spec.#2017-08-1519:39didibusBasically s/keys describes an associative structure which when used with :req says that the structure must have as keys the given list of namespaced keywords. If the keyword has a spec defined, it will be used to validate and conform the value of the key. #2017-08-1615:31scaturrThe todo example does use the :some-ns/id approach#2017-08-1615:32scaturrBut if I were to define (s/def ::entity/id ...)
how can I then import that elsewhere?#2017-08-1615:32scaturrThis might just be a misunderstanding of clojure O_o#2017-08-1615:32scaturr(:require [todos.entity.spec :refer [?])
#2017-08-1615:33scaturror even if you use :as
- since its already namespaced in the spec - is there a way to get at it?#2017-08-1615:33scaturrthank you for the reply by the way š#2017-08-1616:43didibusYou don't need to refer anything, just require it.
(:require [todos.entity.spec])
Specs are always defined globally. All that needs to happen is your app must eval the s/def form before you use it. Require will do that. #2017-08-1616:44didibusAlso, ::entity/id doesn't make sense. It would be :entity/id#2017-08-1616:47didibusSo specs are namespaced. They're globally accessible from anywhere after they've been deffed, but they need to be prefixed with a namespace. This helps reduce name clash.#2017-08-1616:49didibusSo if you are in namespace todos.entity.spec and you s/def a spec, and you use the double colon ::, then you created :todos.entity.spec/your-spec#2017-08-1616:51didibusNow if you are in say todos.app namespace, you can require todos.entity.spec and refer to the spec as :todos.entity.spec/your-spec#2017-08-1616:54didibusIf what you want it :entity/id instead of :your-ns/is you have two options#2017-08-1616:56didibusEither define the spec like that, so don't use the double colon, you can (s/def :entity/id int?)
for example#2017-08-1616:58didibusThis will be make it less portable though, since the chance of name clash are higher.#2017-08-1617:01didibusOr you can define it prefixed with the full namespace using ::, and then when you use it elsewhere, you can alias the namespace as entity.#2017-08-1617:15didibus`(ns ns1)
(s/def ::id int?)`
Then in ns2
`(ns ns2
(:require [ns1 :as entity])
(s/def ::user :entity/id)`
#2017-08-1617:18didibusAs far as I know, you can't refer a spec yet. But you can kinda manually do it if you want#2017-08-1617:18didibus(s/def ::id :some-other-ns/id)#2017-08-1518:06csmSo I have a cat
spec that contains a UUID for one field, and another field is a coll-of maps, and each map has a :guid->UUID; the requirement is that one of the maps contain the UUID. I can express this in a spec, but Iām not sure how to make a generator for such a scenario#2017-08-2506:23seancorfieldBut for basic stuff you can just use inst?
#2017-08-2506:24thedavidmeisteroh yeah, i'm slowly working through moving between js dates, goog dates, joda dates and iso8601 strings#2017-08-2506:24thedavidmeisterjust wanted something basic for now though š#2017-08-2506:24thedavidmeisterah inst?
looks great#2017-08-2506:25thedavidmeisteralthough hmmm#2017-08-2506:25thedavidmeisterit seems to strongly prefer 1970-01-01#2017-08-2506:26thedavidmeistereven if i do (first (rand-nth (spec/exercise inst? 1000)))
it rarely moves away from 1970#2017-08-2506:27seancorfieldYeah, the inst?
generator is pretty conservative.#2017-08-2506:27thedavidmeisterhmmm#2017-08-2506:27thedavidmeisterknow of anything like that regex lib?#2017-08-2506:28thedavidmeisteri feel like even using a random int as a timestamp would be better š#2017-08-2506:29gklijsmaybe with dates you want one based on the current date and add/substract some random year/month/seconds etc?#2017-08-2506:30seancorfieldHence our age range spec / generator š#2017-08-2506:31thedavidmeisteryeah#2017-08-2506:31thedavidmeisteri think i'm going to go for a jog and think about how i can adapt to my stuff š#2017-08-2506:33seancorfieldOne thing I've found important with clojure.spec
is to be careful not to over-specify things and to override generators to produce more "sane" data (for your problem domain) -- generating just a small subset of possible conforming values sometimes.#2017-08-2506:34thedavidmeisteri think that will be ok for me atm#2017-08-2506:34thedavidmeisteras i'm replacing hard-coded values in tests anyway#2017-08-2506:34thedavidmeisterit will be an incremental process#2017-08-2506:34thedavidmeisteras i gradually introduce more complexity of what is specced#2017-08-2509:19thegeez@thedavidmeister spec generator have the notion of a size parameter (100 here). However, this is not a minimum length: (clojure.test.check.generators/generate (clojure.spec.alpha/gen string?) 100)
and clojure.test.check.generators/resize
#2017-08-2509:21thedavidmeister@thegeez wow that is great#2017-08-2509:21thedavidmeisterthanks!#2017-08-2515:47martinklepschhow do you express that all keys of a map should be keywords? I want to spec a structure that looks like this:
{vec {keyword {string {:some-known-key ::x
:another-known-key ::y}}}}
#2017-08-2515:49seancorfield@martinklepsch map-of
?#2017-08-2515:50seancorfield(map-of keyword? ::value-spec)
#2017-08-2515:51martinklepsch@seancorfield thanks, Iāll look into map-of
#2017-08-2518:19ikitommi@martinklepsch map-of
is the right answer, but your pseudo looks much like the thing that the data-specs are eating š (and generating map-of
s & friends)
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.data-spec :as ds])
(s/def ::x string?)
(s/def ::y int?)
(let [martin {vector? {keyword? {string? {:some-known-key ::x
:another-known-key ::y}}}}
value {[1 2 3] {:a {"b" {:some-known-key "nice"
:another-known-key 18}}}}
spec (ds/spec ::martin martin)]
(s/valid? spec value))
;; true
#2017-08-2518:24martinklepsch@ikitommi ah that indeed seems like a nice utility!#2017-08-2519:46eraserhdSo I just discovered that (s/def :foo/bar :baz/quux)
doesn't work if :baz/quux
hasn't already been defined. Is this new?#2017-08-2519:46eraserhdIs it just s/keys
that defers lookup?#2017-08-2519:48eraserhdAnd if so, does that mean that I can't redefine :baz/quux
later?#2017-08-2519:52Alex Miller (Clojure team)almost everything defers lookup. this is a known (and unresolved) issue.#2017-08-2519:53eraserhdIf I wrap it with s/spec
, will it defer?#2017-08-2519:54Alex Miller (Clojure team)maybe? try it#2017-08-2519:55eraserhdapparently... more testing in half an hour or so#2017-08-2622:46souenzzothere is some wayt to (s/def :req-un [:foo/bar :as :new-name])
?#2017-08-2623:55Alex Miller (Clojure team)no#2017-08-2717:00hmaurerHi! Is there a way to get only the keys specified in a spec (req and opt) and nothing else after validating a map? And to do so with nested maps too? #2017-08-2717:00hmaurerI expected conform to do this but it doesn't appear to #2017-08-2721:16hmaurerthe spec-tools package seems to offer a solution to my problem#2017-08-2718:36ikitommiHi. The s/conformer
seems to take the optional unf
function as a second argument for unforming. It defaults to nil
currently. Would an identity
be a better default?#2017-08-2718:38ikitommie.g. this would not fail by default:#2017-08-2800:35Alex Miller (Clojure team)yes, it would still fail by giving you the wrong result#2017-08-2800:36Alex Miller (Clojure team)given that conformers are used to transform the input, using identity is virtually always as wrong as nil. much better to fail and tell you how to fix imo#2017-08-2914:01ikitommithanks.#2017-08-2718:38ikitommi(require '[clojure.spec.alpha :as s])
(s/def ::spec (s/conformer str))
(s/conform ::spec 1)
; => "1"
(s/unform ::spec (s/conform ::spec 1))
; CompilerException java.lang.IllegalStateException: no unform fn for conformer
#2017-08-2721:45hmaurerAnybody familiar with spec-tools here?#2017-08-2807:21ikitommi@U5ZAJ15P0 I am, one of developers of the lib.#2017-08-2819:28hmaurer@U055NJ5CC Hi! I wanted to ask if there is a utility function to walk a (deeply nested) map together with its associated spec#2017-08-2819:28hmaurerI just realised itās precisely what has been discussed here https://github.com/metosin/spec-tools/issues/65#2017-08-2819:29hmaurerdoes such a function exist now?#2017-08-2905:01ikitommi@U5ZAJ15P0 Not yet. Just commented on the issue. If you have ideas, please share them on the issue. Contributions welcome too#2017-08-2803:21danielcomptonIs there a way to spec a nested map that dispatches on type? e.g.
(s/def :app/command (s/keys :req [:command/id
:command/params
:request/id
:idempotency/id]))
(defmulti command-params :command/id)
(s/def :command/params (s/multi-spec command-params :command/id))
(defmethod command-params :test/command [_]
(s/keys :req [:material/id]))
;; This is the kind of map shape I want
{:command/id :test/command
:request/id (random-uuid)
:idempotency/id (random-uuid)
:command/params {:test/id (random-uuid)}}
#2017-08-2803:22danielcomptonI thought I could do this with a multi-spec, but I'm not sure how to define a multi-spec where the data is at one level, and the tag is at the parent level#2017-08-2807:28ikitommiIdea: s/conform
could be changed into s/walk
that could be parametrised to do either :conform
(as now) or :coerce
(like conform, but would not return the branch-information, e.g. same results as with conform + unform).#2017-08-2807:29ikitommiI think this would yield much better perf, which matters at runtime.#2017-08-2813:38scaturrIs there any sort of consensus on where to put specs? Separate ns? Same ns as things being specced?#2017-08-2815:41souenzzo+subscribe+#2017-08-2815:59seancorfieldIt Depends(tm).#2017-08-2816:00seancorfieldData specs can usually be in their own ns, possibly with a few predicate functions or closely related helper functions.#2017-08-2816:01seancorfieldFunction specs probably should be in the same ns as the functions they are spec'ing -- unless you need to support Clojure 1.8 (or earlier) as well as Clojure 1.9, such as for a library, in which case they must go in a separate, optional ns, that only 1.9 users will load.#2017-08-2816:38scaturrthat is super helpful. thank you!#2017-08-2813:39scaturrTooling makes me feel like the same ns might be more convenient - not necessarily better?#2017-08-2813:40scaturrfor instance ciderās doc command doesnāt work unless you explicitly have the spec required#2017-08-2813:40scaturrĀÆ\(ć)/ĀÆ#2017-08-2817:49hmaureris there a way to āoverrideā an already defined spec?#2017-08-2817:49hmaurer(with s/def
)#2017-08-2817:50hmaureritās a no-op for me if the spec is already defined#2017-08-2818:37seancorfieldRemember that s/def
will only update the spec -- not all the other specs that have been defined in terms of that spec.#2017-08-2900:48Alex Miller (Clojure team)yeah, this should work (once you re-def the other specs)#2017-08-2817:58hmaurerFound my answer: hacky (reset! @#'spec/registry-ref {})
#2017-08-2818:00hmaureractually that seems to break thingsā¦#2017-08-2820:17joshjonesiām not shocked#2017-08-2900:48Alex Miller (Clojure team)there is a ticket on the way such that (s/def ::foo nil)
will be sufficient to remove a spec#2017-08-2910:53hmaurer@alexmiller thank you!#2017-08-2910:55hmaurer@ikitommi I just started working with spec-tools but I am finding it a bit hard to write functions dispatching on (s/form spec)
as most are wrapped in spec-tools.core/spec
. Am I doing something wrong?#2017-08-2910:57hmaurer@ikitommi ah, I just looked at https://github.com/metosin/spec-tools/blob/master/src/spec_tools/visitor.cljc and it seems you are handling that case by extracting the inner spec#2017-08-2911:24ikitommi@U5ZAJ15P0 wrapping into spec records is unfortunate, there is https://dev.clojure.org/jira/browse/CLJ-2116 to enable runtime coercion natively. But, yes the original spec is found in :spec
#2017-08-2912:19hmaurer@ikitommi thank you! Iāll take a look. Overall great work on spec-tools by the way; it appears to solve a lot of the issues I had with clojure.spec š#2017-08-2912:57hmaurerOut of curiosity, what is the consensus regarding issues described here @ikitommi @alexmiller ? https://dev.clojure.org/jira/browse/CLJ-2116. More specifically @alexmiller , you mention
> While I think some interesting problems are described in the post, I donāt agree with most of the approaches being taken there.
Do you have an alternative approach to suggest? I am a beginner with clojure/clojure.spec but I did find that spec-tools answered some real needs of mine (e.g. strip extra keys when conforming a map)#2017-08-2913:42Alex Miller (Clojure team)if you want to transform data, then transform data. why does that need to be in spec?#2017-08-2913:42Alex Miller (Clojure team)if the selective conforming only happens at the end, why does it need to be in spec?#2017-08-2913:46Alex Miller (Clojure team)the purpose of conforming is not transformation or coercion. itās to tell you how a value conforms to a spec,#2017-08-2913:58hmaurerAh I see. So these sort of tools should be built on top of spec, not as a part of it#2017-08-2913:59hmaurerHow would you approach annotating a spec with additional data that can be used later on by utility functions?#2017-08-2913:59hmaurer(what spec-tools does by wrapping specs in a Spec record)#2017-08-2914:53Alex Miller (Clojure team)I would like to have arbitrary meta on specs and we have some tickets around that#2017-08-2914:54Alex Miller (Clojure team)but barring that, specs are named with a fully-qualified name - anyone can make a registry with those keys to store arbitrary stuff#2017-08-2914:59hmaurer@alexmiller good point; I guess those things will come more naturally to me as I get familiar with specs š#2017-08-2922:08tap@alexmiller Then this is a wrong use case example for conforming? Transforming some numbers to fizz, buzz, fizzbuzz. https://gist.github.com/stuarthalloway/01a2b7233b1285a8b43dfc206ba0036e#2017-08-2922:33Alex Miller (Clojure team)Fizzbuzz is not a use case#2017-08-3001:21tapDo you mean itās a fun little problem which we can use whatever we want, right?
Itās good to know for me because I misunderstood the purpose of conformer
by this. Already did a csv validating and parsing using conform
and additional conformer
s#2017-08-2916:40seancorfieldA use case we've found very valuable is using spec for conforming REST API parameters and form parameters -- which all start off as strings but we need numbers or Boolean etc. For those specs, we coerce string input to conformed values. Then we have a single step to s/conform
the input map to a validated parameters map.#2017-08-2916:41seancorfieldSo at the core, we'll have things like (try (Long/parseLong v) (catch Exception _ ::s/invalid))
as a named spec (predicate).#2017-08-2916:43seancorfieldI would agree with @alexmiller that coercion/transformation beyond that level is probably "doing it wrong".#2017-08-2916:46ikitommi@seancorfield So you allow clients to send numbers as strings over JSON too?#2017-08-2916:47ikitommi(and over EDN or Transit too)#2017-08-2916:54seancorfieldWe accepted form-encoded POST bodies so it isn't always JSON.#2017-08-2916:54hmaurer@seancorfield well, a simple case which brought me to try spec-tools was having the ability to strip all keys from a map that were not part of the spec#2017-08-2916:55hmaurerI assume that, were I to do this myself, I would have to write a function to traverse my spec together with the data and strip keys appropriately?#2017-08-2916:57seancorfieldWhy do you need to strip other keys? The only case I can think of is if you send the full hash map to a 3rd party that would object to the extra keys -- and it's safer to explicitly build the map you need at those points.#2017-08-2916:58seancorfieldIf your code never uses :x
in a map, why do you care that you're passed :x
in the first place, is my question.#2017-08-2916:58hmaurerprecisely that; I am transacting the data to Datomic#2017-08-2916:58seancorfieldBut you'd be building a namespace-qualified map specific to your schema at that point -- I wouldn't imagine the namespace-qualifier would be the same as your JSON input?#2017-08-2916:59hmaurerno, but I have a step which namespace-qualifies my JSON input#2017-08-2917:01ikitommiI think it's a good default to strip extra keys when reading external input. Like rest/http parameters from client#2017-08-2918:17souenzzoBit off, but I have a experiment that I use for example /user/%5B%3Auser%2Fuuid%20%23uuid%22123%22%5D
(encoded form of /user/[:user/uuid #uuid"123"]
) to avoid transformation problems (simple edn/read-string transform types).
Not sure if it's a good solution/it's sacale, but it's working well on my tests.#2017-08-3102:11tjscollinsI think I'm slowly losing my mind trying to get my function specs to work right.
(s/fdef example-fn
:args string?
:ret string?)
(defn example-fn [s] s)
This does not work if I try stest/check example-fn
. Why? I'm clearly missing something. I've been re-reading the guide for two days, but something's not clicking, and I can't see where why s/fdef's are going wrong.#2017-08-3102:20seancorfield@tjscollins :args
is always a sequence of argument specs so you want (s/cat :s string?)
there.#2017-08-3102:25tjscollinsHow does s/cat work? Do the keys have to match the variable names, or are they just used as informational labels in the output of s/explain?#2017-08-3102:28seancorfieldHave a read of this https://clojure.org/guides/spec#_spec_ing_functions -- it has some examples that should help.#2017-08-3102:29tjscollinsYeah, I've read through that repeatedly. It's just not making sense to me how s/cat and s/alt work.#2017-08-3102:29seancorfieldThe keywords in s/cat
don't have to match the argument names but it's common convention that they do (otherwise your error messages will be less than helpful).#2017-08-3102:30tjscollinsSo what is actually happening when the args are checked against s/cat? Do the args pass if they match any spec that's "s/cat"ed together? Match every spec?#2017-08-3102:31seancorfields/cat
is a "sequence regex" so it matches the sequence of arguments passed into the function#2017-08-3102:31tjscollinsThe guide calls it a regex op, but I don't know what that means in this context#2017-08-3102:31tjscollinsSo the first pair passed to s/cat has to match the first arg?#2017-08-3102:31tjscollinsAnd the second pair matches the second arg?#2017-08-3102:32seancorfieldSort of... consider (s/fdef clojure.core/declare
:args (s/cat :names (s/* simple-symbol?))
:ret any?)
#2017-08-3102:32seancorfieldThat matches zero or more simple symbols as an argument sequence and labels that sequence as :names
#2017-08-3102:33seancorfieldSo in (declare foo bar quux)
, (s/* simple-symbol?)
will match the three symbols and :names
will be (foo bar quux)
#2017-08-3102:33seancorfieldIt's quite literally "regex" for sequences (instead of strings).#2017-08-3102:34tjscollinsSo something like (s/cat :s string? :i int?)
would match both ("1" 1)
and (1 "1")
?#2017-08-3102:34seancorfieldNo, it's sequential.#2017-08-3102:34seancorfieldIt would only match "1" 1
#2017-08-3102:35seancorfieldHow familiar are you with regex for strings?#2017-08-3102:35tjscollinsAlright, would (s/cat :s (s/* string?) :i int?)
match both ("1" 1)
and ("1" "1" 1)
?#2017-08-3102:35seancorfieldYes, I believe it should#2017-08-3102:35tjscollinsYeah, I'm familiar with string regexes#2017-08-3102:36tjscollinsOkay, that makes more sense now#2017-08-3102:37seancorfieldYou can try it out directly in the REPL, without worrying about arguments: (s/conform (s/cat :s (s/* string?) :i int?) ["1" "1" 1])
#2017-08-3102:37seancorfieldand (s/conform (s/cat :s (s/* string?) :i int?) [1])
=> {:i 1}
because there were no strings at the beginning#2017-08-3102:39tjscollinsAlright that makes a lot more sense now. Is s/alt basically s/or for a sequence?#2017-08-3102:43seancorfieldYup.#2017-08-3102:44tjscollinsThat helps a lot. Thanks.#2017-08-3102:44seancorfieldAlso from that guide: (s/cat :prop string? :val (s/alt :s string? :b boolean?))
#2017-08-3102:47tjscollinsFirst element has to be a string, second can be boolean or string, right?#2017-08-3113:48favilaI'm trying to spec a sequence where there is a fixed predicate/spec based on position, but the sequence may omit things at the end; and I'd like it to conform to a flat map, one key per position, nil or no-key for the omitted items#2017-08-3113:50favila(s/conform ??? [1 ""])
=> {:pos1 1 :pos2 ""}
#2017-08-3113:50favila(s/conform ??? [1 "" :kw])
=> {:pos1 1 :pos2 "" :pos3 :kw}
#2017-08-3113:51favila(s/conform ??? [1 :kw])
=> ::s/invalid
#2017-08-3113:51favilaI can't use cat plus nested s/?
because that backtracks (last example would be accepted)#2017-08-3113:53favilathe best I can think of is a linked-list type structure of nested s/or
that a conformer flattens#2017-08-3113:53favilaany other ideas?#2017-08-3113:58Alex Miller (Clojure team)s/or of s/tuple ?#2017-08-3113:59favilas/tuple won't work because variable-width#2017-08-3113:59Alex Miller (Clojure team)Iām saying you could s/or multiple s/tuple together#2017-08-3113:59Alex Miller (Clojure team)itās unclear how much variability there is here#2017-08-3114:00Alex Miller (Clojure team)I guess maybe s/alt of s/cat would be better for your conformed value#2017-08-3114:00Alex Miller (Clojure team)I think Iād say: donāt fight the conformed value. write the right spec, then transform the conformed value to what you want.#2017-08-3114:01Alex Miller (Clojure team)always come back to: how can I state the truth about the data?#2017-08-3114:01favilathat's the part that is difficult#2017-08-3114:02favilaI essentially want s/tuple, with keyed names for each slot (not numbers), and possible truncation on the right side for optional values#2017-08-3114:03favilafor background, this is parsing an edi segment, if you are familiar with those#2017-08-3114:04Alex Miller (Clojure team)nah#2017-08-3114:04Alex Miller (Clojure team)sounds like you really want s/cat with nested s/?#2017-08-3114:05Alex Miller (Clojure team)(s/cat :1 int? (s/? (s/cat :2 string? (s/? (s/cat :3 keyword?)))))
etc#2017-08-3114:05favilawait that's legal?#2017-08-3114:05Alex Miller (Clojure team)sure#2017-08-3114:05Alex Miller (Clojure team)the downside of that is the result will be nested maps#2017-08-3114:05favilai thought it had to be key+spec pairs#2017-08-3114:06Alex Miller (Clojure team)oh, I missed the keys in there (always do that)#2017-08-3114:06Alex Miller (Clojure team)(s/cat :1 int? :more (s/? (s/cat :2 string? :more (s/? (s/cat :3 keyword?)))))
#2017-08-3114:07favila(s/conform
(s/cat
:a number?
:tail/next (s/? (s/cat
:b string?
:tail/next (s/? (s/cat :c keyword?)))))
[1 "" :k])
#2017-08-3114:07favilawas the best I could come up with#2017-08-3114:07Alex Miller (Clojure team)yeah#2017-08-3114:07favilaand I was thinking maybe an s/& could flatten the keys out#2017-08-3114:07favila?#2017-08-3114:08Alex Miller (Clojure team)I would separate that from the spec, but yes should be able to regularize both the creation of these (with a macro) and the transformation to something you want#2017-08-3114:32bbrinck@alexmiller When using a predicate as a spec (which can occur in, say, s/assert
), the value of :clojure.spec.alpha/spec
is a function. However, if a proper spec is defined, the :pred
is a symbol. https://gist.github.com/bhb/de31b73133d6322158a3c0a5d47d67a2#2017-08-3114:32bbrinckDo you happen to know why there is a difference?#2017-08-3114:32bbrinck(My goal is to provide a consistent printer that can say ā<value> did not match <predicate>ā in all cases)#2017-08-3114:33bbrinckI did not see a JIRA ticket, but would be happy to create one if it is helpful š Iām not sure if there is a technical limitation here.#2017-08-3114:35bbrinckI suspect this is related to the fact that s/def
is macro whereas explain-data
is a function, so it only gets the function value#2017-08-3114:39bbrincki.e. the symbol is already resolved by the time explain
and explain-data
see it. Hm. I wonder if the right solution is to spend time making a pretty-printer for functions that works reliably across CLJ and CLJS#2017-08-3114:44Alex Miller (Clojure team)there is a ticket and a patch for that#2017-08-3114:44Alex Miller (Clojure team)but your reasoning is correct#2017-08-3114:48bbrinckMy ticket searching skills need work š#2017-08-3114:48Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2068 is what Iām thinking of#2017-08-3114:48Alex Miller (Clojure team)but maybe itās not covering what youāre asking#2017-08-3114:49Alex Miller (Clojure team)you should not have any expectation about the concrete type of :clojure.spec.alpha/spec, other than that it can be used as a spec#2017-08-3114:50bbrinckNo, youāre correct, this is what I want: I am not concerned about the concrete type, I just want a way to consistently pretty print the problem.#2017-08-3114:51Alex Miller (Clojure team)that ticket fixes the :pred in the explain-data#2017-08-3114:51Alex Miller (Clojure team)by unmunging the function name#2017-08-3114:51bbrinckAlthough in my case, Iād probably print the set differently (instead of a set literal, Iād enumerate options), but that should not be a problem#2017-08-3114:51Alex Miller (Clojure team)Iām hoping it will be in the next spec release#2017-08-3114:52bbrinckDoes it apply the same unmunge to :clojure.spec.alpha/spec
at the top level of the explain-data?#2017-08-3114:52bbrinckIOW, would it work for (s/explain odd? 10)
?#2017-08-3114:52Alex Miller (Clojure team)user=> (s/explain odd? 10)
val: 10 fails predicate: odd?
#2017-08-3114:53Alex Miller (Clojure team)(running with the patch)#2017-08-3114:53bbrinckMuch appreciated! I will definitely be using this in expound#2017-08-3114:57Alex Miller (Clojure team)I have been too busy to look at expound, but hope to soon#2017-08-3115:01bbrinckDonāt look too closely: the internals are a bit of a mess, but I hope to refactor soonish š . Most importantly, I want to cleanly separate the pretty printing part (hopefully useful to end users, but not really reusable) from the stuff I did related to adding information to the āinā paths, which could be useful for any library that wants to consume explain-data
#2017-08-3115:06Alex Miller (Clojure team)seems wise#2017-08-3116:14didibusCan I trust describe to be consistent on specs of the same kind. Like will all s/keys describe itself in a consistent way?#2017-08-3116:15didibus I'm going to build reflective logic on spec descriptions, but I want to be sure its reliable.#2017-08-3116:41hmaurerHi! Is there a built-in function to conform a spec and throw an error if the spec does not conform?#2017-08-3116:42hmaurerMy use-case is basically this: I want to pipe data through a series of steps, and I would like to throw an error if a spec does not conform#2017-08-3116:42hmaurerI could easily write a function to do this#2017-08-3116:42bbrinck@hmaurer Would s/assert
work?#2017-08-3116:42hmaurerjust wondering if itās in clojure.spec#2017-08-3116:43bbrinckfollowed by conform?#2017-08-3116:43hmaureractually s/assert
does exactly what I want; I donāt care too much about conforming#2017-08-3116:43hmaurerthank you!#2017-08-3116:45bbrinck@hmaurer Just make sure to call (s/check-asserts true)
to turn on assert checking for spec#2017-08-3116:46hmaurer@bbrinck ok. Is it sensible to use s/assert
when validating user input, e.g. an API payload?#2017-08-3116:50bbrinck@hmaurer I would prefer to just use explain-str
in that case personally, since a) assert
will raise an exception that youāll need to catch and b) turning on asserts will enable asserts throughout your program, which you probably donāt want. Iād only use assert
for dev-time checks/invariants#2017-08-3116:51bbrinckIām assuming the flow is check data against spec if it works, proceed else, show message to user?#2017-08-3116:52hmaurerpretty much#2017-08-3116:52hmaurerI would like to have some flow like this:
check data again spec => validate through authorization filters => assign UUID => transact to database#2017-08-3116:52bbrinckIf your user is a dev at dev-time, I think assert is fine. But if you want to show end-user error messages, Iād avoid it#2017-08-3116:52hmaurerany step can throw an error which should be reported back to the user#2017-08-3116:52hmaurerend-user, not dev#2017-08-3116:53bbrinckRight, in that case, Iād use explain-str
and check to see if it success.#2017-08-3116:53bbrinckThen return that string to whatever is going to show it to a user#2017-08-3116:54hmaurerI am quite new to Clojure; would you in general avoid to throw errors and return error conditions instead? Aka use a āEitherā-like type, similar to Haskell/etc?#2017-08-3116:55hmaurere.g. return {:ok ...}
or {:fail ...}
#2017-08-3116:56bbrinckGood question. I donāt see a lot of use of āeitherā in Clojure code. Exceptions are fine if itās actually an exceptional case , but AIUI, in your case, you want to validate user input so you expect things to not conform#2017-08-3116:56bbrinck(correct me if Iām wrong)#2017-08-3116:57hmaurerYou are right. I would like to be able to chain my pipeline steps with (-> ...)
, but breaking out of the pipeline on error seems hard to do without exceptions#2017-08-3116:57bbrinckI donāt have great advice here for general error patterns, but Iād lean towards āuse exceptions for exceptional cases, not control flowā#2017-08-3116:57bbrinckRight#2017-08-3116:57bbrinckThere are libraries for this, but I donāt know how idiomatic they are. I havenāt used them myself#2017-08-3116:58bbrinck@hmaurer Some discussion here: https://www.reddit.com/r/Clojure/comments/6wmnfm/thoughts_on_failjure_vs_funcoolcatseither_for/#2017-08-3116:59didibusIt's more idiomatic to use exceptions with ex-info. I would wrap explain-str or explain-data in a function which throws an ex-info, where I handle it at the place where my code can return an error msg to the user. There I'd parse out what I need from the exception into the msg I want to show the user and return it.#2017-08-3116:59hmaurerThanks! Iāll read this. I like the philosophy of using errors for exceptional cases only though; considered following that approach in a previous javascript projct#2017-08-3117:00hmaurer@didibus yep, that sounds a bit more idiomatic/practical#2017-08-3117:01didibusWhat you do is in the ex-info map, you put a key such as :type :validation-error and so where you catch it, you know this isn't a bug, but a validation error. Normally, the exceptions should not log error or push failure metric, as they are actually expected behaviour#2017-08-3117:01bbrinckEven if you use exceptions in this case (not necessarily bad IMO), Iād still probably throw an exception manually instead of relying on assert
since turning on/off asserts is a global thing. IOW, do you want to enable all asserts in production?#2017-08-3118:22hmaurerThat seems reasonable#2017-08-3117:02bbrinckThatās a valid question, but you probably donāt want it tied to your user-facing errors messages#2017-08-3117:02didibusI mean exceptions of type validation-error#2017-08-3117:02didibusAgree with bbrinck#2017-08-3117:02bbrinckGood suggestion @didibus#2017-08-3117:03bbrinck@hmaurer Shameless plug, but if you are considering showing spec error messages to users, take a look at https://github.com/bhb/expound#2017-08-3118:24hmaurerI stumbled on it yesterday actually! Itās looking very useful#2017-08-3117:06didibusBig fan of expound, the default spec errors are so full of noise.#2017-08-3117:08didibus I'm actually disappointed in spec for improving clojure error msgs. They promote the idea, but it really doesn't help because the error is so long. Its only slightly friendlier then a stack trace#2017-08-3117:09bbrinckIMO, the default error messages are a good default, since they need to be useful āout of the boxā in a variety of contexts: shown to a end user, shown to a dev, logged, returned as an API response, etc. Expound makes a different set of tradeoffs that are very much optimized for the ādisplay to devā case - *much* more verbose, and omits some information, but hopefully helps devs find the root problem faster#2017-08-3117:10bbrinckLuckily we can swap out the default for custom printers like expound when we want to customize behavior š#2017-08-3117:11bbrinckExpound is probably still not quite optimized for end users, but I am looking to make it more suitable for this in the future.#2017-08-3117:11didibusI don't know. Id like more beginner friendly defaults in Clojure. Easier to swap out to something else when you're not a beginner anymore. Its a lot to ask to a beginner to learn how to setup expound to figure out why his code doesn't work.#2017-08-3117:13bbrinck@didibus I agree with you that itād be nice to get something easier for beginners. I have been wondering if it would be possible to create a lein plugin or library that would set up beginner-friendly REPL / logs by default#2017-08-3117:14bbrinckAdding a single dependency that loads core specs, turns on instrumentation (with orchestra?) and configures spec to use expound without any further config could be really useful#2017-08-3117:16bbrinckmaybe pretties up/shortens stack traces as well#2017-08-3117:17didibusThat would be really nice. Maybe if it can also add a coloured repl and preload clojure.repl #2017-08-3117:19didibusI've heard of https://github.com/venantius/ultra/blob/master/README.md#2017-08-3117:19didibusBut since at my work we use Ant, it doesn't help me promote Clojure to other teams#2017-08-3117:22bbrinckWhat editor would you recommend for other teams? Just curious about the constraints.#2017-08-3117:24didibusWell, I tend to recommend eclipse + counterclockwise or cursive with intelliJ.#2017-08-3117:25didibusThe latter less so, because of its paid nature#2017-08-3117:27didibusYou'd just be surprised how many top class engineers don't bother to learn how to setup any environment properly.#2017-08-3117:28didibusSo when I tell them, if you want to do Clojure and like it, you need to learn at least how to setup a working repl. And already I've lost the interest of half of them#2017-08-3118:12gdeer81@didibus I think @sekao is the best person to talk to about this since he has written three IDEs so far and each one tries to lower the burden of getting everything set up#2017-08-3118:16gdeer81this is a side topic not really related to spec so I broke it out into this thread#2017-08-3120:21didibusNightlight looks pretty cool #2017-08-3118:15gdeer81my favorite is nightmod since I can send someone the download link and they can be in a new project with a repl, starter code, and a live game environment in less than 5 minutes#2017-08-3118:38favilaIn clojure.spec.alpha, s/invalid? is defined like (identical? ::invalid ret)
. I thought we couldn't rely on keyword interning anymore?#2017-08-3118:40ghadiit still is a thing#2017-08-3118:40ghadi(not in cljs afaik)#2017-08-3118:40favilaI thought at some point even in clj it was no longer guaranteed#2017-08-3118:41favilaso (identical? (sym|kw) x)
became a bad idiom#2017-08-3118:41ghadisymbols are not interned#2017-08-3118:41ghadijust keywords#2017-08-3118:41favilaah maybe that's what I'm thinking of#2017-08-3118:41favilaor I'm just confused with cljs#2017-08-3118:41favila(I do a lot of cljs)#2017-08-3118:41ghadithough the codepath goes still through a method called intern#2017-08-3119:05ikitommihi. I have a problem creating a recursive spec. the s/spec
requires the spec to exist:
(require '[clojure.spec.alpha :as s])
(def data
["/api"
["/ping" {}]
["/pong" {}]])
(s/def ::route
(s/cat
:path string?
:arg (s/? (s/map-of keyword? any?))
:childs (s/* (s/spec ::route))))
; CompilerException java.lang.Exception: Unable to resolve spec: :user/route
#2017-08-3119:06ikitommiSetting something temporary as a spec and double-evaluating the original spec seems to work thou, but donāt think this is the way to do this š#2017-08-3119:06ikitommi;; declare'ish
(s/def ::route int?)
(s/valid? ::route data)
; false
;; resetting the spec works now (but the childs is wrong)
(s/def ::route
(s/cat
:path string?
:arg (s/? (s/map-of keyword? any?))
:childs (s/* (s/spec ::route))))
(s/valid? ::route data)
; false
;; now it might work?
(s/def ::route
(s/cat
:path string?
:arg (s/? (s/map-of keyword? any?))
:childs (s/* (s/spec ::route))))
;; ....
(s/valid? ::route data)
; true
#2017-08-3119:59favila@alexmiller My attempt to spec the s/cat s/tuple hybrid with optional trailing positions https://gist.github.com/favila/67c869b3c92b1a40559f639b065734e2#2017-08-3120:02favilaworks for me (so far)#2017-08-3120:04ikitommiOh, my recursive spec works when I wrap the child spec into something, like s/and
(s/def ::route
(s/cat
:path string?
:arg (s/? (s/map-of keyword? any?))
:childs (s/* (s/spec (s/and ::route)))))
(s/valid?
::route
["/api"
["/ping" {}]
["/pong" {}]])
; true
Is this the correct way to do this?#2017-08-3121:33hmaurerHi! Quick question: I have a spec for a map which has required and optional keys (`(s/keys :req [..] :opt [..])`). One of my functions takes āpartialā versions of those maps, which essentially means that every key is optional (and every key that was optional in the full spec can be nil
, but I can handle that separatly)#2017-08-3121:34hmaurerit there a neat way to do this, how should I define two versions of the map specs? (e.g. :foo/my-map
and :foo.partial/my-map
)#2017-08-3121:42plinshello everyone, im reading the spec docs (`https://clojure.org/guides/spec`)
is it possible to achieve with spec/fdef
something similar to this? (catching spec errors at run time):
(defn person-name
[person]
{:pre [(s/valid? ::person person)]
:post [(s/valid? string? %)]}
(str (::first-name person) " " (::last-name person)))
#2017-09-0104:20didibusYou can instrument, but it doesn't do the post#2017-09-0104:40didibusYou can use orchestra a 3rd party library, that one will instrument the post also#2017-09-0114:17plins@didibus#2017-09-0114:18plinsany recommendations on libraries that already do that?#2017-09-0221:59didibus@U3QUAHZJ6 https://github.com/jeaye/orchestra#2017-08-3122:04wilkerluciohello, I'm having an issue trying to define the following specs:#2017-08-3122:04wilkerlucio(s/def :user/accessors (s/keys :opt [:user/id :user/name :user/address]))
(s/def :user/id string?)
(s/def :user/name string?)
(s/def :user/address :address/accessors) ; <- trouble maker
(s/def :address/accessors (s/keys :opt [:address/street :address/user]))
(s/def :address/street string?)
(s/def :address/user :user/accessors)
#2017-08-3122:05wilkerluciothis is a simplified case where I need to set a spec to something that still are going to be defined#2017-08-3122:05wilkerlucioright now this throws an error, as:#2017-08-3122:05wilkerlucioCompilerException java.lang.Exception: Unable to resolve spec: :address/accessors, compiling:(2017.clj:10:1)
#2017-08-3122:06wilkerlucioI remember reading that the definitions should be lazy to allow this kind of things, am I missing something here?#2017-09-0104:12didibusI always known it as not being lazy. Can't you just reorder your definitions? Being lazy would make it risky I feel, what if specs are accross namespaced and lazy, and you forgot to load another namespace.#2017-08-3122:17wilkerlucio@hmaurer you can define multiple versions, create a simpler one and then you can merge it with other to create a more fully-featured, example:
(s/def :foo.partial/my-map (s/keys :req [:foo/a :foo/b]))
(s/def :foo/my-map (s/merge :foo.partial/my-map (s/keys :req [:foo/c])))
#2017-09-0108:43lmergeni have a question about project organisation and specs ā specifically, iāve settled on the use of a separate specs.clj
file to put all my specs in, that works quite well.
but, i have multiple small subprojects, of which some share the same specs (i.e. the output of one project is the input of another).
whatās the best strategy here ? just copy/paste these common specs into these projects (because theyāre different domains), or share the code using a common library ?#2017-09-0108:52mpenetseparate repo for shared specs works well#2017-09-0108:52mpenetdepends what it is#2017-09-0109:00ikitommicould there be something like s/lazy
which would work like s/spec
but would not fail if the given qualified keyword doesnāt link to registered spec? could be used with the recursive specs (my example above). (s/lazy ::route)
(vs (s/spec (s/and ::route))
. The error messages be easier to understand. Happy to do a PR if.#2017-09-0113:59wilkerlucioI have the same problem when trying to spec some distant relations where I can't guarantee the load order#2017-09-0413:02wilkerlucio@ikitommi just one thing, you can do (s/and ::route)
instead of (s/spec (s/and ::route))
and it will work the same#2017-09-0413:08ikitommi@U066U8JQJ indeed, thanks!#2017-09-0109:21lmergen@mpenet in this specific instance itās the spec of what a job input should look like ā itās produced by one component, and consumed in another.#2017-09-0109:21lmergenso it strikes me that a common spec library might be a good approach for this, but it also feels like it might create more spaghetti / overhead#2017-09-0113:04aisamuHi!
Is there a non-macro alternative to specās sequence operators? (`cat`, *
, ?
ā¦).
To avoid the XY problem: I have to generate medium-sized specs at āruntimeā, and my current solution is giving me ERROR java.lang.RuntimeException: Method code too large!
. Iāve read that it happens because there is an eval
generating classes too big for the JVM. The eval
exists because I couldnāt use the spec macros at runtime without wrapping them with make-fn
.
Is it unfeasible or just not currently implemented?
Or am I approaching this backwards?
;;
(defmacro make-fn [m]
`(fn [& args#]
(eval
(cons '~m args#))))
#2017-09-0114:13ghadi@aisamu what specifically are you evalling that is causing the error?#2017-09-0114:17aisamuThe sequence macros, as in (make-fn s/?)
, which later gets called with apply
and a vector of specs#2017-09-0114:28dealyHi, I've been trying to write a spec for a function which takes another function as a parameter. However it always fails with an Unable to construct gen error, is there a way to spec a function like this?#2017-09-0114:33ghadi@aisamu Are you doing something like calling eval on a long collection [(s/?....)......]
? How many before it blows with an error?#2017-09-0114:57aisamu@ghadi No. It's a small collection (~4), but the specs inside can get rather large. (The last input collection I managed to trace before the blow-up had 50k chars)#2017-09-0115:05ghadiyeah... spec macros can generate a lot of code#2017-09-0115:05ghadithe clojure compiler doesn't do method splitting#2017-09-0115:06ikitommi@aisamu had the same problem. See the source code, most specs have non-documented functional versions beneath. Hopefully will be part of the final public api.#2017-09-0115:11aisamu@ikitommi I looked into that, but it quickly became clear I'd have to bring along a dozen private functions to get the top level to work =/#2017-09-0115:23Alex Miller (Clojure team)the downside of the functional versions is that you lose forms for reporting#2017-09-0115:23Alex Miller (Clojure team)as those are captured by the macro#2017-09-0115:24Alex Miller (Clojure team)I have talked here in the past about how specs for spec forms (CLJ-2112) can be used to programmatically construct specs via unform#2017-09-0115:24Alex Miller (Clojure team)still a work in progress#2017-09-0115:47ikitommiI don't mind that the forms need to be manually passed to the functions. I'm using a pred->form multimethod to auto-resolve the forms, with most/all core preds mapped.#2017-09-0115:50ikitommiAnd looking forward on progress on the CLJ-2112.#2017-09-0116:33hlshipI think clojure.spec could do a bit better job of reporting failures to dynamically load clojure.test.check.generators. My production code doesn't include the org.clojure/test.check dependency, but does instrument a particular function at runtime; that seems to be forcing the code down a path that involves org.clojure/test.check ... possibly because some of the values being specs are functions? Very hard to say.#2017-09-0116:33hlshipPossibly this can be circumvented by providing an explicit generator for those function specs within our giant spec.#2017-09-0118:04mpenetfunctions as argument of instrumented fn get tested via gen yep#2017-09-0118:05mpenetyou can overwrite :gen on them with ifn? but it kinda defeats the purpose after all, personally I would prefer if we could make it so that the argument (which is a fn) has its args checked at "invoke" time#2017-09-0118:05mpenetthere's also a ticket on hold somewhere about this if I recall#2017-09-0119:03hlshipI've basically changed my spec from a useful fspec
to just fn?
. https://github.com/walmartlabs/lacinia/pull/112 That'll do for now.#2017-09-0412:32mpenetseems like I hit an interesting bug: direct-linking breaking a multispec : [...] foo is not a fn, expected predicate fn
#2017-09-0412:33mpenetstill trying to understand the why/how#2017-09-0412:33mpenet :at [clojure.spec.alpha$dt invokeStatic alpha.clj 756]}]
#2017-09-0412:33Alex Miller (Clojure team)and you canāt reproduce without direct linking?#2017-09-0412:33mpenetthe bug goes away without dl yes#2017-09-0412:33Alex Miller (Clojure team)well would be good to see a jira on that if you can build a reproducible case#2017-09-0412:34mpenetthe code is quite involved I ll try to create a minimal repro#2017-09-0412:34Alex Miller (Clojure team)I donāt think multimethod calls are direct linked#2017-09-0412:34Alex Miller (Clojure team)the line you have above is part of regex specs, not multispec#2017-09-0412:35mpenetbut it mentions a fn that's used as first arg to a multi-spec#2017-09-0412:36mpenet(it's a multimethod to be more precise)#2017-09-0412:36Alex Miller (Clojure team)sorry, just guessing from too little info#2017-09-0412:36mpenetnp#2017-09-0419:54gklijsmanaged to get spec working on clojurescript also after some searching/trying, but did not find any direct help on the internet. Iān now using a cljc file starting with `(ns m-venue.spec
(:require [#?(:clj clojure.spec.alpha
:cljs cljs.spec.alpha
:default clojure.spec.alpha) :as s]))` and on clojure I only need to source path, for clojurescript i need to put it in the source paths and require it. Now I can use the same spec front and back-end š.#2017-09-0422:45Alex Miller (Clojure team)cljs automatically rewrites the clojure namespace as the cljs one, so you shouldnāt actually need any of this afaik - just require the clojure ns#2017-09-0420:42plinshi everyone, im trying to use :pre
and :post
to produce a more detailed error in run time
(defn person-name
[person]
{:pre [(if-not (s/valid? ::person person)
(throw (IllegalArgumentException. (s/explain ::person person)))
true)]
:post [(if-not (s/valid? string? %)
(throw (Exception. (s/valid? string? %)))
true)]}
(str (::first-name person) " " (::last-name person)))
(person-name 42)
;;=> CompilerException java.lang.IllegalArgumentException
val: 42 fails spec: :payment-gateway.util/person predicate: map?
:clojure.spec.alpha/spec :payment-gateway.util/person
:clojure.spec.alpha/value 42
;; works fine, s/explains works properly
(person-name {::first-name "Elon" ::last-name "Musk" ::email "
#2017-09-0502:50souenzzomissing # ?#2017-09-0420:44plinswhy this works for :pre
but not for :post
?#2017-09-0422:47Alex Miller (Clojure team)shouldnāt both of those pre and post be functions? I think theyāre both getting evaluated at compilation time, not runtime#2017-09-0509:32AvichalHello,
I am trying to spec some higher order functions using fdef
and fspec
For eg:
(s/fdef adder
:args (s/cat :x number?)
:ret (s/fspec :args (s/cat :y number?)
:ret number?)
:fn #(= (-> % :args :x) ((:ret %) 0)))
is it possible to get the value of argument x
inside the spec of the inner function which is returned ?#2017-09-0511:49Alex Miller (Clojure team)No #2017-09-0514:38fothaggendaHello there! Quick question: is it possible to create an "inline" s/keys
inside a s/cat
in fdef
s? Like:
(s/fdef something :args (s/cat :first-argument (s/keys :req-un [::a ::b ::c])))
#2017-09-0514:38Alex Miller (Clojure team)no#2017-09-0514:38fothaggendaš Ok, thank you!#2017-09-0514:39Alex Miller (Clojure team)or wait, maybe? what are you trying to do?#2017-09-0514:39Alex Miller (Clojure team)you might want s/keys*
#2017-09-0514:39Alex Miller (Clojure team)if youāre matching varargs in a function#2017-09-0514:39fothaggendaHmmm, I'll check that out š#2017-09-0514:40Alex Miller (Clojure team)and even my prior answer is wrong - you can use s/keys there (I just donāt think itās what you want)#2017-09-0514:40Alex Miller (Clojure team)are you taking a map or a series of key/value options in something?#2017-09-0514:40fothaggendaOdd; I tried that way before and my generative test exploded#2017-09-0514:40fothaggendaI'm taking a map#2017-09-0514:41Alex Miller (Clojure team)oh, then you should be fine with the above#2017-09-0514:41Alex Miller (Clojure team)if you want to test it, run (s/exercise (s/cat :first-argument (s/keys :req-un [::a ::b ::c])) 1000)
- thatās basically what youāre getting with a generative test on something
#2017-09-0514:42fothaggendaNice catch, I'll inspect this way#2017-09-0514:43Alex Miller (Clojure team)often pushing that out to 1000 samples reveals more :)#2017-09-0514:43Alex Miller (Clojure team)if a/b/c are big colls, that can provoke some problematic samples - bounding their size often helps#2017-09-0514:43Alex Miller (Clojure team)with :gen-max
#2017-09-0514:44fothaggendaActually that's not the case, in my scenario ::a
, ::b
and ::c
are just very specific definitions, but I got your point#2017-09-0518:47arohnerare there any projects for associating docstrings with spec definitions?#2017-09-0518:52Alex Miller (Clojure team)we will eventually get around to implementing it in spec, just kinda far down the list atm#2017-09-0608:41gklijs@arohner I have a private project which I use to serialize specced data, where I also store the used spec in either the bytes or the string. I might make it public. It also makes it possible to set defaults, in order to get correctly specced data back, in case you added required keys.#2017-09-0608:47gklijsIt works both with clojure and clojurescript, but it really needs some more work (liks tests for the clojurescript) before I put it public#2017-09-0611:46ikitommi@gklijs can you re-create the spec instances from serialized format in cljs without (form) eval?#2017-09-0614:06gklijs@ikitommi Doentāt know what you mean exactly, i use the form function to re-create the instance, but by assuming the spec is a map with either keys, or merge, and by storing the data in a certain way, I only store the values. If there is an additional required key which wasnāt there when the instance was serialized, either nil is filled in, or the default is it is set.#2017-09-0712:14seantempestaIs there a version of s/or
that just conforms the value and doesnāt tell you what value it is? I have a map where certain entries can be either a string or a function that returns a string and I just want to get back either a conformed map or :cljs.spec.alpha/invalid
. Am I supposed to traverse the returned map and remove the named vectors?#2017-09-0713:54misha@seantempesta
(s/def ::key (s/or :int int? :str string?))
(s/conform ::key 1)
=> [:int 1]
(s/conform (s/nonconforming ::key) 1)
=> 1
(s/conform (s/nonconforming ::key) :foo)
=> :clojure.spec.alpha/invalid
#2017-09-0713:55seantempesta@misha oh cool. so just call s/nonconforming
first?#2017-09-0713:56mishawrap spec you don't want vectors from in nonconforming
#2017-09-0713:58seantempestaOh wait, s/nonconforming
says that it returns the original value. I want the conformed value though.#2017-09-0713:59mishawhat do you think original and conformed values are?#2017-09-0713:59seantempestaoriginal is the original value, but a conformer function can return a modified value.#2017-09-0714:00mishaif you have custom conformer, I think you can "avoid vector" there#2017-09-0714:02seantempestaWell I was using all custom conformers. Writing separate functions like str-or-fn-ret-str
, but I realized I was duplicating a lot of code. Also, I think that requires me to write custom generators for everything.#2017-09-0714:03mishacan you share str-or-fn-ret-str
?#2017-09-0714:03seantempestaIt looks like (s/unform ::key (s/conform ::key val))
is working#2017-09-0714:04seantempestaWell, itās actually string-or-react-class. Hereās what it looks like:
(defn string-or-react-class?
"Accepts either a string, react class, or a fn that returns a react class. If it's a fn, props will automatically
convert (js->clj) when the fn is called."
[s-or-c]
(cond
(helpers/is-react-class? s-or-c) s-or-c
(string? s-or-c) s-or-c
(fn? s-or-c) (fn [props & children]
(let [clj-props (js->clj props :keywordize-keys true)
react-c (s-or-c clj-props children)]
react-c))
:else :cljs.spec.alpha/invalid))
#2017-09-0714:04mishaif you supply custom unformer - it might work.
But I have a feeling you are doing something redundant#2017-09-0714:05mishawhy do you have it as a conformer, rather than spec composed from predicates?#2017-09-0714:05seantempestaThatās what I was asking. If I do it as s/or
predicates I get back vectors#2017-09-0805:42didibusFrom my perspective, you conform to get back a vector, or a custom projection of the specced data.#2017-09-0805:43didibusIf you want to know if something validates, you do (when (s/valid? ...) do), or one of the explain fns.#2017-09-0714:08mishatry to wrap s/or
in nonconforming
. this way you can have more granular conformer for just (fn? s-or-c)
case.#2017-09-0714:08seantempestaOkay. Iāll give that a shot. Thanks.#2017-09-0714:14mishaalthough I'd like to see examples of coercion in spec done right, myself#2017-09-0714:17misha@seantempesta it seems like nonconforming or
prevents any conformation inside its branches#2017-09-0714:18mishaduh... opieop#2017-09-0714:26bbrinck@seantempesta Can you talk more about why a call to conform
is useful in this case? If you just want the original value, could you just do something like (if (s/valid? ::string-or-int x) x :cljs.spec.alpha/invalid)
#2017-09-0714:27mishaI think to wrap fn? in (fn [props & children]
#2017-09-0720:57cgrandHow do you prevent a :ret
spec to be described by doc
? I have a big spec and Iād like to be displayed by name rather than dumping it on the user. Currently I wrap it into spec/and
. Is there a more elegant solution?#2017-09-0721:04Alex Miller (Clojure team)name it?#2017-09-0721:05Alex Miller (Clojure team)then youāll just see the name in the doc#2017-09-0721:14cgrandIāll stick with spec/and
then. (btw I also abuse spec/and
for late-bound aliases)#2017-09-0721:25Alex Miller (Clojure team)I guess I donāt understand why thatās better#2017-09-0722:07cgrandIf I put it in only in the doc then it's not in the spec anymore. If I put it in doc and in spec it doesn't prevent doc
from dumping the whole description. So s/and
is the hack to have :ret spec'ed without doc being verbose#2017-09-0722:50Alex Miller (Clojure team)sorry. I went and tried it and I understand what youāre saying now. So the current behavior is not the intention. there is a ticket for the early aliased spec resolution, donāt know the #.#2017-09-0721:28ddellacostawhat is the best approach for validating a tuple that has optional fields? E.g. [:required :required :optional]
?#2017-09-0721:35Alex Miller (Clojure team)use (s/cat :a int? :b int? :c (s/? int?))
etc#2017-09-0721:37ddellacostaah thanks Alex, s/?
was the missing piece I was trying to find.#2017-09-0721:39Alex Miller (Clojure team)itās like regex - s/* s/+ s/? s/cat s/alt#2017-09-0721:30misha@alexmiller can you comment on @seantempesta's question, please?
what is a better way to do coercion with spec?
what is expected way to work with conformed compound values (map conformed to some the nested spec)? e.g. when you get map values as an or-conform-vector [:branch conformed-value]
. Is "conform-client" code supposed to know the spec structure, and be able to walk the conformed data structure?#2017-09-0721:35Alex Miller (Clojure team)donāt do coercion with spec#2017-09-0721:36Alex Miller (Clojure team)use conform to tell you how a value is parsed. then use clojure to transform that to what you want.#2017-09-0721:38Alex Miller (Clojure team)for the specific case of wanting to avoid conforming an s/or (one of the most common), spec as (s/def :foo (s/nonconforming (s/or :a int? :b string?)))
#2017-09-0721:39Alex Miller (Clojure team)that is, use s/nonconforming at the leaves, not at the top#2017-09-0721:40Alex Miller (Clojure team)for this particular case, Iāve come to believe that nonconforming or is a reasonable use case and either we should keep s/nonconforming or add that as a variant/option on or#2017-09-0721:41mishaso, going back to Sean's example, conform map to spec (will return [:fn? s-or-c]
for some key), and then post-process conformed map with a coerce-function, which will replace [:fn? s-or-c]
with (fn [props & children] ...
?#2017-09-0721:42Alex Miller (Clojure team)without having completely read or grokked all of the backchat, sure#2017-09-0721:42Alex Miller (Clojure team)Clojure is like, pretty good at transforming data. use it.#2017-09-0721:48mishaI think the issue/question is: "should one wrap coerce-fn in s/conformer
and conform to that, or should one just apply coerce-fn after conforming data to simpler spec?"#2017-09-0722:01Alex Miller (Clojure team)the latter#2017-09-0722:01Alex Miller (Clojure team)if you do the former, you are registering specs that throw away information about the original value#2017-09-0812:32ikitommiwe are running our spec libs tests also with :advanced
cljs optimizations. Seems that the cljs.spec.test.alpha/instrument
doesnāt work under it. Is there a way to enable it?#2017-09-0822:05jjttjjlet's say the spec decimal-in
defined here: https://github.com/SparkFund/useful-specs/blob/master/src/specs/number.clj#L24 is exactly what i need, except that the generator it comes with results in a bigdec, where I need a double. Is there a way to make a new generator that just coerces the output of an existing generator?#2017-09-0822:06jjttjjbasically i just need a spec and a generator for a double with a specified precision and scale#2017-09-0822:16Alex Miller (Clojure team)Make a spec with s/decimal-in#2017-09-0822:16Alex Miller (Clojure team)Then make a spec that wraps it with s/with-gen#2017-09-0822:17Alex Miller (Clojure team)If you want build on the existing gen, use gen/fmap over the s/gen of the first spec#2017-09-0923:58samedhiI have the following
;; This ::post does not work, recursion limit
;; (spec/def ::post (spec/keys :req [::post-content ::user-id ::posts]))
;; This ::post does work, never seems to recurse though...
(spec/def ::post (spec/keys :req [::post-content ::user-id]
:opt [::posts]))
(spec/def ::posts
(spec/with-gen
(spec/map-of ::post-id ::post)
#(spec/gen (spec/map-of ::post-id ::post :min-count 0 :max-count 1))))
which defines a spec recursively; ::post
refers to ::posts
.However, if I remove :opt [::posts]
and add ::posts
to the :req
fieldās ::post
, I get a Maximum call stack size exceeded
. Is this the expected behavior?#2017-09-1000:04samedhiSeems like the two should be equivalent, since ::posts
would (probabilistically) stop its recursion when it generates the empty map (`{}`).#2017-09-1000:09samedhiPleased to see that the generated data is in fact recursive! I guess I will just let it be, just to close out, my question mostly has to do with why ::posts
must be put in the :opt
and cannot be included in the :req
.#2017-09-1000:11samedhiAlso, it is rather interesting that ::posts
can be referenced before being used, which I guess implies that spec/keys
does not ādereferenceā ::posts
at its declaration, but waits to do so until the ::post
spec is used.#2017-09-1000:11samedhiCuriouser and curiouser#2017-09-1118:48bfabry@samedhi well... they're not equivalent, because a optional key is different to a required key which is a (possibly empty) map. it seems like the generator produced by map-of
just doesn't produce empty maps often enough to avoid blowing the stack#2017-09-1118:51bfabryyou need to be able to reference something before it's defined in order to support recursive specs, which is an intended feature#2017-09-1118:52samedhiHmm, interesting. It appears that I am referring to ::posts
from within ::post
before ::posts
actually existā¦#2017-09-1118:53bfabryyes, that's intended to be supported#2017-09-1118:59bfabryI don't know enough about how generators work, I'm guessing it's more sophisticated than just random so that's why it's managing to find the pathological case of a very deeply nested tree every time#2017-09-1120:57tbaldridgeYes, a lot of the complexity in generators comes from the feature of shrinking. When you get a test failure with a generator, test.check will try to shrink the possible data set to give you the smallest failing test. So instead of giving you "Failure when I tried: [123123121 55 3 9 5 23 4]" It may say: "Failure when I tried: [0 2]". And that tells you a lot about went wrong:#2017-09-1120:57tbaldridgefor example you know that the vector has to be at least two items long (it didn't fail when there was only one value in the vector). And that the second number has to be larger than one: (since 0 and 1 didn't cause it to fail).#2017-09-1120:58tbaldridgeThat sort of information is very helpful in debugging, but requires a lot of backtracking in the generator, and that's what makes them so hard to write/understand.#2017-09-1123:15bfabryI would love to know the "right" way to make the generators work for the recursive structure above#2017-09-1200:04gfredericksyou could try writing your own with gen/recursive-gen
#2017-09-1200:05gfredericksor...is this a DAG?#2017-09-1200:06gfredericksor I guess it's a proper tree?#2017-09-1200:07bfabryI did try with recursive-gen and got the same error. seems like it should be a proper tree yeah, which I guess is a type of DAG#2017-09-1200:07gfredericksyeah I think recursive-gen
would work#2017-09-1200:07gfrederickscan you share the recursive-gen
code?#2017-09-1200:07bfabrythis is what I tried:
boot.user=> (spec/def ::posts
#_=> (spec/with-gen
#_=> (spec/map-of ::post-id ::post)
#_=> #(tc-gen/recursive-gen (fn [post-gen] (tc-gen/map (spec/gen ::post-id) post-gen)) (spec/gen ::post))))
#2017-09-1200:08gfredericksI think the problem there is the (spec/gen ::post)
#2017-09-1200:09gfredericksyou can't use that generator as part of what you're building, since it's your original problem#2017-09-1200:09gfredericksif you had a ::post
that had everything but the ::posts
key, so that it didn't recurse, then you could use that#2017-09-1200:09gfredericks::post-with-empty-posts
#2017-09-1200:17bfabrybut a post without the ::posts key is not valid, the valid value for the leaf would be {}#2017-09-1200:20gfredericksYeah I meant a version where the key always has {}#2017-09-1205:59levitanong@gfredericks out of curiosity, could you try:
(s/with-gen ::post #(s/gen (s/or :post ::post :empty #{{}})))
#2017-09-1210:50gfredericksno, because it will still try to generate the default generater for a ::post
, which can overflow the stack or whatever it was#2017-09-1207:20lmergenwhat would be the best way to spec a collection of tuples, e.g.
[[:k1 :v1] [:k2 :v2]]
i'm thinking of just doing an into {} ...
before speccing and speccing it as if it were a map, but is there perhaps a more elegant way to do this ?#2017-09-1207:20lmergenthe alternative i came up with was (s/* (s/or (s/cat ...) (s/cat ....)))
, but that's also ugly#2017-09-1207:24lmergen(as a bit of background, the reason i'm not using a map is that the ordering of the elements is important)#2017-09-1207:30tapMaybe this?
(s/coll-of (s/tuple keyword? keyword?))
#2017-09-1207:31lmergenyeah that's similar to the cat use case... the problem is that for a specific key, i need a specific value
so then you would end up with
(s/or :k1 (s/tuple #(= :k1 %) ::v1)
:k2 (s/tuple #(= :k2 %) ::v2)
#2017-09-1207:31lmergenetc#2017-09-1207:32lmergenthat is, if you want to have a similar effect as
(s/keys :opt-un [:k1 :k2])
#2017-09-1207:40tapOk. I also donāt know a better way than your s/or
way#2017-09-1207:42lmergenok. guess i'll wrap this in a macro or something to make it pleasant.#2017-09-1207:44lmergenmaybe a multi spec is the answer though...#2017-09-1207:44lmergenyes, i definitely think that can work#2017-09-1210:01mpenetyou can shorten it a bit with (s/tuple #{:k1} ::v1)
#2017-09-1210:02mpenetotherwise maybe keys* can be used#2017-09-1210:07mpenetbasically (s/keys* :req-un [(or ::k1 ::k2)])
#2017-09-1210:07mpenetconforming it will return a map tho#2017-09-1210:07mpenet(s/valid? (s/keys* :req-un [(or ::foo ::bar)]) [:foo 1]) -> ok (::foo is #{1} here for the example)#2017-09-1210:08mpenetbut it will happily take "longer" tuples so not sure it's the best thing tbh#2017-09-1210:10mpenetseems like keys* exists (among other things) to validate unspliced kw args#2017-09-1210:34lmergen@mpenet thanks! that looks like what i want#2017-09-1211:12Jon:data #:clojure.spec.alpha{:problems [{:path [:args], :reason Extra input, :pred (clojure.spec.alpha/cat :docstring (clojure.spec.alpha/? clojure.core/string?) :attr-map (clojure.spec.alpha/? clojure.core/map?) :clauses :clojure.core.specs.alpha/ns-clauses), :val ((:require-macros [respo.macros :refer [defcomp div span input <> cursor->]]) (:require [clojure.string :as string] [hsl.core :refer [hsl]] [respo.app.comp.task :refer [comp-task]] [respo.core :refer [create-comp]] [respo.comp.space :refer [=<]] [respo.comp.inspect :refer [comp-inspect]] [respo.app.comp.zero :refer [comp-zero]] [respo.app.comp.wrap :refer [comp-wrap]] [polyfill.core :refer [text-width* io-get-time* set-timeout*]] [respo.app.style.widget :as widget])), :via [], :in [1]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x60330e39
#2017-09-1211:12Jonhow to read this error please?#2017-09-1212:33ikitommi@jiyinyiyong try https://github.com/bhb/expound#2017-09-1212:38ikitommiThere is a good sample of it in https://github.com/metosin/reitit/blob/master/README.md#validating-route-trees. Original :problems
was totally unreadable (recursive specs). Would like it just to report the deepest error, but still it's awesome.#2017-09-1217:04seancorfield@jiyinyiyong According to the :args
and :in [1]
, it's not expecting that :require-macros
form -- so I'm thinking you tried to run this in Clojure, rather than ClojureScript? (I think :require-macros
is ClojureScript-only?)#2017-09-1223:52Jonit turned out shadow-cljs compiler did not handle it correct š#2017-09-1217:04seancorfield(but, yes, some of ns
spec output is a bit overwhelming)#2017-09-1217:05seancorfieldExtra input
generally means a sequence (`s/cat`) ran into a form that was not recognized by the syntax specified in the spec.#2017-09-1217:07seancorfield:value (respo.app.comp.todolist (:require-macros [...
is the value that failed and :in [1]
gives you the position within that sequence.#2017-09-1218:57kevin.lynaghIs validating data with runtime context in the purview of spec? For example, instead of just saying that :foo/id
is an integer, can I specify that "it's an integer that corresponds to an entity in the provided database?"#2017-09-1218:57ghadinot recommended#2017-09-1218:57kevin.lynaghFrom what I can tell, even though specs are being checked at runtime, there's no idiomatic way (short of dynamic var based hacks) to pass additional context into the validation logic.#2017-09-1218:58kevin.lynagh@ghadi Is there any kind of rationale or discussion on this topic somewhere that you know of? I haven't been able to find anything in my research more specific than "don't use spec as a runtime transformation system"#2017-09-1218:58kevin.lynaghwhich is sort-of-related, but not quite.#2017-09-1218:59ghadiif you look up in a DB your predicates / specs might change their contract#2017-09-1219:00ghadithe meaning of a spec might change based on state, which loses a whole bunch of value (inverting a spec into a generator)#2017-09-1219:00ghadiI think specs should not depend on any ambient state#2017-09-1219:01kevin.lynaghbesides the implicit spec database#2017-09-1219:01ghadie.g. just check that the email address is valid looking, don't try to open up an SMTP connection#2017-09-1219:02kevin.lynaghYeah, but that's something that can't be technically enforced by the API of spec#2017-09-1219:02kevin.lynaghgiven that it accepts arbitrary predicate fns#2017-09-1219:02ghadii get that, it's just a terrible idea to curry in state into your predicate#2017-09-1219:03ghadinothing I can do to prevent it ĀÆ\(ć)/ĀÆ#2017-09-1219:03kevin.lynagh= )#2017-09-1219:03kevin.lynaghRight, I don't mean to challenge the underlying design decision#2017-09-1219:03kevin.lynaghas a consumer, it just wasn't clear to me whether what I wanted to do was in the design space at all#2017-09-1219:04kevin.lynaghI thought it might be, since conceptually it's just further validation#2017-09-1219:04ghadii think you'd use spec to check data at the door, then do extrinsic checks afterwards#2017-09-1219:04ghadii don't think it's a design goal to be a total system#2017-09-1219:04kevin.lynaghAt that point, it's easier to do the entire validation / generation myself, rather than using spec.#2017-09-1219:05kevin.lynaghsince in my domain pretty much all of the incoming data is just ids#2017-09-1219:05ghadithere's so many more benefits you'd be dropping#2017-09-1219:05kevin.lynaghso I need to realize those IDs into entities, then check properties of those entities.#2017-09-1219:05kevin.lynaghditto on the generation front, just getting integers won't help. I need to be able to generate integers that correspond to entities of type X#2017-09-1219:05kevin.lynaghor whatever.#2017-09-1219:06ghadidon't you need to check the entities?#2017-09-1219:06kevin.lynaghunless I'm missing something where spec might still be able to help in here?#2017-09-1219:07ghadiyou're right the IDs are trivial, but I doubt that's the extent of the applicability of spec for your use case#2017-09-1219:07ghadiI'd certainly want something to describe one of the "entities" and get a free generator#2017-09-1219:08ghadiA non-trivial generator#2017-09-1219:08kevin.lynaghI can walk you through a concrete example if you have 5 minutes#2017-09-1219:08ghadigo for it; I might not be able to help, but there are many š in here#2017-09-1219:09kevin.lynaghcool, thx = )#2017-09-1219:09kevin.lynaghI'm using datascript as the db, so it's all in-memory.#2017-09-1219:09kevin.lynaghthe application architecture is event sourced#2017-09-1219:10kevin.lynaghbasically, I have (step db event) ;;=> new-db
#2017-09-1219:10kevin.lynaghbasically, create a new database from the old database, given an event#2017-09-1219:11kevin.lynaghThe events implement a few protocol methods like (valid? [this, db])
which says whether or not the event is valid for a given database.#2017-09-1219:13kevin.lynaghSo, for example, #events.AddChild{:parent-id 5}
is a record that represents the event "add a child to node 5". The valid?
function needs to look and see if node 5 exists in the db and, if it does, make sure it's the type of node that is allowed to have children#2017-09-1219:13kevin.lynaghAll this make sense so far?#2017-09-1219:18kevin.lynaghAnyway, I don't understand how I can implement valid?
w/ spec, since everything depends on having the runtime db
value. Likewise, I cannot generate these events with spec, because it'd need to only generate integer IDs that correspond to nodes that can have children.#2017-09-1219:19kevin.lynaghIf wrote some specs to define what it means for a node to be allowed to have children, then I could turn the ID into a datascript entity at runtime and validate the entity using spec.#2017-09-1219:23kevin.lynaghbut it doesn't seem like those specs would help me generate events to test against the system#2017-09-1219:24ghadiyour scenario makes sense. I think the fact that you and spec named something the same thing (valid?) but has a different contract is confusing the situation#2017-09-1219:24kevin.lynaghAh, I only did that once I gave up on spec =P#2017-09-1219:30ghadiYou might make a spec to check a JWT. (s/def :my.security/jwt ....)
Doesn't mean that the token is good enough to get through authentication. Spec will check that it is structurally sound#2017-09-1219:31ghadiThe spec might be pretty involved, but you can't have something that is valid? one minute and not the next#2017-09-1219:31ghadispec predicates should not be hooked up to state. a point of the specs is to have enduring, sharable meaning#2017-09-1219:31kevin.lynaghRight, that makes sense#2017-09-1219:32ghadiI wish I had a more succinct rationale. Maybe @alexmiller will chime in later#2017-09-1219:32kevin.lynaghI didn't realize that it was a deliberate decision to prioritize that sharable meaning by disallowing runtime context.#2017-09-1219:33kevin.lynaghyeah, I hope I don't sound cranky about spec at all --- I think it's a really interesting idea and solves some hard problems.#2017-09-1219:33ghadiit doesn't disallow it, it's the runtime context (AKA mutable state) that will work against spec's goals#2017-09-1219:33kevin.lynaghthat's why I'm trying to use it in this context#2017-09-1219:33kevin.lynaghI don't want to rely on implicit runtime context or mutable state#2017-09-1219:34kevin.lynaghI'm just trying to pass that in#2017-09-1219:36kevin.lynaghAnyway, thanks for the chat and all your work on this stuff#2017-09-1219:36kevin.lynaghThere are some other places where I've very happily used spec for validation/generation = )#2017-09-1220:24Alex Miller (Clojure team)I donāt have a lot to add to Ghadiās comments which I mostly agree with. Some people are doing runtime generation and registration of specs (by loading values from a db for example). That seems fine to me (it does get a little trickier with s/keys).#2017-09-1220:26Alex Miller (Clojure team)in constrast, specs that use stateful stuff (dynvars, atoms, etc) seem bad to me#2017-09-1220:27Alex Miller (Clojure team)what is spec buying you in this case vs just writing code to assert whatever predicate you have?#2017-09-1221:12kevin.lynagh@alexmiller I have the same kinds of problems that spec solves very well: composing predicates and giving back specific errors about why things failed validation. E.g., the AddChild
event I mentioned earlier actually needs to take multiple parent-ids, and it would be great to get the detailed validation errors from spec.#2017-09-1221:13kevin.lynagh"Cannot add child because the third item in the list is of type X, but you can only add children to types U, V, W"#2017-09-1221:13Alex Miller (Clojure team)ok#2017-09-1221:13ghadiyou can easily express that in spec if you write a predicate#2017-09-1221:14ghadiit's the db extrinsic stuff that is a anti-recommendation#2017-09-1221:16kevin.lynagh@ghadi The predicate can only run against reified entities from the DB. Are you suggesting something like (->> parent-ids #(d/entity db %) #(s/valid? (s/coll-of ::parent))
?#2017-09-1221:17kevin.lynaghWhere ::parent
is a spec that has the relevant predicates?#2017-09-1221:18ghadisure, that's better -- fetch the data before asking spec#2017-09-1221:19ghadi(I was suggesting just building plain, offline predicates)#2017-09-1221:21ghadiif there are different types of nodes, some that have parent-ids and some that don't, you should be able to represent that in the data directly, without querying something external#2017-09-1221:21kevin.lynaghJust to clarify, all of this is happening in an in-memory database, so there is no network fetching or anything like that.#2017-09-1221:21ghadiRight but if that thing can change, YMMV#2017-09-1221:21kevin.lynaghit's immutable.#2017-09-1221:22ghadiok as long as a payload that is valid?
right now, is valid?
in the future, that's good.#2017-09-1221:22ghadiIf that can flip, bad kitty#2017-09-1221:23ghadii'm assuming you wouldn't call it a database if the data could never change#2017-09-1221:25ghadithis will probably get me in trouble: specs should only check stuff intrinsic in the data.#2017-09-1221:28ghadi(s/def :my.expiration/date #(re-matches #"\d{4}-\d{2}-\d{2}" %)
Spec can tell you if something conforms to :my.expiration/date
... but not if it's expired#2017-09-1221:28ghadi(aside: are regexes valid specs?)#2017-09-1221:29ghadiSpec will generate you a bunch of :my.expiration/date's, some might be expired, some not#2017-09-1221:30kevin.lynaghAnother way to phrase that question: Should spec allow you to validate {:time #inst"..." :expr #inst"..."}
#2017-09-1221:30kevin.lynaghpart of the spec is that both values should be dates#2017-09-1221:30kevin.lynaghbut should you check in spec whether the :expr
date is before or after the :time
date.#2017-09-1221:31ghadisure#2017-09-1221:31ghadiit's a bad idea to call (Clock/now) while validating the expiration#2017-09-1221:32ghadibecause that's extrinsic#2017-09-1221:32kevin.lynaghYeah, I understand that part#2017-09-1221:32kevin.lynaghI've been drinking the clojure kool aid for a while, don't worry = )#2017-09-1221:32ghadiheh#2017-09-1221:32ghadiThen you should have no problem extrapolating (Clock/now) to a database š#2017-09-1221:33kevin.lynaghcalling (d/entity some-db some-id)
essentially turns my integer id into an immutable view into the whole database#2017-09-1221:33kevin.lynaghso reifying the IDs into entities, then calling spec makes sense.#2017-09-1221:33ghadiit's weird though, when you check {:time #inst"..." :expr #inst"..."}
it's a totally different entity that you're checking. Something like a token-request
#2017-09-1221:34ghadiwe're in sync#2017-09-1221:34ghadii think originally I was thinking the db lookup was happening within the spec definition#2017-09-1221:35kevin.lynaghwell, that was my question, if it'd be possible to do that.#2017-09-1221:35kevin.lynaghbut you can't without dynamic var tomfoolery#2017-09-1221:35kevin.lynaghwhich I don't want to do.#2017-09-1221:35kevin.lynaghIt's still awkward, though, since I can't really spec out my events, since I can't spec their integer-id arguments#2017-09-1221:36kevin.lynaghThe best I can do is write my own validation fn that hydrates the ids into immutable entities, then calls spec on those entities.#2017-09-1221:36kevin.lynaghditto for generation, I can't use spec to generate anything.#2017-09-1221:37kevin.lynaghThe best I could do is, for a given DB, enumerate all entities, see which ones are valid?
against a specific spec, then return their ids#2017-09-1221:38kevin.lynaghAnyway, I'll give that a shot and see how tidy I can make things. I may come back in a week or two with a follow up report + gist example#2017-09-1221:38kevin.lynaghThanks again for the conversation!#2017-09-1313:14thedavidmeisterhow would i put together a spec for a map with string keys and values that are vectors of strings?#2017-09-1313:15Alex Miller (Clojure team)(s/map-of string? (s/coll-of string? :kind vector?))
#2017-09-1313:19thedavidmeisteris (spec/map-of string? (spec/+ string?))
ok?#2017-09-1313:21Alex Miller (Clojure team)I would recommend coll-of in this case#2017-09-1313:21Alex Miller (Clojure team)it gives you more control over size constraints, generation, etc and is a better match for what you described#2017-09-1313:22thedavidmeisterok cool#2017-09-1313:22Alex Miller (Clojure team)s/+ is for describing sequential collections with internal structure along with the other regex ops. while what you have will work, youāre not really using any of those features#2017-09-1313:22thedavidmeisterthanks#2017-09-1318:21bfabryno doubt this has been asked 1k times, but it's probably evolving. what are people currently doing to turn on instrumentation for their test run? top of the file in test_helpers.clj that's required by every test, fixture that's used by every test?#2017-09-1318:44Alex Miller (Clojure team)once fixture seems like a good place for that#2017-09-1318:54bfabrymakes sense#2017-09-1319:35joost-diepenmaat@bfabry weāre using :once fixtures,#2017-09-1319:35joost-diepenmaatworks ok, unless you forget to use the fixture š#2017-09-1320:53jumar@bfabry we're using lein profiles and injections - something like this:
:injections [(require 'clojure.spec.test)
(clojure.spec.test/instrument)
(.println System/err "Instrumented specs")]
#2017-09-1420:18akielIāve created a lib to support things like form validation using spec. I would be super happy if someone can look at it and give me feedback. Thanks! https://github.com/alexanderkiel/phrase#2017-09-1506:59mbjarlandI have a left field question about spec. There are a number of libraries in clojure for describing binary formats (buffy, octet, etc). Essentially you create a "binary spec" which describes the byte layout of your binary format and then ask the library to parse a byte array, byte buffer etc and back come strings, numbers etc parsed data types. The wish list for the language where you create these binary specs is that they be composable, support references from one spec to another (string length stored in another field), be terse etc. It strikes me that this wish list has a lot of overlap with spec. Would it be sane to try to use spec as the language for describing these kinds of binary formats or is it just a bad fit? You would get a fair amount of power for free and in years to come you would be using a language people are already familiar with...so it seems there are some upsides here.
One example of a good fit is multi-specs which in spec parse maps of different types depending on a 'type' field. There is a very close correspondence in a lot of binary formats with a type int16 or int32 followed by some binary data conforming to the type specified. Collections also match well. Anyway, somebody please call me crazy and tell me why this doesn't work : )#2017-09-1510:35gklijsIt would be great if you would have a higher-level format describing the data, which could be serialized to bytes, and deserialized to language-specific data-structures of different languages. However it is not easy, I had some experience with avro, https://avro.apache.org/ but even for bigintegers it doesnāt work for java out of the box..#2017-09-1512:40Alex Miller (Clojure team)I do think there is some overlap in goals and you could build custom spec types for this similar to the regex ops part of spec. Byte level stuff has some special considerations so I don't think you'd want to use what's in spec now for that.#2017-09-1516:48cgrand@alexmiller when/if you have time could you explain how my hack ruins s/form
? #2017-09-1516:51Alex Miller (Clojure team)if you have doc, then I guess itās fine. really Iād just rather actually fix this.#2017-09-1611:30OlicalAfternoon š I was wondering what sort of performance I could expect from s/fdef
? Like, if I don't instrument, is it just a normal defn
?#2017-09-1611:32OlicalAlso, is it "normal" to automatically enable instrumentation during development within a library? Wondering how much spec goodness I can expose to the consumer of the library automatically without the performance hit in production builds.#2017-09-1611:35gfredericks@olical the fdef
is independent from the defn
until instrumentation, so it should have no effect at all#2017-09-1611:35OlicalAwesome, that's what I was hoping for. Apologies if it was mentioned in the documentation and I didn't spot it.#2017-09-1611:38OlicalI was thinking of wrapping (s/instrument)
within some sort of (when dev? ...)
. Not sure if that's possible or a good idea.#2017-09-1611:39Olical(also I realise I was being dumb, you do defn and fdef next to each other, fdef does not replace defn)#2017-09-1816:11bbrinck@olical Keep in mind you can place the fdef
above the defn
(which I prefer for readability). Sometimes poeple donāt realize this#2017-09-1819:43christianromney@alexmiller love these useful type predicates in clojure 1.9! I'm putting (ex-info?)
on my wish list for Clojure Claus... š#2017-09-1819:46Alex Miller (Clojure team)Ticket and patch would be useful#2017-09-1820:14christianromneyhttps://dev.clojure.org/jira/browse/CLJ-2237 as requested! Thanks for considering this...#2017-09-1819:48christianromney@alexmiller will do!#2017-09-1911:56stathissideriswhy does this result in the function to be called several times?
(s/valid? (s/fspec :args (s/cat))
(fn []
(println 'foo)))
#2017-09-1911:56stathissideris21 times in fact#2017-09-1911:58gfredericksI think it's quickchecking your function to make sure it matches the spec#2017-09-1912:01stathissiderisok, is there any way to say that itās fine if the function throws an exception?#2017-09-1912:02stathissiderisbecause:
(s/valid? (s/fspec :args (s/cat))
(fn []
(throw (ex-info "foo"))))
#2017-09-1912:05stathissiderisfalse
#2017-09-1912:46Alex Miller (Clojure team)no, exceptions should be ā¦ exceptional#2017-09-1912:47danielnealCongrats on the 1.9 beta release! I'm just wondering how much of clojure is planned to be specced? I'm guessing it's just the macros - or are there plans to spec clojure core functions as well?#2017-09-1912:48Alex Miller (Clojure team)tbd both from a short-term and long-term perspective#2017-09-1912:49Alex Miller (Clojure team)there are several categories of fns that canāt currently be instrumented (protocols, multimethods, primitive fns, inline fns) so that limits some things#2017-09-1912:49Alex Miller (Clojure team)Iāve worked on others and I think it is valuable to have them. it is also pretty tricky to spec some of them, esp the higher order stuff#2017-09-1912:50danielnealyeah that makes sense#2017-09-1912:50danielnealso "everything" - probably out of scope - but there are plans over the long term to spec what would be useful, including the functions#2017-09-1912:51Alex Miller (Clojure team)I would hesitantly say yes as there are also performance things to be aware of when you load a large number of specs#2017-09-1912:52Alex Miller (Clojure team)right now, the core.specs.alpha has only macro specs in it and thatās loaded implicitly on first macro expansion#2017-09-1912:52Alex Miller (Clojure team)I think for fns, we may want to put those in a separate ns that you explicitly load if you want to instrument core#2017-09-1912:52danielnealah I see that makes sense#2017-09-1912:53Alex Miller (Clojure team)since core.specs.alpha is a separate lib, this is something that can evolve and be released in between clojure releases too. so if it doesnāt get done for 1.9 ga, can still make progress before the next major release#2017-09-1912:53danielnealThanks alex#2017-09-1915:46seancorfield@danieleneal I have almost everything spec'd in clojure.java.jdbc
and run the test suite with it instrument
d -- it's quite an overhead. When I get to my desk I can get numbers. #2017-09-1916:59seancorfieldSo I have rough numbers now: clojure.java.jdbc
has about 600 assertions. Against 1.8 it takes about 30 seconds (about 14 seconds "real" time). Against 1.9 with instrumentation it takes about 100 seconds (about 70 seconds "real" time). @danieleneal /cc @alexmiller#2017-09-1917:14Alex Miller (Clojure team)what does it take on 1.9 without instrumentation? :)#2017-09-1917:14Alex Miller (Clojure team)comparing apples to orange juice there#2017-09-1917:33seancorfieldLet me just comment out the instrument
call to check that...#2017-09-1917:36seancorfieldPretty much identical. 1.8 looks a hair faster but I'd have to run the test suite a lot more times to be certain.#2017-09-1917:36seancorfieldAnd the 30/100 seconds above is "user" (cpu) time. The "real" time is elapsed.#2017-09-1919:53jeayeYeah, jdbc was heavy for me, when instrumenting as well. Much, much heavier than all of our intrumenting combined.#2017-09-1919:53jeayeHad to disable it during development.#2017-09-1919:55seancorfield@jeaye It has specs for all the public API and JDBC stuff tends to get called a lot in DB-centric apps š If you think it's doing something suspect in its specs that causes undue performance overhead, let me know.#2017-09-1920:47richiardiandreahello folks! is there somewhere a wrapper for passing spec keys directly to generate? I mean without the extra nested call to s/gen
, ideally (somewhere/generate :my-ns/key)
?#2017-09-1921:15bfabry@richiardiandrea exercise?
boot.user=> (s/def ::foo integer?)
:boot.user/foo
boot.user=> (s/exercise ::foo 1)
([0 0])
#2017-09-1922:40jeaye@seancorfield Will do. I did end up looking, when I saw how long it was taking, and the specs seemed sane to me. It might be worth digging deeper, to see why these specific specs are taking so long. If I had to guess, it's the extensive usage of s/or
and s/cat
with s/*
, wheras just about every spec we have is a s/keys
or a clojure.core predicate.#2017-09-2003:43seancorfieldGood point. I might look at refactoring the specs to make them more efficient...#2017-09-2007:04odinodinIs the JVM opt to disable spec asserts still -Dclojure.spec.compile-asserts=false
or did this change with clojure.spec.alpha
?#2017-09-2007:49mpenetit's a compile time thing#2017-09-2007:49mpenetso doing that at runtime with compiled asserts will do nothing I believe#2017-09-2007:50mpenetotherwise you might be looking for clojure.spec.check-asserts
if asserts were compiled#2017-09-2012:44Alex Miller (Clojure team)I believe in both cases we did not put alpha in the property name, but you can check the source#2017-09-2012:47mpenetno alpha yes#2017-09-2014:12wagjoHi, let's say I have a function that takes a nested map and adds a new key somewhere deep.
(defn add-bar
[m]
(assoc-in m [:model/foo :model/bar] 42))
Given the spec for the input argument (and its :model/foo
key), what's the best practice to spec out the the return value with :model/bar
as required key for :model/foo
?#2017-09-2019:40dbushenkoHi all!#2017-09-2101:12richiardiandreaHello! I have probably asked this already at some point in the past, but is there a way to create a Generator
that basically uses a function of mine? I am trying with (tcgen/generate (tcgen/->Generator #(faker/name.findName)))
but no luck, it always returns nil
#2017-09-2101:14gfredericks@richiardiandrea what sort of thing are you trying to generate?#2017-09-2101:15richiardiandreafake data generated from a function...it looks like this works:
(def fake-generators
{:order/customer #(tcgen/return (faker/name.findName))})
#2017-09-2101:16richiardiandrea(gen/generate (s/gen :order/customer fake-generators))
#2017-09-2101:17richiardiandreaI was basically thinking of it in terms of repeatedly
#2017-09-2101:17gfredericksif you're doing property-based testing at all, it's generally better to build things from the combinators#2017-09-2101:17gfrederickswhat does faker/name.findName
do?#2017-09-2101:17richiardiandreawell, I am not using it for property testing with this specific case#2017-09-2101:18richiardiandreaI am bending it a bit, so that I can generate intelligible data structures#2017-09-2101:19gfrederickstest.check can generate intelligible data structures š#2017-09-2101:20richiardiandreaso now I get a very nice:
{:name "saladman",
:key "ia",
:timestamp #inst "1970-01-01T00:00:00.942-00:00",
:aggregate "order",
:data {:customer "Velma Doyle"}}
#2017-09-2101:22gfredericksis this because you want to use the generators for friendly examples?#2017-09-2101:22richiardiandreayep exactly#2017-09-2101:22gfredericksI almost feel like that should be a different piece of functionality all together#2017-09-2101:26gfredericksmaybe not; I guess it's just the strings that were bothering you?#2017-09-2101:28richiardiandreayes I completely agree with you, it could be a separate piece functionality, but generators by definition can be a good fit for that as well#2017-09-2101:29richiardiandreaif I have a restricted set of things I can use tcgen/elements
and that's nice, but for names...something like the above works well#2017-09-2111:01gfredericksNow I'm thinking this belongs in spec, if anywhere. A function called s/example
which will use custom code if registered and fall back on the generator otherwise#2017-09-2118:35bbrinckIf often find myself redefining fdefs a few times to get them right and AFAICT, you need to call instrument
after each redefinition. Has anyone else run into this? I suppose I could define a new macro named fdef!
that did both steps and then just remove the !
before I commited the code?#2017-09-2118:36bbrincksome sort of auto-instrumentation mode that worked as I redefined things would be handy. Has anyone written something like this?#2017-09-2120:40danielcompton@bbrinck I have instrument
called in my user
namespace that is reloaded after all of the other dependencies are loaded#2017-09-2120:41danielcomptonSo whenever I reload the code, (with clojure.tools.namespace) there is an instrument call at the end#2017-09-2120:41danielcomptonI still haven't figured out what's the best way to do this for testing though#2017-09-2120:43bbrinckAh, youāre right clojure.tools.namespace would be useful here. Thanks!#2017-09-2121:15danielcomptonI've got a lazy way of doing it, which is just to put the instrument call directly in the user
ns, but a fancier way to do it would be to attach it to the c.t.n.r/refresh :after
handler#2017-09-2200:29jeayeGiven a fn like (defn foobar ([a] (inc a)) ([a b] (str a b)))
where arity 1 expects a number and, let's say, arity 2 expects two strings, can I spec this so that arity 1 doesn't accept a single string?#2017-09-2200:30jeayeEverywhere I see people spec'ing multiple arities, they'll just use s/or
. That leaves all sorts of open holes.#2017-09-2200:30jeayeI'd see something like this: (s/fdef foobar :args (s/cat :a (s/or :int integer? :str string?) :b (s/? integer?)))
#2017-09-2200:30jeayeBut that permits (foobar "bad")
#2017-09-2200:39seancorfield(s/fdef foobar :args (s/or :one (s/cat :a number?) :two (s/cat :a string? :b string?)))
#2017-09-2200:41jeayeDamnit, that worked perfectly. Cheers, @seancorfield. I think I had tried s/alt
there, with two s/cats
, but it failed.#2017-09-2200:41seancorfields/alt
will alternate multiple sequence regexes <-- this is poorly worded but I'm not quite sure how to describe its interaction with s/cat
#2017-09-2200:42jeayeThis would be a good thing to have in the spec guide, I think; multiple arities aren't covered at all.#2017-09-2200:44seancorfieldIt probably doesn't help that the docstrings for s/alt
and s/or
look so similar. Seems to be a common point of confusion.#2017-09-2200:44jeayeYep, agreed.#2017-09-2200:49seancorfieldI just tried it with s/alt
in place of s/or
and it worked -- but the spec failure you get for (foobar "a")
is different... boot.user=> (foobar "a")
clojure.lang.ExceptionInfo: Call to #'boot.user/foobar did not conform to spec:
In: [0] val: "a" fails at: [:args :one :a] predicate: number?
val: () fails at: [:args :two :b] predicate: string?, Insufficient input
:clojure.spec.alpha/spec #object[clojure.spec.alpha$or_spec_impl$reify__810 0x55365ffb "
compared to boot.user=> (foobar "a")
clojure.lang.ExceptionInfo: Call to #'boot.user/foobar did not conform to spec:
val: () fails at: [:args] predicate: (alt :two (cat :a string? :b string?)), Insufficient input
:clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x373ea576 "
#2017-09-2200:50seancorfieldIn the first case, it fails to match :args :one :a
because "a"
is not a number?
and it also fails to match :args :two :b
because only one string was provided.#2017-09-2200:51seancorfieldIn the second case, it only reports the failure to match on alt :two
(again, because only one string was provided).#2017-09-2200:51seancorfieldThe spec I used was (s/fdef foobar :args (s/alt :one (s/cat :a number?) :two (s/cat :a string? :b string?)))
#2017-09-2200:53seancorfield^ @jeaye#2017-09-2201:15jeayeAh, I was getting the second one, but I was also likely testing with invalid inputs and ... bah, I don't remember. Awesome to see it works well; thanks again.#2017-09-2201:16jeayeNot having to calculate the various alternations between arities for use with s/or
is going to make my macro writing significantly simpler as well.#2017-09-2205:11jeayeIs there a good spec floating around for specs themselves? They can be ifn?
but also things like s/coll-of
return something else. I've also tried s/spec?
.#2017-09-2211:50Alex Miller (Clojure team)See https://dev.clojure.org/jira/browse/CLJ-2112 for wip#2017-09-2211:50Alex Miller (Clojure team)For spec forms that is#2017-09-2211:51Alex Miller (Clojure team)Currently spec instances should just be considered to be opaque and respond to true for s/spec?#2017-09-2217:03jeayeAwesome, thanks. Good to see there's work being done here.#2017-09-2205:36seancorfieldThat would be a good question for Alex when he's around tomorrow @jeaye#2017-09-2207:17jeayeAlright, I'll bug him in the morning.#2017-09-2208:12dbushenkowill Alex be today?#2017-09-2209:23vikeriI canāt seem to get my mutimethods to become intstrumented. Anyone else also having this issue?#2017-09-2209:39rovanionMultimethods are weird, one of those things I sometimes restart my repl for and it just works.#2017-09-2209:48andrewmcveigh@vikeri see above (hope the link works)#2017-09-2209:51vikeri@andrewmcveigh Ok thanks!#2017-09-2218:43richiardiandrea@gfredericks is it possible to write a generator that continues to generate until the sum of an operation on the generated stuff meets a condition?#2017-09-2218:43richiardiandreafor instance generate integers until their sum is 5#2017-09-2218:46tbaldridge@richiardiandrea that doesn't make a lot of sense, since there's a almost unlimited set of numbers that would sum up to 5#2017-09-2218:46gfredericksyou want a collection of integers in particular?#2017-09-2218:46richiardiandrearight, let's say I restrict the domain#2017-09-2218:47gfredericksand you want the sum to be =5, or <=5?#2017-09-2223:25richiardiandreaI see now the reason of this question, I am basically now generating a list of things such that their sum <= total randomly, if < then the last element is manually crafted#2017-09-2301:11gfredericksyeah, that's definitely a legit way to do it#2017-09-2218:47richiardiandreabasically I would want to have a such-that
, but where I accumulate results first, then I apply pred
on the result#2017-09-2218:47gfredericksone idea is to generate larger sequences and fmap
it to the largest prefix that meets your criteria#2017-09-2218:47gfrederickswould that be sufficient?#2017-09-2218:48richiardiandreauhm, let me try that, I will be back with the result š#2017-09-2319:43jstewWhere is the canonical place to put specs? I read that clojure.core has them in a clojure.core.spec namespace . I'm thinking of making a separate namespace of .spec for each namespace in my app. But I will have some common specs (boilerplate stuff), but don't know where to put those. I'm not sure it matters since all of the defined specs are global.#2017-09-2320:03gfredericksclojure is canon-poor#2017-09-2320:57bbrinckI donāt know if there is a standard, but Iāve been putting them in the same namespace as the functions that manipulate the data. That way, other namespaces can require a single ns and get functions+specs#2017-09-2322:04jstew@bbrinck What about specs that are shared between namespaces? Say you have a spec definition for a core.async channel or a map structure that's shared all over your app, do you duplicate that everywhere or keep those sorts of things in a single namespace#2017-09-2322:07bbrinckIn that case, Iād probably extract to a common āspecsā namespace like youāve suggested.#2017-09-2322:08bbrinckItās kind of like generic util functions. Iāll put them in a generic ns until some natural grouping emerges, then extract a more well-defined ns. YMMV#2017-09-2513:12acronI notice that a value which returns :clojure.spec/invalid
under conform
does not necessarily return false
from a valid?
call - is this reasonable? Am I misunderstanding something about valid?
?#2017-09-2513:16acronuser> (spec/valid? (spec/conformer #(if (string? %) :clojure.spec/invalid %)) "an invalid string")
true
#2017-09-2513:20Alex Miller (Clojure team)itās :clojure.spec.alpha/invalid
now#2017-09-2513:21Alex Miller (Clojure team)user=> (s/valid? (s/conformer #(if (string? %) ::s/invalid %)) "an invalid string")
false
#2017-09-2513:22Alex Miller (Clojure team)Iād recommend using the auto-resolved keyword here ^^ as this will change again when spec eventually comes out of alpha#2017-09-2513:22acron@alexmiller picard-facepalm thanks#2017-09-2613:22benhI want to generate strings of length 64.
This code does the job, but seems a bit unwieldy. Is there a nicer way to do this?
(gen/generate (gen/fmap clojure.string/join
(gen/vector clojure.test.check.generators/char-ascii
64)))
#2017-09-2613:23mpenetsame question as yesterday, I dont think so no#2017-09-2613:23gfredericksif you like gen/let
better syntactically, you can
(gen/let [chars (gen/vector test.check.generators/char-ascii 64)]
(apply str chars))
#2017-09-2613:24gfredericksmodulo getting all the namespaces right#2017-09-2613:24mpenetI guess we could have something better in test.check for this#2017-09-2613:25mpenetin the same vein as gen/vector#2017-09-2613:25gfredericksSomething specialized for strings? #2017-09-2613:25mpenetyeah#2017-09-2613:26gfredericksThe whole string thing needs an overhaul if anybody can think of a reasonable approach to unicode#2017-09-2613:26mpenetI know we can abuse string-for-regex in test.chuck for some of it#2017-09-2613:31gfredericksThat is a good example of the issues. Try generating matches for #".*" and see how much you like it#2017-09-2616:36uwois it likely or unlikely that s/def
will accept docstrings in the future?#2017-09-2617:55Alex Miller (Clojure team)Likely#2017-09-2617:13bfabryI wish there was a way to mark a function as impure in an fdef/fspec that had the consequence of turning on :ret validation for instrumentation and turning off validation of things against that fspec#2017-09-2617:16fedregHi all, is there a way in spec to define a map with something like this: (s/def ::my-map :req-un [::name ::id (one-of ::role ::title)]
?? ...So the data could be {:name "tom" :id "1" :role "mgr"}
or {:name "tom" :id "1" :title "ceo"}
for example. Thx!#2017-09-2617:18bfabryso xor?#2017-09-2617:19bfabryit's been a while but what little is left of my Maths for CS class in my brain is saying you can't create an xor with just or and and. so I'd say you'd have to just use or + a predicate#2017-09-2617:21fedregI need an s/or but for the keys instead of just the vals#2017-09-2617:23bfabrywell s/keys supports or, so (s/def ::my-map :req-un [::name ::id (or ::role ::title)]) is perfectly valid#2017-09-2617:24bfabrymy point was just that {:name "tom" :id "1" :role "mgr" :title "ceo"} would be valid with that spec (as would your two examples)#2017-09-2617:25fedregahh, missed that. Thanks for clarifying. Didn't realize it was that simple. Thx!#2017-09-2617:27bfabrynp#2017-09-2620:00mrchanceHi, is there an automated way to get a spec for the output of conform, called with another given spec?#2017-09-2620:03mrchanceI mean:
> (def sp (s/cat :x double? :y double?))
> (s/conform sp [1.0 2.5])
{:x 1.0, :y 2.5} <- I'd like a spec describing this map
#2017-09-2620:15Alex Miller (Clojure team)no, although thatās been asked a few times.#2017-09-2620:18mrchanceThanks! So if I wrote one myself, how would I check that they fit together? generative tests?#2017-09-2620:20Alex Miller (Clojure team)sure!#2017-09-2620:20mrchanceOk, will play with it more, thanks š#2017-09-2701:28waffletowerIs there a better way to parameterize generators than to use macros? In this case, Iād like to reuse a generator in different specs with different parameters:
(defmacro limited-string-m [lim]
`(gen/such-that (fn [s#]
(not (blank? s#)))
(gen/sized
(fn [size#]
(gen/resize (dec ~lim) gen/string-alphanumeric)))))
#2017-09-2710:56gfredericks@waffletower at a glance that looks like a macro that could trivially be a function
you don't generally need to write any macros to build generators#2017-09-2711:38stathissideris@mrchance maybe you could do it with https://github.com/stathissideris/spec-provider#2017-09-2711:45mrchance@stathissideris oh nice, that looks like a helpful recommendation in any case!#2017-09-2716:24waffletower@gfredericks Thanks! works as a function as well#2017-09-2716:24waffletower(defn limited-string-fn [lim]
(gen/such-that (fn [s]
(not (blank? s)))
(gen/sized
(fn [size]
(gen/resize (dec lim) gen/string-alphanumeric)))))
#2017-09-2716:45gfredericks@waffletower btw, if you want to retain the gradual-growth property of the builtin generator, you could replace (gen/resize (dec lim) ...)
with (gen/scale #(min % (dec lim)) ...)
#2017-09-2716:46waffletowerthanks, I had noticed that side effect#2017-09-2802:28shark8meHi! is there a (short) way to reference a spec defined in a different namespace?
(ns a.b.c
(:require [cljs.spec.alpha :as s]))
(s/def ::foo int?)
;;; in a different ns
(ns a.d
(:require [cljs.spec.alpha :as s])
(s/valid :a.b.c/foo 10)
Is there a way to avoid typing the fully qualified :a.b.c/foo by using :require :as ?#2017-09-2802:38gfredericks(require [a.b.c :as a-really-short-name-that-doesn't-take-a-while-to-read]) ::a-really-short-name-that-doesn't-take-a-while-to-read/foo
@shark8me#2017-09-2802:40shark8meThanks! š#2017-09-2802:41shark8meI couldn't find this documented, I assumed it would be :ns/foo instead of ::ns/foo#2017-09-2806:54jumarNo, it's not, the proper syntax really is ::specs-ns/my-spec
#2017-09-2806:20witekHello. I have a function: (defn f [m])
. I expect m
to be a map containing a value unter the key :name
. The value has to be a keyword. How do I spec this function using (s/fdef f :args #!%&)
? What do I have to put in :args
?#2017-09-2807:01jumar@witek If I understand your problem, then s/keys
should work just fine:
(s/def ::name keyword?)
(s/valid? (s/keys :req-un [::name]) {:name :john}) ;=> true
(s/valid? (s/keys :req-un [::name]) {:name "john"}) ;=> false
#2017-09-2807:04jumarOr rather for the function:
(defn f [m]
(:name m))
(s/fdef f
:args (s/cat :m (s/keys :req-un [::name])))
#2017-09-2807:26hkjelsit seems that coll-of
messes with ordering, how do I specify a sorted set of maps#2017-09-2809:57curlyfry@hkjels What do you mean by "messes with ordering"? Do you mean when conforming?#2017-09-2812:00hkjels@curlyfry yeah#2017-09-2815:50bfabrykinda seems like a sorted-map?
predicate that takes the comparator makes sense#2017-09-2817:43souenzzomaybe (s/coll-of string? :kind set? :sorted true)
. It will be able to sort map, set, vec, list...#2017-09-2823:35jeaye:sorted? true
-- now I'm just nitpicking#2017-09-2900:02johanatans/gen
(or s/with-gen
) seems to unnecessarily use such-that
#2017-09-2900:02johanatancausing its associated spec's gen to be usable for only a few elements#2017-09-2900:02johanatan(s/def ::kebab-str
(s/with-gen
#(= %1 (csk/->kebab-case %1))
#(gen/fmap clojure.string/join (gen/list (gen/frequency [[1 (s/gen #{\-})] [9 gen/char-alpha]])))))
#2017-09-2900:03johanatan[`csk` is camel-snake-kebab]#2017-09-2900:03johanatancore> (s/gen ::kebab-str)
#clojure.test.check.generators.Generator{:gen #function[clojure.test.check.generators/such-that/fn--17362]}
#2017-09-2900:03johanatanthe such-that
above is suspicious#2017-09-2900:04johanatanand the result is usuable only for a few elements:
core> (gen/sample (s/gen ::kebab-str) 10)
("" "" "i" "rc" "sh" "s" "gh" "q" "" "od")
#2017-09-2900:07johanatanand not for many:
core> (gen/sample (s/gen ::kebab-str) 100)
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
#2017-09-2900:29seancorfield@johanatan Generators always validate against the spec. Your generator probably generates values that do not satisfy your predicate.#2017-09-2900:30seancorfieldWhat does your predicate return for "-a"
or "---"
?#2017-09-2900:37johanatan@seancorfield ah, good call. looks like first digit needs to be non-dash#2017-09-2900:39johanatan@seancorfield interesting that I wasn't getting a more obvious exception though (which I did get with a more egregious disparity between the validator and generator)#2017-09-2900:47johanatanFYI... this version works:
(s/def ::kebab-str
(s/with-gen
#(= %1 (csk/->kebab-case %1))
#(gen/fmap (comp clojure.string/lower-case clojure.string/join)
(gen/such-that (fn [c] (and (not-empty c) (not= \- (first c)) (not= \- (last c))))
(gen/list (gen/frequency [[1 (s/gen #{\-})] [9 gen/char-alpha]]))))))
#2017-09-2901:01seancorfieldYou probably ought to look at Gary Fredericks' test.chuck library -- it contains a generator for regex strings which would make this a lot simpler!#2017-09-2901:03seancorfieldAside from anything else, you could then define ::kebab-str
as a regex predicate and use test.chuck's regex generator directly on the same regex.#2017-09-2902:20amarHello! Any ideas on what Iām doing wrong here. (s/valid? ::bar-path ["foo" 1])
is ok, but not so in an s/fdef
(s/def ::foo-path
(s/cat :foo (partial = "foo")))
(s/def ::bar-path
(s/cat :foo ::foo-path :idx nat-int?))
(s/valid? ::foo-path ["foo"]) ; => true
(s/valid? ::bar-path ["foo" 1]) ; => true
(s/fdef my-fn
:args (s/cat :path ::bar-path)
:ret int?)
(defn my-fn
[path]
(second path))
(stest/instrument)
(my-fn ["foo" 1]) ; => my-fn did not conform to spec
#2017-09-2905:34seancorfield@amar The s/cat
calls combine to make one sequence regex. I think you can say (s/spec ::bar-path)
as the :path
argument spec and it will work.#2017-09-2905:35seancorfieldYou might also want to look at s/tuple
for your ::bar-path
spec instead of s/cat
.#2017-09-2911:57amarseancorfield: Thanks. The following worked.
(s/fdef my-fn
:args (s/cat :path (s/spec ::bar-path))
:ret int?)
#2017-09-3017:44eggsyntaxI feel as though I remember reading that it's OK to have advance spec definitions (ie to define specs whose s/def
refers to a spec defined later in the same ns. Can anyone confirm or deny that?#2017-09-3019:16Alex Miller (Clojure team)yes (although there are some known cases where that does not yet work right)#2017-10-0209:21borkdudeIs it me or is clojure.future.spec not checking macro specs at compile time?#2017-10-0211:04gfredericksit couldn't without patching some monkeys, right?
I have no idea if it attempts to do that or not.#2017-10-0221:27ajsIs it typical to use spec during runtime in production apps, or is more of a debugging tool in development to ensure data is flowing as expected in your app, but turned on off in production?#2017-10-0221:34bfabryturned off in production#2017-10-0221:34bfabryexcept possibly in very specific spots (system boundaries etc)#2017-10-0221:34bfabrywe use it in development, and in our master and staging environments#2017-10-0221:39guyHow big a performance inpact is it to have it running?#2017-10-0221:48ajsit would seem to me that spec/conform is designed with runtime use in mind for coercing data into a desired format, is this not accurate?#2017-10-0221:49bfabry@ajs no it absolutely is designed for that, those are the "specific spots" I was talking about#2017-10-0221:50bfabryie it'd be odd to be validating/conforming data somewhere other than a system boundary, unless you're looking for bugs in your own code#2017-10-0221:50bfabrywhich is what instrument/test.check are for, but they're expensive so just use in lower envs#2017-10-0221:50ajsis something like goog.debug used for turning it off in production, or is there a spec-specific way to do so#2017-10-0221:51bfabryyou still need to validate/conform at system boundaries though, because you can't trust other people's code#2017-10-0221:52bfabrythere's a few different levers to tweak. instrument
is used to add input validation to every function with an fdef. so just not calling that in production. spec/assert is a more explicit validation check, which can be disabled using compile-asserts or clojure.spec.check-asserts#2017-10-0311:06dbushenkohi all!#2017-10-0311:07dbushenkoI'm specifying an anonymous function, and its return value does not match the spec. Instead of normal error report or something it says like this: clojure.lang.ExceptionInfo: Can't convert path {:form (#function[.../fn--20377/fn--20384]), :val {}, :in [0], :in' []}#2017-10-0311:07dbushenkois it normal or I'm doing something wrong?#2017-10-0321:51bbrinckIt sounds like a bug in Expound? https://github.com/bhb/expound/blob/b9fc0d46dc3534bd77cdfebcf771cfecf163f058/src/expound/paths.cljc#L160-L166 If you have a repro and some time, a bug report would be appreciated š#2017-10-0313:01Alex Miller (Clojure team)hmm, not sure. repro?#2017-10-0315:00dbushenkosorry? do you want a gist?#2017-10-0315:04Alex Miller (Clojure team)yeah, want to know how to reproduce that error#2017-10-0315:05Alex Miller (Clojure team)and then if you can follow it with a (pst *e)
#2017-10-0315:05dbushenkowhat is this (pst *e) ?#2017-10-0315:07guyprevious stack trace ?#2017-10-0315:08guymy guess#2017-10-0315:08Alex Miller (Clojure team)print stack trace#2017-10-0315:08guydang#2017-10-0315:08Alex Miller (Clojure team)it is a function in clojure.repl#2017-10-0315:08dbushenkoknow what, looks like spec went ok, it was test who produced this strange error report...#2017-10-0315:08dbushenkoš#2017-10-0315:09Alex Miller (Clojure team)good, because I didnāt know where the āconvert pathā error was coming from :)#2017-10-0315:14dbushenko@alexmiller are there any plans to support specing the protocol implementations in records?#2017-10-0315:17Alex Miller (Clojure team)not near term. If you could spec them, Iām not sure you have the opportunity to actually do useful stuff with those specs.#2017-10-0315:25dbushenko@alexmiller maybe you remember I tried to invite you to our conf in December and you couldn't make it š Can you probably recommend someone from clojure core developers who might be able to visit us with a talk?#2017-10-0315:28Alex Miller (Clojure team)lots of fine people - take a look at any of the recent clojure confs (EuroClojure, Clojure/west, Clojure/conj, ClojureX, clojureD, clojuTRE, etc)#2017-10-0315:28dbushenkook, thanks#2017-10-0315:28Alex Miller (Clojure team)maybe post something here in #events, on the mailing list, on the reddit, etc#2017-10-0318:37jonasIs there a function in spec similar to spec/assert
but that conforms the value? (spec/assert ::a-spec x)
returns x
(or throws) and not the conformed value of x
.#2017-10-0318:39jonasPerhaps the reason is that you can set *compile-asserts*
to false
which results in all assertions to compile to x
?#2017-10-0318:49souenzzoinstrument?#2017-10-0318:41bfabry@jonas that would seem like a good reason why that wouldn't work#2017-10-0507:39dbushenkohi all!#2017-10-0507:39dbushenkodo we have any means of specifying records?#2017-10-0507:41mpenets/keys should just work on them#2017-10-0508:17dbushenkothanks!#2017-10-0515:19guyWhats the best way to test a spec'd function that has side effects?#2017-10-0516:04jaymartin@guy Iām not sure about the best way, but mocking can help speed things along if thatās what youāre after.
https://github.com/benrady/specific#2017-10-0516:26guyThanks @jaymartin have you tried it with the latest version of clojure.spec.alpha (17) ?#2017-10-0516:26guyit looks like specific is using alpha13#2017-10-0518:05jaymartinNo, Iāve just messed about with it playfully. But the ideas should be rather generic.#2017-10-0518:08guykk thanks š#2017-10-0522:20aengelbergIs there an equivalent of declare
for specs?#2017-10-0522:20aengelbergalternatively, is there a way to disable the check to see if a spec name exists when used in other specs?#2017-10-0522:24aengelbergbasically I would like to structure my code in the following order:
(s/def ::x ::y)
(s/def ::y ...)
#2017-10-0522:24aengelbergreplacing the first ::y
with (s/and ::y)
seems to work.#2017-10-0523:08bfabryfeels kinda odd that it doesn't just work, seeing as most of things in spec are lazy evaluated#2017-10-0523:08bfabryfeels kinda odd that it doesn't just work, seeing as most of things in spec are lazy evaluated#2017-10-0602:07Alex Miller (Clojure team)There is a ticket about this. The intent is that it should work #2017-10-0615:35bfabrygood to know, thanks alex#2017-10-0619:59aengelbergOh cool, I'm glad that that is the intent#2017-10-0523:36guy@aengelberg what do you mean#2017-10-0523:38bfabry@guy he means
boot.user=> (s/def ::foo ::bar)
clojure.lang.Compiler$CompilerException: java.lang.Exception: Unable to resolve spec: :boot.user/bar, compiling:(boot.user1422927036988753503.clj:1:1)
#2017-10-0523:39guySorry im probably being slow#2017-10-0523:39guyhe wants to use a different spec and call it something new?#2017-10-0523:43bfabryhe wants to say that the spec for ::foo is the same as the spec for ::bar, and it be ok that the spec ::bar has not been defined yet#2017-10-0523:44bfabry@aengelberg when I've wanted that I've just used (s/def ::bar any?)#2017-10-0620:02aengelbergthat works too. thanks#2017-10-0607:39guykk thanks @bfabry#2017-10-0609:16kennethkalmerI need some advice, Iāve hit a strange situation while trying to writes specs for data coming in from an external sourceā¦ The source returns a map that roughly translates to this: {:productStatus {:productStatus "active" ...} ...}
#2017-10-0609:17kennethkalmerhow on earth would I spec :productStatus
the map, and the string value? I have no control over the output generated by the service#2017-10-0609:18mpeneteither you overspecify by creating separate nses for each, or use something more generic like map-of, if it's only one key its fine, if that s a lot of keys that s so so#2017-10-0609:23kennethkalmerthink separate nsās are maybe the way to go here, will test that out, thanks!#2017-10-0609:28kennethkalmerthanks @mpenet! simple, it works, donāt know why I got myself stuck in another loop#2017-10-0617:58jaymartinThis may be highly subjective, but will Spec replace :pre
and :post
conditions as the idiomatic way to validate inputs and outputs? For example, something like this:
(ns rna-transcription)
(def m
{\G \C
\C \G
\T \A
\A \U})
(defn to-rna [xs]
{:pre [(every? #(contains? m %) xs)]}
(apply str (map m xs)))
#2017-10-0618:00jaymartinOr will :pre
and :post
still have their place and time?#2017-10-0618:04bfabryI could still see myself using :pre and :post to check non-local invariants, but that'd be pretty uncommon#2017-10-0618:23seancorfield@jaymartin You could define specs for your DNA/RNA stuff and use s/valid?
in :pre
... (s/def ::dna m)
(s/def ::dna-sequence (s/coll-of ::dna))
(defn to-rna [xs]
{:pre [(s/valid? ::dna-sequence (seq xs))]}
(apply str (map m xs)))
#2017-10-0618:24guythat looks nice#2017-10-0618:24seancorfieldor go further and define ::dna-string
as (s/and string? #(s/coll-of ::dna (seq %)))
and then {:pre [(s/valid? ::dna-string xs)]}
#2017-10-0618:27seancorfield(I hardly ever use :pre
or :post
, I must admit -- but I'm an advocate of leaving assertions on in production if you're going to use them at all)#2017-10-0618:27guywhat do you think about instrument in production? š#2017-10-0618:28seancorfieldinstrument
is a different beast, not least because it can trigger generative testing on higher-order function.#2017-10-0618:29seancorfieldIn addition, instrument
can introduce a big performance overhead depending on how much is spec'd and how they're written. The specs for java.jdbc
are a case in point: they introduce a huge overhead.#2017-10-0618:29jaymartinSuper helpful!#2017-10-0618:29seancorfield(Hmm, that reminds me that I forgot to add :keywordize?
to the optional specs in release 0.7.3!)#2017-10-0618:30guythanks!#2017-10-0618:47jeaye@guy We're using it in production.#2017-10-0618:47jeayeGiven our specs, I've measured that we spend about 2% of our time with instrumentation.#2017-10-0618:48jeayeAs Sean said, that can very wildly, depending on how the specs are written. Primarily whether or not there are a lot of alternate paths which need to be considered.#2017-10-0618:48jeayeFor us, it's almost entirely clojure.core predicates and s/keys
though, so it ends up being very fast and entirely practical for our production builds.#2017-10-0619:40guynice thanks @jeaye!#2017-10-0707:18ajsSince instrument does not check the return value, it would seem to me that a :post condition is still quite useful.#2017-10-0708:10mpenetI tend to use s/assert instead nowadays #2017-10-0709:08ajswith :post
at least, you donāt have to let
your fnās return value so you can assert it before you return it#2017-10-0709:25ajsdoes s/valid? work on all components of fspec?#2017-10-0709:26guywhat do you mean#2017-10-0709:27guyoh right got you#2017-10-0709:27guyhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/fspec#2017-10-0709:29ajsi know that instrument only looks at args, but i havenāt tried out how valid? works on it. will try that when i get home.#2017-10-0709:29ajswould be cool if that worked#2017-10-0709:32guywell instrument works on the return value too ?#2017-10-0709:32guythats why u have :args, :ret :fn#2017-10-0711:59ajs@guy no it doesn't. Are you sure? Check the docs. Only args are supported for instrument, though you can specify ret and fn for other purposes. #2017-10-0712:01ajs"Note that the :ret and :fn specs are not checked with instrumentation..."#2017-10-0712:08jaymartin@ajs Where exactly are you seeing that note, as I canāt find it.#2017-10-0712:09ajsThe spec guide itself, in instrument section. Why do you think libraries like Orchestra exist?#2017-10-0712:10jaymartinSilly me, I was looking in the doc string for instrument.#2017-10-0712:11jaymartinSo then, is fdef
orthogonal in some way. Its doc string says: āOnce registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests, and (if a macro) used to explain errors during macroexpansion.ā#2017-10-0712:12jaymartinI think I need to start over at the spec guide itself and re-read.#2017-10-0712:12ajsThe are tested by instrument for :args only I think#2017-10-0713:39Alex Miller (Clojure team)correct#2017-10-0714:00ajsOther than tests, I gather there's really no way to perform validation on the entirety of a function's signature, including its return value, and also validating that the entire fn received in a higher order function, is what is expected, is that right?#2017-10-0715:50Alex Miller (Clojure team)stest/check
is the only thing in spec that validates :ret and :fn specs#2017-10-0715:51Alex Miller (Clojure team)Iām not sure I fully understand the question re hof#2017-10-0716:25ajsThat answers it!#2017-10-0713:47taylorIām trying to write a spec for a collection where I need the first item to conform to another spec. s/cat
works great, but when I fdef
and check
an unary function of that collection it looks like test.check is apply
ing the collection to the function, and so I get arity exceptions. Should I change the function to take varargs?#2017-10-0713:48gfredericksare you using s/cat
to express the whole arglist or just the first arg?#2017-10-0713:49gfredericksI think to use a regex for a single arg you'll need to wrap in s/spec
, maybe that's what you're missing#2017-10-0713:49gfredericks(s/cat :first-arg (s/spec (s/cat ...)))
something like that#2017-10-0713:49tayloryeah, my :args
is (s/cat :clause (s/spec ::group))
#2017-10-0713:50gfredericksI don't know then. You certainly should be able to spec that.#2017-10-0713:56tayloractually it does work when I wrap with s/spec
, but I think I need to limit the test cases because itās running forever#2017-10-0713:57taylorthe spec is recursive and I guess is getting really complex samples#2017-10-0713:57gfredericksyeah š the Big Problem of spec/test.check#2017-10-0713:59taylorwell the good news is this is a toy problem š#2017-10-0714:13taylor(binding [clojure.spec.alpha/*recursion-limit* 1]
(stest/check `clause-str {:num-tests 1}))
#2017-10-0714:13tayloris āļø what recursion limit is there for?#2017-10-0714:15tayloror I could read the docsā¦ https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/*recursion-limit*#2017-10-0714:13taylorit runs pretty quickly like that#2017-10-0714:18gfredericksI think so. what's it default to?#2017-10-0714:20tbaldridgeThe depth of spec checking is somewhat limited by the JVM stack size, that keeps the generators from blowing the stack. Without it, youād get a stack overflow from almost any recursive structure.#2017-10-0714:25gfredericks@tbaldridge you're saying *recursion-limit*
refers to checking rather than generation?#2017-10-0714:28tbaldridgeIt only refers to generation. Checking is assumed to be bounded by the depth of the input#2017-10-0714:29gfredericksokay.
I think there could be a more automatic approach to handling the recursion, but I'm not sure.#2017-10-0714:29gfredericksI have to come up with something for the vanilla test.check collection generators, so whatever works there might apply to how spec assembles generators#2017-10-0714:30tbaldridgeThere probably could be. That dynamic var got added in one of the pre-alpha builds when I tried to build a recursive spec and it would stack overflow on the first structure generated.#2017-10-0714:31tbaldridgeChecking is fine since that's GIGO, so it's on the user not to check data that's millions of levels deep.#2017-10-0714:31tbaldridge@gfredericks any insight in the past year about shrinking recursive generators?#2017-10-0714:32tbaldridgeThey currently shrink by width (and a b) -> (and a), but not by depth: (and (and (and a))) -> (and a). I know we discussed this once, not sure if anything has changed in the past year#2017-10-0714:33gfredericksI don't think I have any thoughts that I didn't put on the ticket#2017-10-0714:34gfrederickshttps://dev.clojure.org/jira/browse/TCHECK-110#2017-10-0714:59taylorHa. This is exactly the type of structure Iām generating too#2017-10-0715:04gfredericks@U3DAE8HMG important to note here that spec doesn't use recursive-gen
#2017-10-0715:12gfredericksBut spec's approach has the same problem, but even worse#2017-10-0714:35gfredericksthe unfortunate part is that after generating a collection, test.check can't (in general) tell what parts the children are#2017-10-0714:36gfredericksand you'd need their shrink-trees in any case, not just the data#2017-10-0714:37gfredericksoh ummmm#2017-10-0714:39gfredericksI was thinking that there's an unsound approach where you wrap the child generator in an observer that keeps track of how it was called and use that to add shrinking info
I thought it was unsound because you can't tell just because the generator was called, that the result actually appears in the final structure
but I just realized that that probably doesn't matter. anything generated by the child generator should be a valid value from the final generator, so even if it doesn't appear in the actual data, it's not an invalid thing to shrink to in the strict sense#2017-10-0714:41gfredericksso now the worst thing about this approach is that it's a bit messy#2017-10-0714:42gfredericksbut I think everything still ends up being deterministic#2017-10-0714:42gfredericksI'll go comment on the ticket. I don't have time to work on it until at least thursday#2017-10-0714:48gfredericks@tbaldridge ā#2017-10-0715:12gfrederickshttps://clojurians.slack.com/archives/C1B1BB2Q3/p1507389135000078?thread_ts=1507386899.000005&cid=C1B1BB2Q3#2017-10-0715:14gfredericksWell maybe not worse....#2017-10-0715:30tbaldridgeah right, I forgot about that. Spec's stack overflow happened during the creation of the generator#2017-10-0715:48gfredericksah yeah#2017-10-0715:49gfredericksthe thing about spec's approach is that it's committing to a particular depth in the generator rather than describing the recursion the way the spec itself does#2017-10-0715:49gfredericksit might be making other commitments in the process, I'm not sure#2017-10-0715:49gfredericksI'm not sure this is a significant problem, it just complicates things#2017-10-0801:04taylorthanks for the advice & discussion! FWIW hereās the code I was working with https://gist.github.com/taylorwood/232129ccd3cb809281fea591d46f1b8a#2017-10-0910:05jwkoelewijnHi all, does someone know if there is a way to limit generators? I have created specs with generators, and generating a top-level entity now takes multiple seconds, which in turn seems to limit me from using generative testing#2017-10-0911:06gfredericks@jwkoelewijn I'm not familiar with all the entry points, but test.check has a :max-size
option that spec will pass through, that you could set to 50 or 15 depending on what you're doing exactly when the multiple-second thing happens#2017-10-0911:13jwkoelewijncool, will look into that options, thanks!#2017-10-1001:50erichmondIs there a good doc/faq/something on using spec's ability to generate mock data? Generating a string like "001212314" in particular? (meaning, n length, chars consisting of just 0-9?#2017-10-1002:47Alex Miller (Clojure team)spec itself does not really have something like that built in#2017-10-1002:47Alex Miller (Clojure team)you can use fmap to build a set of characters, then apply str to them#2017-10-1002:48Alex Miller (Clojure team)or use the regex gen support in the test.chuck library#2017-10-1003:09erichmondI had been using test.chuck, but was curious if there was a more spec-native solution. thanks, I'll keep going with what I have!#2017-10-1003:31Alex Miller (Clojure team)test.chuck is rad#2017-10-1013:53jwkoelewijnquestion, when i run (clojure.spec.test/check 'merge-users {:clojure.spec.test.check/opts {:num-tests 15 :max-size 2}})
it seems the num-tests option seems to be ignored. Am I passing the options for quick-check wrong? or is this to be expected and should I address my own expectations? š#2017-10-1013:55jwkoelewijnwhere I use the backtick instead of the quote (to help in markdown slack formatting)#2017-10-1014:06taylor(stest/check `foo {:clojure.spec.test.check/opts {:num-tests 1}})
this works for me#2017-10-1015:13borkdudeIn Clojure spec, how could I generate symbols or strings of max length 2 that contain only lowercased alphabetic characters?
This works, but maybe it can be simplified?
(s/def ::varname
(s/with-gen symbol?
#(gen/fmap (fn [[i j l]]
(symbol
(str/lower-case (.substring
(str i j)
0 l))))
(gen/tuple (gen/char-alpha)
(gen/char-alpha)
(gen/choose 1 2)))))
#2017-10-1015:14gfredericksmin length 1?#2017-10-1015:14borkdudeyes#2017-10-1015:14gfredericksI don't think that's terrible#2017-10-1016:28nwjsmith(s/def ::varname
(s/with-gen symbol?
#(gen/fmap (fn [characters]
(symbol
(string/lower-case
(apply str characters))))
(gen/vector (gen/char-alphanumeric)
1
2))))
#2017-10-1017:05jonasWith spec/keys
(or spec in general), whatās the best way to express: One and only one of the keys ::foo
or ::bar
is required?#2017-10-1017:06jonasOr alternatively āEither ::foo
or ::bar
or both is requiredā#2017-10-1017:06bfabry(s/keys :req [(or ::foo ::bar)]) and then and'd with a spec to enforce only one if you want that#2017-10-1017:07jonasThanks, let me try that#2017-10-1017:07bfabryThe :req key vector supports 'and' and 'or' for key groups:
(s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
#2017-10-1017:07jonasI think this is exactly what I was looking for :+1:#2017-10-1020:40borkdude@nwjsmith Thanks š#2017-10-1020:47borkdude@nwjsmith https://github.com/borkdude/aoc2015_day7/commit/936987c86451d9032e28e6151f0dd061ab58a0be#2017-10-1020:48nwjsmithCrazy, I was just reading your blog post. Great stuff!#2017-10-1023:59nwjsmithHow are you folks debugging Couldn't satisfy such-that predicate
errors?#2017-10-1100:00nwjsmithIām having trouble figuring out which generator is causing the error#2017-10-1100:06bfabry@nwjsmith one option would be to walk the registry using s/registry and try and generate an example for each one until you fail#2017-10-1100:06bfabryI believe there's a jira open to figure out a way to add the offending spec to the error message you're seeing#2017-10-1100:07gfrederickss/and
is a common culprite#2017-10-1100:08gfredericksthe way to add the offending spec to the error message is implemented in the latest alpha of test.check, but I doubt has been incorporated into spec. I assume there's an open ticket for that.#2017-10-1100:12nwjsmith@bfabry @gfredericks thanks!#2017-10-1100:12nwjsmithIāve narrowed my current run-in with that error down to this:
(s/def ::header-name
(s/with-gen
(s/and string?
(complement string/blank?)
utilities.string/lower-case?
utilities.string/ascii?)
#(gen/fmap (comp string/lower-case (partial string/join "-"))
(gen/vector (gen/not-empty (gen/string-ascii))
1
100))))
#2017-10-1100:16nwjsmithAh, figured it out#2017-10-1100:16nwjsmithmy utilities.string/ascii?
function is broken#2017-10-1100:18gfredericksphew#2017-10-1107:27msolliIām specing a function that calls another function, stubbing out that second function with instrument
. When Iām doing stest/check
on the function, itās really slow. Takes ~30 s with :num-tests 1000
. With num-tests 100
, itās sub-second. What gives?
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])
(defn fetch-foo-from-somewhere
[a]
[,,,])
(defn get-foo
[b]
(fetch-foo-from-somewhere b))
(s/def ::foo string?)
(s/fdef fetch-foo-from-somewhere
:args (s/cat :a any?)
:ret ::foo)
(s/fdef get-foo
:args (s/cat :b any?)
:ret ::foo)
(stest/instrument `fetch-foo-from-somewhere {:stub #{`fetch-foo-from-somewhere}})
(stest/summarize-results (stest/check `get-foo {:clojure.spec.test.check/opts {:num-tests 1000}}))
#2017-10-1110:58gfredericks@msolli how about {:num-tests 1000 :max-size 100}
?#2017-10-1111:07msolliYeah, that took the time down to ~4 s.#2017-10-1111:09msolliSo I understand :max-size
somehow controls the way generated values āgrowā as the number of tests increase. How does this explain why larger sizes takes so much more time?#2017-10-1111:09gfredericksit's probably this: https://dev.clojure.org/jira/browse/TCHECK-106#2017-10-1111:10gfredericksprobably stemming from the use of any?
in this case#2017-10-1111:12gfredericksI suppose you could argue that the generator underlying any?
should be designed to never get very big#2017-10-1111:16msolliAh, I see. Iām just getting into Spec, and Iām following along to https://clojure.org/guides/spec#_combining_code_check_code_and_code_instrument_code. It seemed strange that the official docs endorse something this slow.#2017-10-1111:17msolliI can confirm that the values generated for any?
really do become huge.#2017-10-1111:20msolliSo I guess itās the GC-ing of those large objects that is the bottleneck.#2017-10-1111:22Alex Miller (Clojure team)There are some tickets and work to do in this area still#2017-10-1111:25Alex Miller (Clojure team)@gfredericks the most critical fix was the bug in s/coll-of that went into the spec release last week actually. That was the cause of one set of problems (not the one here though).#2017-10-1111:27gfredericks@alexmiller this thing? https://github.com/clojure/spec.alpha/commit/739c1af56dae621aedf1bb282025a0d676eff713#2017-10-1111:27gfredericksI can't see anything that looks particularly relevant in the last 6 months of commits#2017-10-1111:28gfredericksoh was it generating a bunch of stuff and throwing it away?#2017-10-1111:28msolliThe conj, well, enjoy yourselves! And thanks for the explanation and all the great work youāre doing.#2017-10-1111:32gfredericks@alexmiller after reading the ticket, I assume CLJ-2103 is what you were referring to
Any thoughts (ha!) about any?
? The underlying gen/any-printable
in test.check could have its size scaled down. I feel like that would make it more useful for almost any conceivable use case.#2017-10-1111:38Alex Miller (Clojure team)Yeah that would be good#2017-10-1111:39Alex Miller (Clojure team)And yes 2103#2017-10-1111:41gfredericksalthough now that I think about it, making sizing better in collections and gen/recursive-gen
might take care of the gen/any-printable
problem#2017-10-1113:30mmerIs there anyway to get the context of a element being validated by a spec. I need to get hold of the overall datastructure that is being validate by a set of specs so that I can do some cross validation in my own predicate?#2017-10-1114:13taylorI commented on your Stack Overflow question too. I think maybe the solution to your problem is to enforce that invariant from your outer spec instead of your inner spec?#2017-10-1114:20taylorposted example on SO#2017-10-1114:32taylor(s/def ::tag string?)
(s/def ::inner (s/keys :req-un [::tag]))
(s/def ::outer
(s/and
(s/keys :req-un [::inner ::tag])
#(= (:tag %) ;; this tag must equal inner tag
(:tag (:inner %)))))
(s/conform ::outer {:tag "y" ;; inner doesn't match outer
:inner {:tag "x"}})
;=> :clojure.spec.alpha/invalid
(s/conform ::outer {:tag "x"
:inner {:tag "x"}})
;=> {:tag "x", :inner {:tag "x"}}
#2017-10-1114:33taylorhttps://stackoverflow.com/questions/46685778/clojure-spec-accessing-data-in-hierarchical-spec/46690677#46690677#2017-10-1115:46mmerThank you - I kind of understand. I need to work through some details.#2017-10-1122:05mmerWhat seems strange is that with the whole conformed model that it is not easier to walk the model. In other words it woul dbe good to be able to apply the equivalent of an xpath to the specs to be able to find associated data and perform link and checks between them#2017-10-1119:13ikitommiHi. Wrote an suggestion of the spec coercion: https://dev.clojure.org/jira/browse/CLJ-2251#2017-10-1121:45dealytrying to spec a function that takes 2 params, a keyword and a set of string (which can be nil), but my spec always fails when a set is passed to the function, how should it be spec'd#2017-10-1121:46dealy(s/fdef cs!
:args (s/cat :mis keyword? :table-names (or nil? (s/coll-of string?))))
is the spec I was using#2017-10-1121:50bfabryyou want (s/or (s/coll-of string? :kind set?) nil?) but really you want (s/nilable (s/coll-of string? :kind set?)#2017-10-1204:24uwoIs there a way to ask of a spec, āwhat keys can you have?ā#2017-10-1204:25uwoI know we can call (s/describe :my/spec)
and get back a form. I would need to parse that s expression to get that answer - which is fine of course#2017-10-1214:48guyHi i've got a newbie spec question.
Should you reuse specs from one ns to another.
Say i have some data that has :name :email and wanted to validate it with spec.
Would i create a ::name ::email in each namespace that checks it, or would i create a spec namespace and import it from there.
Or should i do something else?#2017-10-1215:29taylorYes, itās normal and sometimes necessary to use specs across namespaces. If you have common/shared specs then you should definitely consider defining them in one place and referencing them where theyāre needed.#2017-10-1215:31guyok thanks!#2017-10-1218:51the2bearsHi, I'm starting out with spec and adding it to a small project we have. However, the project uses Incanter and a 'lein compile' now fails.
Caused by: clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec:
{:clojure.spec.alpha/args (matrix "\n Returns a matrix or vector, in a valid core.matrix format. You can use the slices function to\n access the rows.\n\n Equivalent to R's matrix function.\n\n Examples:\n (def A (matrix [[1 2 3] [4 5 6] [7 8 9]])) ; produces a 3x3 matrix\n (def A2 (matrix [1 2 3 4 5 6 7 8 9] 3)) ; produces the same 3x3 matrix\n (def B (matrix [1 2 3 4 5 6 7 8 9])) ; produces a vector with 9 elements\n\n ; since (plus row1 row2) adds the two rows element-by-element\n (reduce plus A) ; produces the sums of the columns\n\n ; and since (sum row1) sums the elements of the row\n (map sum A) ; produces the sums of the rows\n\n " ([data] (m/matrix data)) ([data ncol &] (m/matrix (partition ncol (vectorize data)))) ([init-val rows cols] (m/compute-matrix [rows cols] (constantly init-val))))}
#2017-10-1218:51the2bearsI imagine Incanter has not been updated, but I also don't read the spec errors very well yet.#2017-10-1218:54bfabrywhat version of incanter?#2017-10-1218:54bfabryand do you have more error/stacktrace?#2017-10-1218:54the2bears1.9.0#2017-10-1218:55the2bearsI do have more trace, hold on#2017-10-1219:00the2bearsShoot, that cut off some#2017-10-1219:02guyu cud try putting it in a gist#2017-10-1219:03the2bearsyeah, would be better I think.#2017-10-1219:06bfabryah, yes, bug in incanter#2017-10-1219:06bfabrythis is not a valid args list
([data ^Integer ncol &]
(m/matrix (partition ncol (vectorize data))))
#2017-10-1219:08bfabryI'll open a PR#2017-10-1219:08the2bearsThanks!#2017-10-1219:09bfabryoh actually it's fixed in master#2017-10-1219:09bfabrytry the latest snapshot maybe#2017-10-1219:09the2bearsokay, I can do that for testing this out. Thanks again for the help.#2017-10-1219:09bfabryoh no I'm on the wrong branch#2017-10-1219:09bfabry1s#2017-10-1219:10bfabryyeah fixed on 'develop' branch#2017-10-1219:12the2bearsokay, I'll pull it down and test a local.#2017-10-1309:49mishagreat talk, @gfredericks, thank you!
now we need to persuade you into writing a book on test.check and gen testing#2017-10-1312:03Alex Miller (Clojure team)that is a fantastic idea#2017-10-1501:24codonnellam I missing something about conformers? I'm confused as to why (s/valid? (s/conformer (fn [_] :clojure.spec/invalid)) nil)
is true
. If the conformer function returns :clojure.spec/invalid
, shouldn't the value be invalid?#2017-10-1501:27codonnellah, it should be clojure.spec.alpha/invalid
or just ::s/invalid
:duck:#2017-10-1507:26jumarwhat exactly is the relationship/difference between clojure.spec.gen.alpha
and clojure.test.check.generators
?
So far, I've found two differences:
1. missing nat
generator in clojure.spec.gen.alpha
2. cannot reference generators directly, but has to call them as functions:
e.g. with clojure.test.check.generators I can do this:
(gens/hash-map
:boolean gens/boolean
;; note: `nat` generator is not in spec
:small-integers (gens/vector gens/nat)
:large-integer gens/large-integer
:double gens/double
:color (gens/elements [:red :green :blue])
:uuid gens/uuid
:string-or-keyword (gens/tuple gens/string gens/keyword))
But with clojure.spec.gen.alpha
I have to do this:
(gen/hash-map
:boolean (gen/boolean)
;; note: `nat` generator is not in spec
:small-integers (gen/vector gens/nat)
:large-integer (gen/large-integer)
:double (gen/double)
:color (gen/elements [:red :green :blue])
:uuid (gen/uuid)
:string-or-keyword (gen/tuple (gen/string) (gen/keyword)))
#2017-10-1507:34jumarclojure.test.check.generators/let
is also missing in clojure.spec.gen.alpha
namespace#2017-10-1507:37jumarRelated to the 2.) When I actually try to use (gens/double)
it throws an exception so the usage pattern is a bit inconsistent#2017-10-1513:28gfrederickswhat kind of exception?#2017-10-1518:13jumar1. Unhandled java.lang.ClassCastException
clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn
#2017-10-1518:14jumarI don't have anything against either approach, I just thought that they behave in the same way O:-)#2017-10-1518:37gfredericksoh, you mean between the two namespaces it's inconsistent
yes absolutely, and that's unfortunate#2017-10-1507:51jumarbtw. @gfredericks thanks a lot for the great Conj talk. I missed it at the conference but watched it now. Really useful!#2017-10-1507:58jumarjust a small thing. imho, gen/large-integer
doesn't accept min/max options. I had to use gen/large-integer*
(slide 43).
Funny thing is that if you use clojure.spec.gen.alpha/large-integer
it doens't throw an exception but generates wrong data#2017-10-1513:28gfredericksthat's disappointingly error-prone#2017-10-1513:29gfredericksthe other thing is definitely a slide typo š#2017-10-1513:30gfredericksmy org-modeābeamer workflow is not yet sophisticated enough to automatically verify that the code works#2017-10-1513:48stathissideriswhat do people normally do for specāing var-args#2017-10-1513:48stathissiderisdo they put the whole thing in a s/?
#2017-10-1513:49stathissiderisit feels slightly wrong because that means that itās all or nothing#2017-10-1513:51gfredericks@stathissideris does s/*
not work?#2017-10-1513:54stathissiderisok some context: Iām trying to extend spec-provider to support inference of function specs, and when it comes to varargs, if I see an example of a function call where the varargs happen to be 1 "foo"
Iām not sure if I should infer a spec of (s/cat ... :rest (s/? (s/cat :integer integer? :string string?)))
#2017-10-1513:54stathissideriswhich says that there are up to 2 varargs but theyāre both optional#2017-10-1513:55stathissiderisfeels like itās more restrictive than the function signature#2017-10-1513:55gfredericksif all you know is one example call, I don't think you can know very much at all for sure#2017-10-1513:56stathissiderisagreedā¦#2017-10-1513:56stathissiderisbut even if I get more calls, Iām not sure whatās the ābestā spec to infer#2017-10-1513:56stathissiderisfor example letās say I get one more example where the varargs is just 20
#2017-10-1513:57stathissiderisdoes it then become (s/cat ... :rest (s/? (s/cat :integer integer? :string (s/? string?))))
#2017-10-1513:58stathissiderisor maybe I always wrap the individual var-args with s/?
because theyāre optional#2017-10-1514:01gfredericksthere's no right answer, it's all heuristics#2017-10-1514:01stathissiderisI know š#2017-10-1514:03gfrederickswhich I guess means I have no further advice š those kinds of problems make me just throw up my hands and give up#2017-10-1514:04stathissideristhey are hard, but they make you think hard about idioms and coder expectations#2017-10-1514:05stathissiderisI had to do a lot of that for spec-provider#2017-10-1514:05stathissideris@gfredericks great talk btw#2017-10-1514:05gfredericksthanks#2017-10-1514:54stathissideriswhatās the difference between s/alt
and s/or
?#2017-10-1514:54stathissiderisI saw Ambrose used alt
for multiple arities#2017-10-1514:54stathissiderisIāve always used or
#2017-10-1514:55taylor;; unlike `or`, `alt` works on sequence elements, not on values themselves!
#2017-10-1514:56taylorthe example here https://clojuredocs.org/clojure.spec/alt makes that a little more obvious#2017-10-1515:00stathissiderisok, so itās like with alt you have an implicit s/cat
around each option#2017-10-1515:00stathissiderisok, so itās like with alt you have an implicit s/cat
around each option#2017-10-1515:08taylorI think it's more like all the "regex" spec types (`cat`, alt
, *
, +
, ?
) are specifically meant to specify elements of a sequence. I wouldn't say there's an implicit cat
around alt
or any of the regex specs, they all work on sequences whether you use them alone or in combination#2017-10-1515:11taylorI'm probably missing some context for your use case though? Writing specs for multi-arity functions?#2017-10-1515:18taylorThe fdef
docstring has a nice example:
(s/fdef clojure.core/symbol
:args (s/alt :separate (s/cat :ns string? :n string?)
:str string?
:sym symbol?)
:ret symbol?)
#2017-10-1515:18taylorUsing an alt
case for each arity#2017-10-1515:19stathissiderisok, but apart from conciseness, whatās the benefit over s/or
in this case?#2017-10-1515:19stathissideris(Yes, Iām writing specs for multi-arity functions)#2017-10-1515:22taylor(s/fdef clojure.core/symbol
:args (s/or :separate (s/cat :ns string? :n string?)
:str (s/cat :str string?)
:sym (s/cat :sym symbol?))
:ret symbol?)
#2017-10-1515:22stathissiderisyeah, itās longer, but are there any other disadvantages?#2017-10-1515:27taylorNot sure what other disadvantages there might be. The :args
spec isn't just longer with or
, it's more complex. They also produce slightly different output e.g. when incorrectly calling a instrument
ed function#2017-10-1515:28taylorThe or
version has additional, unnecessary tags in its explanation#2017-10-1515:30taylorfrom the :clojure.spec.alpha/problems
: :path [:args :sym]
vs :path [:args :sym :sym]
#2017-10-1515:36Alex Miller (Clojure team)Regex specs compose together to describe a single sequential context and are generally preferred for describing args#2017-10-1515:37Alex Miller (Clojure team)Generally it's best to use a top level cat as you will then get a map conformed to component names that you can use in the :fn spec#2017-10-1515:38stathissideris@U064X3EF3 but are different arities a āsingle sequential contextā?#2017-10-1515:38Alex Miller (Clojure team)For any given case, yes#2017-10-1515:38Alex Miller (Clojure team)I would actually write that example now as#2017-10-1515:39Alex Miller (Clojure team)Sorry, I can't type it all, on the phone#2017-10-1515:43stathissiderisno problem, would be grateful to have your example when you get some time#2017-10-1515:44Alex Miller (Clojure team)Use alt for alternatives, ? for optional args, * for varargs etc. you can describe any set of arities I've encountered with a regex spec#2017-10-1515:45stathissiderisalright, thanks#2017-10-1515:45stathissiderison a bit of tangent: what about keyword args?#2017-10-1519:34Alex Miller (Clojure team)Use keys* - that's what it's for#2017-10-1608:21stathissiderisok, many thanks!#2017-10-1613:58bbrinckIām unclear on how conformers are intended to be used.#2017-10-1613:59bbrinckIāve seen cases like:
- converting a string into a seq so spec regex operators can validate a string
- converting a string into an int
- converting a string into a UUID#2017-10-1614:01bbrinckBut from my understanding of https://dev.clojure.org/jira/browse/CLJ-2116, conformers arenāt intended to be used for transformation#2017-10-1614:02bbrinckAre the above cases unintended uses of conformers? What would be an intended use case? I ask because Iām trying to figure out how to include support for conformers in Expound#2017-10-1615:31Alex Miller (Clojure team)the intended use case was for writing new composite spec types (like s/keys*)#2017-10-1615:31Alex Miller (Clojure team)so basically not what most people try to use them for#2017-10-1907:41jrychterFWIW, I also use them for things like converting strings to keywords in JSON.#2017-10-1907:42jrychterI think that while this might not have been the intended usage, it shows that there is a need here. I am having hopes that Rich will take this on as something to think about in the future.#2017-10-2322:13bbrinckFor what itās worth, using conformers in this way also makes it harder to give good error messages. Iām not sure what the best solution is, but perhaps two phases where first the data is validated, then transformed#2017-10-1614:03mpenetto give you information about what you are validating for reuse "later", this makes more sense in the context of validating/destructuring data passed to a macro such as defn#2017-10-1614:05bbrinckDo you happen to have an example? My main source of examples of macros specs is https://github.com/clojure/core.specs.alpha, but I donāt see any conformers there#2017-10-1614:07mpenetmy bad I read conforming and I got carried away#2017-10-1614:10mpenetI am guilty of using it for transformation, not sure my example is clean but here you go: we have a dsl a bit like sql for a rule engine, we validate the syntax with spec and use a conformer to spill the ast (parsed form)#2017-10-1614:14bbrinck@mpenet Hm, that is interesting. Is the DSL originally a string then? Do you use spec to parse the string?#2017-10-1614:14mpenetalso guilty of using it do to json transforms, "old" stuff#2017-10-1614:14mpenetyes#2017-10-1614:14mpenetthe spec is basically a conformer that wraps the parse fn#2017-10-1614:14bbrinckRight#2017-10-1614:15mpenetin reality it s a lot hairier because we have custom explain and whatnot but that s the idea#2017-10-1614:15bbrinckI see, thanks for the example. One reason itās hard to add support for conformers is that a) Iām unclear on the intended use and b) in practice, it seems like they are often uses for transformation. Iām not sure if itās practical to not support a common use case.#2017-10-1614:15bbrinckRight#2017-10-1614:16mpenetit s made for transformations, I guess their validity depends on the context#2017-10-1614:21bbrinckThis is the part I donāt fully understand: conformers do transform values, but what are examples of recommended transformations? @alexmiller said: āI donāt think we are interested in turning spec into a transformation engine via conformersā, but Iām not sure what that includes/excludes#2017-10-1614:22mpenetit's a good question indeed#2017-10-1614:24mpenetneither the guide or rationale pages mention it#2017-10-1614:24bbrinck@mpenet In any case, I appreciate you explaining how you use them in practice. This is very helpful for my research, since I may end up wanting to support different use cases#2017-10-1614:25mpenetyou're welcome. love expound. thanks for that#2017-10-1615:32Alex Miller (Clojure team)the guide page intentionally does not mention conformers as we consider them to be primarily useful for writing new custom composite spec types (not for general data transformation)#2017-10-1615:44bbrinckBy ācomposite spec typesā do you mean new spec types that work similar to or
or regex specs? i.e. where I would want to name the parts?#2017-10-1619:11bbrinckWhoops, sorry I missed your comment addressing exactly this question in the thread above. Thanks!#2017-10-2010:31carocad@alexmiller would you be so kind to put an example of the āexpected useā in
? š
Currently only the āunintented useā is documented so I guess that this leads people in the wrong direction (and might continue to)
http://clojuredocs.org/clojure.spec/conformer#2017-10-1615:33Alex Miller (Clojure team)I think having a tool for transforming data informed by specs would be awesome. I donāt think conformers are that tool.#2017-10-1617:24ikitommi@alexmiller any ideas how could we find such a tool? It looks like we could do that with conformers, with just small changes to how it works. Or is there something new coming from Rich?#2017-10-1615:50danielnealI'm trying to use spec to document a map that contains a :key-fn
that returns a collection
i.e. on this data {:aliases #{:bob :robert}}
, {:key-fn :aliases}
would be suitable.
However this happens
(s/def ::key-fn
(s/fspec :args (s/cat :item any?)
:ret (s/coll-of any?)))
(s/def ::my-map (s/keys :req [::key-fn]))
(s/valid? ::my-map {::key-fn :aliases}) ;;false
(s/valid? ::key-fn :aliases) ;; false
Am I doing this all wrong?#2017-10-1615:50danielnealI'm trying to use spec to document a map that contains a :key-fn
that returns a collection
i.e. on this data {:aliases #{:bob :robert}}
, {:key-fn :aliases}
would be suitable.
However this happens
(s/def ::key-fn
(s/fspec :args (s/cat :item any?)
:ret (s/coll-of any?)))
(s/def ::my-map (s/keys :req [::key-fn]))
(s/valid? ::my-map {::key-fn :aliases}) ;;false
(s/valid? ::key-fn :aliases) ;; false
Am I doing this all wrong?#2017-10-1615:56taylorI think the problem is that a keyword (`:aliases`) wonāt conform to your fspec
#2017-10-1615:57taylorYou could add an s/or
case for keywords in your ::key-fn
spec#2017-10-1615:57danielneal#2017-10-1615:57taylor(s/or :kw keyword? :fn (s/fspec ...))
#2017-10-1615:58danielnealthe thing is it's the fact that the function returns a collection that is the important thing I'm trying to capture#2017-10-1615:58danielnealI wouldn't mind if I have to help it along with a generator or something#2017-10-1615:59danielnealbut I don't know where or how or if I'm approaching the whole thing in a stupid way#2017-10-1616:00danielnealoh no wait#2017-10-1616:00danielnealthe inlined function with constants worked#2017-10-1616:01danielnealI would like to specify that if you're gonna give a keyword it's got to look up data that is a collection#2017-10-1616:05taylorhmm maybe need another approach if you need that guaranteeā¦ a keyword could return anything that happens to be in the map#2017-10-1616:08danielnealmm yes#2017-10-1616:09danielnealas far as I can make out specifying the return types of functions <as> parameters is a bit of an edge case#2017-10-1616:09danielnealbut would love to be able to provide it to help readers understand the contract#2017-10-1616:10taylorso I guess youād need more context if you were going to validate that the supplied keyword would return a collection; youād need to know 1) what the keyword is and 2) the value for that keyword in the other map itād be applied to?#2017-10-1616:14danielnealyeah you're right#2017-10-1616:14danielnealI suppose I could just put it in the doc string#2017-10-1616:15taylorone weird idea: if you had all this info (including the āsubjectā map) in one data structure, you could write a spec for the whole structure to ensure any ::key-fn
refers to a collection value in the āsubjectā map#2017-10-1616:15danielnealooh interesting#2017-10-1616:32taylorAnyway, the return value spec for your function isnāt going to matter for valid?
. Itād be useful if you were using test.check to check the function though #2017-10-1620:38akhudekIs there a way to write a spec that checks properties of pairs of elements in a sequence? E.g. in [{:a 1 :b 2} {:a 2 :b 3} {:a 3 :b 4}]
Iād like to check that :b
is equal to :a
for each pair of items from the sequence.#2017-10-1620:38akhudekI can do this as a function on the sequence.#2017-10-1620:39akhudekBut then on failure explain returns the entire sequence as the value causing the problem.#2017-10-1620:39akhudekerm, let me fix that#2017-10-1620:40ghadiyou can do it but not as a function on the sequence, but each element of it#2017-10-1620:40ghadiunless I understand you wrong#2017-10-1620:41ghadiDo you mean there is some relationship between adjacent maps? or within individual maps?#2017-10-1620:42akhudekI mean that for {:a 1 :b 2}
and {:a 2 :b 3}
:b
of the first and :a
of the second must match, and then for {:a 2 :b 3}
and {:a 3 :b 4}
, we have the same check and so on#2017-10-1620:42ghadigotcha#2017-10-1620:42ghadiyou have a tradeoff here#2017-10-1620:43ghadiYou can individually check pairs by running (map #(s/valid? ....) (partition 2 1 collection))
#2017-10-1620:43ghadiiff you want more targeted error messages#2017-10-1620:44ghadiotherwise you can continue to check the whole collection and get errors at the collection level#2017-10-1620:44ghadidepends on what you want#2017-10-1620:44akhudekok, thanks, thatās what I thought but wanted to make sure there wasnāt some other way š#2017-10-1620:44ghadiI don't believe that (at this time) there is a have-your-cake-and-eat-it-too...#2017-10-1705:51talgiatAre there ways to validate a form with spec, with user friendly error messages (not developer friendly). Iāve found this: https://github.com/alexanderkiel/phrase but I donāt want to do field by field validation but validate a whole spec that is a map with field dependent error messages.#2017-10-1713:54bbrinckCan you talk more about the map of fields? Do you have an example?#2017-10-1706:24seancorfield@talgiat See if this might be closer to what you want https://github.com/bhb/expound#2017-10-1712:07tatuthow do I override a generator for something thats deep within some nested maps, without doing s/def on that thing?#2017-10-1712:07tatutam I missing something#2017-10-1713:36Alex Miller (Clojure team)You can override by path#2017-10-1713:37Alex Miller (Clojure team)But it is tricky to find exactly the right path to use #2017-10-1722:24taylordo you need the same spec to work for both those test cases, or just the first?#2017-10-1722:29taylorclojure
(s/def ::field
(s/cat :field-name string?
:field-type #{'Currency 'Text}))
(s/def ::entity
(s/cat :entity-name keyword?
:fields (s/+ (s/spec ::field))))
(s/valid? ::entity '(:columns ("Price" Currency) ("Description" Text)))
;;=> true
#2017-10-1722:31taylorcould replace s/+
with s/*
if you donāt require anything after the initial keyword#2017-10-1722:32fentonJust the first#2017-10-1722:33fentonOk I'll try that thanks#2017-10-1809:59joost-diepenmaathow can I combine two specs like s/or
but without the labels so it conforms as a single value?#2017-10-1810:14andrewmcveigh@joost-diepenmaat you could wrap it in a s/nonconforming
spec https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1761#2017-10-1810:17joost-diepenmaatthatās interesting. thanks @andrewmcveigh#2017-10-1817:39seancorfield@joost-diepenmaat bear in mind that s/nonconforming
is undocumented and may disappear. I believe the Clojure/core folks have said the most likely "solution" to this is that they may provide a specific non-conforming version of s/or
, rather than exposing general non-conforming machinery... /cc @andrewmcveigh#2017-10-1817:47ikitommicalling s/unform
on the conformed value will also remove the branching info.#2017-10-1817:49ikitommis/nonconforming
also stops all conforming, so the conformers on the or branches are not run.#2017-10-1817:50joost-diepenmaatI could possibly use a multi spec. For now I could make it work with a single spec #2017-10-1907:46jrychterI am having problems with circular dependencies. Spec strongly encourages splitting your data entities into separate namespaces. I have projects (`project`) which contain entries (`entry`). But entries also refer back to :project/id
via a :entry/project-id
attribute. If I split entries into their own namespace, I have to refer to it when defining what a project
is, but the entry namespace also needs to know what a :project/id
is.#2017-10-1914:01Alex Miller (Clojure team)the qualifiers used in spec names do not have to correlate to an actual namespace nor do they have to be defined in that namespace. The word ānamespaceā is dangerously overloaded in this area.#2017-10-1914:09Alex Miller (Clojure team)I think itās most useful to declare stuff like this in some totally different ādomain dataā namespace#2017-10-1915:36jrychterInteresting. But if they don't correlate to namespaces, I lose the (rather convenient) ::
syntax. I guess the namespaced map reader syntax could make up for that.#2017-10-1915:45bbrinckschec lets you create aliases to non-existing namespaces, and then you could use the alias to shorten writing specs#2017-10-1915:45bbrinckhttps://github.com/gfredericks/schpec#things-it-has#2017-10-1917:30Alex Miller (Clojure team)you can do this without a library too#2017-10-1917:31Alex Miller (Clojure team)(create-ns 'my.cool.thing)
(alias 't 'my.cool.thing)
::t/foo
#2017-10-1917:32Alex Miller (Clojure team)support for handling keyword aliasing in better ways (without doing that ^^) is something Rich is working on for future release btw#2017-10-1907:53jrychterAlso, I'm not sure about the new reader syntax for namespaced maps, specifically how it should work with destructuring:
(let [{a :some.ns/a} {:some.ns/a 1}] a) => 1
(let [#:some.ns{:keys [a]} {:some.ns/a 1}] a) => 1
(let [#:some.ns{a :a} {:some.ns/a 1}] a) => error
Is map destructuring with key renaming not supported?#2017-10-1911:15gfredericksa problem with that last one is that the binding form is read in as {some.ns/a :a}
which is sort of backwards from what you want, because the symbol gets namespaced but not the key#2017-10-1912:29jrychter@gfredericks Right. But what now, do we give up key renaming? Is that expected/intended?#2017-10-1912:32gfredericksyou can do it the long way, at worst#2017-10-1912:34gfredericksI have no idea whether there was any intention to make something like this work#2017-10-1912:36jrychterHmm. In that case, the push to use namespaced keys everywhere seems slightly premature.#2017-10-1913:57Alex Miller (Clojure team)thatās not valid syntax#2017-10-1913:58Alex Miller (Clojure team)the āleftā part of destructuring is always the unqualified name of the local being bound#2017-10-1913:59Alex Miller (Clojure team)so you want: (let [{a :some.ns/a} {:some.ns/a 1}] a)
#2017-10-1913:59Alex Miller (Clojure team)(this has worked since Clojure 1.6 I believe)#2017-10-1914:11jrychter@alexmiller I am moving to ns-qualified keys everywhere and have many places where I do {:keys [a b c d e f]}
. I was hoping I could do #:some.ns{:keys [a b c d e f]}
and this works well, but what if I additionally want to extract {renamed-g :some.ns/g}
?#2017-10-1914:11Alex Miller (Clojure team)you can do both#2017-10-1914:12Alex Miller (Clojure team){:some.ns/keys [a b c d e f], renamed-g :some.ns/g}
#2017-10-1914:13Alex Miller (Clojure team)I think using the #:some.ns
prefix is kind of confusing in this context - I usually put it inside the map#2017-10-1915:31jrychter@alexmiller Oh, so that's the syntax! Thank you, I've never seen it until now.#2017-10-1915:49Alex Miller (Clojure team)Means the same thing - just alternate syntax#2017-10-1915:50Alex Miller (Clojure team)You can do multiple keys destructuring from different namespaces too#2017-10-1916:44jrychterThank you for your time and help!#2017-10-1918:58radscan anyone explain to me why spec doesn't check :ret
specs during instrumentation by default? I know there was a discussion about it a while back, but I'm curious if anyone has a succinct explanation for the decision#2017-10-1919:09bbrinckIIRC, the rationale is that instrumentation is for checking that you called function X correctly, not for checking that the function X implemented correctly#2017-10-1919:09bbrinckThe recommendation is to use generative testing to confirm that function X works correctly (based on the :ret
and :fn
specs)#2017-10-1919:10bbrinckI personally disagree with that distinction, since I have functions that canāt be realistically verified with generative testing due to a) side effects or b) disproportional work to get the specs perfectly tuned#2017-10-1919:10bbrinckWhich is why I use https://github.com/jeaye/orchestra š#2017-10-1919:12bbrinckIn my limited experience, instead of using check
to test function X, we ended up just not speccing :ret
or :fn
because they were unverified and I preferred no documentation rather than documentation that was potentially incorrect or out of date#2017-10-1919:16bbrinckIn particular, using orchestra to instrument functions during non-generative testing gives me confidence that my :ret
and :fn
functions are correct, which makes them more valuable as documentation#2017-10-1919:19radsthanks, that's very helpful#2017-10-1921:50eriktjacobsen@alexmiller Is there any plan to allow multiple, or nested namespaces in spec? Example: A spec ::bar/type
declared in the foo.core
namspace that could be referenced as :foo.core/bar/type
? We are currently struggling with a seeming limitation of spec, and while the underlying problems deserves a longer form post, before heading down that path, was just curious if this was something being thought about.#2017-10-1922:03seancorfieldA spec of ::bar/type
would need a namespace alias of bar
available in order to be legal. That could alias any namespace @eriktjacobsen but could be foo.core.bar
if you wanted it to be. Then ::bar/type
would be :foo.core.bar/type
.#2017-10-1922:04seancorfieldSince namespaces are (supposed to be) globally unique and can be arbitrarily "nested" already, I'm not sure what you're asking for...#2017-10-1922:16eriktjacobsenThank you Sean. I realize what the current system provides, and how to work around it. I was mostly curious if that specific solution has been talked about... will write up an explanation in a bit.#2017-10-1922:21Alex Miller (Clojure team)There is no plan (or need) for any kind of nesting#2017-10-1922:21eriktjacobsenThanks for answer. put another way, has there been any thought to allowing a component of a spec identifier that isn't tied to a namespace or part of the unqualified key#2017-10-1922:22Alex Miller (Clojure team)No, why?#2017-10-1922:23Alex Miller (Clojure team)That is, please help me understand the problem you're trying to solve#2017-10-1922:23eriktjacobsenwriting the why up now, mostly due to tradeoffs in workflow my team is seeing. Perhaps it just needs a better framework for how we're structuring things, but it could also be solved by what I mentioned.#2017-10-1922:25Alex Miller (Clojure team)I think sometimes it's also unclear that keyword qualifiers can refer to Clojure namespaces but don't have to have any relation to them#2017-10-1922:25Alex Miller (Clojure team)There is an unfortunate overload and common use for the word "namespace"#2017-10-1922:27Alex Miller (Clojure team)Currently namespace aliases also have the unfortunate downside of being tied to the particular case where a qualifier is actually a namespace (I do not expect this to be as constrained in the future)#2017-10-2000:03eriktjacobsen@alexmiller Thanks for the quick answers. Explained: https://redd.it/77if23 If the answer is "you need a separate namespace per data type with colliding keywords", so be it.... just wanted to have the discussion.#2017-10-2000:42seancorfield@eriktjacobsen Yes, the answer is to provide unique namespace-qualifiers for each of the colliding keywords -- remember that those namespaces do not have to exist, but they should have names that represent the different concepts being modeled.#2017-10-2000:47seancorfieldThe whole point of namespaces is to provide unique ways to name things that are distinct (but otherwise have the same unqualified name).#2017-10-2001:01eriktjacobsenRight, it's just unfortunate to lose the relationship of between using the spec and knowing where it's defined, losing ability to use the ::
and aliases (until patched), and other limitations mentioned. I understand that might be the answer, its just my team is unhappy if that's the final answer.#2017-10-2001:07gfredericks:foo.bar/baz.hullabaloo
is a valid keyword I think#2017-10-2001:07gfredericksI don't suppose it's likely to make anybody happy though#2017-10-2001:09eriktjacobsenCorrect, but when used in a (spec.keys :req-un)
it will require the keyword to be :baz.hullabaloo
, correct? Rather than :hullabaloo
. That is basically the same solution as my first proposal, just with .
instead of /
#2017-10-2001:11gfredericksyep. it's just sadness all over#2017-10-2001:16eriktjacobsenit does seem like differentiation between specspace and namespace seem slightly confused, or dare I say complected.#2017-10-2001:26gfredericksOTOH it would be weird and confusing to have many parallel namespace systems
e.g. those lisps that keep functions separate from the other kind of thing#2017-10-2001:10potetmWhy use :req-un
for a namespaced key?#2017-10-2001:13eriktjacobsenBecause the incoming data I'm trying to spec is generally using unqualified keys. Such as from parsing json.... I suppose I could add a step before doing spec validation that transforms all the unqualified keys into qualified keys using some mapping scheme, validates with spec, then unqualifies them again for usage downstream, but that's basically the functionality i'm looking for from spec and seems like a lot of overhead#2017-10-2001:33potetmAh, okay. It sounded like you had control over the keyword. That makes sense though.#2017-10-2007:03jrychter@eriktjacobsen FWIW, I am struggling with similar issues. I tried (briefly) using spec keywords in "fictional" namespaces (e.g. not corresponding to clojure ns), but then I end up with an explosion of long namespaced keywords. Lots of typing, lots of things to read and process when looking at code. The ::
really helps in reducing that.#2017-10-2007:05jrychterSomewhat related: I'm also thinking about dropping :req-un
and moving to fully-qualified keys everywhere. But there is a price to be paid in storage (JSON database), transmission (websocket trafic isn't compressed, Transit helps but I'm not sure to what extent), code complexity (need to include keyword namespaces in all destructuring code), and ClojureScript code building keywords from strings. I'm not sure what that price is.#2017-10-2008:05mpenetif you throw in some utils it's not so bad#2017-10-2008:07mpeneti use a macro that creates "relative" aliased ns and tend to spec maps using it following a pattern like ::map ::map/foo ::map/bar ::map/baz etc#2017-10-2008:07mpenet(defmacro rel-ns [k]
`(alias k (create-ns (symbol (str k))))))#2017-10-2008:10mpenetyou can just do (rel-ns 'foo.bar.baz.bad) if deeply nested and then use it as ::foo.bar.baz.bad/xxx, it's quite readable imo#2017-10-2018:06eriktjacobsenThank you! We have thought of similar workarounds, mostly wanted to bring the discussion up for a solution from the core.specs team, though we could incorporate this if we have to workaround it.#2017-10-2016:45metametadataHi! Given ::allowed-val
"enum spec", is there a way to print all the allowed values in the error message? Currently it prints (into #{} allowed-vals)
.
And let's consider it's the requirement that I need allowed-vals
extracted as a var because I need to iterate over it in other places.
(def allowed-vals [1 2 3])
(s/def ::allowed-val (into #{} allowed-vals))
(s/explain ::allowed-val 5)
=>
val: 5 fails spec: :cljs.user/allowed-val predicate: (into #{} allowed-vals)
:cljs.spec.alpha/spec :cljs.user/allowed-val
:cljs.spec.alpha/value 5
#2017-10-2017:24hiredmanuser=> (s/def ::foo #{:a :b :c})
:user/foo
user=> (s/form ::foo)
#{:c :b :a}
user=> (doseq [i (s/form ::foo)] (prn i))
:c
:b
:a
nil
user=>
#2017-10-2017:49metametadatayeah, it's a bit backwards but should work, thanks#2017-10-2020:04ajsThere is no forwards or backwards with a set, right? Unordered. #2017-10-2100:03metametadataright š I meant that it's backwards in a sense that I'd like the variable to be the source of allowed values instead of putting them into spec#2017-10-2018:13metametadataand what if the enum values cannot be hardcoded like that in spec? e.g. when I need to read them from file#2017-10-2018:46ghadiright now calling eval
to build specs is probably the best choice, but work is apparently underway on making spec more "programmable" -- make specs from external stuff#2017-10-2020:04bsimahow would I spec a function with variadic keyword args like (myfn :a 1 :b 2)
? Iām trying to use (spec/* (spec/keys :req-un [::a ::b]))
but thatās not right#2017-10-2020:04taylorkeys*
#2017-10-2020:05taylorhttp://clojuredocs.org/clojure.spec/keys*#2017-10-2020:06bsimaah thanks#2017-10-2022:19falakCan somebody help me out with writing a fdef
for a multi-method using defmulti
?#2017-10-2022:22taylorI might be able to in a little while. Do you have an example?#2017-10-2022:26falakI have a function which takes 2 maps as input and returns a vector with 2 maps.
defmulti my-function (fn [map1 map2] (:type map2)
defmethod my-function :type1
[map1 map2]
(let ....
...
...
...)
[ret-map1 ret-map2]
defmethod my-function :type1
[map1 map2]
.
.
.
#2017-10-2022:31falakThis is what I tried for fdef
-
(s/fdef my-function
:args (s/cat :map1 map? :map2 map?)
:ret vector?)
#2017-10-2023:02taylorNot sure exactly whatās not working but this works for me:
(defmulti my-function (fn [map1 map2] (:type map2)))
(defmethod my-function :type1 [map1 map2]
[map1 map2])
(defmethod my-function :type2 [map1 map2]
[map2 map1])
(s/fdef my-function
:args (s/cat :map1 map? :map2 map?)
:ret vector?)
(stest/instrument `my-function)
#2017-10-2100:11falakSo how do I test the dispatch-value
of the dispatch-fn
in the defmulti
definition?#2017-10-2102:55James VickersI'm trying to make a spec for a function that takes in a map with un-qualified keys (I know how to do this with qualified keywords, just use s/keys). The function arguments look like this: [{:keys [interest term balance] :as m}]
, and for that I make a function spec with (s/cat :m (s/keys :req-un [::interest ::term ::balance]))
- I have specs named ::interest
,`::term`, ::balance
. Is there a more direct way to make a s/fdef
spec for :args
? I don't use the whole map (`:m`) in the function body and only added that part to use in the spec. I tried the example in the clojure.spec documentation for s/keys
with unqualified keywords but couldn't get it to work. Alternatively, am I just way off by using un-qualified keywords as keys in the map the function takes?#2017-10-2102:55James VickersI'm trying to make a spec for a function that takes in a map with un-qualified keys (I know how to do this with qualified keywords, just use s/keys). The function arguments look like this: [{:keys [interest term balance] :as m}]
, and for that I make a function spec with (s/cat :m (s/keys :req-un [::interest ::term ::balance]))
- I have specs named ::interest
,`::term`, ::balance
. Is there a more direct way to make a s/fdef
spec for :args
? I don't use the whole map (`:m`) in the function body and only added that part to use in the spec. I tried the example in the clojure.spec documentation for s/keys
with unqualified keywords but couldn't get it to work. Alternatively, am I just way off by using un-qualified keywords as keys in the map the function takes?#2017-10-2102:57taylorthe :as m
in the function argslist shouldnāt matter to spec, you should be able to remove it if youāre not using it#2017-10-2103:01James VickersThanks for answering so fast. So s/cat
needs key/pred argument pairs, so if I remove :m
from the definition of the function spec, what would I put instead?#2017-10-2103:07taylorthe :m
in your function spec doesnāt have any relation to the :as m
in your arglist, it could be named anything really#2017-10-2103:07taylors/cat
requires each āelementā be tagged with some keyword though, so you canāt omit it. The actual keyword name doesnāt matter so much unless youāre interested in the conformed version of it i.e. it could be (s/cat :tgif ::map-spec)
and itād still work#2017-10-2103:07taylorit looks like youāre doing it right#2017-10-2103:10taylorif it doesnāt work when you remove :as m
from the fn arglist, something else must be wrong#2017-10-2103:11James VickersYou are totally correct - took out :as m
in the function args list, still works. Can change :m
to any keyword (e.g. :foo
) in the :args
spec and that works too. But that's so weird! In this usage, I guess spec doesn't use the keyword arg for this spec?#2017-10-2103:13James VickersThat's weird. I wonder if there'll be other functions coming in spec like s/cat
that don't have that setup, it seems weird that for this case (which seems common?), it doesn't matter what keyword you put there - just a placeholder. Thanks!#2017-10-2103:14taylors/cat
does use the keyword to tag conformed outputs, it just doesnāt matter for your use case#2017-10-2103:16taylor(s/def ::opts (s/* (s/cat :opt keyword? :val boolean?)))
(s/conform ::opts [:silent? false :verbose true])
;;=> [{:opt :silent?, :val false} {:opt :verbose, :val true}]
example from spec guide, notice the output has :opt
and :val
ātagsā#2017-10-2103:23James VickersI guess the reason I didn't understand is I don't quite get s/cat and the other regex ops in spec.#2017-10-2104:57seancorfield@jamesvickers19515 So your function has one arg, a hash map? (s/cat :m ::account)
perhaps, with (s/def ::account (s/keys :req-un [::interest ::term ::balance]))
#2017-10-2105:10James VickersThanks @seancorfield. @taylor showed me that when using s/cat
in this instance, the first argument actually didn't matter - could be :foo
for all it mattered.#2017-10-2105:10seancorfieldRight, s/cat
takes a sequence of (whatever) argument names and specs.#2017-10-2105:11seancorfieldIt's convention to use the same (keyword) name for each argument as the function but there's no reason to.#2017-10-2105:11seancorfieldIn your case, you have a destructuring as the first (only) argument so its name is somewhat arbitrary anyway.#2017-10-2105:12James VickersI guess what was surprising was that there wasn't a spec function for this case that doesn't require the unused keyword as the first arg - I was under the impression that functions that take a single map were common in Clojure.#2017-10-2105:13seancorfieldYes... not sure what you're asking...#2017-10-2105:13James VickersSorry, wasn't a question š just a comment#2017-10-2105:14seancorfield(s/cat ...)
is how you specify an argument list.#2017-10-2105:18seancorfieldSpecs name things, as part of their conformance. The names don't have to correspond to anything in the source code (`s/or` is a good example, s/cat
is similar).#2017-10-2105:23James VickersI think I sort of see now. I did something like this at the REPL:
(s/def ::account (s/cat :m (s/keys :req-un [::interest ::term ::balance])))
(s/conform ::account [{:interest 4.25 :term 360 :balance 261250}])
=> {:m {:interest 4.25, :term 360, :balance 261250}}
#2017-10-2105:27seancorfieldYeah, argument lists are sequences. s/cat
matches a sequence of (named) specs. So, in this case :m
is the name and the s/keys
is the spec for it.#2017-10-2105:27James VickersAnd this call to valid? with a vector that looks like the function signature:
(s/valid? ::account [{:interest 4.25 :term 360 :balance 261250}])
=> true
#2017-10-2105:27seancorfieldRight.#2017-10-2105:31James VickersThanks, I think I understand s/cat
better after playing with it in the REPL a bit.#2017-10-2300:11Drew Verlee@alexmiller in the clojure spec workshop you said you wouldnt use constantly with with-gen
in this example:
(def id-gen
(gen/fmap #(str "ID-" %)
(s/gen (s/int-in 100000 999999))))
(s/def ::id
(s/with-gen (s/and string? #(re-matches id-regex %))
(constantly id-gen)))
what was your advice on that?#2017-10-2319:46Alex Miller (Clojure team)Well first, nothing actually wrong with this example. But I think I would prefer (fn [] id-gen)
now.#2017-10-2300:11Drew Verleeugh, i shouldn't ping ppl at 8pm on sunday. got excited.#2017-10-2302:51seancorfield@drewverlee It's OK to be excited at weekends š Just needs to be tempered with a little patience until most folks get to their desks on Monday morning š#2017-10-2315:44mmerIssues with : Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs.alpha/bindings at: [:args :bindings :init-expr] predicate: any?, Insufficient input - Should I be using clojure.spec or clojure.spec.alpha in my namespace?#2017-10-2315:45taylorif youāre using the latest version, the namespace is clojure.spec.alpha
#2017-10-2315:46mmerThanks - but ...why am I seeing this error?#2017-10-2315:47taylorcan you post a code snippet of your let
?#2017-10-2315:51mmerThanks Taylor - I had been getting errors all the time just when I added the clojure.alpha module to my project that I assumed the spec error was in spec not my code! But of course it was my code!#2017-10-2317:53hiredmanI am using spec as a parser, and have just noticed that when I parse a seq with 40 entries (all already realized in memory) I get out of memory errors, any ideas how to avoid this?#2017-10-2317:55bfabryseems weird, lots of alternates?#2017-10-2317:57hiredman2 alternates with 2 and 4 cases#2017-10-2317:58hiredmanI haven't played with changing how the spec is structured yet to see if that improves things#2017-10-2317:59hiredmanI do use s/&
a few times, which I can imagine being expensive for backtracking#2017-10-2318:48ghadi@hiredman any way you can file an issue?#2017-10-2318:49hiredmanmaybe#2017-10-2319:43Alex Miller (Clojure team)donāt know that I can help without a repro case or more info#2017-10-2322:12Drew VerleeAny idea how you can get the keys from a spec?
(s/def ::foo (s/cat :name ::name))
(??? ::foo)
=> [:name]
#2017-10-2322:14bfabry@drewverlee there's a few different options. s/form will give you the spec definition as data that you could walk#2017-10-2322:32Alex Miller (Clojure team)Spec form specs will eventually make this easier to answer (see clj-2112)#2017-10-2322:53hiredmanwhile I was fiddling with coming up with a minimal test case, I realized I could change the format slightly to have an explicit delimiter instead of an (s/& ...)
check, which seems to have solved my performance issues#2017-10-2408:52msolliWhere is a good place to put (test/instrument)
so that all my specāed functions are instrumented in dev? I have a fairly standard Luminus-based webapp.#2017-10-2411:25stathissideris@drewverlee I had to do this for the spectacles library, see here: https://github.com/stathissideris/spectacles/blob/master/src/spectacles/impl.clj#L4-L36#2017-10-2411:55ikitommi@stathissideris @drewverlee there is also a parse-spec
in spec-tools doing about that: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/parse_test.cljc#L28-L39#2017-10-2411:55ikitommispec parsing is the new black š#2017-10-2411:58stathissideris@ikitommi do you think we could use spec to parse spec forms or would reality collapse under the weight of self-reference? š#2017-10-2412:01ikitommi@stathissideris https://dev.clojure.org/jira/browse/CLJ-2112 ?#2017-10-2412:01stathissiderisway ahead of us š#2017-10-2412:07ikitommibut even when the spec-of-specs ships, we still need to utilities to do something with the parsed data, e.g. collect keys out of (s/merge ::a-map ::another-map)
, spec-tools is one bin collecting this kind of stuff. And there are many others.#2017-10-2414:29taylorIām curious about this too#2017-10-2414:43andre.stylianosso am I#2017-10-2415:17jeayeI'm just using re-frame, not Luminus, but it's in my-app.core
behind a macro.#2017-10-2415:23jeayehttps://gist.github.com/jeaye/9a57740fc474a60a050ae68acf714c4d#2017-10-2415:24jeayeThat'll do the trick, allowing you to specify bits in your leiningen profiles and then conditionally bring in code by reading the profile using environ in a macro.#2017-10-2415:24taylorthanks, the only caveat is that any namespaces you want to instrument must have already been loaded before thatās called, right?#2017-10-2415:24jeayeThat's right.#2017-10-2415:25jeayeI don't think you can get around that though.#2017-10-2415:26jeayeOur my-app.core
requires every model ns we have. Our my-app.core-views
requires every view ns we have, as well as requiring my-app.core
(so view code isn't needed for unit testing).#2017-10-2415:26taylorI wouldnāt imagine so, was just curious what other peopleās approaches to that might be. I was thinking another approach would be to just put the (conditional) instrument
calls at the bottom of each specād namespace#2017-10-2415:26jeayeHm, that would be every single ns for us, which seems less manageable. Sounds like it would work though, if that's your preference.#2017-10-2418:00msolliThanks, @U4986ECDQ, seems like a reasonable solution. Those are some handy macros! š#2017-10-2415:55tony.kayI have a project where I use tools-ns to reload code. If I donāt carefully set refresh dirs then suddenly defn
will start causing spec failuresā¦as in defn-
doesnāt conform to spec. Is this a known issue? Narrowing refresh dirs seems to fix it.#2017-10-2512:53Alex Miller (Clojure team)I havenāt heard of that before#2017-10-2513:22robert-stuttafordis the right way to spec a map that must have at least one of several keys (s/or (s/keys :req []) (s/keys :req []) (s/keys :req []))
?#2017-10-2513:22tayloryou can use a single keys
spec with or
inside of the :req
vector#2017-10-2513:22robert-stuttaford(s/keys :opt [])
says 0..n. i want 1..n#2017-10-2513:22tayloroh sorry, misread#2017-10-2513:23mpenetsounds ok, but if they are sharing the same context and have something like a :type key it might make sense to make a multi-spec out of it#2017-10-2513:24robert-stuttafordit truly is a situation where you can provide 1, 2 or 3 of the keys, but providing 0 of them makes the whole map redundant#2017-10-2513:25robert-stuttafordmodelling a rule system where there are 3 possible outcomes - no-value-yet, yes, or no. you have to model at least one outcome, or thereās no point to testing for the rule#2017-10-2513:26taylorthose three states should be mutually exclusive?#2017-10-2513:27robert-stuttafordthe rule can be reused for multiple cases in a doc#2017-10-2513:27robert-stuttafordsome may 0, some may 1, some may 2#2017-10-2513:27robert-stuttafordiāll try s/or + s/keys !#2017-10-2513:33robert-stuttaford@taylor you were right!#2017-10-2513:33robert-stuttaford(s/keys :req [(or :a :b :c)])
, plus the fact that all ks are validated gives me what i need#2017-10-2514:17ajslooking at this example from @cemerick do i read correctly that specs default to passing valid? even if the namespaced keyword was never def'd? https://twitter.com/cemerick/status/875748591310168065#2017-10-2514:19taylorIāve only seen this behavior w/`key` specs, but yeah it doesnāt require that the keyword have a registered spec#2017-10-2514:19mgrbytenope. (require '[clojure.spec.alpha :as s]) (s/valid? ::some-undefined-spec [}) will throw an exception (unable to resolve some-undefined-spec)#2017-10-2514:20mgrbyteyep, (s/valid? (s/keys :req [::some-spec]) {})
will pass tho#2017-10-2514:20mgrbyteeven when ::some-spec
has not been defined yet#2017-10-2514:21ajshmm, i wonder what the reasoning is for allowing that keys
#2017-10-2514:21ajsthat's doesn't make any sense to me#2017-10-2514:21taylormaybe so that you can easily spec required map keys without writing specs for the keysā values, or having to define the key specs before the keys
spec#2017-10-2514:22taylorotherwise youād have to do (s/def ::foo any?)
for every key before using it in a keys
spec?#2017-10-2514:22ajslot of room for user error though, as @cemerick has noted. as i am very prone to typos (perhaps we all are), i can see this biting me#2017-10-2514:23tayloritās true, and Iāve actually seen some code in the last week that checks map spec keys for ātyposā but I canāt remember where š#2017-10-2514:23ajsi'm currently evaluating a variety of validation libraries like funcool/struct and truss and others. the one that most ably catches my typos is the one i will use.#2017-10-2514:24taylorfound it https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2017-10-2514:24taylor;; The example program below lets you identify "missing" keys specs at
;; the time and place of your choosing, and then handle them as you
;; deem appropriate, without imposing those decisions on other
;; users of spec.
#2017-10-2514:25ajsi find it a little bizarre that you have to jump through those hoops and write that code just to have spec do what you are expecting; perhaps that leaves room for a little library on top of spec to help mitigate disaster#2017-10-2514:28ajsi don't understand why specifying the presence of a key uses the same syntax/naming as specifying the presence of an actual spec/predicate -- those are two different things, but they are expressed the same in spec, hence the room for easy errors#2017-10-2515:01bbrinckIt seem like most of my typos are around keywords. It seems like a tool that would macroexpand code and look for keywords that are unique would catch a large number of my typos.#2017-10-2515:11andre.stylianosFrom https://clojure.org/about/spec#_sets_maps_are_about_membership_that_s_it
Sets (maps) are about membership, thatās it
As per above, maps defining the details of the values at their keys is a fundamental complecting of concerns that will not be supported. Map specs detail required/optional keys (i.e. set membership things) and keyword/attr/value semantics are independent. Map checking is two-phase, required key presence then key/value conformance. The latter can be done even when the (namespace-qualified) keys present at runtime are not in the map spec. This is vital for composition and dynamicity.
#2017-10-2515:14andre.stylianosFrom what I understand there are two different things.
-- Membership:
"This map has (required or optional) such and such keys"
(s/def ::map (s/keys :req [::foo]))
-- Value:
"Anything with this key should conform to this value"
(s/def ::foo string?)
#2017-10-2515:16andre.stylianosThat's the way I understood this at least#2017-10-2516:55ajsPerhaps one could consider spec a lower level tool on top of which you could build an API to better express concrete structural requirements that eliminate some of that ambiguity, and use that instead of spec directly. #2017-10-2516:56ajsOtherwise there seems ample room for user error, which is typically what you're trying to avoid in the first place by using a validation library#2017-10-2815:04metametadata@U5YHNV0EA I also questioned this behavior on the mailing list recently: https://groups.google.com/forum/#!topic/clojure/i8Rz-AnCoa8
And put my current solution into the gist here:
https://gist.github.com/metametadata/5f600e20e0e9b0ce6bce146c6db429e2#2017-10-2516:38mmerApart from expound is there a way to associate an error message with spec definition so you can force an error message that makes sense and also allows for globalisation of error messages?#2017-10-2516:47bbrinckIs this for devs or end users?#2017-10-2516:47bbrinck@mmer If the messages are for end users, you probably donāt want Expound. https://github.com/alexanderkiel/phrase is likely a better fit#2017-10-2516:54mmerWould it not be reasonable to include some method of defining the error message directly alongside the definition?#2017-10-2517:16bbrinckHow would you want to use those errors messages? In explain
?#2017-10-2517:21bbrinckor would you want to show the error messages to end users?#2017-10-2517:25bbrinckMy understanding is that specs are getting metadata, so if you just need a place to add some info and you want to later pull that out, you could potentially use metadata? https://dev.clojure.org/jira/browse/CLJ-2194#2017-10-2518:46mmerI guess I need to get a usable error message for end users. My usecase revolves around validating yaml files edited by people. As these are nested structures the explanation from explain? get complex. I was hoping for a simple way to add a message to def that explain in human terms what is required. In my case the errors from spec are about a format the user does not see. Why use spec? I have found no other suitable tool that allows me to define in effect a schema for Yaml.#2017-10-2518:51bbrinckOK, that makes sense.#2017-10-2518:52bbrinckSo, you could assign an error message to each spec. You could then call explain-data
and, for each problem, convert the failing spec into the error message.#2017-10-2518:53bbrinckThat may be sufficient for your use case. However, remember that problems are flat - if you have a āorā spec, youāll get two different problems for the same value#2017-10-2518:54bbrinckAdditionally, itās not trivial in my experience to show where the non-conforming is located within the larger data structure#2017-10-2518:55bbrinckSpec also will print errors in terms of the Clojure data structure, which may not be useful if your users are expecting to see errors in terms of Yaml#2017-10-2519:03bbrinckIt really depends on your spec, but probably the simplest approach is to assign errors to each spec (you could just build a static map for now, since each spec name is a unique keyword), then call explain-data
, then take the first problem for each in
, then print out the in
path + your custom string error (you can find the relevant spec in via
). That might be sufficient, although possibly misleading if you have a lot of āorā specs.#2017-10-2519:39stathissideris\|||||||||#2017-10-2522:31Brendan van der EsAnyone know a better way to conform an or spec without keywords? This is the best I got:#2017-10-2522:33mmer@bbrinck Thank you sir for a long and consider answer - I like the map of spec errors approach as even with an or you can explain the options. . Much appreciated.#2017-10-2522:41mmerA follow up from my previous question - is it posible to get the set of specs that are currently in registered?#2017-10-2600:37bbrinck(require '[clojure.spec.alpha :as s])
(s/registry)
#2017-10-2522:42hiredmanthere is a registry function that returns the registry (a map of names to specs)#2017-10-2522:51Brendan van der Es@hiredmanThanks. I am trying to s/or arbitrary specs. Essentially I'm trying to confom different argument lists, e.g.[& args], to the same format of input.#2017-10-2522:53Brendan van der EsSpecifically for datomic, [db entity-hash-map] conforms to EntityMap & EntityMap conforms to EntityMap. Maybe this approach is flawed anyways š, I'll keep playing around with it. Thanks#2017-10-2522:54hiredmanI would use spec to recognize (tag) different formats so you know which formats to deal with, not to try and smoosh all the formats in to one#2017-10-2523:06Brendan van der EsMakes sense, you do than have to handle the different inputs in your function, which arguably is the better place for this logic anyways. I'm hoping this logic might be more reusable if I can embed the fact that the different arg-list formats essentially carry the same information in the spec.#2017-10-2600:39bbrinck@b.j.vanderes FWIW, my understanding is that conformers are not intended to be used to change the shape of the data, so you may run into problems here. Perhaps itād be easiest to just conform as normal, then write a transformation function that uses the conformed values and converts them#2017-10-2601:59Brendan van der Es@bbrinck @hiredman Yea, this is all probably too counter to intention to be usable (performance might also be an issue). But just for fun this is what I came up with as far as defining functions that process the conformed arguments:#2017-10-2602:04Brendan van der EsCan combine with above to mitigate an outer layer of branching in the function.#2017-10-2608:23tapI get OutOfMemoryError when calling instrumented function with wrong number of arguments where one of the argument is a huge collection. I know itās considered user mistake but itās a kind of mistake that is quite hard to figure out the cause. Should clojure prevent that, or break circuit or something?#2017-10-2612:49Alex Miller (Clojure team)Would help to know more about the spec and understand why its generating huge collections#2017-10-2612:49Alex Miller (Clojure team)There are some known issues in this area#2017-10-2702:16tapSure. Iāll try to come up with a sample project replicating the issue I found this weekend#2017-10-2611:01danielnealWhat is the best way of getting the specs of arguments to an fdef'd function?#2017-10-2611:01danielnealI've got an approach here#2017-10-2611:01danielnealbut it uses eval and is probably all wrong š#2017-10-2612:44Alex Miller (Clojure team)Yeah, donāt do that :)#2017-10-2612:45Alex Miller (Clojure team)Just (-> sym s/get-spec :args)#2017-10-2612:46Alex Miller (Clojure team)Or do you want the spec of each argument separately? #2017-10-2612:48Alex Miller (Clojure team)In the most general sense (considering multi arity and variadic), I think that is difficult#2017-10-2612:52danielnealyeah - the spec of each argument separately#2017-10-2612:53danielnealthanks for the get-spec tip š#2017-10-2612:55danielnealjust doing a very rough and ready thing to see if I can go from a value to the (specced) functions that you can invoke that take that value as an arg#2017-10-2612:55danielneallike intellisense, but stupid#2017-10-2612:55danielnealstupidisense, if you will#2017-10-2614:04Alex Miller (Clojure team)this is kind of whatās happening inside of spec regex validation checking, but itās really happening left to right and every possibility is being evaluated and narrowed based on the prior matches (of which there may be multiple)#2017-10-2614:05Alex Miller (Clojure team)so a priori before starting the parse there is not necessarily a single spec but really a set of specs that may apply at each point#2017-10-2614:05Alex Miller (Clojure team)that said, there is the very common case that the args spec is a single non-variadic arity and each arg matches up 1-for-1 with an s/cat args spec#2017-10-2614:06Alex Miller (Clojure team)in which case, you can just pull the s/cat spec apart to match it up#2017-10-2614:07danielnealmm yes I follow you. Technically the args spec for fns could be any regex (or anything) but in practice there will be a common case like you described#2017-10-2614:10danielnealthis is what I've got so far - https://gist.github.com/danielneal/d82c142c9eab9f8caec0fa93b87ff7f3 - I'd like to see if I can get it completing in emacs to see if it 'feels' useful. Dumb but proof of concept, like when IDEO demonstrated the talking kitchen with people hidden behind the fridge saying stuff and responding to voice commands.#2017-10-2619:17zcljSometimes I get errors such as:
`java.lang.IllegalStateException: Attempting to call unbound fn: #'clojure.test.check.generators/choose, compiling:(generators.cljc:499:3)
Exception in thread "main" java.lang.IllegalStateException: Attempting to call unbound fn: #'clojure.test.check.generators/choose, compiling:(generators.cljc:499:3)`
It happens with different test.check unbound fns, it is not the same every time and a rebuild usually resolves the error. The file that is beeing compiled is a quite basic cljc file with some specs in it.
Does anyone have any advice to point me in the right direction?#2017-10-2621:49Alex Miller (Clojure team)Any chance this is happening during check?#2017-10-2621:51Alex Miller (Clojure team)There is a dynamic load race that can happen with loading the test check generator namespace that we have a ticket on. I donāt remember /cdn-cgi/l/email-protection presents, but it was weird looking#2017-10-3012:03zclj@U064X3EF3 Sorry for the late reply. The stack trace only show line 1 for my files i.e. in the ns loading. The file do include a stest/check test but from the stack trace I do not seem to hit that. It blow up on the first line. I'll try and find the ticket you mentioned and see if that holds any clues#2017-10-2619:26gfredericksO_O#2017-10-2619:35billin an s/fdef :fn
spec for a tree function of mine, it seems like I need to get access to the un-conformed argument (a tree) to the function so I can pass it to some other tree functions to build up my validation.
is there a way to gain access to the un-conformed args or perhaps to un-conform the conformed args?#2017-10-2619:44billis the right word for āun-conformā āunformā?#2017-10-2619:45billyessss unform
https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/unform#2017-10-2619:45gfredericksdeconformificate#2017-10-2708:53mmerIs there a way to associate a string of text with a spec definition. I would like to annotate my definitions#2017-10-2709:17gklijsYou can do something like (def label (s/and (s/spec string?) #(> (count %) 3) #(< (count %) 40))) and then use hat spec somewhere else like (s/def ::nl-label label)#2017-10-2709:45danielnealI think doc-strings for specs are coming later, too#2017-10-2713:47souenzzohttps://dev.clojure.org/jira/browse/CLJ-1965
https://dev.clojure.org/jira/browse/CLJ-2194#2017-10-2819:19mike706574Is it intentional that the functions in clojure.spec.gen
that alias functions in clojure.test.check.generators
aren't considered generators?#2017-10-2819:19mike706574Or am I doing something wrong?#2017-10-2819:20mike706574(clojure.test.check.generators/generator? clojure.spec.gen.alpha/int)
seems to be returning false#2017-10-2819:22mike706574In my case, I'm trying to write a spec for an 5-10 character alphabetic string with a
custom generator that uses clojure.spec.gen.alpha/char-alpha
#2017-10-2819:23mike706574Sampling is failing with an AssertionError
with a message of Assert failed: First arg to vector must be a generator (generator? generator)
#2017-10-2819:24mike706574It works fine when I use clojure.test.check.generators/char-alpha
- just seems counterintuitive that the aliased function doesn't work#2017-10-2819:40hiredmanthe spec aliases are actually single argument functions that return generators#2017-10-2819:41mike706574oh#2017-10-2819:41mike706574that makes more sense#2017-10-2819:41hiredmanthis is to support lazy loading of clojure.test.check#2017-10-2819:41mike706574so i shouldn't be using them?#2017-10-2819:42hiredmanyou can use them, you just have to be aware of that fact when mixing with plain test.check generators#2017-10-2819:43hiredmanand if I recall correctly, most places in clojure.spec where you can supply a custom generator, you supply a no arg function that returns a test.check generator#2017-10-2819:43mike706574yup#2017-10-2819:44mike706574i just put parens around it - (gen/vector gen/char-alpha 5 10)
to (gen/vector (gen/char-alpha) 5 10)
- and it worked#2017-10-2819:44hiredmanthat's lisp for you#2017-10-2819:45mike706574cool, thanks#2017-10-2910:17Oliver GeorgeQuick sanity check. I think this code should stub a function (in second case). It does on CLJ but doesn't on CLJS.#2017-10-2910:17Oliver George(defn add2 [a b] 1.1)
(s/fdef add2 :args (s/cat :a int? :b int?) :ret int?)
(println ::a (add2 1 2))
(stest/instrument `add2 {:stub #{`add2}})
(println ::b (add2 1 2))
(stest/unstrument `add2)
(println ::c (add2 1 2))
#2017-10-2910:18Oliver GeorgeAm I doing something obviously wrong?#2017-10-2915:24eoliphantIs it possible to do ācrossā specification/validation with spec?
If I have a map
{:a "even"
:b 2}
Where say :b
should be odd?
if :a
is "odd"
and vice versa?#2017-10-2915:25mingp@eoliphant Perhaps treat them as separate cases and combine with spec/or
.#2017-10-2915:26mingpIt would depend more on what the more immediate problem you're trying to solve is.#2017-10-2915:26eoliphanthmm ah I see, when youāre doing the āmapā spec cover both cases with the or#2017-10-2915:26eoliphantiāll give that a try#2017-10-2916:08mrchanceHi, is there a way to generate self contained specs? I want to turn a swagger.json into a spec and run it on some data at runtime, and it looks like that requires eval with the way the spec registry currently works?#2017-10-2916:08mrchanceOh, and https://clojure.github.io/core.specs.alpha/ appears to be broken, I arrived there from the overall clojure api page https://clojure.org/api/api#2017-10-3016:26Alex Miller (Clojure team)This is a known issue in the doc build process#2017-10-3016:41mrchanceOk, just wanted to let you know. Any insights into the other problem? I feel pretty dirty if I have to fiddle with global mutable state just to get some spec validation working...#2017-10-2922:17Drew VerleeIf i want to use spec in order to pass the user of my application a error msg how would i do that? Is there a place to override the error msg with a custom one?#2017-10-2922:18Drew VerleeDoes that even fit with the rational? Like if i have a command line app and i want to validate the input and send back an error msg, would spec fit into?#2017-10-2923:03seancorfieldThe idea is to use the explain-data result and map it to whatever error messages you want in your app @drewverlee#2017-10-2923:15bbrinck@drewverlee Perhaps https://github.com/alexanderkiel/phrase would be a good fit?#2017-10-2923:16bbrinckBut yes, if your specs are fairly simple, it might be easies to just map explain-data to error messages. It gets a little trickier if your specs are more deeply nested#2017-10-2923:24Drew Verlee@bbrinck phrase is in the spirit of what i want. Its odd i couldn't search it down.#2017-10-3013:26Drew VerleeI poked around with the idea of phrase and iām not sure it make sense for what iām doing. As @seancorfield suggested i can pull the information with explain-data
.
And if i want to reference something inside the spec it might make sense to pull it into a var to use across multiple domains rather then pulling it out via regex matching (which is what phrase seems to do).
Iām would be interested to hear how something like phrase worked in a larger project, as it seems prescriptive.#2017-10-3014:36mpenetdid anyone create a web based spec browser thing?#2017-10-3014:39bbrinck@mpenet Whatās your use case? Do you want to browse specs at dev time, or, say, publish specs online similar to API docs?#2017-10-3014:40mpenetyep exactly#2017-10-3014:40mpeneti know of https://github.com/jpmonettas/inspectable but i d like to ship a browser based thing#2017-10-3014:41mpenetI guess this could be adapted, but just asking in case somebody did that work already#2017-10-3014:42bbrinckHm, not that I know of. On a related note, I wonder if itās be useful to start a wiki that collects these spec helper libraries as a resource#2017-10-3014:50jebberjeb@drewverlee Iāve been working on something like this which tries to map a human readable message by a problemās spec keyword, and falls back to a symbolic predicate match (like Phrase) if it canāt. Trying to embloginate it now.#2017-10-3015:00Drew VerleeI was thinking this over last night. I donāt think there is any generic relationship between a human readable message and the code.
The only way i could see to do this that was better then plain old clojure would be some sort of semantic analysis of the function words. I feel like a neural net might be able to make something out of the function names and values.
I was curious about phrase, but felt it wasnāt doing much for me that i couldnāt get from core spec and a mapping to human readable message.
Iām curious about your work.#2017-10-3015:06jebberjebI suspect its there, but Iām not even close to finding it. My work is mostly a straight dispatch based on the problemās spec keyword, falling back to a non-macro based Phrase implementation if that fails. And only the latter out of necessity. s/keys specs for example, generate problems that really require you to look at the predicate to know exactly what went wrong.#2017-10-3015:07jebberjebSo the predicate dispatch is mostly to catch cases where :via just doesnāt tell you enough (in the s/keys case, I mean the exact field) about what went wrong.#2017-10-3015:10jebberjebIām hoping that approach casts a wide enough net to handle most of the problems generated by our specs. Chopping the specs up, and defāing each piece seems to help.#2017-10-3015:29Drew VerleeI think dispatching on the problem specās keyword and maybe some context importer to the end user might make sense. The advantage to just handing them as data and passing them around is that you could organize your code around them.#2017-10-3015:29Drew VerleeIs that in line with your thinking? I should get back to you tomorrow when im feeling better.#2017-10-3015:30jebberjebI think it is š Yeah, would love to talk more.#2017-10-3014:51jebberjebItās in use now in a fairly large project, but was introduced only about a month ago. So still waiting to see how it pans out.#2017-10-3015:06amannNot sure if this is the correct place to ask this question (if not, feel free to boot me to another channel!), but I recently stumbled across some performance related things with respect to spec generative testing specifically regarding every
and the :kind
and :into
parameters.
So the spec I had is loosely similar to this:
(s/def ::x (s/every string? :kind vector?))
(time (some-gen-testing ::x))
=> "Elapsed time: 93083.882115 msecs"
(s/def ::x (s/every string? :into []))
(time (some-gen-testing ::x))
=> "Elapsed time: 293.882115 msecs"
As you can see, :into
is orders of magnitude faster than :kind
for generative testing.
My question is this: when should I be using :kind
and when should I be using :into
?#2017-10-3016:20Alex Miller (Clojure team)Are you using latest spec? This problem has been fixed.#2017-10-3016:21Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2171 fixed in spec 0.1.134#2017-10-3016:22Alex Miller (Clojure team)I would recommend preferring kind if thatās sufficient#2017-10-3016:53bostonaholic@alexmiller I think you may have linked the wrong JIRA#2017-10-3016:53Alex Miller (Clojure team)Yup#2017-10-3016:54Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2103#2017-10-3016:54bostonaholicmore better!#2017-10-3017:08amann@alexmiller looks like there was a very fun dependency error with pedantic being enabled in leinengen. I was pulling alpha17 when my deps required beta2. Rerunning things I got the following (which looks acceptable to me):
(s/def ::x (s/every string? :kind vector?))
(s/def ::y (s/every string? :into []))
(s/def ::n nil?)
(defn xfn
[x]
nil)
(defn yfn
[y]
nil)
(s/fdef
xfn
:args (s/cat :x ::x)
:ret ::n)
(s/fdef
yfn
:args (s/cat :y ::y)
:ret ::n)
(time (tu/stest-w-report `xfn))
(time (tu/stest-w-report `yfn))
#2017-10-3017:08amann(time (tu/stest-w-report `xfn))
"Elapsed time: 974.417512 msecs"
=> true
(time (tu/stest-w-report `yfn))
"Elapsed time: 522.806027 msecs"
=> true
#2017-10-3017:08amannie, off by half a second rather than minutes#2017-10-3017:08amannmany thanks for calling that out!#2017-10-3015:25mpenet@alexmiller is that voluntary or a leftover https://github.com/clojure/spec.alpha/blob/739c1af56dae621aedf1bb282025a0d676eff713/src/main/clojure/clojure/spec/alpha.clj#L237 ? just lost a bit of time tracking success! spurious messages in logs that where coming from here#2017-10-3015:27mpenetthe originating code was a bit weird anyway (just calling s/explain-data in a when-let to validate a piece of content and grad the error in one go), but maybe printing here is not something desirable#2017-10-3016:22hiredman@mpenet that is what explain does, if you look at the code up above it as full of prs, the default for explain is to print stuff out#2017-10-3016:27mpenetNo prob with explain printing. Just found a bit odd that -data would#2017-10-3016:23Alex Miller (Clojure team)Youāll get that if you ask for explain on something valid#2017-10-3016:23Alex Miller (Clojure team)Which leads me to why you would do that... :)#2017-10-3016:30mpenetThe code in question is gone. But since there an if branch in that code for the printing in case of success maybe it's expected people would do that#2017-10-3016:23mpenetyes that was the case, not very nice #2017-10-3016:23mpenetIndeed#2017-10-3016:25mpenetI was not expecting it to trigger a print regardless#2017-10-3016:25mpenetex-data doesnt for instance#2017-10-3016:29hiredmanare you sure you aren't just calling explain, not explain-data?#2017-10-3016:31mpenetNot in front of my computer anymore. I think it was -data, but not sure#2017-10-3016:39mpenetAh no it was explain, sorry about the noise š All good then#2017-10-3017:13amannš¬#2017-10-3018:38stexI'm was wondering if the following is possible with clojure.spec, since I'm not sure if it's one of the use-cases it tries to solve. Let's say we have a user which can have two roles: (s/def ::role #{:admin :normal})
and a admin specific field which is only required when the user is an admin, not a normal user. I currently have: (s/def ::user (s/keys :req [::role] :opt [::admin-specific-field])
, but that's only partially true, since the second key is required depending on the value of ::role
. Anyone have an idea how to spec this? Thanks š š#2017-10-3018:39taylorI think thereās a multi-spec
example in the spec guide very similar to that#2017-10-3018:41taylorhttps://clojure.org/guides/spec#_multi_spec#2017-10-3018:44stex@taylor Thanks! Didn't catch that for some reason š#2017-10-3021:55abpHey, considering https://dev.clojure.org/jira/browse/CLJ-2116 is the top two voted ticket for Clojure right now, I thought about this general issue for quite a while last year and as a result wrote a spec that, given a root spec, recursively parses specs into reversed dependency order, to do a spec rewrite to a new root, while replacing certain parts of the spec-tree. The latter algorithm isn't implemented yet, I put it aside for quite a while. It shouldn't be too much work but maybe I'd need to get tools.analyzer into the mix for rewrites of multispecs. And then there was Rich announcing better programmability of specs in his conj keynote. So the question is, is it still worth pursuing this? Any ideas about how deep the rabbit hole might get? I'm aware of some libs doing spec parsing into records etc. but for some reason I'd like to avoid that (you're welcome to convince me). Also I haven't looked into term rewriting at all, but maybe that's what I'm trying to do.#2017-10-3022:07abpAh, https://github.com/jpmonettas/inspectable/blob/master/src/inspectable/spec_utils.cljc#L24 is a nice solution for the multispec-problem.#2017-10-3117:01ikitommi[metosin/spec-tools "0.5.1"]
works now with the latest spec & clojure beta.#2017-11-0109:40hkjelsHow do I get the merged form of a spec?#2017-11-0114:42Alex Miller (Clojure team)You donāt#2017-11-0122:35hkjelshttps://github.com/lab-79/clojure-spec-helpers/blob/f3d7c48c55bbe4dc901cd7f1ace1e02b3777bd1b/src/lab79/clojure_spec_helpers.cljc#L61
This solved my issue for now#2017-11-0112:05rickmoynihanDoes clojure/spec have a predicate that matches g/simple-type
?#2017-11-0112:05rickmoynihanbut no more#2017-11-0112:05rickmoynihani.e. doesnāt match complex (collection) types#2017-11-0114:43Alex Miller (Clojure team)no, but (def simple? (comp not coll?))
is a good first approximation#2017-11-0114:56mpenetOr (complement coll?)#2017-11-0114:56Alex Miller (Clojure team)yep, gooder#2017-11-0114:58Alex Miller (Clojure team)well, mine was shorter :)#2017-11-0114:58Alex Miller (Clojure team)ācomplementā is a long word :)#2017-11-0115:30mpenetš#2017-11-0120:34uwoam I right to understand that you canāt document keys with (s/keys :opt [::some-key])
without also providing the spec for ::some-key?#2017-11-0120:47seancorfieldIt won't generate without a spec for ::some-key
but it will validate just fine -- it will treat missing specs as any?
I believe.#2017-11-0120:50seancorfield^ @uwo is that the scenario you are asking about?#2017-11-0121:06uwo@seancorfield yes! thatās the way I thought/hoped it worked, but I was calling s/assert
in some clojurescript code and it was complaining about some keys that I had declared in s/keys, but that did not have their own spec.#2017-11-0121:55ajs@alexmiller those were some meaningful tweets #2017-11-0203:51thedavidmeisteri have a really basic question...#2017-11-0203:51thedavidmeisterhow do i get#2017-11-0203:52thedavidmeister(def foo [:foo/bar]) (spec/def (spec/keys :req foo))
to not throw errors?#2017-11-0203:52thedavidmeisterDon't know how to create ISeq from: clojure.lang.Symbol
#2017-11-0204:18seancorfield@thedavidmeister That's because spec/keys
is a macro so it doesn't eval its arguments.#2017-11-0204:18thedavidmeisterhmm, is there a way to get around that?#2017-11-0204:18thedavidmeisterit would be nice to reuse foo
elsewhere#2017-11-0204:18seancorfieldWhat problem are you trying to solve?#2017-11-0204:18thedavidmeisterjust generally keeping things DRY#2017-11-0204:19thedavidmeisteri have a list of keys that i'd like to put in a spec, but also use elsewhere#2017-11-0204:19thedavidmeistere.g. putting in a list of options for a dropdown menu#2017-11-0204:19seancorfield(s/def ::foo (s/keys :req [::foo/bar]))
(last (s/form ::foo))
Try that.#2017-11-0204:21thedavidmeisterhmm yes, that does work but#2017-11-0204:21thedavidmeisterthen if i add :opt
in the positions of things in form change#2017-11-0204:21seancorfieldi.e., make the spec the system of record and get the keys from it elsewhere#2017-11-0204:21thedavidmeisteryeah i see, is there a more robust way to extract things from spec?#2017-11-0204:21thedavidmeisterk/v style?#2017-11-0204:21seancorfield(let [[& {:keys [req req-un opt opt-un]}] (rest (s/form ::some-spec))] ...)
#2017-11-0204:22thedavidmeistermmm, because (::foo (s/form ::foo))
didn't work#2017-11-0204:23seancorfieldIt's just data shrug#2017-11-0204:23seancorfieldThe spec should be the system of record tho'...#2017-11-0204:23thedavidmeistermaybe#2017-11-0204:23thedavidmeisteri don't necessarily see why it should be#2017-11-0204:24thedavidmeisterand it seems like trying to use it that way is clunky at best#2017-11-0204:24seancorfieldThere are JIRA issues around providing a more programmatic API at some point.#2017-11-0212:47Alex Miller (Clojure team)Rich is working on a significant spec update that will include more support in this area#2017-11-0215:27uwosuper awesome! around the idea of clj-2112?#2017-11-0215:37Alex Miller (Clojure team)no, this is in the internals#2017-11-0204:25thedavidmeisterif it hasn't been prioritised to date, isn't that evidence against the idea of "should be used in this way"?#2017-11-0204:26seancorfieldSpec is designed for human writing and comprehension right now, not for programmatic writing. But at least it's data so you can programmatically read and comprehend them.#2017-11-0204:26thedavidmeisterit's true, it's better than nothing atm#2017-11-0204:27thedavidmeisteri can manually pick apart the form of the spec and alias it to something more convenient#2017-11-0204:27thedavidmeisterbut if the intention is truly that spec is a "one stop shop" for this type of thing, then this API needs some polishing >.<#2017-11-0204:28thedavidmeister(::foo (:keys (s/xxx ::foo)))
seems to be what i'm reaching for here#2017-11-0204:29thedavidmeisterwhere s/xxx
is not s/form
but something along those lines#2017-11-0204:30seancorfieldI just tried this in the REPL. I think it's clearer: (let [[& {:keys [req req-un opt opt-un]}] (rest (s/form ::foo))] req)
#2017-11-0204:30seancorfieldThat will destructure all four possible keys
options.#2017-11-0204:30thedavidmeisteroh nice#2017-11-0204:30thedavidmeisteryeah something like that š#2017-11-0204:31thedavidmeisterthanks for the help#2017-11-0204:31seancorfieldIt won't deal with s/merge
constructs tho'... but what we do is have a bunch of basic specs with s/keys
and those are what we destructure to get the actual keys, and then we s/merge
them for compound specs.#2017-11-0204:31seancorfield(funnily enough I was writing some code to do exactly this at work today)#2017-11-0204:32seancorfieldAnd we have other code that uses specs as the basis of CRUD-style data functions too...#2017-11-0204:32thedavidmeisteri think atm it's pretty good getting things in to spec but not so much getting them back out#2017-11-0204:32thedavidmeisteractually it would be pretty sweet if i could plug spec straight into my UI#2017-11-0204:33thedavidmeisterbut it seems to not quite be there yet#2017-11-0204:34seancorfieldI think it's mostly a matter of tooling -- a lot of which will come from the community.#2017-11-0204:34thedavidmeistertotally, i've seen a lot of cool tooling already just lurking in this chat#2017-11-0204:34seancorfieldGiven that 1.9 is still in pre-release, not everyone is using it yet, which is holding back tooling.#2017-11-0204:35seancorfieldWe have 1.9 Beta 3 in production and we're heavy users of spec but it's all still a bit of an adventure...#2017-11-0204:35thedavidmeisteryeah i know that hoplon had to make some changes to keep pace with 1.9, and also has started playing around with spec but it's WIP#2017-11-0204:35thedavidmeisteri imagine it is the same for everyone#2017-11-0204:36seancorfieldBeta 4 broke our code due to removing bigdec?
which was added earlier in 1.9. But it was a small change. We expect to have Beta 4 in production on Monday.#2017-11-0204:36thedavidmeisternice š#2017-11-0204:36thedavidmeisterwell i g2g run some errands#2017-11-0204:36thedavidmeisterthanks for the help#2017-11-0216:11uwo@seancorfield I was able to create minimal reproduction. This only occurs in cljs. Should I make a ticket for this?
(s/def ::test (s/keys :opt-un [::unspeced-keyword]))
(s/def ::test-fn
(s/fspec :args (s/cat :an-arg ::unspeced-keyword)))
(s/assert ::test-fn (fn [a]))
;; => #object[Error Error: Unable to resolve spec: :cljs.user/unspeced-keyword]
#2017-11-0216:14uwoguessing this is related to fspec invoking a generator, but dunno#2017-11-0221:28Alex Miller (Clojure team)I believe it is. fspecs will be checked by using the :args generator to verify the function can accept those args. In this case itās trying to generate ::test instances that sometimes include ::unspeced-keyword instances, but it doesnāt know how to do that.#2017-11-0221:28Alex Miller (Clojure team)or at least thatās my guess, maybe something else going on#2017-11-0313:43uwoShould I create a jira cljs ticket for this?#2017-11-0314:06Alex Miller (Clojure team)sure#2017-11-0216:54seancorfieldIf that works in Clojure but fails in cljs, raise a JIRA issue. If it failed in both I'd say it was related to generators. If it's only failing in cljs, I'm not so sure.#2017-11-0216:55seancorfieldConfirmed it works in Clojure, so that looks like a cljs bug.#2017-11-0216:58uwothanks. will do#2017-11-0219:07bmabeyHow do I use a sequence spec as a function argument? This is what I've tried so far without any luck:
(s/def ::int-then-strings (s/cat :num int? :strs (s/+ string?)))
(defn blah [stuff m]
(first stuff))
(s/fdef blah
:args (s/cat :list-of-int-then-strings ::int-then-strings :map map?))
(stest/instrument)
(s/valid? ::int-then-strings [2 "foo"]) ; => true
(blah [2 "foo"] 3)
;; clojure.lang.ExceptionInfo: Call to #'blah did not conform to spec:
;; In: [0] val: [2 "foo"] fails spec: :int-then-strings at: [:args :list-of-int-thenstrings :num] predicate: int?
;; :clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x37194692 "
#2017-11-0219:10taylor(s/def ::int-then-strings (s/spec (s/cat :num int? :strs (s/+ string?))))
(defn blah [stuff m]
(first stuff))
(s/fdef blah
:args (s/cat :list-of-int-then-strings ::int-then-strings :map map?))
(stest/instrument)
(s/valid? ::int-then-strings [2 "foo"]) ; => true
(blah [2 "foo"] {:foo 3})
#2017-11-0219:10taylorāļø this works#2017-11-0219:11taylortwo changes: your ::int-then-strings
spec wrapped in s/spec
to prevent it from getting āflattenedā into one big sequence spec, and your test call was invalid according to your spec: it was an integer and not a map#2017-11-0219:11bmabeyThanks! Why is the extra s/spec
needed around the s/cat
?#2017-11-0221:30Alex Miller (Clojure team)because all regex specs combine to describe a single level of sequential collection. The s/spec forces a boundary such that the outer regex spec includes a collection which has an inner regex spec.#2017-11-0220:58xiongtxBeen getting some problems when using clojure.spec.alpha
, clojure.test.check.generators
, and boot-cljās cljs
compilation.
See issue: https://github.com/clojure-emacs/cider/issues/2104 and minimal repo: https://github.com/xiongtx/reload-error-boot
@alexmiller Is it not recommended to use clojure.test.check.generators
directly when using clojure.spec
? It seems to me thereās weird interaction b/t the lazy combinators, CLJS compilation, and namespace reloading. I suspect we can avoid this problem by using only clojure.spec.gen.alpha
.#2017-11-0221:33Alex Miller (Clojure team)I donāt understand the problem - too much tooling and other stuff for me to get it. Shouldnāt be any issue with using clojure.test.check.generators directly - thatās the same thing clojure.spec.gen.alpha is doing, just with a delayed load step.#2017-11-0221:40Alex Miller (Clojure team)generators in test.check are records iirc - maybe youāre running into a case where the record is being redefined and the old generators are no longer instances of the new (reloaded) record class?#2017-11-0223:19xiongtxGood insight! Yes, that seems to be precisely whatās happening. After a refresh:
(let [spec (s/int-in 0 10)
g (s/gen spec)
h (gen/->Generator (constantly 1))]
(println "clojure.spec.alpha generator's classloader: " (.getClassLoader (type g)))
(println "clojure.test.check.generators generator's classloader: " (.getClassLoader (type h))))
;; clojure.spec.alpha generator's classloader: #object[clojure.lang.DynamicClassLoader 0x73867ca9
The problem is that clojure.test.check.generators
was reloaded (and the defrecord
re-evaluated), but the same was not done for the lazy combinators in clojure.spec.gen.alpha
. This is probably b/c clojure.spec.gen.alpha
does not :require
clojure.test.check.generators
, so the dependency tracking in clojure.tools.classpath...
isnāt working properly.#2017-11-0223:55xiongtxI believe there was some talk of giving clj a way to lazily load :require
ed namespaces. That, if implemented, would seems like the ideal solution here.#2017-11-0301:00James VickersWhat is the shape of the data that gets passed to the :fn
argument of s/fdef
? Does someone have an example of that (or a way to print it)?#2017-11-0301:00tayloritās a map with :ret
and :args
keys#2017-11-0301:01taylorthe :args
value is the conformed value of the args I think#2017-11-0301:02taylor:fn (fn [{:keys [args ret]}]
...
#2017-11-0301:03James VickersThanks. Is that in the docs somewhere?#2017-11-0301:04Alex Miller (Clojure team)That is all correct - the values of the map are the confirmed values of the args and ret specs #2017-11-0301:05Alex Miller (Clojure team)Iām not sure where it would be docāed in the docstrings#2017-11-0301:05James VickersThanks#2017-11-0301:05taylor:fn A spec of the relationship between args and ret - the
value passed is {:args conformed-args :ret conformed-ret} and is
expected to contain predicates that relate those values
#2017-11-0301:05taylorfdef
docstring#2017-11-0301:06James VickersAh, thanks.#2017-11-0301:07James VickersSo, if the function returns a single value (like a number), then (% :ret)
in the :fn
spec should yield the return value?#2017-11-0301:08Alex Miller (Clojure team)Depends on the ret spec#2017-11-0301:08taylorit might be tagged/conformed though I guess?#2017-11-0301:08Alex Miller (Clojure team)If itās a simple pred then yes #2017-11-0301:09Alex Miller (Clojure team)Right#2017-11-0301:10James VickersSweet, thanks.#2017-11-0301:10James VickersI was surprised that putting a do
expression with a println
in the :fn
spec didn't print anything when the function was called (instrumentation on)#2017-11-0301:11taylorthe :fn
spec doesnāt come into play re: instrument
calls#2017-11-0301:11taylorit does get used if you check the function though#2017-11-0301:11James Vickersoh#2017-11-0301:12taylor> Instrumentation validates that the :args spec is being invoked on instrumented functions ā¦#2017-11-0301:12taylor> check
will generate arguments based on the :args
spec for a function, invoke the function, and check that the :ret
and :fn
specs were satisfied.#2017-11-0301:13James VickersCool. I ran a check
and it printed out what was passed to :fn
.#2017-11-0313:17zcljIf I have a production spec of say an email address, it will require a custom generator if I want to generate test data. Since a generator typically will not be used in production would it be a good practice to redefine the production spec in i test ns that will include the generator, keeping any generators out of production code? Can this lead to problems if I have nesting and to be able to generate the top level spec I need to redefine lower levels with the generator in my tests?#2017-11-0314:04Alex Miller (Clojure team)Another thing to consider is using generator overrides at the point of testing. That way you donāt have to redefine a spec, you just supply a set of alternate generators.#2017-11-0314:19zcljBut in the case of a nested spec such as a person having an email, and I am testing the person don't I have to redefine the email spec refered by the person spec?
Is generator overriding done by using with-gen
or are there other ways I have missed?#2017-11-0314:29Alex Miller (Clojure team)no, you can supply custom generators in exercise, instrument, check, etc#2017-11-0314:31Alex Miller (Clojure team)in stest/check
for example, you can supply a :gen map in the options: āmap from spec names to generator overridesā#2017-11-0314:31Alex Miller (Clojure team)so that way you can supply overrides just in the context of a test#2017-11-0314:53zcljah I see, will try that out, thanks for your help!#2017-11-0321:30seancorfieldGood to be reminded of that. So far, we've tended to write wrappers for generators so we can lazy load the testing library (and therefore keep the actual "testing generator" in the production code without needing the testing libraries in production).#2017-11-0313:25wilkerluciohello, has anyone here though about the idea of specing nested structures? currently s/keys
only supports flat structures, if I need a nested I have to define the nested structure ahead of time on that key, this prevents different nesting structures depending on the context#2017-11-0313:26wilkerluciomaybe we could support via a syntax like the datomic pull syntax, eg: (s/keys :req [:user/name {:user/address [:address/line1 :address/city]}])
#2017-11-0314:04ikitommi@U066U8JQJ have you checked https://github.com/metosin/spec-tools/blob/master/README.md#data-specs ? nests also vectors and sets. In the end, just generates specs with alternative (macro free) syntax.#2017-11-0314:07ikitommialso would like to see support for nested keys
-specs in spec itself. #2017-11-0314:07Alex Miller (Clojure team)thatās not in line with the āset of attributeā thinking and no plans for that#2017-11-0314:55wilkerluciothe issue I'm facing is that for the "container" specs I might want different subsets of keys, depending on the context#2017-11-0314:55wilkerlucioso by not having a fixed for the children, it gets tricky#2017-11-0315:06Alex Miller (Clojure team)you donāt need a fixed spec for the children#2017-11-0315:07Alex Miller (Clojure team)an empty (s/keys) is sufficient to cover an open set of attributes#2017-11-0315:07Alex Miller (Clojure team)or use s/multi-spec to select the spec based on the contents#2017-11-0315:07Alex Miller (Clojure team)or s/or multiple choices, etc#2017-11-0315:08wilkerluciothanks for the tips Alex, I like the open one, but at same time, if I want to require different sets of keys depending on the context, that doesn't work, the multi-spec can work, but its a lot more involved#2017-11-0315:11wilkerlucioI love the idea of living by just attributes, but if I need to give a name to a context of sub-attributes, I feel like backing again to the "box" (class, entity, whatever...) constraints again#2017-11-0315:17wilkerlucio@U055NJ5CC thanks for pointing that out, I'll look it up#2017-11-0315:18Alex Miller (Clojure team)what youāre saying is that they key at the top level does not have a stable semantic meaning so I would think about what that means and whether itās a good idea#2017-11-0315:19wilkerlucioyeah, in general terms, any sub-set might be unstable, if what you care is just about the leaf attributes validation#2017-11-0315:21wilkerlucioI've been writing a considerable amount of code regarding to data fetching apis (om.next style), and many times I see that the sub-set of keys of a child element can vary wildly, in my case I'm embracing this and it's working pretty good, but I can't get the specs around to match it in the way it is#2017-11-0315:21Alex Miller (Clojure team)(s/keys) ! :)#2017-11-0315:21wilkerluciothe problem is the children, for example, working in micro-service architecture#2017-11-0315:22wilkerlucioI have many endpoints across services that can return different sub-sets of the data#2017-11-0315:22wilkerlucioand altough they share some root keys, what comes in the children is variable, some endpoints give more, some give less information#2017-11-0315:22wilkerlucioand in current spec way, I can't define what is required for each return (or input) on a case-to-case bases#2017-11-0315:23wilkerlucioin the same way we can't have a good definition of what are the required fields for an user
(a login might be user/password, a signup would require much more)#2017-11-0315:24wilkerlucioI feel the same problem when trying to specify requirements for a nested item#2017-11-0315:24Alex Miller (Clojure team)I think thatās all ok, and you should not try to define every key set aggregation#2017-11-0315:24Alex Miller (Clojure team)lean on the attributes#2017-11-0315:27wilkerluciobut that's the problem, if I have one key like :user/address
, I expect this key to be the same always, but what I expected to be inside of it can change from case to case#2017-11-0315:27wilkerlucioon the top level, just create a new set and we are all good#2017-11-0315:27wilkerluciofor the nested, there is no way to override the requirements#2017-11-0315:28wilkerlucioto say that the sub-set is different a separated case, makes sense?#2017-11-0315:29Alex Miller (Clojure team)why not just (s/def :user/address (s/keys))
?#2017-11-0315:29Alex Miller (Clojure team)then rely on your address attribute specs to do the work#2017-11-0315:34wilkerluciothe only issue there is that we can't make some attributes required for a context that way#2017-11-0315:34wilkerlucioso the validation gets too loose#2017-11-0315:36Alex Miller (Clojure team)if you need that then it sounds like s/multi-spec to me, or state all the possibilities with s/or, or add outer constraints s/andāed at the top level#2017-11-0315:36wilkerluciohumm, the top level constraint sounds like a good path for the cases I'm thinking#2017-11-0315:36wilkerluciothanks, I'll try that and see how it goes#2017-11-0315:36Alex Miller (Clojure team)always come back to āstate the truth about what itās in your dataā - what can occur in your actual data? say that in the spec.#2017-11-0313:27wilkerluciohad anyone felt the desire to specify the kinds on that way?#2017-11-0520:26Drew VerleeI just spenT a day thinking about how to use specs for human error msgs. i think phrase is interesting but it seems to lack a way to do some rather useful things and makes others somewhat more difficult then then should be.
The one problem i can't think around is naturally combining your "for user erorrs" together when you concat specs together.#2017-11-0613:27dominicm@drewverlee I think the human error messages also need to include a location which is separate from their spec position. Sometimes you need to do validation across keys for humans, but place the error message on a particular key (password & password_confirm must match, but if they don't, the error is in password_confirm)#2017-11-0614:24gfredericks#2017-11-0618:13seakoi find them noisy and like that test.chuck turns them off when using the checking
macro#2017-11-0616:51Drew VerleeIs there a rational for the :reason
key in the problem map produced by spec/explain-data?
. I cant find any discussion on it#2017-11-0620:34Drew Verleeis there a function that you can call on a spec to get the code for the spec?
(s/data some-spec)
;;=> (s/def ::some-spec string?)
#2017-11-0620:35taylorhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/form#2017-11-0620:36taylor(s/form some-spec)
in your example should work @drewverlee#2017-11-0620:37Drew VerleeIt seems to only return part of the spec.
(s/def ::kid (s/keys ::req [::name]))
(s/form ::kid)
;;> (spec/keys)
#2017-11-0620:38taylorI think that keys
spec is invalid#2017-11-0620:39taylortry :req
instead of ::req
#2017-11-0620:39Drew Verleegood catch#2017-11-0621:09Drew VerleeHmm, how about something similar for spec functions? those defined using fdef?#2017-11-0621:10taylor(s/form `foo)
where foo
is your function#2017-11-0716:14Drew VerleeHow does that work? backtick stops evaluation of things and qualifies their namespace. Why does that help?#2017-11-0716:17taylorhttps://clojure.org/guides/weird_characters#syntax_quote#2017-11-0716:17taylor> However, symbols used within a syntax quote are fully resolved with respect to the current namespace#2017-11-0716:19taylor(defn foo [x] x)
=> #'sandbox.core/foo
foo
=> #object[sandbox.core$foo 0x111da369 "
#2017-11-0716:38Drew Verleegotcha. I was actual experiencing another mis understanding the same time concerning lists.#2017-11-0621:12taylors/fdef
is using s/def
ābehind the scenesā to register the spec to the function symbol, so you can use the function symbol to resolve the spec (instead of a namespaced keyword)#2017-11-0709:26mattiaswmap-like? I have an application and I use maps a lot. Added spec, most of them using s/keys. In some cases I want to apply operations to the maps like filter. But this breaks the specs, since it isn't a map any more, but a sequence of pairs. The simple solution is of course to convert the result of filter back to a map, but is there a better way? Here is a small sample
(s/def ::test-id int?)
(s/def ::test-data string?)
(s/def ::test1
(s/keys :req-un [::test-id ::test-data]))
(s/fdef spec-test
:args (s/cat :m ::test1)
:ret int?)
(defn spec-test
[m]
(count m))
(def test1-sample {:test-id 1 :test-data "hello"})
(defn works
[]
(spec-test test1-sample))
(defn works-not-which-is-ok
[]
(spec-test (dissoc test1-sample :test-data)))
(defn works-not-which-is-not-ok
[]
(spec-test (filter (fn [_] true) test1-sample)))
#2017-11-0713:17Alex Miller (Clojure team)You can spec them as s/coll-of an s/tuple of key value pairs. Thatās not great if you are still relying on attribute keys. I guess you could also use s/keys* on the kv tuple.#2017-11-0714:19mattiaswIf I understand it correctly, s/keys* wants the structure [:a 1 :b 2]
, but I have [[:a 1][:b 2]]
#2017-11-0813:18Alex Miller (Clojure team)Yeah, youād want coll-of keys*#2017-11-0712:08rickmoynihanWhat is the best way to spec a map which has a single required key (s/keys :req-un [::id])
(s/def ::id int?)
where every other (optional) map key is a string?
with a string?
value? Such that (s/valid ::spec {:id 123}) ;; => true
, (s/valid ::spec {"foo" "bar" :id 123}) ;; => true
, (s/valid ::spec {"foo" "bar"}) ;; => false
#2017-11-0713:08taylorhereās a really naive way to do it:
(s/def ::my-map
(s/and (s/keys :req-un [::id])
#(every? (fn [[k v]]
(or (= :id k)
(and (string? k) (string? v))))
%)))
maybe thereās a better way, but Iām not sure how youād combine keys
+ map-of
specs like this#2017-11-0713:13Alex Miller (Clojure team)These are sometimes called hybrid maps - I have a blog about the spec for destructuring which covers the techniques for handling them. http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-11-0713:36taylorvery nice, then something kinda like this might work:#2017-11-0713:36taylor(s/every (s/or :id (s/tuple #{:id} int?)
:str (s/tuple string? string?)))
#2017-11-0713:37rickmoynihantaylor: yes I had something similar to your first, but the use of and
& or
in a single monolothic predicate bothered me, as it kills error message granularity further down the tree.#2017-11-0713:38rickmoynihanI think as youāve discovered the use of every
and or
looks to be how to do it š#2017-11-0713:38rickmoynihanThanks @U064X3EF3 for the pro tips#2017-11-0712:31sushilkumarHow to write spec for āstring of integerā? I want to create a generator of "string of integer" which should only generate valid string of integer (value within Integer/MIN_VALUE
and Integer/MAX_VALUE
).#2017-11-0712:31sushilkumarHow to write spec for āstring of integerā? I want to create a generator of "string of integer" which should only generate valid string of integer (value within Integer/MIN_VALUE
and Integer/MAX_VALUE
).#2017-11-0712:45tayloryou could write a predicate function int-str?
that returns true/false if the given string can be parsed as an integer, then itās trivial to use that predicate as a spec#2017-11-0712:45gfredericksThe generator would be (gen/fmap str an-appropriate-int-generator)
#2017-11-0714:36sushilkumarThanks @U3DAE8HMG and @U0GN0S72R for sharing your ideas. Now I am able to do it as follows and it will also help in writing fdefs
. (defn- in-integer? [x]
(and (>= x Integer/MIN_VALUE) (<= x Integer/MAX_VALUE))) (s/def ::in-integer?
(s/and number? in-integer?)) (s/def ::str-long?
(s/spec string?
:gen #(gen'/fmap str (s/gen ::in-long?))))
#2017-11-0718:54seancorfieldQuick Q about double-in
-- if I specify :min 0.0
and :max 1.0
, does that automatically exclude NaN and infinity? Or do I also need to specify :NaN? false :infinity? false
?#2017-11-0718:56taylorlooking at https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L1630 it doesnāt look like specifying min/max has any effect on NaN/infinity (and they both default to true)#2017-11-0718:57taylorbut (s/valid? (s/double-in :min 0 :max 1) Double/NaN)
=> false
#2017-11-0718:59taylorso I guess it implicitly excludes NaN/infinity by virtue of those not passing the range comparator checks?#2017-11-0719:01taylor(s/valid? (s/double-in :min 0.0) Double/POSITIVE_INFINITY)
=> true
#2017-11-0719:24seancorfieldThanks @U3DAE8HMG That's sort of what I intuitively expected to happen but I wasn't sure how "special" NaN was... I guess that begs the question of what do you do if you want 0.0 .. 1.0
or NaN
? I suppose you have to :or
two specs together... but what would that second spec look like, i.e., how would you allow only NaN
or a range?#2017-11-0719:27taylor#2017-11-0719:27taylora more qualified person could very well have a better answer!#2017-11-0719:59seancorfieldThat allows any double, it seems.#2017-11-0720:33taylorha! yeah it doesā¦ disregard#2017-11-0720:34taylorlooking at double-in
impl. :NaN? true
is a no-op#2017-11-0720:36taylor(s/or :range (s/double-in :min 0.0 :max 1.0)
:nan #(and (double? %) (Double/isNaN %)))
#2017-11-0817:12akhudekWhat is the rational for s/cat returning a map when conformed?#2017-11-0817:13akhudekI suppose there is no way to conform a sequence and get a sequence rather than a map?#2017-11-0817:13akhudekI know you can use coll-of but that doesnāt work if you want a sequence rather than a collection#2017-11-0817:16bfabryyou can make it non conforming#2017-11-0817:17bfabrythe rationale is that s/cat starts a regex spec, and regexes have alternates, so you need to give everything names to know which part of the regex it matched#2017-11-0817:18bfabrythere's a non-documented not currently supported function that makes a spec nonconforming https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1761#2017-11-0817:30rickmoynihanare there any docs/guides etc on conforming / conformers and tweaking output etc? I understand roughly how they work, but am curious about how to apply them in practice. What itās ok to use them for, what itās not a good idea to use them for etcā¦#2017-11-0818:13Alex Miller (Clojure team)conformers exist primarily to build custom composite spec types (s/keys* for example)#2017-11-0818:27rickmoynihaninteresting hadnāt seen s/keys*
before#2017-11-0819:04Alex Miller (Clojure team)s/nilable was originally written this way too although I ended up rewriting a custom impl for better performance#2017-11-0818:13Alex Miller (Clojure team)generally I would say you should not use them for data coercion or tweaking output#2017-11-0818:13zcljI have a case where I try to spec a HOF but I can not get check
to satisfy. Here's a toy example of the problem:
(def uuid-regex #"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
(s/def ::name string?)
(s/def ::age pos-int?)
(s/def ::id (s/and string? #(re-matches uuid-regex %)))
(s/def ::person (s/keys :req-un [::name ::age ::id]))
(s/fdef do-stuff
:args (s/cat :person ::person
:lookup (s/fspec :args (s/cat :p ::person) :ret int?)))
(defn do-stuff [person lookup-fn]
(let [result (lookup-fn person)]
:important-work-on-result))
;; works
(s/exercise ::person 1 {::id (fn [] (gen/fmap str (gen/uuid)))})
;; do not work
(s/exercise `do-stuff 1 {::id (fn [] (gen/fmap str (gen/uuid)))})
;; do not work
(stest/check `do-stuff {:gen {::id (fn [] (gen/fmap str (gen/uuid)))}})
I am also open to that there are better ways to run check
on HOFs that I am missing#2017-11-0818:20Alex Miller (Clojure team)youāll need to supply a generator on s/fspec#2017-11-0818:22Alex Miller (Clojure team)the thing thatās breaking down is really the gen of ::id I think#2017-11-0818:25zcljis there anyway to define an override? For example if the fspec was provided by a lib?#2017-11-0818:32Alex Miller (Clojure team)what youāre doing should be working - I think this is a bug#2017-11-0818:34Alex Miller (Clojure team)(s/exercise (s/fspec :args (s/cat :p ::person) :ret int?) 10 {::id #(gen/fmap str (gen/uuid))})
is minimally sufficient#2017-11-0818:37zcljGood to know, then I can stop scratching my head. Yes, that example yields the same result#2017-11-0818:50Alex Miller (Clojure team)yeah, I see the bug in the code. canāt say I know how to fix it though.#2017-11-0818:52Alex Miller (Clojure team)when the fspec is conformed by exercise (or by check), it āchecksā the generated function spec, but that conform does not have access to the gen overrides so it gens without them.#2017-11-0818:54zcljI see, that explains the observed behavior#2017-11-0818:59zcljshould I file a ticket? Have not done so before but I am willing to try if that would be helpful#2017-11-0818:59Alex Miller (Clojure team)Iām filing one#2017-11-0819:02Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2264#2017-11-0819:05zcljthanks, will follow the activity, and thanks for your help and explanation of the problem#2017-11-0819:07Alex Miller (Clojure team)itās kind of a deep problem - weāve got other tickets that are related but I never took the time to really get it.#2017-11-0819:26zcljI would be happy to help, but it sounds like its above my knowledge of the code base if it is a deep problem, not a novice ticket#2017-11-0818:15seancorfield@rickmoynihan Where I've found conformers useful is dealing with input data that is all strings, but is expected to conform to numeric, boolean, date, etc types.#2017-11-0818:16rickmoynihan@seancorfield: yeah I understand the input coercion at boundaries case#2017-11-0818:16seancorfieldThat's pretty much the only place I'd "recommend" them, having worked with spec in production since it first appeared.#2017-11-0818:19rickmoynihanok thanks, thatās useful. I just remember rich saying that conformed values are basically the parsed/labelled output you want; and itās very close to a shape I want for a specific caseā¦ thinking itās better to conform then post-process, rather than use the conformers to do it.#2017-11-0818:20rickmoynihanbut it occurred to me that if I used conformers then Iād really need to write unformers too, and my spec output would no longer be spec outputā¦ so what youāre saying seems to agree my gut feeling#2017-11-0818:39rickmoynihanso one thing Iād quite like (for myself) is to write a variant of s/keys
and s/def
that uses URIs as keys instead of clojure keywords.
I work with RDF, so mapping to keywords just to spec something feels somewhat redundant, when the URI straight out of the database is basically the same thing so Iād like to basically save a redundant mapping and write:
(def rdfs:label (URI. "http://,,,/label"))
(rdf/def rdfs:label string?)
(rdf/keys :req [rdfs:label])
And have it do the same thing as keys. Is there an easy way to do this? Thinking I need to extend s/Spec
/`s/Specize` to URI
and then rewrite s/keys
? s/keys
looks pretty hairy, is there any easy way?#2017-11-0819:03Alex Miller (Clojure team)I think this is roughly how you would do this, yes, and it would not be super easy. Also, I expect some of that plumbing to possibly change soon.#2017-11-0819:11rickmoynihanIām assuming there are no plans/proposals in the pipeline to let you plugin and you use arbitrary values as keys/keywords?#2017-11-0818:40rickmoynihanor am I thinking the thoughts of a madman?#2017-11-0818:43rickmoynihanI think what rich says about the design influence of RDF on clojure and spec is very clear; in many ways theyāre almost identical. Iād basically like to reduce the friction of working in clojure with RDFā¦ Also compare SHACL/SHEX to spec, theyāre almost just different syntaxes for the same abstract model.#2017-11-0818:58akhudekok, thanks for the comments bfabry and alexmiller, Iāll avoid using spec for transforms.#2017-11-0900:02rickmoynihanI think Iāve found a bug in clojure 1.9 / clojure.core.specs.alpha:
(if-let [foo 1] foo :clojure.spec.alpha/invalid)
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/if-let did not conform to spec:
In: [2] val: :clojure.spec.alpha/invalid fails at: [:args :else] predicate: any?
#:clojure.spec.alpha{:problems ({:path [:args :else], :pred clojure.core/any?, :val :clojure.spec.alpha/invalid, :via [], :in [2]}), :spec #object[clojure.spec.alpha$regex_spec_impl$reify__1188 0xeabf22e "
#2017-11-0900:05rickmoynihanissue appears to be that you canāt return the value :clojure.spec.alpha/invalid
from macros. Itās kinda like a macro hygiene issue, but for s/invalid
#2017-11-0900:06bronsait's known#2017-11-0900:07rickmoynihancool. Figured it would be#2017-11-0900:08bronsasee https://dev.clojure.org/jira/browse/CLJ-1966#2017-11-0900:12rickmoynihanthanks#2017-11-0912:26sushilkumarHello everyone, I am learning Spec'ing higher order functions.
Here are the examples which I am experimenting with (actual HOFs I am working on are quite complex) :
(Works fine with exercise-fn.)
(defn hof1 [x f] (f x))
(s/fdef hof1
:args (s/cat :x number?
:f (s/fspec :args (s/cat :y number?)
:ret number?))
:ret number?
:fn #(= ((-> % :args :f) (-> % :args :x)) (:ret %)))
(exercie-fn doesn't work :(. Here I am expecting input f to be either + or - or * fn.)
(defn hof2 [f x] (f x 1))
(s/fdef hof2
:args (s/cat :x number?
:f (s/fspec
:args (s/cat :y number?)
:ret number?))
:ret number?
:fn #(= ((-> % :args :f) (-> % :args :x) 1) (:ret %)))
How write spec to ensure correctness in the samples generated using exercise-fn for HOFs? (I want to use spec's check fn to test HOFs on generated inputs based on written spec.)#2017-11-0914:29Shantanu KumarHi, Iām a spec noob here ā can somebody tell me an easy way to automatically (st/instrument)
in all namespaces?#2017-11-0914:30joost-diepenmaatyou only need to call st/instrument once#2017-11-0914:30joost-diepenmaatbut you have to make sure youāve loaded all the namespaces before you do#2017-11-0914:30joost-diepenmaatif itās in your dev setup, just call st/instrument in your main or dev namespace at the bottom#2017-11-0914:31joost-diepenmaatif itās in tests, your best bet is probably to use a fixture#2017-11-0914:31Shantanu Kumar@joost-diepenmaat Ah OK, thanks!#2017-11-0915:07Shantanu KumarAlso, how do I generally make ādid not conform to specā errors easier to read? Iām using Orchestra+Expound and have specified the following, but to no avail:
(st/instrument)
(set! s/*explain-out* expound/printer)
as mentioned here: https://github.com/bhb/expound#using-orchestra#2017-11-0915:20bbrinck@kumarshantanu Can you post an example of the code youāre calling and the ādid not conform to specā message you get?#2017-11-0915:22bbrinckSetting expound/printer
seems to work for me https://gist.github.com/bhb/f538f2b1a47eb7a1514da0fe62d46265#2017-11-0915:34Shantanu KumarThis gist works for me at the REPL. @bbrinck The error reporting Iām dealing with extracts the exception message and stack trace and sends it as HTTP response. Is Expound supposed to work in that use case?#2017-11-0916:03bbrinckYes, it should. For instance, building on my previous example, here is code to catch the instrumentation exception and return the message as a string: (try (foo "") (catch Exception e (.getMessage e)))
#2017-11-0916:04bbrinckAnother option is to use Expound to validate incoming data directly. For instance, letās say the HTTP endpoint takes some JSON as incoming data#2017-11-0916:06bbrinckThen you could validate that data with something like:
(binding [s/*explain-out* expound/printer]
(s/explain-str :my-app/http-data-spec http-data))
#2017-11-0916:08bbrinckBy default, Expound writes out all the Clojure specs that failed, but there is an (undocumented, but soon to be documented) option to omit this#2017-11-0916:09bbrinckif youāre interested in that, I can give some example code#2017-11-0918:49bbrinckIf anyone else is interested, the issue was the dynamic var *explain-out*
is only set in the current thread, and servers like HTTP Kit spawn new threads for requests. The fix is to replace (set! s/*explain-out* expound/printer)
with (alter-var-root #'s/*explain-out* (constantly expound/printer))
#2017-11-0917:38naomarikthere a better way of having a generator that just executes a function with no regard to the input? this does what i want, but notice the argument here is unnecessary #(gen/fmap (fn [_] (utils/uniform 1e12 1e13)) (s/gen pos-int?))
#2017-11-0918:58taylorI was going to suggest gen/return
but do you even need the output of the pos-int?
generator?#2017-11-0919:04taylorfor example, doesnāt matter what the previous generator is doing if youāre just constantly returning the same thing regardless of input
(gen/sample (gen/fmap (constantly -1) gen/pos-int))
=> (-1 -1 -1 -1 -1 -1 -1 -1 -1 -1)
#2017-11-0919:23gfredericksthis is a nonstandard generator construction
The "right way" would be to use combinators somehow rather than calling a function that supplies its own randomness.
That might be a bit more effort, depending on what you want.
If you aren't doing things the "right way", then there's not really a better way.#2017-11-0919:24gfredericksdo you really need a uniform distribution between two doubles?#2017-11-0919:24gfredericks(gen/double* {:min 1e12 :max 1e13})
would not be very nonuniform I don't think#2017-11-1305:56naomarik@U3DAE8HMG yeah my point was i didnāt need the output of the pos-int? generator. gen/return
was EXACTLY what i was looking for, thanks š#2017-11-1305:57naomarik@U0GN0S72R yeah i need uniform distribution between two large numbers because made a spec that expected a number beteween those two ranges and this issue causes it to fail without a custom generator https://dev.clojure.org/jira/browse/CLJ-2179#2017-11-1305:58naomariki also stumbled upon #(gen/choose 1e12 1e13)
that does uniform distribution#2017-11-1312:55gfredericksthat ticket is interesting#2017-11-1312:55gfredericks@U0DHHFEDP what is your spec for exactly?#2017-11-1312:55gfredericksI'm having trouble seeing why something would fail#2017-11-1313:27naomarik@U0GN0S72R itās a ghetto spec for a unix timestamp in ms, the generator fails because it attempts to make ints starting at very low numbers. hereās the code
(s/and pos-int?
(s/int-in 1e12 1e13))
#2017-11-1313:30naomarikit passes most the time when ran by itself but i have it nested in another datastructure and it fails 100% of the time#2017-11-1313:30gfredericksoooooh; this is an s/and
issue, not an s/int-in
issue#2017-11-1313:30gfredericksif you swap the order of those two you should have it succeed#2017-11-1313:31gfredericksor set the generator to be the default generator that the s/int-in
expression generates#2017-11-1313:31gfredericksI think CLJ-2179 is a red herring here#2017-11-1313:33naomariktried swapping the order, didnāt work#2017-11-1313:35naomariki think the issue is legit, if you try running the code (gen/sample (s/gen (s/int-in 0 100)))
you get a ton of low numbers#2017-11-1313:35naomarik(0 0 0 1 0 4 0 52 3 34)#2017-11-1313:35naomarikthings like this#2017-11-1313:36naomarikmy final code works well enough though
(s/with-gen (s/and pos-int?
(s/int-in 1e12 1e13))
#(gen/choose 1e12 1e13)
))#2017-11-1313:36naomariksorry for formatting š#2017-11-1317:29gfredericks@U0DHHFEDP getting low numbers that are in the range is one thing, but I thought you were describing getting numbers that were out of range#2017-11-1402:33naomarik@U0GN0S72R thatās what I thought was going on, i think my issue is that i was plugging in doubles using the short notation. It seems (s/int-in) doesnāt cast those for you but will verify that the input is in range, this is why I needed to pair it with pos-int#2017-11-1402:36naomarik;; Fails reliably
(s/def ::test2 (s/and pos-int? (s/int-in 1e12 (- 1e13 1))))
;; Fails sometimes
(s/def ::test3 (s/and pos-int? (s/int-in 1e9 (- 1e10 1))))
(s/def ::works (s/int-in (encore/as-int 1e13) (encore/as-int (- 1e14 1))))
#2017-11-1402:38naomarikbumping the numbers up in the first example makes it fail a lot more often, I guess pos-int is the only thing generating numbers here?#2017-11-1402:43naomarikin any case was my fault for plugging doubles in something that is clearly for ints š#2017-11-1412:06gfredericksyeah, when you use s/and
the generator is taken from the first spec and the remaining specs are used to filter#2017-11-1019:02uwoI know this question comes up all the time, and that multi-spec is a way to solve it. but just asking again to see if thereās another solān:
we want to ensure that a data structure at some point in our system conforms to a certain shape. The data is nested and the keys already have fixed namespaced names. Itās easy to specify required keys at the top level, but at every nested level the key already has a fully qualified name, and that fully qualified key has different requirements in different system contexts.
The only two solāns I see atm are multi-specs and key-name rewriting. Using an empty (s/keys) is not a solān in this case, because we need conform to ensure that required fields at nested levels are present.#2017-11-1019:49Alex Miller (Clojure team)when you say āfully qualified key has different requirements in different system contextsā you have started off by violating specās assumption#2017-11-1019:49Alex Miller (Clojure team)unless you spec it to cover all possible contexts#2017-11-1020:52uwo@alexmiller gotcha. I figured that was the answer. Those fully qualified key names are the ones we place in datomic, and they get shuffled all over the app. Itās just different requirements for the āsameā data in different contexts#2017-11-1020:55qqqI'm looking at:
(s/def ::ingredient (s/cat :quantity number? :unit keyword?))
(s/conform ::ingredient [2 :teaspoon])
;;=> {:quantity 2, :unit :teaspoon}
I'm starting to get the impression spec is NOT ONLY for unit testing / debugging, but also something intended to be used in live production code ?#2017-11-1020:55qqqIt's not clear to me how this s/conform is useful, unless it is meant to be used in production to 'reshape' data#2017-11-1021:14taylorI donāt think itās intended to arbitrarily reshape data, but the ātaggedā output from conform
can be useful. For example, if youāre expecting different types of inputs, s/or
will tag the matched spec in the conform
output, otherwise itād be hard to tell which spec matched#2017-11-1021:15taylorI do use it in production for similar cases, one example is a message handler that receives a few different types of messages that donāt have a natural ākeyā. conform
output tells me exactly which kind of message it is#2017-11-1021:38seancorfield@qqq We use spec very heavily in production code. We s/conform
input values from forms and REST APIs to the desired shape/types.#2017-11-1021:40seancorfieldWe define a spec for each API endpoint, and we s/conform
the raw Ring :params
to the spec. If it's s/invalid?
then we use s/explain-data
to guide us in returning error code and messages, otherwise we have a validated parameters map.#2017-11-1021:41seancorfieldWe limit the coercions tho'... We do string->long, string->boolean, string->date, and very little else.#2017-11-1023:17qqq@seancorfield: interesting, I was thinking that something vaguely along those lines would be possible ; nice to see someone do it in practice#2017-11-1023:17qqqunrelated spec question: how does s/or and s/alt differ ?#2017-11-1023:19qqqI'm running them with s/conform ... and getting identital results#2017-11-1023:31seancorfields/alt
is for alternatives in a (sequence) regex; s/or
is for specific alternatives in a spec.#2017-11-1023:31seancorfield(so s/alt
mostly goes with s/cat
etc)#2017-11-1023:41qqqIs there a way to define a spec, then selectively, on valid? calls turn "recursion on / recursion off" ?
Practically, imagine we are speccing SVG.
For a given SVG node, there are (1) properties of this node and (2) the children of this node.
I want to be able to define the spec once, and then selectively say:
1. check that this node itself is valid (but don't care much about its children)
vs
2. check that this node itself is valid, and that all its children (and all descendants) are valid too
so one is a "shallow, just this level check"
another is a "deep, check the entire tree check"#2017-11-1023:55bfabryI'm 99.4% sure that the answer is "define 2 specs"#2017-11-1102:07qqqthe problem with two specs is that in the case of maps, the key is used to pic, the spec for the value#2017-11-1102:07qqqwhich mean the data 'decides' whether the check is recursive or non-recursive#2017-11-1105:16seancorfield@qqq But that's exactly the point: with two separate specs you control whether the keys are deep or shallow -- assuming you have unqualified keys in the map and can use different namespace-qualifiers for shallow vs deep.#2017-11-1105:25qqq@seancorfield: but I want to determine whether to make a shallo/dep check at the function call, not at the point when I construct the data#2017-11-1105:26qqqI want a single piece of svg data
in some functions, I want to make an expensive deep check taht it's an valid svg tree
ikn other functions, I want to make a shallow check that "current node is good; we know nothing abut chiklren"#2017-11-1109:30mattiaswI need just 1 running one call to the function to check at a time. Inside the function, there is a db call, so I need to run checking in single-thread, so (check 'test-many)
doesn't work. I tried to use check-fn, but this doesn't even start calling the function. (stest/check-fn 'test-many (s/fspec :args (s/cat :samples (s/coll-of :hashdb.db.commands/data)) :ret nil?))
(I changed backquote to quote when publishing to slack.#2017-11-1201:52James VickersIs there a 'normal' way to use function specs in your automated test suite? I feel like I haven't found a good solution to this - likely because I haven't written a regular test.check
suite either.#2017-11-1201:54gfrederickswrapping a call in clojure.test/is
is pretty easy#2017-11-1202:08James VickersI was wondering if there was already (`test.check`?) library function that would do something like that (some 'idiomatic' way).#2017-11-1202:09James VickersIs using function specs in the test suite a normal thing that people do, or are folks just using spec
for development time help?#2017-11-1202:15gfredericksI would certainly put them in the test suite if I were using spec#2017-11-1202:31seancorfield@jamesvickers19515 We use spec in all sorts of ways. We use it heavily in production code for validating and conforming input parameters, specifying data structures (and then leveraging the form of the spec to "reflect" and get the set of keys that are "permitted", e.g., when storing hash maps in a SQL table), in tests via s/instrument
, in tests via s/exercise
to generate conforming test data.#2017-11-1202:32seancorfieldIn some cases we use test.check
as part of a regular test, but generally we have those as separate tests we run apart from our unit tests -- generative testing often takes quite a while so it probably shouldn't be part of your unit test suite.#2017-11-1203:44jeayeI'm seeing an inconsistency between Clojure and ClojureScript for multi-arity fdefs. https://gist.github.com/jeaye/eaa8da70171a538e23b3c5dbfd9797fd#2017-11-1203:45jeayeThat spec catches the errors just fine, on Clojure, for all the arities. On ClojureScript, bad calls don't get caught by instrumentation at all.#2017-11-1203:50jeayeUpdated the gist; looks like a simple defn
, with the same spec, throws for both cases (as it should). So this is specific to something happening in ClojureScript's multi-arity defn
.#2017-11-1216:35qqq(def color-regex #"^#[0-9a-z][0-9a-z][0-9a-z][0-9a-z][0-9a-z][0-9a-z]")
(s/def ::stroke-color (s/and string? #(re-matches color-regex %)))
(s/valid? color-regex "#ffaa88")
what am I doing wrong here, it is complaining that regex can not be converted into a function#2017-11-1216:45qqqthe last line should be:
(s/valid? ::stroke-color ...)
#2017-11-1304:53qqqis there a way in spec to say
::x is a vector of numbers
::y is a vector of numbers
::s is a string
and the three have the same length (thisxis regarding to an s/keys)#2017-11-1304:55qqqthis is what I have so far:
(s/def ::text-x (s/coll-of number? :kind vector?))
(s/def ::text-y (s/coll-of number? :kind vector?))
(s/def ::text-str string?)
but it's not clear to me how to specify the three have the same length#2017-11-1306:24seancorfield@qqq Are you using them in a context together? Sounds like you could s/and
a length check on top of whatever structure uses all three?#2017-11-1306:29qqq@seancorfield
(s/def ::text-x (s/coll-of number? :kind vector?))
(s/def ::text-y (s/coll-of number? :kind vector?))
(s/def ::text-str string?)
(s/def ::svg-text (s/keys :req [::text-x ::text-y ::text-str ]))
is what I have so far#2017-11-1306:30qqqYou are recommending (s/and (s/keys ...1) ...2)
?#2017-11-1306:30qqqin ...2
, how do I access the text-x, text-y, text-str fields ?#2017-11-1307:34jumar@qqq something like this?
(defn- vals-same-length?
[m]
(->> (vals m)
(map count)
(apply =)))
(s/def ::svg-text-same-length (s/and (s/keys :req [::text-x ::text-y ::text-str ])
vals-same-length?))
(s/valid? ::svg-text-same-length {::text-x [1 2 3]
::text-y [10 20 30]
::text-str "abc"})
(There might be more straightforward implementation of vals-same-length?
)#2017-11-1307:38qqq@jumar: I did not realize I can just pass in an arbitrary clojure function.#2017-11-1307:38qqqThis is very useful, thanks!#2017-11-1308:07seancorfield(sorry, was distracted by the amazing availability of early 80's German electronica on iTunes these days... yes, @qqq what @jumar said... when you use s/and
you can have any predicate and it has access to the entire form being processed, so you could access just ::text-x
, ::text-y
, ::text-str
if you wanted, or vals
to get all the map's values.#2017-11-1308:07seancorfieldI was thinking you were using this with s/cat
for argument specifications, but the same principle applies. Specs are just predicates. It's all functions, all the way down š#2017-11-1319:37degI need to define a spec for a map that holds keys owned by multiple parts of my code. So, looking a a tasteful way to build it incrementally from multiple files. (So that each namespace adds in the parts it understands).
This strikes me as non-trivial, especially since I can't just have a defonce'd atom that I assoc into. Because s/keys is a macro, I need to assemble only after I've gathered all the parts.
Sounds like I need to think about file loading order and other issues that I've ignored until now in my Clojure/Script.
Is there a straightforward way to do this? Or am I barking up the wrong trees?#2017-11-1319:44hiredmanI am not sure what you are asking?#2017-11-1319:45hiredmanif each key is owned by a different part of your code, and you want to spec each key do that#2017-11-1319:45hiredmanand then in whatever central place, create your keys spec from each key#2017-11-1319:46hiredmanthe structure of clojure requires you to have a central place, the way the ns macro and everything else works requires you to have a defined load order and explicitly require the code you need access to#2017-11-1319:52qqqdoes s/coll-of check every element, or does it randomly pick a few and test them ?#2017-11-1319:52taylor> coll-of will exhaustively conform every value#2017-11-1319:53tayloras opposed to every
:
> Note that āeveryā does not do exhaustive checking, rather it samples
coll-check-limit elements.#2017-11-1320:05bbloombeen away for a bit - i remember core had some private convenience macro for checking macro invocations. whatās the latest recommendation on that? i actually want to use the spec for parsing inside the macro, but itās a bit awkward to conform and then if that fails explain, etc#2017-11-1320:23deg@hiredman - I was looking for a way to avoid having to know about all the parts at one central place.#2017-11-1320:25degThat is, I don't want to say (s/keys :opt [ns1/key1 ns2/key2 ,,,])
, but instead create the spec dynamically.
This seems trickier than it should be, since keys
is a macro, so I can't just do a trivial atom/assoc/apply dance.#2017-11-1320:46hiredmanit sounds like you want the already existing feature in spec where a s/keys spec will check specs for keys that exist in the map that have a spec even if they aren't part of the s/keys spec#2017-11-1321:15qqqI assume this is more appropriate for blog post reading rather than discussion:
is there a good writeup somewhere of qualified vs unqialified names for specs ?
Everything I've done so far in clojure is unqualified, i.e. :foo kws, but looking at spec, it seems like the default / preferred wa yis to be qualified i.e. ::foo or :blah.namespace/foo#2017-11-1321:45arohnerI have a uuid
. Is there a smart way to bias the generator so I can control the distribution of uuids? i.e. completely random vs. power law?#2017-11-1321:46arohnerIām aware of e.g. gen/elements
, was wondering if thereās something else#2017-11-1321:46gfredericksyou're trying to get collisions?#2017-11-1321:46arohneryes, some of the time#2017-11-1321:47arohnerbut Iād really like e.g. power laws on repeat UUIDs, so out of 1000 uuids, one appears 100 times, one appears 20 times, etc.#2017-11-1321:48arohnerreading the spec source was informative. Iāll just with-gen
with a UUID constructor#2017-11-1321:49gfrederickswhere will you get the args for the constructor?#2017-11-1321:50arohnerhttps://github.com/danlentz/clj-uuid#2017-11-1321:50arohnertype 5s are deterministic, and I suspect I can create v1s by specifying a unix time#2017-11-1321:50gfredericksif you use gen/large-integer
and gen/fmap
that to the type-3 uuid constructor, that will give you something like the distribution you want#2017-11-1321:51gfrederickskeep in mind how distribution relates to size
#2017-11-1321:51arohnercan you expand on that?#2017-11-1321:53gfrederickshave you seen this? https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2017-11-1321:55gfredericksI also talked about it some here if you liked faces and voices and slides and things https://youtu.be/F4VZPxLZUdA?t=25m26s#2017-11-1321:56arohnervery useful, thank you#2017-11-1400:58qqq(defn tag-is [t]
(fn [x]
(= (:tag x) t)))
;; ** vec2
(s/def ::x number?)
(s/def ::y number?)
(s/def ::vec2 (s/and (s/keys :req [::x ::y])
(tag-is ::vec2)))
;; ** bbox
(s/def ::x0 number?)
(s/def ::x1 number?)
(s/def ::y0 number?)
(s/def ::y1 number?)
(s/def ::bbox (s/and (s/keys :req [::x0 ::x1 ::y0 ::y1])
(tag-is ::bbox)))
(defn vec2-new [{:keys [x y]}]
{:tag ::vec2
::x x
::y y})
(defn bbox-new [{:keys [x0 x1 y0 y1]}]
{:tag ::bbox
::x0 x0
::x1 x1
::y0 y0
::y1 y1})
Am I using spec right nere? the vec2-new and bbox-new seem very UN-clojurish 'every is data' approach.
However, given that I'm using ::x ::y ::x0 ::y0 ... I don't know how else to nicely create vec2/bbox objects#2017-11-1401:15taylormulti-spec
seems like it might better fit that use case https://clojure.org/guides/spec#_multi_spec#2017-11-1401:17taylorthat would obviate the tag-is
thing. the *-new
fns at the bottom seem like theyāre just converting map keys from unnamespaced to the current namespace, why not use the namespaced keys upstream?#2017-11-1401:17taylor(using unnamespaced keys in your spec is also an option)#2017-11-1401:33qqq@taylor: because namespace 'object' is constructed != namespace 'object' is specced#2017-11-1402:08seancorfield@qqq In the upstream code, do you already have maps with unqualified keys? If so, where did they come from?#2017-11-1402:09seancorfieldIf you don't want the keys namespaced, change :req
to :req-un
(then bbox-new
becomes just (assoc data :tag ::bbox)
)#2017-11-1402:11qqq@seancorfield: all code is code I have written (and can change -- am rewriting with spec -- and trying to figure out best practices)#2017-11-1402:11qqqI actually think qualified makes more sense than unqialified, since a word may mean different things n different namesapces#2017-11-1408:16deg@hiredman I did not know that, thanks!. Was it always the case that s/keys
checks other keys too? Where is this documented?
Meanwhile, I realized that another part of the solution for me is to use s/merge
. That way, the central spec needs to only about the existence of each module, not its behavior.#2017-11-1413:05madstap@deg It's documented here https://clojure.org/guides/spec#_entity_maps#2017-11-1420:06ajmagnificoQuestion: In general, do you find itās better to define specs in the namespaces that use them? Or in a separate āappname.specsā namespace?#2017-11-1420:07ghadipersonally, separate#2017-11-1420:35ajmagnificoI think that the advantage of defining them in the namespaces where they are used is one of brevity.#2017-11-1420:35ajmagnificoFor example, with nested data structures#2017-11-1420:35ajmagnifico#:longappname.specs{:a 1 :b 2}
#2017-11-1420:36ajmagnificothis is less verbose than {:longappname.specs/a 1 :longappname.specs/b 2}
#2017-11-1420:36ajmagnificoBut the #:
syntax only applies to the top level of the map#2017-11-1420:36tayloryou can alias the namespaces for brevity#2017-11-1420:41ajmagnificoSo, Iām loading a config file that looks like this#2017-11-1420:41ajmagnificoand Iām trying to spec it#2017-11-1420:41ajmagnificoI know I can use (spec/keys :req-un ā¦)
#2017-11-1420:41ajmagnificoBut Iād like to use namespaces#2017-11-1420:43ajmagnificoAnyway, all of the pieces of this config are currently in my appname.specs
namespace#2017-11-1420:43ajmagnificoWhen I load it and try to validate it, it fails, because all of my specs are namespaced#2017-11-1420:43ajmagnificoBut Iād rather not have to put #:appname.specs
in front of each map in the config#2017-11-1420:44ajmagnificoSo if I use the alias strategy like taylor said, Iād need to choose an arbitrary alias ahead of time that I would use in the config file, and that would need to match up with an alias that I would create in the namespace where Iām actually using and validating this config data.#2017-11-1420:45ajmagnificoOr, if I def
ed the specs in the namespace where the data will be loaded and used,#2017-11-1420:45ajmagnificothen I can just put a ::
in front of each of the keys and everything will work as intended#2017-11-1420:46ajmagnificoBut am I going to get burned down the road if I have all of my spec definitions scattered throughout a bunch of namespaces instead of keeping them all in one location?#2017-11-1420:46taylorI guess Iād ask myself what problem am I solving by namespacing all the keywords in my config map#2017-11-1420:46tayloror is it instead creating more problems#2017-11-1420:47taylorwhereās your config defined, how is it loaded?#2017-11-1420:48ajmagnificoOkay, thatās a fair question taylor. I guess at this point itās me still trying to learn the idiomatic way of doing spec as it was intended by its creators. And they seem to think that namespaced keywords are a beautiful thing. Iām still trying to figure out how theyāre beautiful. š#2017-11-1420:48ajmagnificobest practices#2017-11-1420:48ajmagnificoso Iām wondering if ābest practicesā here are, like you said, creating more problems#2017-11-1420:48ajmagnificoor if Iām just doing it wrong.#2017-11-1420:49taylorI use specs for maps with unnamespaced keywords all the time, personally. I donāt think thereās anything necessarily wrong with it#2017-11-1420:49ajmagnificoAnyway, the config is stored in edn format in a hand-edited file, loaded with edn/read-string
during app initialization#2017-11-1420:50taylorbut then again most of the maps I work with are deserialized from some format that doesnāt support namespaces#2017-11-1420:52taylorand FWIW you could use any āmade upā namespace for your config keywords, it doesnāt necessarily have to match a real namespace in your project#2017-11-1420:53ajmagnificothatās an interesting idea#2017-11-1420:53ajmagnificothanks for the discussion, taylor#2017-11-1420:53ajmagnificoIāll probably just end up using :req-un#2017-11-1420:54taylorthatās what Iād do, but Iām lazy š#2017-11-1420:54ajmagnificothe hallmark of every great programmer#2017-11-1508:40tbaldridge@ajmagnifico something to consider, Spec also recommends flat maps:#2017-11-1508:42tbaldridge{:cnx-info.source.db/host :env/DB_HOST
:cnx-info.source.db/port :env/DB_PORT
:cnx-info.dst.db/host "localhost"
...
:job/dir "some-directory}#2017-11-1514:35uwo@tbaldridge interesting, is that implicit in its design, or did I miss that recommendation somewhere?#2017-11-1514:39slipsetHow would you spec a map of ints to maps, eg#2017-11-1514:39slipset{1 {:foo "bar"} 2 {:foo "qux"}}
#2017-11-1514:39taylorhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/map-of#2017-11-1514:40slipset@taylor Thanks!#2017-11-1514:41taylor(s/map-of int? map?)
but you could get more specific (s/map-of int? ::some-complex-keys-spec)
#2017-11-1514:41slipsetI did the second š#2017-11-1514:50ajmagnificothanks @tbaldridge. Thatās a really good idea, and it will actually solve a couple of other problems for me as well. I didnāt recall that keyword namespaces donāt actually need to correspond to real namespaces, even though thatās exactly what taylor told me! I guess I just needed to see it concretely for it to click.#2017-11-1514:50ajmagnificoI guess the only thing you miss out on when you make up namespaces is use of the ::
syntax#2017-11-1514:50ajmagnificobut in my case, thatās a small price to pay#2017-11-1523:27dpkpI'm trying to spec a variadic function that has a gnarly java-esque method signature like (f str int)
(f str int bool)
(f str int bool int)
. I've tried to model this w/ s/cat but I'm having trouble building a spec that says (f str int int)
is invalid but (f str int bool int)
is valid. Example:
(s/def ::foo (s/cat :a string? :b int? :c (s/nilable boolean?) :d (s/nilable int?)))
#2017-11-1523:29dpkpthis works if args are supplied as nil. but I can't figure out how to also support the smaller arg list. (s/valid? ::foo ["a" 1])
is false#2017-11-1523:29dpkpany pointers?#2017-11-1602:45Alex Miller (Clojure team)Other suggestions were fine but also could use nested s/? for optional stuff#2017-11-1605:13dpkpSomething like this?
(s/def ::foo (s/cat :a string? :b int? :opts (s/? (s/cat :c boolean? :opts (s/? (s/cat :d? int?))))))
#2017-11-1612:47Alex Miller (Clojure team)Yep#2017-11-1523:29xandrewsgive https://clojuredocs.org/clojure.spec/alt a try#2017-11-1523:30dpkpok, will do! thanks#2017-11-1523:31xandrewslike (s/def ::foo (s/alt :two-arg-version (s/cat :a string? :b int) :three-arg-version (s/cat :a string? :b int? :c boolean?) ...etc)
#2017-11-1523:31xandrewslike (s/def ::foo (s/alt :two-arg-version (s/cat :a string? :b int) :three-arg-version (s/cat :a string? :b int? :c boolean?) ...etc)
#2017-11-1523:41xandrews@U1NCSDGDQ in my haste I forgot to include s/cat
in the mix#2017-11-1523:41xandrewshere was an example that works for me#2017-11-1523:41xandrews(s/def ::foo
(s/alt
:two-args (s/cat :a string? :b string?)
:three-args (s/cat :a string? :b int? :c int?)))
(s/conform
::foo
'("hey" "foo"))
(s/conform
::foo
'("hey" 1 2))
#2017-11-1523:32dpkpaha!#2017-11-1612:39stathissiderisdoes anyone have a good way to instrument everything before running lein test
from the command line?#2017-11-1612:47gfredericks@stathissideris leiningen injections ought to do it#2017-11-1612:59stathissideris@gfredericks thanks, Iāll read up on that š#2017-11-1614:27souenzzo(defmulti mkop :op)
(defmulti mkop' :op)
(s/fdef mkop
:args (s/cat :args (s/multi-spec mkop' :op)))
(defmethod mkop :sum
[{:keys [a b]}]
(+ a b))
(defmethod mkop' :sum
[_]
(s/keys :req [::a ::b]))
(defmethod mkop :inc
[{:keys [a]}]
(inc a))
(defmethod mkop' :inc
[_]
(s/keys :req [::a]))
There is something wrong with this "pattern" for spec on multispec? Someone else doing something like this? Other options?#2017-11-1616:37gklijsNot really sure what your trying to do. @souenzzo#2017-11-1617:16souenzzo@gklijs I'm trying to make a spec to a multimethod#2017-11-1620:00jumar@souenzzo I don't have an experience with multi-spec per se, but if you are trying to to instrument multimethod, that's not currently possible with spec. You'll need to create a wrapper function for multimethod and spec that one.#2017-11-1617:17gklijsok, no experience with that, I was busy using multi methods handling specs, so I can edit certain data based on itās spec in the browser#2017-11-1708:14otfromwhen writing the :fn bit of an fdef, doesn anyone have any tips or examples to figure out which bit of the fn failed if there are multiple properties that should hold?#2017-11-1708:15otfromor
with labels almost gives what I want, but obviously isn't logically correct. I want something like all
#2017-11-1708:30tcouplandsomething abit like:
(s/all [{{:keys [first]} :args second :ret}]
:value-mismatch #(= (:value first)
(:value second)))
would be very tidy for checking a value was copied correctly in the function#2017-11-1710:43trissis there a way to write a spec that says: a collection containing two or more integers and anything else?#2017-11-1710:46gklijswould became something like, and it would need to be a collection, and when filtered on number? the size is minimal of 2#2017-11-1711:09tcoupland@triss sounds like an s/cat which makes use of some of the regexp elements#2017-11-1712:15OlicalDoes anyone know if you can spec a map and have it conformed into multiple parts depending on the key. For example, conform {:A 1 :B 2 :a 1 :b 2}
into something like {:upper {:A 1 :B2} :lower {...}}
. Basically a group-by during a conform.#2017-11-1712:15OlicalI essentially want different rules for different kinds of keys in a map. So certain styles of keys would have different value requirements.#2017-11-1712:17andre.stylianosFrom what I've been seeing of the usage of spec, I'd guess not. Seems like it falls under the "data transformation" camp which is not something they intend spec to be used for.#2017-11-1712:18andre.stylianosbut I might be wrong#2017-11-1712:18OlicalYep, that's how I was beginning to feel about it. I'm currently doing that transformation after the fact, but it's hard to find them in a recursive spec, so I can't do the transform up front ahead of time.#2017-11-1712:18OlicalI don't want to walk the tree multiple times is all.#2017-11-1712:19OlicalI would quite like different specs for different kinds of keys though, not sure if I can do that.#2017-11-1712:19OlicalIn my case, if it starts with on-
I want it to be restricted to a function value. If it's anything else it should be a string.#2017-11-1712:20andre.stylianosmakes sense. Does it have to be a tree? You could {:upper/A 1 :upper/B 2 :lower/a 1...}
#2017-11-1712:21andre.stylianosand not quite sure what you mean by "different specs for different kinds of keys though, not sure if I can do that."#2017-11-1712:23OlicalYup, I'm not sure either š It's like, I don't care about the actual keys, as long as they're keywords. But if it starts with on-
it should be a function. I wonder if I can combine map-of
and keys
. It's almost like I want two stages, conform as much as you can with this spec, THEN conform with this spec.#2017-11-1712:23OlicalRight now I'm just doing map-of keywords to string or function.#2017-11-1712:24OlicalI just need constraints between key and value. Otherwise if I exercised this spec it would generate invalid runtime data, but it would match the spec.#2017-11-1712:24andre.stylianosYeah, for what you're saying it seems like map-of
would be a fitting choice#2017-11-1712:26andre.stylianosBut I'm not sure if there's a way to treat the key/value pair as a whole with map-of
#2017-11-1712:28OlicalI think I need a hybrid that doesn't exist š
#2017-11-1712:28andre.stylianosAnother option might be coll-of
if map-of
doesn't work, since that way you could probably spec each entry as [k v]
pairs#2017-11-1712:31OlicalFor context, this is an attribute map that will be applied to DOM nodes in this little Reagent/React like thing I'm building.#2017-11-1712:31OlicalSo most keys will just be strings that go to setAttribute on a DOM node, but event listeners are special things that are handled separately.#2017-11-1712:31OlicalI'm using spec as a parser of my hiccup like data structure basically.#2017-11-1712:35OlicalAh, hah! https://groups.google.com/forum/#!topic/clojure/WiMV5EEAVhM#2017-11-1712:35OlicalPeople with the same issue, linking back to this Slack.#2017-11-1712:35OlicalFound a strange loop š#2017-11-1712:36OlicalAaaaand bingo https://stackoverflow.com/questions/38151446/how-can-i-spec-a-hybrid-map#2017-11-1712:36OlicalAlex to the rescue.#2017-11-1714:01andre.stylianosnice one!#2017-11-1714:42otfromanyone out there have good examples of complex fn's on a :fn for a fdef?#2017-11-1714:42otfrom(esp ones that do good reporting of what bit of the fn has failed)#2017-11-1714:47otfromwhich is really just me bumping this: https://clojurians.slack.com/archives/C1B1BB2Q3/p1510906494000281#2017-11-1800:45qqqdoes spec have a way to capture:
this object is an ATOM and when you deref the ATOM, it should satisfy this spec#2017-11-1800:57taylorI havenāt seen anything like that. I suppose you could wrap an atom and hook spec on derefs?#2017-11-1800:57qqqWe can set a validator-function on the atom, so it'll auto reject on swap! / reset! if it fails the spec.#2017-11-1800:57taylorinteresting#2017-11-1800:57qqqHowever, I now want a way to get the validator function of an atom, so we can say things like "assert that this atom has FOOBAR validator"#2017-11-1801:01danielcomptonYou also have the same problem for things like manifold deferreds#2017-11-1801:03qqqwtf are manifold deferreds? š#2017-11-1801:03qqqELI-don't-have-math-PhD#2017-11-1801:10danielcomptonhttps://github.com/ztellman/manifold/blob/master/docs/deferred.md#2017-11-1801:15qqq@danielcompton: this looks interesting ; does it also have cljs support ? [ I can't find mention of it anywhere on the page ]#2017-11-2217:03jeroenvandijkhttps://github.com/dm3/manifold-cljs#2017-11-2217:29qqq@U0FT7SRLP: thanks for the manifold-cljs link; much appreciated#2017-11-1803:15qqq1. Please ignore the camelCase.
2 [(s/assert ::appState 23)
(s/valid? ::appState 23)]
(comment
[23 false])
3. How is this possible? if 23 is not a valid ::appState, shouldn't assert throw an exception ?#2017-11-1803:57qqqResolved: I forgot to use (s/check-asserts .... )#2017-11-1807:35danielcomptonThereās a CLJS port of it somewhere#2017-11-1807:48mattiaswI have a simple function that takes data, and creates test data from it. Currently, it uses (rand-int ..), what minimum changes do I need to do to this code, in order to use the same seed for random as the rest of the generators.
(defn create-update-operation
"Change or delete or add a new key to `m`.
In 30% of the operations, then value will be nil, i.e. delete."
;; Do not change or delete any of the pre-defined keys
([m] (let [res (create-update-operation (filter #(not (#{:id :tenant :entity :k :updated :version} (key %))) m) {})]
;; (println (count res))
res))
([m changes]
(if (<= (count m) 3) changes
;; find the next key to change (update or delete)
(let [nxt (rand-int (count m))
next (nth m nxt)
rest (nthrest m (min 1 nxt)) ;0 just mean that I am regenerating the first key.
k (key next)
_ (assert (some? k))
v (update-or-delete-value k)
changes2 (assoc changes k v)]
;; see if we should add a new key to the map
(if (< (rand-int 10) 2)
(recur rest (let [k2 (keyword (random-string))
v2 (update-or-delete-value k2)]
(assoc changes2 k2 v2)))
(recur rest changes2))))))
where update-or-delete-value
also uses rand-int#2017-11-1900:00qqqIs there a better way to write: (s/def ::p1 #(s/valid ::vec2 %)) ?#2017-11-1900:00qqq(s/def ::p1 #(s/valid ::vec2 %))
#2017-11-1900:32taylor(s/def ::p1 ::vec2)
?#2017-11-1901:18qqqIn the spec guide, we see:
(s/def ::suit #{:club :diamond :heart :spade})
which asserts that item is ELEMENT OF given set
is there a corresponding builtin for
'item is SUBSET OF given set`#2017-11-1901:38taylorprobably just use https://clojuredocs.org/clojure.set/subset_q as a predicate#2017-11-2005:08qqqthey very act of writing specs is making my code clearer#2017-11-2005:08qqqit's forcing me to think: what condition does this key have to satisfy to be valid, and documenting it as code#2017-11-2007:39qqqthis is kind of insane, but is there a way to (while only holding weak references), tell clj/cljs to memoize calls to
(s/assert SPEC OBJ)
?#2017-11-2012:20gfredericksthe jvm has a hashmap for that sort of use case I think
dunno how easy monkeypatching s/assert
to get that effect would be#2017-11-2012:20gfrederickswhy're you calling multiple times?#2017-11-2020:25johanatanare there any known gotchas involving partial application around a stest/instrument
ed function ?#2017-11-2021:15hiredmanit depends what you mean#2017-11-2021:15hiredmaninstrument works by setting the root value of vars to an instrumented value of the function, but if you have already taken the value of a var before instrumenting, of course that will have effect#2017-11-2021:17hiredmanso if you have something like (def f (partial g 1))
and then later instrument, calling f won't call the instrumented version of g#2017-11-2101:44johanatan@hiredman yea, that makes sense. unfortunately wasn't my problem but i figured it out anyway (something unrelated)#2017-11-2101:45johanatanthe error message i was getting from spec was unintelligible but i just massaged the code, fixed any issues i saw via manual inspection and it eventually went away#2017-11-2103:02johanatannarrowed it down to instrument
actually doing more than I figured it would. looks like it exercises the instrumented with some values that the user itself isn't passing in. this was a bit surprising to say the least#2017-11-2103:02johanatankept trying to find where the ((nil))
value was coming from in my code/ how that was even possible etc etc#2017-11-2103:40seancorfield@johanatan If you instrument
a higher-order function -- one that takes a function as an argument -- then it will use generative testing, as I understand it.#2017-11-2107:58qqqwith coll-of, is there a way to say:
this collection ca be empty, but it can't be nil
i.e. [] is okay, but nil is NOT okay#2017-11-2110:53trisshey all. Is there a spec for something I can use as a predicate for s/valid?
#2017-11-2110:54trissI want a spec that matches both predicate functions and the names of specs.#2017-11-2111:06mpenet@qqq it works like this by default#2017-11-2111:07mpenet(s/valid? (s/coll-of any?) nil)
returns false#2017-11-2117:08qqq@mpenet: yeah, thanks, I should have run that test, turns out bug was elsewhere and I was misattributing it#2017-11-2118:32johanatan@seancorfield it was a higher-order function in this case. any idea the reasoning behind using generative testing for this case?
is it just a 'hack' since the function inputs cannot be inspected to a level of detail necessary for verification?#2017-11-2118:46seancorfieldI thought that was explained in the Spec Guide on the web site @johanatan? I certainly wouldn't say it was a 'hack'. It seems perfectly reasonable to me. How else would you verify that a function you're passing as an argument conforms to its spec in this situation?#2017-11-2118:47mpenetIt s doable at invoke time#2017-11-2118:47mpenetThere is/was a ticket about this#2017-11-2118:49mpenetI personally wished it was like this, now I almost always spec hof as ifn? for instrumentation because of this, which makes it a bit useless#2017-11-2118:50mpenetImho both make sense in some cases, we should just have the choice#2017-11-2118:58mpenetIt was clj-1936 not sure if there is a new ticket started from the discussion there#2017-11-2119:09johanatan@seancorfield it probably is explained but i haven't read the guide from beginning to end so could've missed it. well, yea, in a dynamic language this probably can't be done (except perhaps by inserting metadata onto anonymous functions at their definition sites and then reading it back out at the spec check sites [which now that I mention it, does sound like a better way if it would be possible to impl])#2017-11-2119:10johanatan[but by 'hack' i meant, an unfortunate edge where the abstraction leaks due to limitations of the underlying system being modified]#2017-11-2119:13johanatan@triss isn't spec?
what you are asking for?#2017-11-2122:05triss@johanatan Not quite.#2017-11-2122:05trissIām validating on (s/or :fn fn? :keyword keyword? :spec s/spec?)
is there a neater way?#2017-11-2122:09qqqI'm starting to love spec's dynamic attributes.#2017-11-2122:09qqqI'm doing some GUI layout, and I can write a spec of :assert that the sum of the width of the elemes of this vector is <= width of the screen#2017-11-2122:09qqqhit is nearly impossible to do in static type systems#2017-11-2123:16seancorfield@triss Do you want to validate that something is a registered spec or that it looks like it could be a spec?#2017-11-2123:16trissis a registered spec would be best for me#2017-11-2123:17seancorfieldFor the former, you could use #(try (s/get-spec %) true (catch Exception _))
#2017-11-2123:17seancorfieldThat would be false for predicate functions tho'...#2017-11-2123:17seancorfield...but it would validate that you actually had a named, registered spec (keyword).#2017-11-2123:18trissah i see. thatās a handy snippet fro something no doubtā¦#2017-11-2123:18seancorfield(so I guess you'd still want (s/or :fn fn? :spec #(try ...))
?)#2017-11-2123:19trissIām gonna have to scratch my head for a bit about what I want to match on reallyā¦ I want anything that woks as the first argument for s/valid?
#2017-11-2123:19trisswhere is the spec for s/valid?
#2017-11-2203:37Alex Miller (Clojure team)you canāt actually spec most of the functions in spec without creating an infinite recursion when instrumented#2017-11-2123:20seancorfieldBased on the docstring for s/valid?
, it doesn't look like there's a spec registered for it.#2017-11-2123:22seancorfieldThe source shows it calls specize
on its spec
argument and that in turn is (defn- specize
([s] (c/or (spec? s) (specize* s)))
([s form] (c/or (spec? s) (specize* s form))))
at which point you'll have to dig in the source for specize*
#2017-11-2203:06qqqThe question is NOT "what is wrong with this code."
The question is "how do I get a more useful error msg"
(s/def ::ys number?)
(s/def ::xs number?)
(s/def ::data any?)
(defn mat-of [ys xs q? data]
(assert (vector? data))
(assert (= (count data) ys))
(doseq [y (range ys)]
(assert (= (count (get data y)) xs)
(str "count of row " y " is " (count (get data y)) " not " xs))))
(s/def ::keyPad (s/and (s/keys :req-un [::xs ::ys ::data])
(fn [obj]
(mat-of (:ys obj) (:xs obj) any? (:data obj)))))
(s/explain ::keyPad
{:ys 3
:xs 3
:data [[7 8 9]
[4 5 6]
[1 2 3]]})
returns false. I expect it to return true. It appears to be eating the assertion error. Is there a way to get a more helpful msg on why it's failing the predicate ?#2017-11-2203:15qqqreframing question: when using predicates with clojure spec, is there a way, instead of just returning true/false, also return a error msg on false ?#2017-11-2203:38Alex Miller (Clojure team)currently, no#2017-11-2204:01qqqis this a short coming of current spec design, or is this because I'm mis using spec and should be using spec primitives as much as possible instead#2017-11-2204:16seancorfieldOne thing I've found helpful @qqq is to break predicates down into smaller components and give them names. That way the explain
refers to the named predicates. In your case, I'd write data-is-vector?
, data-has-ys-rows?
, and data-has-xs-cols?
and have those accept the whole hash map and return true/false. Then you'd have (s/and (s/keys :req-un [::xs ::ys ::data]) data-is-vector? data-has-ys-rows? data-has-xs-cols?)
and I believe you'll get better error messages -- without using assert
.#2017-11-2204:19seancorfieldThe other thing we do is take the explain-data
and parse it so we can map symbols and forms to English error messages (well, actually to i18n keys so that we can produce error messages in any language!).#2017-11-2208:50hkjelsSo, Iāve written quite a lot of ui
-components where I use spec
for various things. Mostly checking that parameters used with the components are valid. Iāve come to a stage where Iām doing performance-testing and it seems that spec
is sucking the life out of this project. Is it just a bad idea to use spec
for such a task?#2017-11-2213:36Alex Miller (Clojure team)Yes#2017-11-2209:00andre.stylianosI believe the encouraged way is to use spec for production code only at the boundaries where you receive data from sources you don't control. Beyond that, the validation that you do for internal code should be used only for development.#2017-11-2209:14hkjelsOK.. Then Iāll have to use something like spec/nonconform
all over the place in dev then#2017-11-2209:14hkjelsI guess it makes sense to not have it on in production#2017-11-2210:34ikitommi@hkjels just turn off the validation for real app? e.g. instrument the component functions for dev. Or use assert and disable that for prod. Havenāt used spec for ui but did that at some point with Schema. WIth it, you can disable function validations and the schematized defs and fns will not emit any code related to Schemas => no penalty.#2017-11-2211:26hkjelsWhen I disabled validation, it definitely got a performance boost#2017-11-2211:27hkjelsconform is still one of the heavier tasks in the performance-tab og chrome, but itās also doing quite some work, so that makes sense I guess#2017-11-2211:28hkjelsI have to do some more testing, but this might do#2017-11-2212:00guyWhen you fdef a function with no args, do you just omit the :args key?#2017-11-2213:35Alex Miller (Clojure team)No, you should use (s/cat) to validate 0 arity#2017-11-2213:45Alex Miller (Clojure team)Also, you should be suspicious of 0 arity functions. Either they have side effects (and maybe shouldnāt be specāed) or you can just use a def instead.#2017-11-2213:51guyThanks!#2017-11-2213:38otfrom@seancorfield and @alexmiller are there any good examples out there of using relatively complex :fn functions in fdefs? I saw that and
hints above and wonder if that is the best way to go to figure out which bit of the property is being violated#2017-11-2213:42Alex Miller (Clojure team)Iāve written some here and there but not sure if in anything public. If they get too complicated, then you have to question whether thatās the best way to test. Sometimes straight test.check is better#2017-11-2213:56otfromcool, that makes sense#2017-11-2217:31qqqI realize this is a bad idea in most cases, but is it possible to say:
(s/maps ... key :buttonContainer satisfies spec ::container) ?#2017-11-2218:03Alex Miller (Clojure team)Donāt understand the question#2017-11-2218:07bronsaare you looking for :req-un
?#2017-11-2220:55Alex Miller (Clojure team)FYI, Iāve been working on fix the auto doc junk for Clojure and contrib and the docs have been updated for Clojure, spec.alpha, and links in the spec guide#2017-11-2221:07seancorfield@alexmiller That's great to hear -- thank you! Is there anything I can do to help get https://clojure.github.io/java.jdbc/ updated too?#2017-11-2221:10Alex Miller (Clojure team)nope. I can try to run it manually thoughā¦#2017-11-2221:15Alex Miller (Clojure team)@seancorfield updated#2017-11-2221:17Alex Miller (Clojure team)@seancorfield Iām not sure if you had specs in those vars in the prior docs, but they are showing up now#2017-11-2221:37seancorfieldOh cool! No, I don't think the specs were showing up before.#2017-11-2221:38seancorfieldThat is very nice! Thank you!#2017-11-2312:26danielneal@alexmiller ooh that's awesome#2017-11-2316:26qqqI have a number of specs:
::Foo ::Bar ::Cat ::Dog
if I do (s/keys :req-un [::Foo ::Bar ::Cat ::dog])
the keys of the map become :Foo :Bar :Cat :Dog
. Is there some way to redo this so the keys become :foo :bar :cat :dog
instead ?#2017-11-2316:50misha(s/def ::foo ::Foo) #2017-11-2316:52mishaAnd then use ::foo in :req-un#2017-11-2316:53qqqobvious yet brilliant#2017-11-2320:32qqq(s/valid? any? nil)
what is idiomatic way to say: this can be anything so long as it is not nik ?#2017-11-2320:32madstap@qqq some?
#2017-11-2320:33qqq(some? false)
=> true -- this is surprising#2017-11-2320:47guy(defn some?
"Returns true if x is not nil, false otherwise."
{:tag Boolean
:added "1.6"
:static true}
[x] (not (nil? x)))
#2017-11-2402:52johanatanI'm getting ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4744)
for a spec generator that seemingly should be fairly trivial:
(s/def ::snake-cased-alpha-numeric
(s/with-gen
(s/and string? #(re-matches #"[a-z|\_|0-9]+" %))
#(gen/fmap (fn [v] (apply str v)) (gen/vector (gen/frequency [[1 (gen/return \_)] [9 gen/char-alphanumeric]])))))
(s/def ::required? boolean?)
(s/def ::property-attrs
(s/keys :req-un [::required?]))
(s/def ::events-schema
(s/map-of integer?
(s/map-of ::snake-cased-alpha-numeric
(s/map-of ::snake-cased-alpha-numeric ::property-attrs))))
> (gen/sample (s/gen ::events-schema))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4744)
#2017-11-2402:52johanatan[I added the with-gen
on snake-cased-alpha-numeric
with it being the most complicated primitive in there and still get the error]#2017-11-2402:58danielcompton@johanatan can you generate any of the leaves?#2017-11-2402:58johanatan> (gen/sample (s/gen ::property-attrs))
({:required? false} {:required? false} {:required? true} {:required? true} {:required? true} {:required? true} {:required? false} {:required? false} {:required? false} {:required? true})
#2017-11-2402:59johanatan> (gen/sample (s/gen ::required?))
(false true true false true true false true true false)
#2017-11-2402:59danielcomptonwhat about snake-cased? that looks like the tricky one#2017-11-2402:59johanatan> (gen/sample (s/gen ::snake-cased-alpha-numeric))
("zno" "n7" "_" "3" "_" "c" "8_" "5i" "k7q" "i")
#2017-11-2402:59danielcomptonhuh#2017-11-2403:00johanatanyea š#2017-11-2403:00johanatanoh, this may be it:#2017-11-2403:00johanatan> (gen/sample (s/gen ::snake-cased-alpha-numeric) 100)
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4744)
#2017-11-2403:01danielcomptontry (s/map-of integer? ::snake-cased-alpha-numeric)
#2017-11-2403:01johanatanperhaps there is a bug where my regex doesn't match values generated with the with-gen
-specified generator#2017-11-2403:01johanatani've seen that error when the two are mismatched: i.e., when the spec and the generator for it are not perfectly aligned on what is acceptable or not#2017-11-2403:02gfredericks@johanatan would this help? https://github.com/gfredericks/test.chuck#string-from-regex#2017-11-2403:02danielcomptonTry https://github.com/gfredericks/test.chuck#string-from-regex maybe?#2017-11-2403:02danielcomptonjinx#2017-11-2403:02gfredericksI forget what the implications of jinx are#2017-11-2403:02gfredericksdo I have to not talk now for some time period#2017-11-2408:58danielnealI think you can't talk until someone says your name three times#2017-11-2412:07gfredericksthe same person? back-to-back in quick succession?#2017-11-2413:49danielnealaw you're right that's totally underspecified#2017-11-2413:53danielnealoh wait#2017-11-2413:53danielnealmy mistake#2017-11-2413:53danielnealTraditionally, a jinx is ended when anyone speaks the jinxed person's name. However, a common variation says that only the jinxer can free the jinxee from their obligation to remain silent. (This is sometimes called a "private jinx" or "jinx personal lock".)
#2017-11-2419:08gfredericksa private jinx sounds a lot less interesting#2017-11-2403:03johanatanhmm, yea, i'll give that a try. thx!#2017-11-2403:08johanatanthat fixed it. was able to successfully generate 1000 at once#2017-11-2403:10johanatanthere is a bug in my regex though. noticing that pipes are ending up in the output. probably was the original problem#2017-11-2403:15danielcomptonbuy me a coke I think, although I probably owe you the drink. Edit: this was about the jinxing#2017-11-2403:17johanatanhaha, definitely. i would even get you a beer if you prefer#2017-11-2403:17johanatanthis was my intended regex, btw: #"[a-z0-9\_]+"
#2017-11-2403:17johanatan[no pipes]#2017-11-2405:20qqqdoes clojure have any naming convention regarding fn names that end in a '-' ?#2017-11-2405:21qqqfor ecample, I have
(defn node->svg ...)
and I would like to define a helper function named
(defn node->svg- ...)
but intuitively, functions that end in '-' seem like a bad idea#2017-11-2405:33seancorfieldThe most common convention seems to be to end in a *
#2017-11-2405:33seancorfield@qqq I've seen that in a lot of libraries -- and started following it myself.#2017-11-2412:06gfredericksI have the uncommon preference for foo'
, imitating mathematical notation#2017-11-2413:34guyHi has anyone made a spec for a base64 encoded string before?#2017-11-2413:35gfredericksthat sounds like it could be a regex#2017-11-2413:35guyooh yes#2017-11-2413:35guythats a smart idea#2017-11-2413:35gfredericksthough perhaps not a trivial one if you want to really follow the spec#2017-11-2413:36gfredericksbut of course that's something somebody else has done somewhere, e.g. https://stackoverflow.com/questions/475074/regex-to-parse-or-validate-base64-data#2017-11-2413:36guyyeah was just looking at that! thanks š#2017-11-2423:27hlshipI'm stumbling on something pretty basic. s/cat is not working the way I'd expect.
(s/conform ::field-def [:foo :text])
=> {:field-name :foo, :field-type :text}
(s/conform (s/cat :fd ::field-def) [:foo :text])
=> {:fd {:field-name :foo, :field-type :text}}
I would have expected that to fail; ::field-def is
(s/cat :field-name keyword?
:field-type field-types)
That is, my ::field-def expects a tuple of keyword and field-type (a set of keywords). So I'd expect s/cat
to consume the first value from a list, then apply ::field-def as a predicate and deconstructor to it. Instead, it is applying ::field-def directly.
From my (mis-?) understanding, I'd expect the following to succeed, but it fails:
(s/explain (s/cat :fd ::field-def) [[:foo :text]])
In: [0] val: [:foo :text] fails spec: :com.walmartlabs.genie.launchpad.db-access/field-def at: [:fd :field-name] predicate: keyword?
:clojure.spec.alpha/spec {:clojure.spec.alpha/op :clojure.spec.alpha/pcat, :ps [:com.walmartlabs.genie.launchpad.db-access/field-def], :ret {}, :ks [:fd], :forms [:com.walmartlabs.genie.launchpad.db-access/field-def], :rep+ nil}
:clojure.spec.alpha/value [[:foo :text]]
Again, my expectation is that s/cat
consumes a seq. The first element in the seq should be a ::field-def, which itself is a seq of two elements. Instead, it is matching the treating the entire list [[:foo :text]]
as the field-def tuple, and failing because the first value in the list, [:foo :text]
is not a keyword.#2017-11-2423:31hlshipBasically, my goal is this:
(s/def ::primary-key (s/cat :pk-field-def ::field-def
:cluster-defs (s/* ::cluster-def)))
That is, a single field def, followed by zero or more cluster-def's.#2017-11-2423:35seancorfield@hlship Regex specs "unwind" when they are combined, unless you wrap them in s/spec
(or some other delimiting operation).#2017-11-2423:36hlshipOk, that could be what I'm missing. It's right there in the docs, once I know what to look for!#2017-11-2423:37seancorfields/cat
isn't meant for tuples -- there's s/tuple
for that#2017-11-2423:37seancorfield(although you'll get no labels with that I think...)#2017-11-2423:38hlshipNice. I don't need the conformed value, I'm just validating.#2017-11-2423:39seancorfieldI think it catches everyone out at least once š#2017-11-2510:17qqqis there a way to hijack s/assert 's error messages?
I often have situations where:
1. obj is a giant data sttructure
2. obj fails spec ::foo
3. the fact that obj fails can be verified by just loloking at (keys obj)
4. s/assert prints out all of obj, which actuall makes it more difficult to see what is going on#2017-11-2522:48bbrinckYou can also customize the printer e.g.
(defn my-assert [spec val]
(binding [s/*explain-out* my-printer]
(s/assert spec val)))
my-printer
will take a single argument that is the explain-data
(itās data like the return value from s/explain-data
).#2017-11-2523:22qqqthis looks better than what I asked for#2017-11-2523:22qqqso I get to take the assertion failure, as clojure data, rewrite it, then print it out#2017-11-2523:22bbrinckYep, you can format it however you want#2017-11-2523:23bbrinckAlso, if you happen to want an out of the box experience, you can use expound#2017-11-2523:23bbrinckIt doesnāt actually help in this specific case (it prints out the entire value when it is missing keys), but you can provide a custom function to print out the value in this specific case#2017-11-2523:24bbrinckhttps://github.com/bhb/expound#configuring-the-printer#2017-11-2523:24bbrinckin that case, youāll get the spec-name and some other data, which might be enough to detect this particular case.#2017-11-2523:25bbrinckOr you could even write your own printer which inspects the explain-data
and does something custom for keys
specs, but passes it along to expound otherwise.#2017-11-2523:42qqqwhen an assertion happens, I want it to pop up a new browser window, print the first level of the data, have + buttons next to parts that can be expand (and expands when I click on them), and highlights in a red DOM rectangle the key/value pair that is failing the spec#2017-11-2523:42qqqsomeone needs to build that š#2017-11-2523:45bbrinckWould https://github.com/jpmonettas/inspectable#specs-fail-explain-why work?#2017-11-2523:50qqqthis is amazing#2017-11-2523:50qqqwhy is https://github.com/jpmonettas/inspectable#specs-fail-explain-why not common knowledge ?#2017-11-2600:00bbrinckNo idea, it seems like a great tool, maybe we need a collection of these helper libraries#2017-11-2512:57gfrederickswrap it in a try/catch?#2017-11-2513:23trisshow can I look inside the result of an s/and
? - Iād like to dig out the s/cat
contained within it.#2017-11-2513:24gfrederickss/form
?#2017-11-2513:34trisslovely thanks!#2017-11-2520:40hmaurerHi! I am getting a cryptic error (to me) when attempting to build my ClojureScript project for production (with advanced opts). Could someone please take a look? https://gist.github.com/hmaurer/1d30b39c8e28e646ebdfa524cbcdedb6#2017-11-2520:41hmaurerwrong channel, sorry#2017-11-2611:36qqq(s/keys ::req [])
<- notice the ::
instead of :req
is there a way to have that throw an error ?
sometimes, I define that, then I create objects, and they satisfy the spec ... because it's a map with no requirements since ::req
is silently ighnroed#2017-11-2613:42Alex Miller (Clojure team)Sure, file a jira#2017-11-2614:50gfrederickswould that not go against the "accept arbitrary extra keys" philosophy of spec?#2017-11-2618:08Alex Miller (Clojure team)I wouldnāt necessarily put it in the spec for keys, but itās reasonable to validate it in the function. Itās not like this is getting passed to anything else.#2017-11-2618:09gfredericksI spose I had always interpreted the openness as partially targeting forwards-compatibility#2017-11-2618:10Alex Miller (Clojure team)Itās still forward compatible to accept more stuff in the future#2017-11-2619:17gfredericksright, but then code targeting the future API can't use the older version if it happens to be around
maybe that's not a useful thing? you just want backwards compatibility so you can use the newest of all the versions that each part of the code wants?#2017-11-2621:21Alex Miller (Clojure team)I would say thatās a non-goal#2017-11-2617:55souenzzo(s/fdef s/keys
:args (s/+ (s/cat :k #{:req :opt}
:v coll?)))
=> clojure.spec.alpha/keys
(s/keys ::req [])
CompilerException clojure.lang.ExceptionInfo: Call to clojure.spec.alpha/keys did not conform to spec:
In: [0] val: :user/req fails at: [:args :k] predicate: #{:req :opt}
#:clojure.spec.alpha{:problems [{:path [:args :k], :pred #{:req :opt}, :val :user/req, :via [], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x2fa7a99c "
@qqq
@gfredericks sure. But you can do š
Maybe in future versions of spec, this spec will be wrong. But for now, it's a usefull build-time error.#2017-11-2617:57gfredericksbut there's a difference between spec setting that spec on s/keys
and a user doing it themselves#2017-11-2619:54arohnerBefore I go write one, is there a generator for specs? i..e one that will generate (s/cat :x int?)
#2017-11-2621:24Alex Miller (Clojure team)The spec specs in CLJ-2112 are imo the right direction for that. The specs there are not yet good enough to be used for this in general#2017-11-2620:00hoopeshttps://github.com/stathissideris/spec-provider what you're looking for?#2017-11-2620:03arohnerNot quite. I'm testing https://github.com/arohner/spectrum, so I want to throw arbitrary specs at it#2017-11-2620:56gfredericksonce there's a spec spec you could presumably get a generator from it#2017-11-2620:56gfredericksshouldn't spectrum have a spec spec already anyhow?
I say that as someone who has never used spectrum#2017-11-2623:02arohnerIt only needs s/spec?
as a spec when using spectrum. To test it, I want a spec generator#2017-11-2621:00taylorcurious if thereās been any thought/discussion/work for sharing specs (besides putting them in a shared library)#2017-11-2621:02taylorsomething akin to XSD? I suppose itād be limited to what can be expressed in whatever format#2017-11-2621:22Alex Miller (Clojure team)The whole point of using fully qualified namespace keys is that everyone can share specs#2017-11-2621:35tayloryeah, I guess my curiosity is more about alternative mediums/mechanisms for sharing specs?#2017-11-2621:36gfredericksbut as soon as you use custom predicates, you need to share code as well#2017-11-2622:56Alex Miller (Clojure team)Which is also fully qualified #2017-11-2621:38taylorIām thinking of something conceptually similar to Swagger I guess. And yeah it would be a pretty limited subset of specās abilities.#2017-11-2621:40taylora ha https://github.com/metosin/spec-tools#spec-transformations#2017-11-2704:34fedregIs there a way to get spec information similar to how I can use meta
to get info on a function? So if I have a spec like (s/def ::example string?)
is there some function I can call to see the spec definition? Even better, is there anything that will give me a list of specs in the namespace? Thanks!!!#2017-11-2715:59Alex Miller (Clojure team)yes, you can call (doc ::example)
#2017-11-2715:59Alex Miller (Clojure team)youād have to write something to filter specs per namespace#2017-11-2707:33mishas/form#2017-11-2707:34mishas/registry#2017-11-2707:34misha@fedreg #2017-11-2712:50ikitommi@taylor related to āsimilar to swaggerā - if specs could be read back from their s/form
without eval
(I think it could be done already with undocumented part of the spec - but Rich might be doing something functional for this?), we could have a pragmatic subset of specs that could be safely used between servers & clients. and build swagger-like docs for them.#2017-11-2712:54ikitommiwe could publish the s/form
as vendor extension to the spec-tools swagger-docs. e.f. :x-spec "(clojure.spec/keys :req [:user/x])"
+ :x-specs
on top-level as the spec-registry for the related specs. Non-clojure clients could use swagger/openapi and clojure-clients just the spec-part.#2017-11-2716:34r0manIs this a bug?#2017-11-2716:34r0man(if-let [x false] x :my-ns/invalid)
(if-let [x false] x :clojure.spec.alpha/invalid)#2017-11-2716:34r0manthe first works, the second raises an exceptions from spec#2017-11-2716:35r0manCall to clojure.core/if-let did not conform to spec: In: [2] val: :clojure.spec.alpha/invalid fails at: [:args :else] predicate: any?#2017-11-2716:36r0manthe second form works also in Clojure 1.8#2017-11-2716:39r0manoh, and I was trying the above with Clojure 1.9.0-RC1#2017-11-2720:02bfabry@U0CKBRBD2 known issue https://dev.clojure.org/jira/browse/CLJ-1966#2017-11-2720:04r0man@U050MP39D ah, thanks!#2017-11-2717:00qqqis there a way to say:
::foo is a COLLECTION
such that (map :get-some-key ...) creates an unique list
and if this fails, print out the entire collection#2017-11-2717:23taylor(s/def ::my-map (s/keys :req [::get-some-key]))
(s/def ::my-coll (s/and (s/coll-of ::my-map)
#(= (count %)
(count (set (map ::get-some-key %))))))
(s/explain ::my-coll [{::get-some-key 1} {::get-some-key 1}])
#2017-11-2717:00qqq(as a spec)#2017-11-2721:20fedregThanks! Completely missed that registry function.#2017-11-2722:29j-poIs there a minimally ugly way to compare specs from two projects (like two versions of the same codebase)? One idea that comes to mind is loading one set of specs into a separate registry, but that looks like it's been made deliberately hard. Is there maybe a way to blow away the spec registry entirely to restart from scratch?#2017-11-2722:31bfabryspecs are fully namespaced with the idea that they'll be unique across projects, so caveat is anything you do that's assuming different is going to end up weird. but, if you want, I'm pretty sure you could just reset! the atom that is the spec registry#2017-11-2722:37gklijs@j-po maybe first load the old one, write all the forms to a map, and serialize. Then start again with the new one, and compare against the stored map?#2017-11-2723:05j-poThanks!#2017-11-2810:37otfromI'm creating a look up hash map from a vector of records where the keys will be strings based on what I want to look up. Anyone seen an example of how to spec this? I can't think of how to do it with s/keys
#2017-11-2810:39otfrom[{:a "foo" :b "shiny"} {:a "bar" :b "dull"}]
=>
{"foo" {:a "foo" :b "shiny"}
"bar" {:a "bar" :b "dull"}}
#2017-11-2810:52guyIām not sure but could you use map-of
?#2017-11-2810:53guy(s/def ::scores (s/map-of string? int?))
(s/conform ::scores {"Sally" 1000, "Joe" 500})
#2017-11-2810:53guytaken from https://clojure.org/guides/spec if you ctrl+f map-of#2017-11-2811:23otfrom@guy thx š#2017-11-2811:23guyhopefully thats helpfully! š#2017-11-2811:25otfromit is#2017-11-2821:45sekaois there any library that can "unroll" a spec so i can display a given spec with all its component specs in a single data structure?#2017-11-2821:47taylormight be some interesting stuff in this project https://github.com/jpmonettas/inspectable#2017-11-2821:51sekaohmm the spec browser is neat but does that lib allow you to just get the whole spec as data?#2017-11-2821:51tayloryou can already get that with s/form
if I understand you correctly#2017-11-2822:01sekaos/form doesn't recursively grab specs that a given spec refers to#2017-11-2822:02tayloryeah youāll have to do some additional lookup, but I think that should be pretty easy#2017-11-2822:04taylormaybe using clojure.walk and s/get-spec
?#2017-11-2822:09taylor(walk/postwalk
(fn [v]
(if (keyword? v)
(or (some-> v s/get-spec s/form)
v)
v))
(s/form ::my-coll))
hereās an idea, maybe not a good one#2017-11-2822:10taylordoesnāt cover specs registered to symbols, and probably other cases, etc.#2017-11-2822:29ikitommithere is a spec visitor on spec-tools: https://github.com/metosin/spec-tools/blob/master/README.md#spec-visitors#2017-11-2916:19acronRe spec, should Clojure ensure that keys referenced in s/keys
are registered? I just noticed an unregistered key in one of mine as was surprised I hadn't been made aware by the compiler#2017-11-2916:20mpenetit allows that so that you can define key sets before the key specs themselves, most of the stuff in spec in wrapped in delays#2017-11-2916:21mpenetI could be wrong, but this sounds like the "issue"#2017-11-2916:22acronI understand, kinda makes sense.#2017-11-2916:23madstapYou can use this to find those errors https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2017-11-2916:24acronThanks @madstap this is useful#2017-11-2919:13mishais there a good example of a generator, which generates map (record) where values of 2 fields are dependent?
multispec solves opposite problem: conform/validate records with 2+ dependent fields, can it be leveraged for generating generator?#2017-11-2919:17mishathe nuance in a data model I have, is that :type key on record determines not presence/absence of an attribute, but shape of a particular attribute:
{:type :string :val "foo"}
;;vs.
{:type :media :val {:url "" :content-type"image/jpg"}}
so I am already jumping some hoops to create multispec for this (extra level of spec indirection between data shape and attribute name)#2017-11-2919:20mishawould like to avoid having custom monster generator, if possible at all#2017-11-3000:01shaun-mahoodI'm trying to hook a custom generator to a spec and I can't quite get it working - I suspect it's something simple that I'm missing. Details in thread.#2017-11-3000:01shaun-mahoodI'm trying to hook a custom generator to a spec and I can't quite get it working - I suspect it's something simple that I'm missing. Details in thread.#2017-11-3000:01shaun-mahoodMy custom generator is (def gen-7-digit-plan (gen/fmap #(apply str %)
(gen/tuple (gen/choose 0 9)
(gen/choose 0 9)
(gen/choose 0 9)
(gen/choose 0 9)
(gen/choose 0 9)
(gen/choose 0 9)
(gen/choose 0 9))))
#2017-11-3000:04shaun-mahoodSpec is (s/def ::plan (s/with-gen string? gen-7-digit-plan))
#2017-11-3000:05shaun-mahoodError message starts with #object[TypeError TypeError: self__.gfn.call is not a function]
#2017-11-3000:06shaun-mahoodWhat I'm actually looking for is a 7 digit string with a range from 0000000
to 9999999
- probably there's a way better way to do this.#2017-11-3000:06taylor(s/def ::plan (s/with-gen string? (constantly gen-7-digit-plan)))
#2017-11-3000:07taylorI know test.chuck lib has regex->string generators https://github.com/gfredericks/test.chuck#string-from-regex#2017-11-3000:08shaun-mahoodAwesome, thanks!#2017-11-3006:59misha@U054BUGT4 I might be wrong, but it seems like every gen override place in spec expects no args fn returning actual generator, hence āconstantlyā above. #2017-11-3007:00mishae.g. s/exercise is one of those places#2017-11-3005:09quanI'm trying to spec clojure.string/index-of
(defn- str-or-char? [s] (or (string? s) (char? s)))
(s/fdef clojure.string/index-of
:args (s/alt :idx (s/cat :s string?
:value str-or-char?)
:idx+from (s/cat :s string?
:value str-or-char?
:from-index int?))
:ret (s/nilable int?))
but running into this weird exception when instrument
(stest/instrument)
(clojure.string/index-of "abc" "a" 0)
;=> CompilerException java.lang.ClassCastException: clojure.spec.test.alpha$spec_checking_fn$fn__2943 cannot be cast to clojure.lang.IFn$OOLO
#2017-11-3005:09quannot sure what's wrong here?#2017-11-3006:13seancorfield@quan I believe that happens when you try to instrument a function that has arguments type-hinted to primitives. The IFn$OOLO type is (Object, Object, long) -> Object and is optimized for the primitive argument and that breaks spec. It's a known issue (I don't recall the JIRA issue number).#2017-11-3006:17quanah yes, from-index
has ^long
hint#2017-11-3006:25shaun-mahoodI want to use spec generators to build sample data when I'm prototyping a front end app. The generators return nil when I try to use them at runtime after requiring the file, but they work fine if I run them directly in the repl. Is there a good way to get these to work at runtime?#2017-11-3014:55mikeyjcatHi. Iām using spec to validate data being prepared for transactions on datomic, including tempidās. All works good. There is now a spec registered for :db/id that checks to make sure it is a instance of datomic.db.DbId. Wonderful, and I even created a custom generator. However, now when I am using spec with data read in, I have data with :db/id 123345, rather than a instance of datomic.db.DbId. This obviously fails the spec in instrumentation. It seems as though I need two versions of the spec for :db/id but this cannot be, in my understanding. Anyone tried anything similar?#2017-11-3021:14eriktjacobsen@U1NL85J57
(s/def ::uuid (s/or :int integer? :uuid uuid? :sha256 ::sha256))
#2017-11-3015:59Alex Miller (Clojure team)your spec is not capturing all the possible forms of :db/id#2017-11-3019:18tcouplandhas anyone got a tidy way of writing core.test is checks of spec/check? I've got something that works, but it's pretty ugly!#2017-11-3020:00tayloris it uglier than this? (is (not (:check-failed (st/summarize-results (st/check `foo)))))
#2017-11-3020:41misha->
!#2017-11-3021:15tcouplandVery similar tbh! Do you get good failure messages out of that?#2017-11-3021:16tcoupland(defn check
[sym]
(-> sym
(stest/check {:clojure.spec.test.alpha.check/opts {:num-tests sample-size}})
first
stest/abbrev-result
:failure))
(defmacro is-check
[sym]
`(is (nil? (check ~sym))))
#2017-11-3021:16tcouplandthat's what i'm using at the moment#2017-11-3022:01taylornice#2017-12-0109:21tcouplandwhen i get a minute i might have a play with sumarize, might give a clearer failure message#2017-12-0105:25Shantanu KumarHi, has anybody successfully used spec for web validation and translated the validation failures to HTTP 400 for nested data models? Any example or references would be great to have.#2017-12-0106:47ikitommiDonāt know about success, but there are routing libs like compojure-api and reitit, which support spec as web (backend) validation. There is an example in https://github.com/metosin/c2. For invalid input, produces http 400 with this kind of body:
{
"spec": "(spec-tools.core/spec {:spec (clojure.spec.alpha/keys :req-un [:c2.spec/x :c2.spec/y]), :type :map, :keys #{:y :x}, :keys/req #{:y :x}})",
"problems": [
{
"path": [
"y"
],
"pred": "clojure.core/int?",
"val": "a",
"via": [
"c2.spec/y"
],
"in": [
"y"
]
}
],
"type": "compojure.api.exception/request-validation",
"coercion": "spec",
"value": {
"x": 100,
"y": "a"
},
"in": [
"request",
"body-params"
]
}
#2017-12-0106:49ikitommiso, just the :problems
polished so that it can be serialized:
https://github.com/metosin/reitit/blob/master/modules/reitit-spec/src/reitit/ring/coercion/spec.cljc#L84-L87#2017-12-0106:51Shantanu KumarI was looking at https://github.com/metosin/spec-tools meanwhile and thought of pinging you!#2017-12-0106:54Shantanu KumarWe are using a different data-driven router, so looking for a standalone validation mechanism using spec. Is there any such library?#2017-12-0106:59ikitommiWell, reitit is kinda modular, if you take the metosin/reitit-spec
isās flattened deps tree is: spec-tools, clojure.spec & meta-merge#2017-12-0107:00ikitommiit bring the ring-module but it has no dependencies. Could move the Coercion protocols into own module.#2017-12-0107:02ikitommido you think the coercion should be totally out of reitit (modules)? I would be more polite towards other routing libs at least.#2017-12-0107:04Shantanu KumarThanks, let me try out Reitit-spec. Iām sort of new to spec. And yes, compat with other routing libs would be very useful.#2017-12-0107:05Shantanu KumarThough canāt comment now about code organization.#2017-12-0107:08ikitommithere is already compat, but needing to include another routing lib to get coercion for another is kinda odd. Plan is to ship coercion interceptors too (as they donāt require any new deps), so they should be usable from āpedestal-style interceptor web libsā#2017-12-0107:10ikitommiare you using mw or interceptors? and if interceptors, which kind?#2017-12-0107:11Shantanu KumarI am using Ring with a routing lib, but right now just using plain app code to do validation (itā a new project) - no m/w for now.#2017-12-0107:13ikitommiok, I think I removed the āvanilla ring coercion middlewareā in favour of the reitit ācompiledā one (much faster). Could push them back to support other routing libs. This: https://github.com/metosin/reitit/commit/7979c9de9d42afd611e13da40502ac8ea1c0c0e3#2017-12-0107:18Shantanu KumarI think I can form an opinion about the commit after some more experience with spec.#2017-12-0107:18Shantanu KumarIām looking at https://metosin.github.io/reitit/ ā is there a section that describes reitit-spec in particular?#2017-12-0107:19ikitommihttps://metosin.github.io/reitit/ring/coercion.html#2017-12-0107:22ikitommiI could cook up examples of that with plain ring. Most likely have time on the weekend.#2017-12-0107:23Shantanu KumarThanks! Looking into it. More examples would be cool for sure.#2017-12-0108:51ikitommi:+1:#2017-12-0110:10Andreas LiljeqvistHow would I define keys and sharing them between specs? like `(def base [::id ::something])
(s/keys :req base)`#2017-12-0110:11Andreas LiljeqvistI suppose it fails because of compile-time evaluation#2017-12-0110:14andre.stylianoss/merge
?#2017-12-0110:15andre.stylianoshttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/merge#2017-12-0110:15andre.stylianosYou can define a base spec and merge that into others#2017-12-0111:44Andreas Liljeqvist@andre.stylianos Thank you, that seems to be exactly what I am looking for#2017-12-0113:15mpenetmost of the time you use s/merge you probably want s/and instead#2017-12-0113:15guywhy do you think that is?#2017-12-0113:15guyJust want to understand thats all#2017-12-0113:17mpenetfrom what @andreas862 asked, he wants to be able to combine specs, not necessary merge them#2017-12-0113:17Alex Miller (Clojure team)I donāt think I would agree with that#2017-12-0113:17Alex Miller (Clojure team)This is exactly when you want to merge#2017-12-0113:19mpenetwhat are the advantages ? genuinely curious. I know about gen#2017-12-0113:20mpenetI remember also merge having arguably odd behavior with conforming#2017-12-0113:25Alex Miller (Clojure team)gen and conform#2017-12-0113:27Alex Miller (Clojure team)in that conform doesnāt flow#2017-12-0113:28Alex Miller (Clojure team)gen is the big one though#2017-12-0113:39ikitommiIs there a reason that conform doesnāt flow?#2017-12-0113:39Andreas LiljeqvistJust adding that what I wanted was to merge even if my wording was a bit unclear#2017-12-0113:53ikitommi(s/def ::a string?)
(s/def ::b (s/and (s/conformer str/upper-case) string?))
(s/conform
(s/merge
(s/keys :req-un [::a])
(s/keys :req-un [::b]))
{:a "kikka", :b "kukka"})
; => {:a "kikka", :b "KUKKA"}
(s/conform
(s/merge
(s/keys :req-un [::b])
(s/keys :req-un [::a]))
{:a "kikka", :b "kukka"})
; => {:a "kikka", :b "kukka"}
#2017-12-0114:13mpenetanother thing I prefer not to use lately: custom conformers š#2017-12-0114:15gfredericksspeaking of schema-related transformations, I've been experimenting with reviving the stuff in https://github.com/gfredericks/schema-bijections as a library that only deals with bijections, not referring directly to schema/spec at all
I'm curious how useful something like that would be for transforming data to the format that a spec expects (and back? e.g. jdbc)#2017-12-0114:16gfredericksI think generating bijections from specs would be the ideal result#2017-12-0114:16mpenetthat'd be useful#2017-12-0114:16gfredericksI still have to decide if/how to incorporate surjections though#2017-12-0114:16mpenetI tend to write this kind of stuff with specter these days, but that's quite gory#2017-12-0114:16gfrederickssince those are also useful in some cases#2017-12-0114:19mpenettbh I didn't really like the form it took in schema, but that was practical. I am not sure I have seen a solution I like for these kind of transformations yet#2017-12-0114:19gfredericksthe form what took in schema?#2017-12-0114:19mpenetcoercers#2017-12-0114:20gfredericksoh right -- I don't like them because they represent surjections everywhere, even for uses when bijections can work
and it makes it easy to test your function-that-coerces-its-inputs with the canonical form of the data, since that's easier, and never actually test the production form that has to be coerced#2017-12-0114:22danielnealbijections from specs would be awesome ā#2017-12-0114:23gfredericksI am envisioning that you can define different styles for other representations, like json and jdbc, and then get bidirectional functions for converting your canonical spec form from one to the other#2017-12-0114:23gfrederickse.g., "I want camel-cased keys, I want date-times serialized like this" etc.#2017-12-0114:24gfredericksthat category of things, but hopefully arbitrarily flexible#2017-12-0114:24gfredericksso if you say your json representation has camelCased keys while your internal representation is kebab-cased, then a json input with kebab-cased keys would be considered incorrect and cause an exception rather than just silently be accepted#2017-12-0114:27mpenetThere was a hint of a next iteration of spec coming out at some point (I think in Rich's talk), I am wondering if there are changes related to these kind of things (and others often mentioned).#2017-12-0114:28mpenetI personally am curious to see how spec forms conforming (or whatever replaces it) will take form#2017-12-0114:29mpenetrelated to CLJ-2112#2017-12-0114:33gfredericksoh yeah, I forgot about that#2017-12-0114:59mpenetand specs metadata too, could help for a lot of things#2017-12-0115:17gfredericksyou mean enabling specs to have metadata attached?#2017-12-0115:18gfredericksI assume they already can so you must mean something else#2017-12-0115:19mpenetyou can't attach metadata to specs atm#2017-12-0117:35ikitommi@alexmiller related to the s/merge
problem, there is a PR request in spec-tools of a merge that seems to fix the problem. Could that be pushed to Spec itself? Comments welcome: https://github.com/metosin/spec-tools/pull/91#2017-12-0117:36Alex Miller (Clojure team)fixes what problem?#2017-12-0117:37ikitommiMy comment on 15:53, that canāt be right?#2017-12-0117:38ikitommie.g. merge merges that values, so the unconformed value override the conformed ones.#2017-12-0117:41Alex Miller (Clojure team)it doesnāt override, itās just that only the last spec in the merge controls the conformed result#2017-12-0117:42Alex Miller (Clojure team)that is the intent, not a bug, but it can have some non-obvious effects, particularly with unqualified keys#2017-12-0117:43Alex Miller (Clojure team)I think there is a ticket for this already in jira, but not sure what should be done with it#2017-12-0117:44ikitommioh, so that works with qualified keys. Thatās interesting.#2017-12-0117:45ikitommihttps://dev.clojure.org/jira/browse/CLJ-1981#2017-12-0117:48ikitommiWoudnāt @arttukaās PR fix that? Could the issue be re-opened?#2017-12-0117:51Alex Miller (Clojure team)I wouldnāt re-open that, but if there is a good statement of a problem and a patch, we can take a look at it#2017-12-0117:54Alex Miller (Clojure team)that is, in a new ticket#2017-12-0117:52Alex Miller (Clojure team)I have no idea what that PR is doing at a glance#2017-12-0117:53Alex Miller (Clojure team)Rich has a pretty extensive re-work coming for spec (post 1.9 release) and I suspect we wonāt really look at any fixes till weāre on the other side of that#2017-12-0316:32danielnealAh interesting! What kind of rework? Is it on the implementation side or is there going to be some kind of change to the interface?#2017-12-0117:58ikitommiok, Iāll ask Arttu to post a new issue out of that. And looking forward to the re-work, whatever that is. thanks!#2017-12-0211:53gklijsI had an issue with a spec I defined in a .cljc file. I use an or to make one of two keys required. Thatās working fine. But when I inspect the form in clj the or is without namespace, while I was expecting the namespace to be clojure.core.#2017-12-0213:43Alex Miller (Clojure team)What do you mean by āinspectā?#2017-12-0214:15gklijsI debugged by adding logging. (namespace x) is empty in the .cljc one, but clojure.core when the spec is defined in a .clj file.#2017-12-0215:52Alex Miller (Clojure team)I still donāt understand a) what youāre doing b) what youāre seeing or c) what you expect to see#2017-12-0216:28gklijsWhat I do is have a library which for a subset of whatās possible to spec, transform the namespaced data into a vector without namespaces in such a way that Iām able to get back to namespaced map from the vector. Part of this is checking the form of the spec used. When you use clojure.spec.alpha/keys, you can use clojure.core/or to be able to spec āone of these should be thereā. Now when the spec id defined in a .clj file the or is namespaces eith clojure.core, but when the spec is defined in a .cljc file the or has no namespace. I now solved it by looking only at the name of the or symbol. So there is no real problem anymore, but I was surprised this was happening.#2017-12-0315:56ikitommi@kumarshantanu did an example of normal ring app with the spec-coercion via a middleware#2017-12-0315:57ikitommivia a middleware: https://github.com/metosin/reitit/blob/master/examples/just-coercion-with-ring/src/example/spec.clj#2017-12-0315:59ikitommisame with reitit, where the data is read from route data instead of middleware args: https://github.com/metosin/reitit/blob/master/examples/ring-example/src/example/spec.clj#2017-12-0316:00ikitommiif you have questions, please join #reitit#2017-12-0318:04Shantanu KumarThanks a lot!#2017-12-0414:03andrea.crottiis this actually valid Clojure?
(defn run-query [service query]
(let [{::keys [result error]} (invoke-service service {::query query})]
(or result error)))
#2017-12-0414:03andrea.crottijust noticed in the clojure spec documentation https://clojure.org/guides/spec#2017-12-0414:03andrea.crottishould it not be :clojure
?#2017-12-0414:05bronsathat's valid clojure yes, why does that confuse you?#2017-12-0414:11danielnealmaybe it's those namespaced keywrods#2017-12-0414:47andrea.crottiwell it doesn't compile
1. Caused by java.lang.RuntimeException
Unable to resolve symbol: result in this context
#2017-12-0414:47andrea.crottiwith ::keys
, but it does with :keys
#2017-12-0414:48bronsawhich version of clojure?#2017-12-0414:48andrea.crotti1.8#2017-12-0414:48bronsause 1.9#2017-12-0414:49bronsa::keys
is 1.9 syntax#2017-12-0415:18denikIs there a way to get a function spec as data?#2017-12-0415:19taylor(s/form `my-fn-name)
#2017-12-0415:22bbrinckIf youāre looking for the :fn
part of an fdef
spec, you can use this:
(s/form (:fn (s/spec `my-fn-name)))
#2017-12-0415:31stathissiderisAnd you can also dig deeper and get the :args part:
(s/form (:args (:fn (s/spec `my-fn-name))))
#2017-12-0415:44denikneat! I forgot to call s/form
on the retrieved spec:
(s/form (s/get-spec `foo))
#2017-12-0417:01denikcan the spec from s/keys
be adapted as s/keys*
to avoid repetition?
(s/def :foo/id number?)
(s/def ::foo
(s/keys :req [:foo/id])) ;; <-----------
(defn new-foo [& {:as attr-vals}]
;; add custom attrs
attr-vals)
(s/fdef new-foo
:args (s/cat :attr-vals (s/keys* :req [:foo/id])) ;; <----------
:ret ::foo)
(new-foo :foo/id 1 :foo/bar "baz")
#2017-12-0417:53denik(defmacro spec-keys->spec-keys*
[spec]
(let [form (s/form spec)]
(if-let [[_ & args] (and (seqable? form) (= (first form) `s/keys) form)]
`(s/keys*
#2017-12-0500:03Drew VerleeCan anyone recommend any good resources on clojure spec, or property based testing concerning side effects (like reading, writing from files).
I just watched āA Deep Specification for Dropbox - Benjamin Pierceā
: https://youtu.be/Y2jQe8DFzUM and have to assume the ideas from it were a motivation for clojure spec (given the timing). In the video, their is an argument that that deep specifications lead to more reliable software, which i agree with. He also argues, that this requires a lot of time and effort, which, also makes sense.
My experience with spec so far has been that its hard to see how you can adapt them to handle side effects (writing to files, calling apis, dbs, etcā¦) In the talk, the side effects are handled by observing them, and checking if the observations follow a specification. In general, is the approach you need to take if you want a specification over side effects?
My intuition is that spec is more about modeling the non side effect parts of your application. But it feels like so much work is generally done as side effects, that the cost is fairly high to create the specs and generators that āstubā out the side effect. Or really, what an āeffectiveā stub would be.#2017-12-0500:22hiredmanthat youtube talk is more a long the lines of using test.check directly, not using spec#2017-12-0500:24hiredmanI am sure some people are using spec for that kind of thing, and you might use spec as a nice dsl for creating generators for that, but you wouldn't use spec's instrumentation stuff to do that kind of testing#2017-12-0500:30hiredmanin general, when I am trying to test a whole process, I use test.check to generate operations, feed them in to the process, then compare the result with what my model says it should be. spec (at least how I use it, and there are a lot of parts to spec so it can be used in a lot of different ways) is more about describing data that is moving around in the system. and I end up using those descriptions in assertions to make sure what I expect is there, to parse/recognize which of some alternative data a function was passed, or to generate other descriptions of the data for other tools (schemas, serializers, etc). I pretty much never use spec to annotate a function and then instrument that function.#2017-12-0500:36hiredmanI think generative testing splits in to two classes. Class 1 is what you see in simple examples on line, you have a pure function that does something to a list of integers, so you generate a list of integers and pass it in and assert some property. Class 2 is like what you see in that dropbox video (which is very good), you end up generating a program (a sequence of operations) that you run on the system you are testing and then also against some reference implementation (say for a distributed key value store, the ref impl might be an in memory hashmap), and compare the results.#2017-12-0500:37hiredmanspec's instrument and generative stuff seems mostly aimed at class 1#2017-12-0500:37hiredmanclass 2 is definitely a big investment in testing#2017-12-0500:52Drew Verlee@hiredman thanks, I saved your comment and I'll get around to reading it in a bit š#2017-12-0600:03zalkyHey all: I'd like make some spec assertions regardless of whether check-asserts
has been set. It's important these assertions are always made. To that end I'm looking to use assert*
. It's part of the public api, but the doc-string says Do not call this directly, use 'assert'.
. Is there something important that I need to know? Seems like it fits the bill otherwise.#2017-12-0600:23Oliver GeorgeCould you do something like (asset (s/valid? xxx))
or (binding [s/*assert-flag-thing* true] (s/assert ::x 1))
.#2017-12-0604:02Alex Miller (Clojure team)you can use assert*
but itās api is subject to change as part of the implementation - itās public because it needs to be accessible by the macro when expanded#2017-12-0619:26hlolliI'm a complete beginner in clojure-spec, my question is, why don't I get spec error here?
(s/def ::spec1 string?)
(defn silly []
1)
(s/fdef silly
:ret string?
:fn string?)
(silly)
#2017-12-0619:27taylorif youāre looking to assert that calls to silly
have valid inputs, you can use instrument
. Although thatās not going to care about your :ret
or :fn
function specs#2017-12-0619:28taylorhttps://clojure.org/guides/spec#_spec_ing_functions#2017-12-0619:28taylorhttps://clojure.org/guides/spec#_instrumentation#2017-12-0619:29hlolliI thought that was done with :args within fdef? I was hoping it would tell me that the number 1 is not string#2017-12-0619:29taylorthatās right, but thereās no :args
spec in your example#2017-12-0619:29taylorand you still have to instrument
the function for those assertions to happen#2017-12-0619:30hlolliah ok! thanks, I'm reading trough the manual and therefore had this question, I'll continue to read further.#2017-12-0619:31tayloroh, if youāre only concerned about the return value of your function, instrument
isnāt going to assert on that. You can use check
for that though.#2017-12-0619:32hlolliyes that's what I was looking for, but in real world cases I'd see spec being most helpful in the args.#2017-12-0621:23bbrinckFWIW, Orchestra can be used to check ret/fn specs on instrumentation#2017-12-0621:25bbrinckhttps://github.com/jeaye/orchestra#2017-12-0622:18hlolli@U053S2W0V how is it different from cljs.spec.test.alpha/instrument
that said, that's exacly the reason why I want to use spec, to get these errors right away.#2017-12-0622:19taylorinstrument
only checks the :args
#2017-12-0622:19taylorlooks like you can use Orchestra to get the same effect for :ret
and :fn
specs though#2017-12-0622:19hlolliah, nice I see#2017-12-0622:22hlollialso like their macros, very neat!#2017-12-0702:56eriktjacobsenDoes anyone have any spec->documentation projects? Something that would, for instance, render in a browser a map spec as a map with the values as the underlying spec types, that hides some complexity via being expandable? Thinking something like https://github.com/jebberjeb/specviz but that is more geared towards in-place documentation.#2017-12-0715:33joost-diepenmaat@hlolli because: you need to use clojure.spec.test.alpha/check
a function to test its :ret and :fn specs#2017-12-0715:34joost-diepenmaatif you switch on instrumentation it will only test the :args spec for afn#2017-12-0715:35joost-diepenmaatnever mind. I just noticed you have a bunch of answers.#2017-12-0813:44trissHi allā¦ anyone got a nicer way of doing this?
(defn cat-specs
"Takes a spec and finds the highest level 's/cat' from it and extracts the
specs of the values it matches."
[spec]
(->> (s/form spec)
(tree-seq seq? identity)
(filter seq?)
(filter #(= 'clojure.spec.alpha/cat (first %)))
(first)
(rest)
(partition 2)
(map second)))
#2017-12-0814:20souenzzo(defn cat-specs
"Takes a spec and finds the highest level 's/cat' from it and extracts the
specs of the values it matches."
[spec]
(->> (s/form spec)
(s/conform (s/cat :op '#{clojure.spec.alpha/cat}
:args (s/* (s/cat :name keyword?
:value any?))))
:args
(map :value)))
Some like this.
This issue will help
https://dev.clojure.org/jira/browse/CLJ-2112#2017-12-2111:30trissFantastic thankyou! Apologies I just got round to understanding this#2017-12-0911:40mbjarlandI have a question, Iām using spec to validate the format of a ālayout stringā and in the spec I use conformers to convert from string to seq of chars:
(s/def ::layout-string
(s/and string?
not-empty
(s/conformer seq)
(s/+ (s/alt :col-align-bracket-expr
(s/cat :left-bracket #{\[}
:col-align-char #{\L \C \R \l \c \r}
:right-bracket #{\]})
:col-padding-non-bracket-char
(complement #{\[ \]})))
(s/conformer char-seq->str-conformer)))
, is conformers the way to go or is there some cleaner way to deal with strings as sequences of chars?#2017-12-0911:44mbjarlandI was hoping to use https://github.com/bhb/expound to make the error messages more readable, but expound does not seem to support conformers#2017-12-0911:52mbjarlandI think Iām landing at using instaparse for parsing strings instead of spec. Seems like coercing proper error messages (including the location in the string etc) from spec would be more work than itās worth for this problem#2017-12-0913:10Alex Miller (Clojure team)While spec can be twisted into parsing strings, itās not designed for that and itās going to be way slower than either a proper regex or a parser.#2017-12-0917:09Drew Verleedoes anyone have an example of how you would conform a string into a data structure, like a hasmap: (conform ::foo "8/blue") => {:age 8 :eye-color "blue"}
It seems i would need to specify almost like a regex over the string. This feels doable, but i'm not sure.#2017-12-0917:24mpenetSee s/conformer. You basically pass a fn to it that does the conforming or returns ::s/invalid#2017-12-0917:24mpenetIt can also take another arg to do the inverse (aka unform)#2017-12-0919:35Alex Miller (Clojure team)I would say, donāt use spec for this, use a parser like instaparse#2017-12-0919:36aengelberg:+1:#2017-12-0919:36Alex Miller (Clojure team)spec is about describing the structure of Clojure data, not strings#2017-12-0920:00borkdude@mbjarland I just used Instaparse for a similar problem: https://github.com/borkdude/aoc2017/blob/master/src/day9_instaparse.clj#L14#2017-12-0920:01borkdudeI was going to ask here, Instaparse allows to hide output or hide tags to prevent extra nesting. Is such a thing with Spec also possible?#2017-12-0920:05aengelbergYou could maybe use s/and
and s/conformer
to flatten things as they are being parsed / conformed.#2017-12-0920:06genecwhat's the correct set of dependencies for using spec?#2017-12-0920:06genec:dependencies [[org.clojure/clojure "1.9.0"]
[org.clojure/spec.alpha "0.1.94"]#2017-12-0920:08Alex Miller (Clojure team)Just clojure #2017-12-0920:08Alex Miller (Clojure team)It includes spec so you donāt need to declare that#2017-12-0920:08Alex Miller (Clojure team)https://clojure.org/guides/spec#2017-12-0920:10Alex Miller (Clojure team)Use of generators is optional. If you want that, youāll also need [org.clojure/tools.check ā0.9.0ā]#2017-12-0920:13genec@alexmiller thanks. I'm going to use spec to validate csv files, seems like a good fit#2017-12-0920:15Alex Miller (Clojure team)I donāt think thatās a good fit#2017-12-0920:15Alex Miller (Clojure team)CSV is text with tricky quoting rules #2017-12-0920:16Alex Miller (Clojure team)The right tool is a parser, like instaparse#2017-12-0920:16Alex Miller (Clojure team)Spec is for Clojure data#2017-12-0920:18genechmm... I'll take a look at instaparse. I have fields that can either be an int or "Missing", which I need to create a log of an errors for the user to correct before importing the file, which is why I thought spec would be good for that#2017-12-0920:21guyCould you use instaparse to parse the data, then use clojure spec to just check it after?#2017-12-0920:21genecI suppose I could, I was just going to use one of the csv libs#2017-12-0920:21guyyeah#2017-12-0920:24genechttp://www.metasoarous.com/presenting-semantic-csv/#2017-12-0920:25guyYeah i just used https://github.com/clojure/data.csv which was pretty handy#2017-12-0920:36borkdude@aengelberg thank you ā and thanks for Instaparse, itās a really amazing tool š#2017-12-0921:46aengelbergGlad you find it useful!#2017-12-0922:17borkdudeYes, just used it for the first time to solve a puzzle, inspired by someone who did it in PureScript using parser combinators.#2017-12-1000:12Drew VerleeI'm somewhat confused about the exchange here:
https://dev.clojure.org/jira/browse/CLJ-2116
If specs aren't meant for coercion from non clojure data structures to clojure datastructures. Then what is the recommended solution for coercion? Is the message that we shouldn't mix coercion to clojure data with validation of that data? Someone with more experience chim in please, but i feel like those things might overlap a great degree and if you cant compose the tool that is doing one with the other, then it feels like there is going to be a lot of confusing repetition.#2017-12-1000:39seancorfield@drewverlee The philosophical reason is that if your spec does coercion, no clients of your spec can avoid that coercion, and that can lead to problems.#2017-12-1000:42seancorfieldFWIW, we use spec at work to validate form parameters which all come in as strings, so we have some very limited coercion in a few of our specs. We have ::long
and ::double
specs that accept both a number and a string that can be parsed to a number, a ::boolean
spec that accepts both a Boolean and a string that can be parsed to a Boolean (under a documented set of rules), and a ::date
spec that accepts both an instant and a string that can be parsed to an instant (again, under a documented, and limited, set of rules).#2017-12-1000:43seancorfieldThat's about as far as I'd go with coercions in specs. Anything beyond that should separate the parsing operation out (and use spec to validate the resulting data structure), IMO.#2017-12-1000:50seancorfieldWe also have a REST API where the payloads are parsed (with Cheshire, via wrap-json-params
and wrap-json-body
) and then validated -- separately -- which makes the specs much more robust and more usable, and makes explain-data
much easier to process (for example, using it to turn spec failures into human readable messages). Coercion in a spec tends to make that last part harder (since you have more operations to pick apart).#2017-12-1000:51seancorfieldCoercion in specs also often causes more work in unform and generators since you need to be able to "de-coerce" validated data.#2017-12-1000:56Drew Verlee@seancorfield That's reasonable. I suppose it works against what compjuri-api is going for. More to the point, i can't get compojure api to work anyway. Is there another restful api service i should look at? The more hand holding solution the better probably, as my use case is really bare bones.#2017-12-1000:57seancorfieldWe've built our REST APIs with just bare bones Ring and Compojure. I haven't looked at compojure-api.#2017-12-1000:57seancorfieldWe've built our REST APIs with just bare bones Ring and Compojure. I haven't looked at compojure-api.#2017-12-1008:08misha@borkdude have a look at s/nonconforming
#2017-12-1009:03ikitommi@seancorfield @drewverlee The CLJ-2116 tries to solve just that, by separating the coercion from specs. Specs to define the form and apply different coercion when needed, e.g. in the borders. int?
should be coerced from string->number If provided as a path-parameter, not if provided as JSON body or within your own domain code. Solution to enable this would be simple.#2017-12-1009:04ikitommithere is also CLJ-2251 on too If that which would make things fast.#2017-12-1009:06ikitommi@drewverlee If you have trouble with the spec coercion with c-api, happy to help.#2017-12-1014:23Drew VerleeThanks @ikitommi i'll ask in another channel.#2017-12-1014:37Drew Verlee@seancorfield i suppose its hard to see why i would use a spec on a string i had already successful parsed in most cases
(parse "drew,2")
=> {:name "a" :siblings 2}
I have already discovered this line meets most of the specification
* has two entries
* second entry is a number
If i want to build a spec about the clojure data to assert it has two entries and the first one is a string and the second is a number, i feel like i'm repeating information. I think the key point here is that, if i build a spec that does coercion on a string it wouldn't verify on the resulting structure. So its not really any more flexible or less work then separating out the coercion function and the spec.#2017-12-1015:01genecIs there a way to use spec so that it can type hint function parameters during development? Coming from F#, it's something I really miss during development. Or is there a good post about spec demonstrating the Clojure way to keep track of data / types during development?#2017-12-1015:01genecIs there a way to use spec so that it can type hint function parameters during development? Coming from F#, it's something I really miss during development. Or is there a good post about spec demonstrating the Clojure way to keep track of data / types during development?#2017-12-1015:06taylorI donāt have a good answer, but I also came to Clojure from F# ā¤ļø Personally, there were lots of things I missed early on from F# but over time I found myself not missing them so much#2017-12-1015:08taylorI imagine youāre thinking of something that integrates into the editor? Spec does solve this problem with instrument
but it happens as your code is executed, not when youāre typing it in#2017-12-1019:06genecthanks - yes, I'm missing the type checking from F#. I don't see any reason why there could not be type hints for things that have been spec'd in Clojure. I'll keep looking into this. Out of curiosity, what's your development env? I was using Atom/Proto-Repl but found it a bit flaky so I've switched to Intellij / Cursive and really like it.#2017-12-1021:40taylorI mostly use IntelliJ w/Cursive#2017-12-1022:05Drew VerleeOk some high level questions:
1. Is there value in exchanging name-spaced data across different systems? I feel like rich might have made a big deal about this and its gotten lost in the noise of me working with spec.
2. What would be the best way to transform a map with one namespace into another one with different namespaces.
e.g
(s/def ::zoo (s/keys :car int?)
(??? {:foo/bar 5 :foo/car 6} ::zoo)
=> {:foo/bar 5 :zoo/car 6}
This feels like what conform does, only i want it to not care about the namespace. Also that idea might be imply i'm doing something else wrong, so i'm open to suggestions.#2017-12-1022:10Drew Verleeanother way to handle my problem would be to unnamespace the map#2017-12-1022:12Drew Verleeor i could not apply the namespace in the first place. I feel like these approaches have tradeoffs that probably dont matter in my situation but are interesting to think about.#2017-12-1022:26Drew VerleeWhile i'm thinking about it. Is there a way to combine two specs for a hash-map into one?#2017-12-1023:04Alex Miller (Clojure team)s/merge
#2017-12-1023:05Drew Verleeit would be that easy wouldnt it š thanks @alexmiller#2017-12-1023:08gfrederickspronounced "smerge"#2017-12-1110:17genec(doc s/merge) returns nil, are there docs for spec?#2017-12-1113:22Alex Miller (Clojure team)It shouldnāt be if you load clojure.spec.alpha :as s#2017-12-1113:23Alex Miller (Clojure team)Docs also at https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/merge#2017-12-1110:58guyhttps://clojuredocs.org/clojure.spec/merge this is pretty bare#2017-12-1110:58guybut you also have https://clojure.org/guides/spec#_entity_maps and its got s/merge inside of it too#2017-12-1110:59guyat the bottom of this entity map part#2017-12-1111:14ikitommi@gfredericks @mpenet bumped into the need of bijections too: In routing, if a path-parameter is declared as keyword?
we can easily coerce it from string with string->keyword
. But with reverse-routing, we need to turn that keyword param back to string to create a path. I seem to have resolved that earlier by using a json-encoder to write things to string-like, but this canāt be the right way to do this.#2017-12-1111:18ikitommiI guess a simple (and fast) way would be to introduce a IntoString
Protocol. But I guess there can be N different formats to read from and same N formats to write to. Hmm.#2017-12-1111:24ikitommiActually, I think I can just introduce a new type-conformer for spec-tools for the opposite direction type->string.#2017-12-1111:30ikitommiNo, using the conform requires the end value to be valid. This would require something like the CLJ-2251 with a ājust transform without validatingā options. Or something totally different.#2017-12-1112:21gfredericksso would still be a good use case for a bijections library?#2017-12-1218:04ikitommi@gfredericks Would be best if spec itself provided the needed hooks to implement the bijections, but not likely to happen. I need something now, so will hack something. If you have good ideas how to do this properly, Iām all ears!#2017-12-1111:45misha@alexmiller when will clojure.core/long?
be released? it is in master, but is not in clojure 1.9.0 as I expected (or am I missing something?). Thanks
https://dev.clojure.org/jira/browse/CLJ-1298#2017-12-1111:58bronsa@misha see https://github.com/clojure/clojure/commit/20f67081b7654e44e960defb1e4e491c3a0c2c8b#2017-12-1111:58mishaharold#2017-12-1111:59mishathanks, @bronsa#2017-12-1112:04mishais there out of the box generator for System.currentTimeMillis
?
(apart from clojure.test.check.impl/gen-current-time-millis) opieop#2017-12-1112:11gfredericks@misha how is what you're describing different from generating a long?#2017-12-1112:11mishafar greater than 0? :D#2017-12-1112:12gfredericksthey weren't far greater than 0 in 1970#2017-12-1112:12mishahonestly, I did not think it through yet#2017-12-1112:12mishapos-int?
then?#2017-12-1112:12gfredericksthey were negative in 1969#2017-12-1112:13gfredericksis there a (not clojure) spec for these numbers? I've never heard of one#2017-12-1112:13mishaoh, okay.#2017-12-1112:14mishaI think the "much greater than zero" part of a spec ā is a specialization for my case.#2017-12-1112:14gfredericksanyhow, with time generators you have three options
A) make a generator that is not focused on your current now (cleanest if you can get away with it)
B) call (time/now)
somehow before constructing your generators and pass that in (lots of boilerplate potentially)
C) make a nondeterministic generators (with associated damage to reproducibility and shrinking)#2017-12-1112:15gfredericksfor B) you'd want to log the time you're using, for reproducibility#2017-12-1112:18mishathank you.
at this point I think I might just use int?
with s/with-gen
, and supply (System.currentTimeMillis)
as a generator, as my only goal (so far) with this particular value - is plausible human-readable exercise data.#2017-12-1112:19misha(to the extent time-millis could be "human-readable")#2017-12-1112:20gfredericksconverting to/from time-mills would be a cool savant mental trick#2017-12-1112:20mishadoubt I would want to master it opieop#2017-12-1112:50tjtoltonIs there a megathread somewhere about 1.9 discussions? Did spec get pulled from the release as a finished product? Are macros using spec in the stable release? what's going on?#2017-12-1113:07bronsaspec got pulled out, it's still alpha but no major breaking API changes are expected, currently there's specs for ns
, let
, fn
, defn
#2017-12-1113:20Alex Miller (Clojure team)Hereās the spec split announcement which explains: https://groups.google.com/forum/m/#!msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ#2017-12-1114:15tjtoltonThanks @alexmiller, I'll bet that was a bummer of an internal conversation š#2017-12-1200:43athosAs you might already see it on the Clojure ML, I just released Pinpointer, a clojure.spec error reporter similar to Expound, Inspectable, etc https://github.com/athos/Pinpointer#2017-12-1200:44athosPinpointer formats (and even colorizes!) spec errors in a human-readable way. The difference between Expound and Pinpointer is very subtle from the outside perspective, but Pinpointer is based on a systematic error analysis rather than heuristic approaches, and this makes it possible to report more complicated errors, such that s/conformer would transform part of the input value.#2017-12-1200:46athosHave a look at it if you're interested, and any feedback is welcome š#2017-12-1210:25stathissideris@athos I tried pinpointer and it gave up on my complex error, it just used explain as a fallback. Under what circumstances does it do that?#2017-12-1211:22athosThank you for giving a try to Pinpointer, @stathissideris! š
By default, Pinpointer falls back to s/explain
if something bad happens during the error analysis. That mechanism exists to prevent Pinpointer itself from ruining the original spec error info.#2017-12-1211:23athosIf you would like to turn off the fallback behavior, call pinpoint
with the option {:fallback-on-error false}
like (pinpoint <spec> <input> {:fallback-on-error false})
#2017-12-1211:23stathissiderisok š in my case I have a pretty loose spec that describes a DSL, so if an expression is wrong, the errors include a lot or different options#2017-12-1211:24stathissiderisbecause the actual error is one of many things that can go wrong#2017-12-1211:24stathissiderisalright, Iāll try again with this option and see what happens#2017-12-1211:47stathissideristhanks!#2017-12-1212:18rickmoynihancan pinpointer/expound/etcā¦ handle exceptions with spec errors in them?#2017-12-1212:18rickmoynihanor would I need to hook into the REPLs exception printer?#2017-12-1212:24degIn a re-frame CLJS app, my app-db holds entries created by multiple parts of my system. I want to check total validity at start of each re-frame event. So, I have (s/def ::db (s/merge :ns1/db-keys :ns2/db-keys ,,,))
in a central db.cljs
file and I validate it in each event.
Most of these parts are small and can be reasonably each held by a single file holding its specs, subs, and events.
But, this creates a problem. The events need to require the namespace of the merged spec, while that namespace needs the namespaces of each part. What is the cleanest way to tastefully break this namespace circularity?#2017-12-1212:44rickmoynihanOkā¦ partly wanting this for exceptions integrant raises from pre-init-spec
ā¦ but it looks like integrant actually calls s/explain-out
to generate the message in the exception so it does indeed work#2017-12-1212:51athosYes, s/*explain-out*
plugin mechanism should handle exceptions well.
If you rather want something like clojure.repl/pst
, pinpointer.core/ppt
would be useful.#2017-12-1212:51rickmoynihancool#2017-12-1214:47Andreas LiljeqvistWhat is the recommended approach for shared specs in several projects? One solution is to make the spec a separate project and then require it.#2017-12-1215:03taylorIāve been curious about the same. I guess the options are 1) share a project 2) serialize to some other format which obv. limits what you can do in spec.#2017-12-1215:09Andreas LiljeqvistProbably going to use a project, on the plus side I get versioned specs#2017-12-1216:23misha1) monorepo 2) shared lib 3) send/receive as edn (youād need to make sure specs are cljs compatible, if sharing with ui)#2017-12-1219:00jeayeWe have a separate repo for specs shared between the client and server (edn). It also contains common functions shared between the two, defs, etc.#2017-12-1310:27Andreas LiljeqvistThanks for your input, quite helpfull#2017-12-1214:48Andreas LiljeqvistBut I am not completely happy with that idea#2017-12-1214:53stathissideriswhy not?#2017-12-1215:15Andreas LiljeqvistI suspect it is the best solution, but I am worried about code reloading and such. -- I will just have to try it out#2017-12-1215:29stathissideris@andreas862 donāt forget about the leiningen checkouts setting (if thatās what youāre using)#2017-12-1215:29stathissiderisit allows you to hack on multiple non-deployed projects in parallel#2017-12-1215:30Andreas Liljeqvist@stathissideris ah, that is a nice feature!#2017-12-1215:31stathissiderisš @andreas862 https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md#checkout-dependencies#2017-12-1215:38Andreas LiljeqvistPerhaps I should save my future self a lot of time and read the documentation...#2017-12-1215:38Andreas LiljeqvistThank you#2017-12-1221:05johanatanfor putting a spec on a def
'd var, is the standard way to have a fspec on a self-invoking function?#2017-12-1221:05johanatanis there some other way?#2017-12-1221:07bfabrynot sure I understand what "putting a spec on a def'd var" means. I'm guessing the answer isn't just "use fdef"?#2017-12-1221:07bfabryor do you mean a var that's defined to be something other than a function? if so then specing it isn't supported#2017-12-1221:07bfabryor... isn't useful#2017-12-1221:08johanatanyea, like a var that's bound to the result of a function#2017-12-1221:08bfabryyeah you can't spec that#2017-12-1221:08johanatane.g., (def ^:private keys-validators (events-schema->keys-validators events-schema))
#2017-12-1221:08johanatanor (def ^:private refinements
(merge user-defined-refinements normalized-base-refinements))
#2017-12-1221:09bfabryspec is meant for validating ranges of possibilities, a def'd var is "permanent" in clojure land. but yeah putting your right hand side inside a function and then fdefing that would make the most sense to me#2017-12-1221:09johanatanmm, ok. i'll try that. i think last time i tried it couldn't get it to work with anonymous functions (and wasn't really excited about polluting the global namespace with another defn
)#2017-12-1221:11johanatando you know if it is possible to fspec
an anonymous function?#2017-12-1221:11bfabryI mean seeing as this is all only happening once you could just do (def foo (s/assert ::spec (your code here)))#2017-12-1221:11johanatanhmm, except that it would probably need a doto
so that the result is returned?#2017-12-1221:12bfabryassert returns the result if the result was valid#2017-12-1221:12johanatanoh, ok#2017-12-1221:12johanatanyea, that works. thx!#2017-12-1221:13bfabryno worries#2017-12-1221:18hlolliHow does one do or in spec, I've tried googling
(s/explain (s/or :k1 (s/coll-of ::ar ::ar)
:k2 (s/coll-of ::ar ::kr)) [(AudioSignal. 1) (ControlSignal. 1)])
I also don't need these keys, but this function is probably wrong. Im makeing a transpiler to a language that has it's own dispatch functions, and since I'm automatically generating the metadata, I end up with different possibilites. Essentally, I'd want to be able to say; this vector can be a sequence of specs ::x ::y or sequence of specs ::foo ::bar.#2017-12-1221:19taylors/tuple
?#2017-12-1221:20hlollitroughout the ocean of functions I want to spec, they can be tuple/octet or whatever number of possibilites...#2017-12-1221:20Alex Miller (Clojure team)probably want to look at the regex specs (cat * + ? alt)#2017-12-1221:22hlolliok, will look there. Lot to take in when first stepping into spec š#2017-12-1221:24Alex Miller (Clojure team)if I read what you want to do there for example you could say (s/cat :x ::ar :y (s/alt :a ::ar :k ::kr))
#2017-12-1221:25hlollicat defines a sequence?#2017-12-1221:25Alex Miller (Clojure team)to match sequential collections that start with ar and then have ar or kr. x, y, a, and k are all made up keys that affect the conformed result#2017-12-1221:25Alex Miller (Clojure team)anything sequential#2017-12-1221:25Alex Miller (Clojure team)seq, vector, list#2017-12-1221:27hlolliok nice, this is actually for the input parameters so it's importent, I'm tempted to do this check at the very beginning of my functions.#2017-12-1221:27Alex Miller (Clojure team)they are most commonly used in s/fdef to spec args to functions#2017-12-1221:27Alex Miller (Clojure team)so that is a good match#2017-12-1221:29hlolliyes I may end up doing that, it's just this instrument function that I'm skeptical of, but I should not be. I want immediate error in runtime on wrong spec, or no matching patterns of a provided arguments, it's kinda pattern matching, therefore looking at or
#2017-12-1221:29Alex Miller (Clojure team)s/assert might be useful too#2017-12-1221:30hlolliOn extra question since I'm rambling, is there a way to have any effect on the error message
In: [1] val: #object[csound.core.ControlSignal] fails spec: :csound.core/ar at: [:k1 :0] predicate: audio-signal?
I'm in clojurescript and I know that the object has keys which could help the user locate his problem, as this could be within a nested ast.#2017-12-1221:32taylors/explain-data
gives you a structure you can get useful stuff out of, and there are some libs for transforming that into something that could be useful for you#2017-12-1221:33taylorhttps://github.com/bhb/expound https://github.com/alexanderkiel/phrase#2017-12-1221:33hlolliah yes! in fact the object itself in question.#2017-12-1221:35hlolliThanks @U3DAE8HMG#2017-12-1221:35hlolliOr explain has it too, just didn't consider takeing advantage of that and throw my own error message, suited to help the user find the spec mismatch.#2017-12-1420:01bfaypssst @U0CAUAKCG whatcha doing with csound? Looks cool#2017-12-1420:12hlolliWho's asking @U0MF00YRX? I'm working on some interoperability between clojurescript and csound, mainly transpiler, that would be a good start, already been live-coding with this combination for few years#2017-12-1420:16bfayOh awesome @U0CAUAKCG! I'm trying to do some similar stuff with clojurescript/supercollider. I tried using Overtone for a bit, but found that the startup time is too painful, especially when I tried running on a Raspberry Pi#2017-12-1420:24hlolliYes, Overtone is bit over engineerd for Raspberry Pi. But it's way better software than Sonic Pi in my opinion.#2017-12-1420:34bfayI think Sam Aaron had a different vision for both. The vision for Sonic Pi is about live-coding and education, it's a fun way to teach newcomers how to code. But yeah possibilities are pretty limited compared to Overtone and plain SuperCollider#2017-12-1304:37johanatanhi, does anyone know how to diagnose the following error?
1. Unhandled java.lang.IllegalArgumentException
No implementation of method: :specize* of protocol:
#'clojure.spec.alpha/Specize found for class: nil
#2017-12-1304:42johanatani found this but it seems only tangentially related: https://dev.clojure.org/jira/browse/CLJ-2032#2017-12-1304:44johanatanhere's the code:
(s/def ::json-primitive
(s/or :n nil :b boolean? :n number? :s string?))
(s/def ::json-structure
(s/with-gen
(let [checker (fn inner [primitives? v]
(cond
(map? v) (and (every? string? (keys v)) (every? (partial inner true) (vals v)))
(coll? v) (every? (partial inner true) v)
primitives? (s/valid? ::json-primitive v)
:else false))]
(partial checker false))
#(gen/recursive-gen (fn [inner] (gen/one-of [(gen/list inner) (gen/vector inner) (gen/map gen/string inner)]))
(s/gen ::json-primitive))))
(s/def ::validation-fn
(s/or :func (s/fspec :args (s/cat :x ::json-structure)
:ret boolean?)
:spec s/spec?))
(s/def ::message-fn
(s/fspec :args (s/cat :x ::json-structure)
:ret string?))
(defn- seq->gen
"Takes a sequence of generators and produces a generator of sequences."
[seq]
(apply gen/tuple seq))
(defn- map-seq->gen
"Takes a sequence of values and a function to apply over them
and produces a generator of the sequence of mapped values."
[f val-seq]
(seq->gen (map f val-seq)))
(s/def ::refinements
(s/with-gen
(s/map-of keyword? (s/tuple (s/nilable keyword?) (s/tuple ::validation-fn ::message-fn)))
#(gen/let [kwds (gen/vector gen/keyword 5 25)
refinements
(map-seq->gen
(fn [kwd]
(gen/let [k (gen/frequency [[4 (gen/elements (clojure.set/difference (set kwds) (set [kwd])))]
[1 (gen/return nil)]])
validation-fn (gen/frequency [[8 (gen/return (with-meta (fn [_] true) {:validates? true}))]
[1 (gen/return (with-meta (fn [_] false) {:validates? false}))]])
message-fn (gen/fmap (fn [s] (with-meta (fn [_] s) {:msg s})) (s/gen string?))]
[kwd [k [validation-fn message-fn]]]))
kwds)]
(into {} refinements))))
and the invocation:
(gen/sample (s/gen ::refinements) 1)
#2017-12-1304:45johanatanyou'll need appropriate imports; i.e., s
and gen
from the spec guide#2017-12-1313:17lopalghost(s/def ::json-primitive
(s/or :n nil š boolean? :n number? :s string?))#2017-12-1313:18lopalghostshould that be nil?
#2017-12-1313:19guynil? is a predicate whereas nil is not#2017-12-1313:19guyis that what you mean?#2017-12-1313:20guy (s/or :n nil? :b boolean? :n number? :s string?))
#2017-12-1313:20guyi think s/or takes keyword, predicate pairs#2017-12-1313:20lopalghostyes, this was in response to @johanatan#2017-12-1313:21guyah sorry#2017-12-1313:21guyš#2017-12-1313:21guyGood spot then!#2017-12-1317:25johanatan@lopalghost good catch! thx!#2017-12-1317:43johanatandoes anyone know a way for an fspec
to consider relations between its target's inputs? like if one of the parameters is functionally constrained by the value of the other one?#2017-12-1317:44johanatani could do it by stuffing the args into a nested vector:
(defn f [[a1 a2]] ...)
and then writing a spec/generator for that vector which combines two other [independent] spec/generators but i was kind of hoping that there is a better story for this...#2017-12-1317:52lopalghostthe :args
argument already treats the args as a sequence, so you can use (s/and ...)
to spec both individual inputs and relations between them#2017-12-1317:52lopalghostthere's a good example in the spec guide: https://clojure.org/guides/spec#_spec_ing_functions#2017-12-1317:53johanatanah, nice! thx again#2017-12-1318:21jcthalysHi, how can i let this second element of a tuple optional?#2017-12-1318:21jcthalys(s/tuple ::string-not-blank
(s/nilable pos-int?))
#2017-12-1318:22jcthalysthis is accepted ["ChLh2" nil]
but this not ["ChLh2"]
#2017-12-1318:32lopalghosttuple is for a collection of fixed size. for variable size you want to use cat#2017-12-1318:33lopalghosteg
(s/cat :string ::string-not-blank :number pos-int?)
#2017-12-1318:34lopalghostsorry, that should be
(s/cat :string ::string-not-blank :number (s/? pos-int?))
#2017-12-1318:34lopalghostif you want the second term to be optional#2017-12-1319:04jcthalysin my case itās a vactor, with one string and when have another thing its that intā¦#2017-12-1319:05jcthalysthatās s/cat
return a mapā¦#2017-12-1319:53lopalghostafter you conform vector, you can use unform
to turn it back into a sequence#2017-12-1319:53lopalghostkeep in mind you don't need to use conform
at all#2017-12-1319:56lopalghostthe alternative would be something like:
(s/or :two (s/tuple ::string-not-blank)
:one (s/tuple ::string-not-blank pos-int?))
#2017-12-1319:57lopalghostor otherwise, pad the vector with a nil before conforming, if it only has one element#2017-12-1319:57lopalghostI think cat
would be most idiomatic though#2017-12-1320:19johanatan@lopalghost actually s/and
for that purpose will result in a such-that
and you can be at risk of a couldn't satisfy such-that predicate after 100 tries
error#2017-12-1320:20johanatanthere needs to be a way to generate the two values together so that we know the two will conform with one another#2017-12-1320:20johanatanhence my suggestion to use a nested vector of args#2017-12-1320:21johanatando you see any other way?#2017-12-1320:38tristefigureHi. This is my first time using spec, and taking inspiration from clojure.core.specs.alpha, I decided to write my own version to parse/produce macro definitions. I wanted it to return something less detailed in a flatter hash and I wasn't happy with the fact results from conform
mirror the spec' structure and with other petty details, so this is what I came up with:#2017-12-1320:38tristefigureTo which extent is this a misuse of this library ?#2017-12-1321:35lopalghost@johanatan not sure how a nested vector would solve the problem, as any spec you apply to the nested vector could also be applied to :args#2017-12-1321:36johanatan@lopalghost it solves it by letting me introduce a generator specifically for that tuple#2017-12-1321:38lopalghostYou can't do that with the :arts spec? Sorry if I'm misunderstanding you#2017-12-1321:38lopalghost:args*#2017-12-1321:38johanatane.g.,
(s/def ::args-tup
(s/with-gen
(s/tuple ::arg1 ::arg2)
#( ... enforce relations here )))
(s/fdef -refinement-kwd->validator
:args (s/cat :tup ::args-tup)
:ret ::ret-val)
(defn- -helper [[arg1 arg2]] ; tuple allows both inputs to be generated simult.
;; omitted
)
(defn- original [arg1 arg2]
(-helper [arg1 arg2]))
#2017-12-1321:39johanatanyou can do it in :args
but you are liable to hit the cannot satisfy such-that
problem#2017-12-1321:47lopalghostI mean why not just use ::args-tup as the :args spec? Does that cause a problem? I'd try it out myself but I don't have a repl at the moment#2017-12-1321:50johanatans/cat is required to name the arg#2017-12-1321:50johanatanyea, without s/cat, stest/check tries to send in the wrong number of arguments#2017-12-1321:50johanatan[i.e., sends in 2 where 1 is expected]#2017-12-1322:09lopalghostok I cobbled together a stupid example, let me know if this is relevant:
(s/def ::args-tup (s/and (s/tuple pos-int? pos-int?)
#(> (first %) (second %))))
(s/fdef subtract
:args ::args-tup
:ret pos-int?)
(defn subtract
[high low]
(- high low))
#2017-12-1322:10lopalghostdidn't supply a generator of course, but stest/check
works with no problems here#2017-12-1322:32hlolliWould it be wise to spec no argument, if just for the metadata, if so, how would someone spec :args
for no args?#2017-12-1406:59curlyfryJust (s/cat)
!#2017-12-1410:08hlolliThusund takk!#2017-12-1322:35johanatan@lopalghost that use of s/and
has an implied underlying gen/such-that
which depending on circumstances may be impossible or too difficult to satisfy. thus it is always better to generate exactly the right structure you want from scratch rather than relying on such-that
.#2017-12-1322:49johanatan@lopalghost also, your fdef
won't work: :args
is an alternating sequence of name
and type/spec
. thus you need s/cat
#2017-12-1415:03lopalghost@johanatan seems to work fine
> (s/exercise-fn `subtract)
> ([[2 1] 1] [[7 1] 6] [[3 1] 2] [[4 3] 1] [[13 4] 9] [[5 4] 1] [[7 3] 4] [[17 1] 16] [[96 3] 93] [[14 3] 11])
> (stest/check `subtract)
> ({:spec #object[clojure.spec.alpha$fspec_impl$reify__2451 0x2bfc33f4 "
#2017-12-1415:04lopalghostis there a reason you need the args vector to conform to a map?#2017-12-1415:05lopalghostif you supply a generator for ::args-tup, you shouldn't have to worry about failure during generative testing#2017-12-1415:13gfrederickseasy generator for a strictly descending pair of numbers: (gen/let [[a b] (gen/tuple gen/large-integer gen/large-integer)] (let [[a b] (sort [a b])] [(inc a) b]))
#2017-12-1415:47andy.fingerhutI was looking at updating the Clojure cheat sheet for v1.9, now that it is out. I was wondering -- is there a list somewhere of all Clojure predicates for which test.check has generators included, vs. those it does not have generators for?#2017-12-1415:48gfredericksI expect there's a data structure in the spec code mapping specs to generators#2017-12-1415:49andy.fingerhutIf someone beats me to finding a link to that, let me know. I may want to link to it.#2017-12-1415:50gfredericksif you have a list of predicates, then programmatically figuring out which have generators should be easy#2017-12-1415:52andy.fingerhutIs there some small expression you can evaluate in a REPL to determine if a predicate has a generator already?#2017-12-1415:53gfredericksI thought there was a function in the spec API that takes a spec and returns a generator#2017-12-1417:03Alex Miller (Clojure team)s/gen#2017-12-1417:03Alex Miller (Clojure team)There are some things in gen namespace too#2017-12-1417:04Alex Miller (Clojure team)And thatās where the built in mapping is set#2017-12-1417:05lopalghostgen/gen-for-pred#2017-12-1417:47andy.fingerhutThanks, s/gen is very helpful. And dang, I could have really added a lot of value by helping with the generator for zero? š#2017-12-1417:50andy.fingerhutI couldn't have been very helpful, of course. Simply my way of expressing surprise that there was such a generator.#2017-12-1420:41gfredericks(gen/one-of (gen/elements [(float 0) (int 0) 0.0 0 0N (biginteger 0) -0.0]) (gen/let [scale (gen/large-integer* {:min Integer/MIN_VALUE :max Integer/MAX_VALUE})] (.setScale 0M scale)))
I assume?#2017-12-1420:56andy.fingerhutYes, the most complex part about it is if you want to generate every value that can satisfy zero?#2017-12-1420:57andy.fingerhutI probably should have made my snide remark about the predicate nil? instead, since it would be more obvious.#2017-12-1420:57andy.fingerhutI am probably having a senior moment here on my memory -- is there any reasonably common term for any type that isn't a collection?#2017-12-1420:58tayloritās not scalar is it#2017-12-1420:58andy.fingerhutIt is, and I am understanding more every day what memory lapses feel like š#2017-12-1420:59taylorI increasingly find myself forgetting what Iām about to do as I walk across my tiny apartment to do it#2017-12-1421:44gfrederickstattoos are the answer
put "things that aren't collections are scalars" on your left arm#2017-12-1421:47cgoreAnd āthere are no scalarsā on your right arm š#2017-12-1422:00andy.fingerhutOn a related note, I watched the movie "Memento" for the first time recently. Sound advice.#2017-12-1422:00taylorare you sure it was the first time??#2017-12-1422:00gfredericksmore likely he will soon watch it for the last time#2017-12-1422:02andy.fingerhutIt felt like the first time, anyway#2017-12-1422:02andy.fingerhutIf only I can keep forgetting my favorite movies and books, I can get full enjoyment from them multiple times -- if I can remember which ones they are.#2017-12-1422:04andy.fingerhut@U3DAE8HMG LOLAHTEMLTMW#2017-12-1422:04taylorif ignorance is bliss then forgetfulness isā¦ personal growth?#2017-12-1422:04andy.fingerhut(Laugh Out Loud And Had To Explain My Laughter To My Wife)#2017-12-1507:30andy.fingerhutAre there any handy examples someone can link to for writing a spec for a function, where one of its arguments is a predicate function, e.g. a spec for clojure.core/filter ?#2017-12-1507:43taylorthis example isnāt exactly taking a predicate but it demonstrates a HOF taking another fn https://taylorwood.github.io/2017/10/15/fspec.html#higher-order-functions#2017-12-1507:59andy.fingerhutThanks. Nicely written examples there.#2017-12-1507:56andy.fingerhutI've got the doc strings for seq? and seqable?, but I can't for the life of me think what the difference is between them, i.e. why would you pick one over the other in a spec?#2017-12-1508:05curlyfry@andy.fingerhut seqable? is for things that can be made into a seq (but aren't necessarily seqs themselves). For example, (seq? [])
returns false, but (seqable? [])
returns true.#2017-12-1508:08andy.fingerhutGot it. Thanks.#2017-12-1508:09andy.fingerhutseqable? seems far more often useful in :args specs, then.#2017-12-1508:14andy.fingerhutLooks like a spec like this probably needs a custom generator, if I see an error message like "Couldn't satisfy such-that predicate after 100 tries."? (s/def ::set/relation (s/and set? #(every? map? %)))#2017-12-1508:19tayloryeah with s/and
I believe the generator is based off just the first spec#2017-12-1508:20taylorso, very unlikely it will generate sets with all maps#2017-12-1508:26andy.fingerhutThis page: https://clojure.github.io/test.check/generator-examples.html gives this example of a generator: (def sorted-vec (gen/fmap sort (gen/vector gen/int)))#2017-12-1508:26andy.fingerhutWhen I try to def sorted-vec in my REPL I get an assertion error "Arg to vector must be a generator"#2017-12-1508:28andy.fingerhutSorry, pilot error. I had my gen alias'ed to clojure.spec.gen.alpha, not clojure.test.check.generators as stated at the top of that examples page#2017-12-1508:44andy.fingerhutDoes a generator for predicate any? ever produce values that aren't equal to themselves in Clojure, e.g. Double/NaN ?#2017-12-1512:10gfrederickstest.check's gen/any
sure can
not sure what any?
got wired up to#2017-12-1512:10gfredericksI feel like in both cases we need a secondary concept, like gen/any-sane
#2017-12-1512:11gfredericksjust making NaN opt-in all the time seems like encouraging bad testing#2017-12-1512:30lopalghost@andy.fingerhut have you tried using coll-of
to spec a set of maps?#2017-12-1512:30lopalghosteg (s/coll-of map? :kind set?)
#2017-12-1512:31gfredericksgen/any-equatable
#2017-12-1513:03andy.fingerhutgen/any-anumber#2017-12-1513:03andy.fingerhutbecause, you know, it is not, not a number#2017-12-1513:09gfredericksyeah but I'm talking about for something like gen/any
where you are potentially generating large data structures and if there are any NaNs hiding anywhere in that giant tree then the whole thing becomes sometimes equal to itself but sometimes not#2017-12-1513:11andy.fingerhutright. gen/any-anumber was a bad joke š#2017-12-1513:12lopalghostreminds me of one of my favorite clojure expressions:
> (number? ##NaN)
> true#2017-12-1513:13andy.fingerhutI feel a JIRA coming on ...#2017-12-1513:14andy.fingerhut@lopalghost I have not tried coll-of, but thanks for the suggestion. Will do.#2017-12-1513:16andy.fingerhutI guess the generator for any? also sticks with things considered "values" in Clojure, i.e. it doesn't generate instances of java.util.Set ?#2017-12-1513:17andy.fingerhutthose also play havoc with one's usual ideas for clojure.core/= if they are inside of other sets, or keys of maps.#2017-12-1513:59gfredericksyeah, definitely#2017-12-1514:00gfredericksgenerating literally anything would be quite a tall order#2017-12-1521:28didibusIs there a way to override the generator of the args spec of an fspec? Using instrument and :gen I thought would be able to do it, but I can't figure how.#2017-12-1521:33Alex Miller (Clojure team)I don't think so, or at least it's not easy (and I think there is a ticket about this)#2017-12-1521:35didibusOk, thanks#2017-12-1609:40andy.fingerhutAre there any common practices on how detailed people like to make their specs, in terms of trying to specify precisely which :args should be allowed, vs. which should not? In particular, I've been looking at clojure.set/rename-keys and how some values of the second argument lead to hard-to-predict return values.#2017-12-1609:41andy.fingerhutExamples: (set/rename-keys {:a 1 š 2 :c 3} {:a š :c :b}) => {:b 3}#2017-12-1609:41andy.fingerhutbut: (set/rename-keys {:a 1 š 2 :c 3} {:c š :a :b}) => {:b 1}#2017-12-1609:42andy.fingerhutIf you see happy faces, those should be keywords with "b"#2017-12-1609:42andy.fingerhutI could write a spec simply saying that the second parameter is a map?, but that would allow maps like the above to be included, which have unpredictable behavior.#2017-12-1609:43andy.fingerhutI could also write a more restrictive spec for the 2nd arg that prevents any 2 values in the 2nd argument map from being equal to each other, which would avoid the unpredictable behavior.#2017-12-1609:45andy.fingerhutFor that same function, it is also clear that for generating random inputs that are "interesting", i.e. that don't simply always return their first argument with no changes, the keys in the first and second maps should often contain common values. So a useful generator that tries to exercise bugs in the implementation needs to steer inputs in that direction with high probability.#2017-12-1609:46andy.fingerhutAnyway, looking for thoughts and recommendations there.#2017-12-1610:13gklijs@andy.fingerhut I think a great deal comes down to how much effort you want to put into it. As sone as you start using very specific specs, you will also need to supply generators for those. While if you just want to have some edge cases tested, it would probably be less work to write specific tests for those. Depending on how you use specs, making them complex might also slow things down.#2017-12-1611:51guyyou can use ` to make it ignore emojiās @andy.fingerhut#2017-12-1611:51guydo one ` then another after you have finished#2017-12-1611:51guyit should look like {:a 2 :b 3}
#2017-12-1618:36bbrinckIāve just released Expound 0.4.0. Now Expound will describe missing key specs https://gist.github.com/bhb/5abeff11f5252e915a07ea20acadd7fc#2017-12-1703:18jeaye@bbrinck Sweet. Nice work!#2017-12-1820:12donaldballIām struggling to figure out a good way to express a spec category that has come up before, but I canāt recall what the best recommendations are any longer. The value is a map with namespace keyword keys, two of which each require values in their own sets, but further the pair of values must occur its own set. For example, :pet/type
could be #{:cat :dog :snek}
and :pet/fur?
could be boolean?
, but {:pet/type :snek :pet/fur? true}
is not allowed.
This is trivial with a predicate spec but then I lose any hope that s/explain
can identify the paths to the keys contributing to the violation.#2017-12-1821:27Alex Miller (Clojure team)multispec?#2017-12-1821:35misha@donaldball yeah, doable with multispec, but verbose. Especially if you'd overload second value based on type of the first (different value sets for :pet/fur?
depending on :pet/type
#2017-12-1821:38mishaplus, you'd need to supply custom generators to avoid "could not satisfy predicate after # tries", when amount of valid value invariants is noticeably smaller than count of combinations in cartesian product of both value sets.#2017-12-1821:44donaldballI thought about multispec but I wasnāt even sure how I could tell spec āhey use a smaller spec for this namespaced key in s/keys
this timeā and couldnāt figure out a good way to express it using s/map-of
š#2017-12-1821:44mishaspeaking of "verbose", how you, ladies and gentlemen, organize your specs, when there is fair amount of "intermediate" specs, which are not exactly useful in and of themselves, but are used to build larger composite specs. E.g. when "leaf" specs are "useful", and "root" spec is "useful" (for eyeballing), but "branch" specs in between are sort of just glue and are noisy.
"Just suck it up" ā is an acceptable answer, I guess opieop#2017-12-1821:45misha@donaldball you can "alias" specs#2017-12-1821:48misha(s/def :cat/color #{:grey :white})
(s/def :dog/color #{:black :white})
(s/def :pet/color :dog/color)
#2017-12-1821:51mishathe thing is, when you want to overload a :pet/foo key value based on :pet/type value, you can get away with this in spec by using unqualified keywords in your entity maps {:foo ..., :type}.
but if you want to overload value of a namespaced key ā it will be a mess to spec it, if possible at all#2017-12-1821:57misha@donaldball you still have an option to return these from multispec
(s/keys :req [:pet/type :cat/fur?]) ;; for type=cat
(s/keys :req [:pet/type :snake/fur?]) ;; for type=snake
where
(s/def :pet/fur? boolean?)
(s/def :cat/fur? :pet/fur?)
(s/def :snake/fur? false?)
#2017-12-1822:34donaldballYeah, but all these options seem to require adjusting the form of my inputs. While the input form is not necessarily desirable, Iād like to try to spec is it is, not as I have transformed it to be.#2017-12-1919:38falakI am trying to write specs for a function whose argument can be nil or a collection. However, I am unable to combine those two using s/or
. This is what I tried -
(s/fdef my-func
:args (s/cat :arg1 (s/or :nil nil? :collection list?)))
This definitely won't work but it'd be great to have the option -
..... (s/or nil? list?)
#2017-12-1919:42guycanāt you do
(s/def ::arg1 (s/nilable list?))
then just use that with s/cat :arg1 ::arg1 ?#2017-12-1919:50falakAh! I wasn't aware of s/nilable
. Thanks!
However, I am not sure why but (s/nilable coll?)
works and s/nilable list?
doesn't when the data structure is actually a list of maps (or nil).#2017-12-1919:52falakWhen I run type
on the args, it says clojure.lang.LazySeq
#2017-12-1920:10seancorfield@fravani because list?
is only true for things that implement IPersistentList
-- which a lazy sequence does not.#2017-12-1920:11seancorfieldBut coll?
is true for IPersistentCollection
and LazySeq
does implement that.#2017-12-1920:18falakThat makes sense. No wonder it worked when I converted the lazy seq to non-lazy using into () ...
Thank you so much for the help.#2017-12-2021:10sparkofreasonIf I want to have a macro that will be able to process spec forms defined in ClojureScript, how do I get the forms in the macro definition? s/form
will only get the form at run time, and I need it at compile time.#2017-12-2021:30mishawill you use it instead of s/def
?#2017-12-2023:15sparkofreasonNo, after s/def
. Basically trying to auto-generate destructuring forms based on the spec.#2017-12-2022:02agCan someone give me an insight to the issue I'm having with our project please.
Because of the way how clojure.spec.alpha interacts with clojure.test.check, specifically I'm guessing this part https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L38
it breaks on attempt to use clojure.tools.namespace.repl/refresh
, in a project where clojure.test.check.generators
being used in .cljc file(s).
And that's because Generator
record gets redefined, resulting in two different classes.
Here's a sample repo that repros that bug (created by my colleague) https://github.com/xiongtx/reload-error-boot.
I thought the easy way of fixing it is to stop using clojure.test.check
and just use clojure.spec.gen.alpha
everywhere. But that doesn't seem to include bunch of things: macros like gen/let
and gen/bind
, gen/return
, gen/pos-int
, etc. Basically not using it at all - seems not possible (?)
Because of that, CIDER's refresh and clojure.tools.namespace.repl/refrsh
are broken, that makes it sort of painful to work with. Can someone suggest a workaround?#2017-12-2022:03gfrederickscan you use refresh-all
instead?#2017-12-2022:03gfrederickswait why does Generator
get redefined at all?#2017-12-2022:08gfredericksrefresh
shouldn't be reloading library code#2017-12-2101:00agcan you please take a look at https://github.com/xiongtx/reload-error-boot
there's something strange going on, I think this deserves to be a JIRA ticket (unless someone already filed one)
I can't formulate though exactly what's happening#2017-12-2101:12gfredericksat a glance, my guess is that cider is more enthusiastic about reloading things than tools.namespace is?
certainly seems like bad behavior. I don't think tools should reload libraries under normal circumstances. I've never seen tools.namespace do that.#2017-12-2101:19agit happens also with tools.namespace#2017-12-2101:20agit's not bug in CIDER#2017-12-2101:20agit's because of this https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L38#2017-12-2112:06gfredericksyou're saying the lack of a standard require makes tools.namespace act as if t.c.generators was a project namespace?#2017-12-2116:07agĀÆ\(ć)/ĀÆ#2017-12-2116:07agit seems failing only for .cljc files#2017-12-2116:08agwith shared generators #2017-12-2117:23gfredericksdoes tools.namespace explicitly announce that it's reloading the generators namespace?#2017-12-2117:23gfredericks(it usually prints which things it's reloading)#2017-12-2118:02agthis is in our project, when you do
(require '[clojure.tools.namespace.repl :refer [refresh]])
(refresh)
:reloading ... then list of namespaces
:error-while-loading finops-admin.specs.shared
#error {
:cause "Assert failed: Second arg to such-that must be a generator\n(generator? gen)"
:via
[{:type clojure.lang.Compiler$CompilerException
:message "java.lang.AssertionError: Assert failed: Second arg to such-that must be a generator\n(generator? gen), compiling:(shared.cljc:72:34)"
:at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3700]}
{:type java.lang.AssertionError
:message "Assert failed: Second arg to such-that must be a generator\n(generator? gen)"
:at [clojure.test.check.generators$such_that invokeStatic "generators.cljc" 346]}]
:trace
and long stacktrace
#2017-12-2118:02agnote, there's nothing wrong with that generator in finops-admin.specs.shared
#2017-12-2118:17agthis is a bug that appears in the .cljc code that's trying to use clojure.test.check.generators
#2017-12-2118:17agif you read the readme in https://github.com/xiongtx/reload-error-boot
you'll see detailed explanation#2017-12-2119:13gfredericksBut does the list of namespaces you elided contain test.check?#2017-12-2119:34agyup it does#2017-12-2119:49gfredericksokay, that is what seems buggy to me#2017-12-2121:08agposted here https://www.reddit.com/r/Clojure/comments/7lbplj/clojuretoolsnamespacereplrefresh_tumbles_on_cljc/#2017-12-2121:22xiongtxSeems that this is a known issue w/ tools.namespace
. Thanks to @U09LZR36F for pointing this out! https://dev.clojure.org/jira/browse/TNS-45?actionOrder=desc#issue-tabs#2017-12-2100:15sparkofreasonFound an answer, albeit hacky-feeling, to getting compile-time spec forms for CLJS. At compile time, the CLJS form of the spec is stored in cljs.spec.alpha/registry-ref
, an atom containing a map keyed by spec-name. This works, but feels like I'm coupling to internal implementation details, so if there's a better way, I'd be thankful.#2017-12-2100:15johanatan@lopalghost ah, yea, sorry re: the s/cat: it apparently is redundant/ unnecessary when the spec itself is already a tuple
#2017-12-2100:15johanatanand yea, generating the entire tuple at once is what i'm doing in fact#2017-12-2100:15johanatanwas just hoping there'd be a better way because that implies that i need wrapper functions that take the untupled inputs and call the tupled ones#2017-12-2100:16johanatani'm hitting a weird error where spec itself is trying to construct an ExceptionInfo in a haram way#2017-12-2100:16johanatanthrowing this exception:
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/ExceptionInfo.java#L31#2017-12-2100:17johanatanhere's the full stack:
[[clojure.lang.ExceptionInfo <init> "ExceptionInfo.java" 31]
[clojure.lang.ExceptionInfo <init> "ExceptionInfo.java" 22]
[clojure.core$ex_info invokeStatic "core.clj" 4739]
[clojure.core$ex_info invoke "core.clj" 4739]
[clojure.spec.test.alpha$explain_check invokeStatic "alpha.clj" 277]
[clojure.spec.test.alpha$explain_check invoke "alpha.clj" 275]
[clojure.spec.test.alpha$check_call invokeStatic "alpha.clj" 295]
[clojure.spec.test.alpha$check_call invoke "alpha.clj" 285]
[clojure.spec.test.alpha$quick_check$fn__2986 invoke "alpha.clj" 308]
[clojure.lang.AFn applyToHelper "AFn.java" 154]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.core$apply invokeStatic "core.clj" 657]
[clojure.core$apply invoke "core.clj" 652]
[clojure.test.check.properties$apply_gen$fn__16139$fn__16140 invoke "properties.cljc" 30]
[clojure.test.check.properties$apply_gen$fn__16139 invoke "properties.cljc" 29]
[clojure.test.check.rose_tree$fmap invokeStatic "rose_tree.cljc" 77]
[clojure.test.check.rose_tree$fmap invoke "rose_tree.cljc" 73]
[clojure.test.check.generators$fmap$fn__9199 invoke "generators.cljc" 101]
[clojure.test.check.generators$gen_fmap$fn__9173 invoke "generators.cljc" 57]
[clojure.test.check.generators$call_gen invokeStatic "generators.cljc" 41]
[clojure.test.check.generators$call_gen invoke "generators.cljc" 37]
[clojure.test.check$quick_check invokeStatic "check.cljc" 94]
[clojure.test.check$quick_check doInvoke "check.cljc" 37]
[clojure.lang.RestFn invoke "RestFn.java" 425]
[clojure.lang.AFn applyToHelper "AFn.java" 156]
[clojure.lang.RestFn applyTo "RestFn.java" 132]
[clojure.core$apply invokeStatic "core.clj" 657]
[clojure.core$apply invoke "core.clj" 652]
[clojure.spec.gen.alpha$quick_check invokeStatic "alpha.clj" 29]
[clojure.spec.gen.alpha$quick_check doInvoke "alpha.clj" 27]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.core$apply invokeStatic "core.clj" 661]
[clojure.core$apply invoke "core.clj" 652]
[clojure.spec.test.alpha$quick_check invokeStatic "alpha.clj" 309]
[clojure.spec.test.alpha$quick_check invoke "alpha.clj" 302]
[clojure.spec.test.alpha$check_1 invokeStatic "alpha.clj" 335]
[clojure.spec.test.alpha$check_1 invoke "alpha.clj" 323]
[clojure.spec.test.alpha$check$fn__3005 invoke "alpha.clj" 411]
[clojure.core$pmap$fn__8105$fn__8106 invoke "core.clj" 6942]
[clojure.core$binding_conveyor_fn$fn__5476 invoke "core.clj" 2022]
[clojure.lang.AFn call "AFn.java" 18]
[java.util.concurrent.FutureTask run "FutureTask.java" 266]
[java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1149]
[java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 624]
[java.lang.Thread run "Thread.java" 748]]
#2017-12-2100:20johanatanseems like this may be a bug:
https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L279#2017-12-2100:20johanatanif when-not
returns nil
, ExceptionInfo
doesn't like that#2017-12-2100:26johanatan(apply ex-info (remove nil? ["Specification-based check failed" (when-not ...)]))
would seem to be a fix#2017-12-2100:27johanatanalthough that's going to seriously limit the person on the receiving end's ability to debug š#2017-12-2100:27johanatanso there is likely a higher-level explanation for why we got here#2017-12-2100:27johanatanwhich might be good to pass on to the user#2017-12-2107:24misha@dave.dixon you can call any defined function inside macro as part of the data transformation, canāt you? Which happens at compile time. It is the form macro returns who is evaluated at run time#2017-12-2107:25mishaDid you try to use s/form inside macro, but outside of quoted return value?#2017-12-2111:29trisshey all - how do I turn something like (clojure.spec.alpha/coll-of clojure.core/number?)
back in to spec I can use in s/valid?
. Iām deconstructing my specs and using bits of them elsewhere.#2017-12-2111:42guyCould you just do (s/def ::your-spec (s/coll-of number?))
Then you can do (s/valid? ::your-spec ..)
#2017-12-2111:42guyOr canāt you just use it in s/valid?
#2017-12-2111:42guy(s/valid? (clojure.spec.alpha/coll-of clojure.core/number?) ā¦)
#2017-12-2111:48trissthanks guyā¦ Iāll check that in a momentā¦. I just realised a quick call to eval
does the job for me.#2017-12-2111:49guykk!#2017-12-2114:18sparkofreason@misha Yes. That doesn't find the spec. Looking at the CLojureScript spec code, the form
function is only defined at run-time, and basically just wraps the specize*
method on the Specize
protocol. The reified instance of Specize
wouldn't be available at compile-time anyway. The registry-ref
atom defined in cljs/spec/alpha.cljc
appears to be the mechanism by which the s/def
macro communicates between compile and run-time. I don't see any function which would abstract access to registry-ref
for use from macros.#2017-12-2121:14agpossible to have a "parametrized spec", where you can specify generator parameters? e.g.:
making the following more flexible, by letting it generate dates in specified range (instead of hardcoded values):
(s/with-gen (s/int-in (inst-ms #inst "1980-01-02")
(inst-ms #inst "2050-12-31"))
#(gen/choose
(inst-ms #inst "2015-01-01")
(inst-ms #inst "2016-12-31")))
?#2017-12-2121:15Alex Miller (Clojure team)no, other than via a macro wrapping this#2017-12-2121:16Alex Miller (Clojure team)there is s/inst-in
which does ranges?#2017-12-2121:16Alex Miller (Clojure team)not sure if that would serve your needs#2017-12-2121:17agalright... thanks @alexmiller#2017-12-2121:56seancorfield@ag Yeah, that's been a bugbear for us at World Singles since we have to track a constantly moving time window for certain valid time ranges.#2017-12-2122:11agso this (gen/sample (s/gen (s/inst-in #inst "2017-10-01" #inst "2018-12-31")))
returns bunch of dates, but they are mostly are in 2017-10
, what gives?#2017-12-2122:12shaun-mahood@ag: There was a great talk by @gfredericks at this years conj about generators that had a good section on better generation of datetimes (the most relevant part starts at about 30 mins in if you want to skip right to it) - https://www.youtube.com/watch?v=F4VZPxLZUdA#2017-12-2122:12seancorfield@ag generators for ranges tend to start at the beginning and use small increments at first. Yup, what @shaun-mahood said!#2017-12-2122:13seancorfieldThat talk was super helpful for us at World Singles!#2017-12-2122:13agwe've been also using test.chuck I think it's @gfredericks project, right?#2017-12-2122:19shaun-mahoodYep - I haven't tried it yet but more than once someone has pointed me to it to answer a question about doing something more complicated. I guess I should try it š#2017-12-2122:21seancorfieldWe use it for regex generators. Wonderful!#2017-12-2123:08misha@ag you can provide map of override generators on spec "call" site#2017-12-2123:10mishahttps://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L272-L281#2017-12-2123:10misha...Optionally an overrides map can be provided which
should map spec names or paths (vectors of keywords) to no-arg
generator-creating fns. These will be used instead of the generators at those
names/paths. Note that parent generator (in the spec or overrides
map) will supersede those of any subtrees...
#2017-12-2123:14mishathis way you can sort of dynamically inject generators built specifically for the current circumstances. But I yet to see (or spend time building myself) complex enough dependent generator "trees", so can't point to any caveats or best practices yet#2017-12-2123:15misha(e.g. imagine ETL pipeline: you generate random input file, generate ETL config based on generated input file, and then property-test extracted result against input)#2017-12-2123:18mishamore detailed your specs become ā more branchy and fragile (I guess?) your generators become. And managing that is noticeable overhead in and of itself. Throw in proper shrinking capabilities support, and random seed honoring, and it is suddenly a full time job opieop#2017-12-2123:59cflemingJust to double check my understanding here - if Iām wanting to use spec for macro grammars in a library which also needs to support clients on older versions of Clojure, I can just have the specs and the fdef
s in a separate namespace which is loaded dynamically somehow if Clojure 1.9+ is detected, is that right?#2017-12-2123:59cflemingOr perhaps users of the library can just require that namespace if theyāre using Clojure 1.9.#2017-12-2200:03bfabryseems to be exactly what clojure.java.jdbc does#2017-12-2200:03cflemingThanks, Iāll check it out.#2017-12-2200:03cflemingin fact, the specs could even be in their own library I imagine.#2017-12-2200:04bfabrywell if it works for core š#2017-12-2200:04cflemingYeah, but core is special š#2017-12-2200:06cflemingOk, great - looks like that will work fine - thanks!#2017-12-2200:54bbloomwhatās the idiom for negating a spec? ie if i want to greedly parse excluding something#2017-12-2200:55bbloomjust use #(not ā¦) ?#2017-12-2200:58gfrederickscomplement
?#2017-12-2200:58bbloomyeah or that#2017-12-2200:59bbloomjust wondering if there was any negative match regexes or anything like that#2017-12-2201:05bbloomi guess iām struggling with a non-greedy (reluctant?) s/* pattern#2017-12-2201:08andy.fingerhutHas anyone attempted to write code that determines the 'difference' between two specs yet? That this is possible to do for "vanilla" regular expressions was, I recall, mentioned as a motivating factor for spec to be include regex matching. (I realize, of course, that spec generalizes vanilla regex's in a way, with extra side conditions via and/or, such that it might not always be possible to apply those regex diff algorithms in those cases).#2017-12-2201:08andy.fingerhuts/be include/be based on/#2017-12-2201:11Alex Miller (Clojure team)I donāt think itās going to be efficient to do so without leveraging the guts of the regex spec impl#2017-12-2202:48cflemingIs it considered idiomatic to name things that I donāt care about capturing :_
?#2017-12-2209:46misha@cfleming I've seen it used. I prefer :_actual-name-of-thing-you-ignore both for keywords and symbols (documentations is hard to come by these days)#2017-12-2209:51cfleming@misha Thanks. This is for constant symbols and keywords in macro grammars, I had things like (s/cat <whatever> :from #{:from} <whatever>)
which seems a bit redundant.#2017-12-2209:51cfleming(i.e. when capturing a constant :from
)#2017-12-2210:03mishaof course there are things which are impractical to document (in any way)#2017-12-2216:00trissis it possible to look up the spec for a function?#2017-12-2216:18arohner(s/form (s/spec 'foo/bar))
#2017-12-2216:22athosYou probably mean s/get-spec
rather than s/spec
?#2017-12-2221:15curlyfry@triss clojure.repl/doc
will show you any specs for a function#2017-12-2222:36pablorehello, is there any way to define a spec for a sorted collection?#2017-12-2311:34stathissiderisI gave a talk on Spec and what you can do with it for the Athens Clojure Meetup, itās in Greek with English captions: https://www.youtube.com/watch?v=T1qpIaB6_vM#2017-12-2320:02bbrinck@stathissideris your talk was excellent. I particularly liked the descriptions and diagrams of more advanced generative-testing techniques. Good stuff.#2017-12-2320:06stathissideris@bbrinck thanks a lot, Iām glad you enjoyed it! :)#2017-12-2408:08misha@pablore yes, expensive one would be something like
(s/def ::sorted-coll
(s/and
(s/coll-of any?)
#(= % (sort %))))
#2017-12-2408:11mishabasically, write a predicate function, which will answer your "is it sorted?" question with true or false.#2017-12-2408:11rauh#(apply <= %)
is probably faster.#2017-12-2408:11misha(the one above changes collection type, and will not work most of the time)#2017-12-2408:17mishayeah, ~400times faster opieop
the point is ā basic spec is just a predicate function#2017-12-2414:27Alex Miller (Clojure team)just use sorted?
#2017-12-2414:27Alex Miller (Clojure team)if you are checking whether itās a sorted collection or not#2017-12-2416:28tayloris there a predicate for checking if something is a lazy seq#2017-12-2416:37taylordisregard, I really just want to convert all sequences to vectors#2017-12-2523:53flyboarderHey guys, trying to figure out how I can serialize a clojure spec exception. java.io.NotSerializableException: clojure.spec.alpha$regex_spec_impl$reify__2436
#2017-12-2607:45gklijs@flyboarder The Class java.lang.Exception extends Throwable implements Serializable, to get it to work you probably need to create your own exception which either extend from the java exception, or implement Serializable, thatās how far I can help you with my java background, never did anything yet in clojure for exception handling, so I donāt know how itās different from java.#2017-12-2609:11stathissideris@flyboarder looks like the serialization breaks for the spec itself which I believe is included in the ex-data of the ex-info, under the key :spec
#2017-12-2609:11stathissideriscould you dissoc it as a quick test and try serializing again?#2017-12-2619:58flyboarder@stathissideris that seems to work, I need to remove the entire namespaced key :clojure.spec.alpha/spec
is there a way to future proof the removal for when the namespace changes?#2017-12-2713:23Alex Miller (Clojure team)Alias clojure.spec.alpha to s in your namespace definition, then use ::s/spec. That way, you will just need to change the namespace in one spot.#2017-12-2718:54flyboarder@U064X3EF3 thanks!#2017-12-2619:59flyboarder@gklijs I needed to wrap the exception in my own ex-info
while removing the non-serializable parts, thanks!#2017-12-2704:40shdzzlI'm trying to work out what is happening with the following snippet, I think I must be misunderstanding how s/cat
works:#2017-12-2704:43shdzzlIt makes no sense to me that it would be matching the first vector element on ::bound
, so I'm not sure how that's the spec it's failing.#2017-12-2704:43flyboarder@shdzzl I had issues wrapping my head around s/cat also, it kinda explodes the argument, is the best way I can describe it#2017-12-2704:47shdzzlI'm not sure I follow, it explodes which argument? I visualize it as kind of zipping predicates to a sequence, so my s/cat
example should match any seq of length three and check the first element against ::needs
, the second against ::bound
and the third against ::body
. What am I missing?#2017-12-2704:48flyboardertry (s/and vector? (s/cat .....))
#2017-12-2704:49shdzzlSame result.#2017-12-2704:50flyboarderLoading up a repl, one sec#2017-12-2704:50shdzzlThanks.#2017-12-2705:10flyboarder@shdzzl well to start with your :needs spec fails#2017-12-2705:11flyboarderthats the real issue#2017-12-2705:11flyboarder(s/def ::needs (s/coll-of symbol?))
#2017-12-2705:12flyboarderboot.user=> (s/explain (s/cat :needs ::needs :bound ::bound :body ::body) [['a 'b] {} '(+ a b)])
Success!
#2017-12-2705:15shdzzlUhuh, that does fix it. ::needs
wasn't failing when tested on it's own. Is there some flattening or something going on? I'm going to have read over s/*
again. Thanks a ton.#2017-12-2705:16flyboarder@shdzzl thats what s/cat is doing (s/cat :needs (s/* symbol?))
is different than when you move that into itās own spec#2017-12-2705:17shdzzlOoohhh! When written like that it's more obvious.#2017-12-2705:18flyboarderrule of thumb for (s/cat)
build it up one arg at a time#2017-12-2705:18flyboarderotherwise itās hard to figure out where it fails#2017-12-2705:19shdzzlGood advice.#2017-12-2705:29shdzzlWhile I'm here, I had another question. Is it possible/advisable to spec constructors (`MyRecord.` for example)?#2017-12-2705:35shdzzlIt touches on it in the spec guide, but I was curious if someone had tried it with fdef
or similar.#2017-12-2705:37johanatanuser> (time (stest/check `a-symbol))
"Elapsed time: 2.153437 msecs"
does anyone have an explanation for why the above completes almost instantly but my repl does hang during the 10s of seconds that it actually takes to check the symbol in question?#2017-12-2705:40tbaldridge@johanatan what does that code print?#2017-12-2705:40johanatanwhich code? it just prints the elapsed time line, then hangs until the check actually completes#2017-12-2705:40johanatanthen i get my REPL prompt back to execute more lines#2017-12-2705:40tbaldridgeand what does it return when it completes?#2017-12-2705:42johanatanlet me check. didn't wait for it to complete before because i was trying to debug why a loop/recur was spinning constantly instead of waiting for check
to complete#2017-12-2705:45johanatan@tbaldridge it returned the typical map result from check
#2017-12-2705:46johanatan({:spec #object[clojure.spec.alpha$fspec_impl$reify__2451 0x6d92492f "
#2017-12-2705:48tbaldridgeinteresting#2017-12-2705:48tbaldridgeSo one last thing to try:#2017-12-2705:49tbaldridge(do (time (stest/check
a-symbol)) nil)`#2017-12-2705:49tbaldridgebleh you get the point, wrap it in a do, nil. that will remove any questions about REPL printing taking a long time#2017-12-2705:49tbaldridgewhich is normally what it is in cases like this#2017-12-2705:50johanatanhmm, that actually returned immediately like before but also gave me back control at the REPL prompt (i'm assuming the check
is happening in the bkg and i do have a process called main
doing a lot of CPU work in my procmon)#2017-12-2705:51johanatan[i think main is the cider repl server if i'm not mistaken-- last time i killed it, cider immediately printed out a bunch of stuff indicating unhappiness lol]#2017-12-2706:04johanatanah#2017-12-2706:04johanatanfound it#2017-12-2706:04johanatancheck
returns a lazyseq#2017-12-2706:04johanatanso wrapping it in doall
seems to have done the trick#2017-12-2706:05johanatan[i suppose the result wasn't being dropped on the floor previously because the repl itself was waiting for the value to be returned so that it could print it]#2017-12-2706:19johanatanthanks for the help!#2017-12-2706:19flyboarder@shdzzl sorry I havnāt tried that one#2017-12-2706:21shdzzlNo problem, I'm going to look into it when I get the time.#2017-12-2814:57romniis there a way to spec a map using a coll of types/keywords (instead of a single type/keyword as with multi-spec)? the idea is that all specs corresponding to matched types are merged and used to validate the map#2017-12-2815:02romniwhat i have is { ::types [:first :second] }
and what i'm looking for is a way to create something like a multi-spec which dispatches on the values in the coll corresponding to ::types
and which will build a spec like (s/merge ::first ::second)
#2017-12-2913:27souenzzo(defmulti types ::types)
(defmethod types :default [{::keys[types]}]) (eval
(apply s/merge types)))`
Some lke this?#2017-12-2815:12ikitommiHi. I have a vector of s/keys
specs and I would like to merge them, without eval. As s/merge
is a macro, I guess I should use merge-spec-impl
?#2017-12-2819:53ikitommiIām trying to combine stuff with s/merge
. Is this intentional or a bug?
(s/explain any? {:x 1, :y 2})
; Success!
(s/explain (s/merge any?) {:x 1, :y 2})
; val: {:x 1, :y 2} fails predicate: any?
#2017-12-2819:55ikitommiok, this works thou:
(s/explain (s/merge (s/spec any?)) {:x 1, :y 2})
; Success!
#2017-12-2820:04ikitommiBut as the s/spec
is macro too, with my arbitrary vector-of-specs input, I would have to call s/spec-impl
instead to avoid eval and figure out the forms somehow. Any news on the new functional core for clojure.spec?#2017-12-2913:31souenzzoI think that this is a bug @ikitommi#2017-12-2914:34ikitommithanks @souenzzo, wrote CLJ-2302. As a workaround, Iām forcing all the incoming spec to be Specs (or names of Specs):
(defn merge-specs [specs]
(when-let [non-specs (seq (remove #(or (s/spec? %) (s/get-spec %)) specs))]
(throw
(ex-info
(str "Not all specs satisfy the Spec protocol: " non-specs)
{:specs specs
:non-specs non-specs})))
(s/merge-spec-impl (vec specs) (vec specs) nil))
#2018-12-3110:09ikitommiHi. Is there or is someone working with a spec error reporter that could hint on misspelled keys or non-defined (directly on s/keys
) map keys?#2018-12-3118:16taylorsomething like this? https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2018-12-3110:11ikitommiI have a s/keys
spec for configuration with all the keys optional. Normal s/keys
is not helping much here.#2018-12-3110:15ikitommispec-tools has the utilities for pulling out all the keys & merges and could built on that, but would rather use something ready for this.#2018-12-3111:21dominicm@ikitommi figwheel has one#2018-12-3111:21dominicm@ikitommi strictly-specking-standalone#2018-01-0115:34ikitommiThanks @dominicm for the pointer. Looks great, seems to lean on custom strict-keys
Spec to catch the typos. Coudnāt get it working as it uses really old version of spec, but will poke around to see if I could get it working. Different artifact name btw in Clojars and in Github.#2018-01-0119:12drcodeHi everyone- I have a clojure DSL that comes in two variants that are about 95% the same and I want to write a single spec that validates both variants- What is the best way of doing this? Basically what I want to do is write about a hundred clojure.spec/def
declarations that are the same for both variants, but a few deeply nested items will need idiosyncratic logic, looking something like:
(s/def :deeply/nested-item
(if variant1?
variant-1-specification
variant-2-specification))
I know I can solve this through some nasty macrology (that maintains two separate namespaces of specs for the two variants, so that the idiosyncratic differences can be "spliced in") but usually it is a bad idea to lean heavily on complex macros... Thanks for any tips!#2018-01-0119:43gfredericksaaron brooks was just talking to me on twitter about this sort of thing#2018-01-0119:43gfrederickshttps://twitter.com/0x1B/status/946499269279846400#2018-01-0119:44gfrederickspaging @abrooks#2018-01-0120:45drcode@gfredericks yep, that's my situation...#2018-01-0120:49drcodeyes, I think I'll just use @abrooks' cheat until "spec specs" are available- Thanks for the pointer!#2018-01-0215:37pabloreHow would you spec this function?
(defn reducer
([] {})
([state action]
(...)))
Where state and action are specced#2018-01-0215:38taylor(defn such-arity
([] "nullary")
([one] "unary")
([one two & many] "one two many"))
(s/fdef such-arity
:args (s/alt :nullary (s/cat)
:unary (s/cat :one any?)
:variadic (s/cat :one any?
:two any?
:many (s/* any?))))
#2018-01-0215:43pablorethanks#2018-01-0215:46pabloreI thought that could solve my validation problem with my function, but it still failing :c#2018-01-0215:46pabloreCouldn't satisfy such-that predicate after 100 tries. {}
#2018-01-0216:00taylorthat could be due to one of your specs using an s/and
and not being able to generate something that conforms to the whole spec#2018-01-0216:02taylor(s/def ::foo-map (s/and map? #(:foo %)))
(gen/sample (s/gen ::foo-map))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4739)
#2018-01-0216:03taylorfor example āļø that s/and
is generating based off that first map?
predicate, but the generator is unlikely to produce a map with :foo
and so it gives up after so many tries#2018-01-0216:29pabloreCannot fix it for now. Will put it on hold until errors are more specific#2018-01-0216:29pablorethanks anyway for the info#2018-01-0402:34cflemingIs there a way to make the spec optional and star operators greedy?#2018-01-0402:34cflemingI have the following spec:
(s/cat :binding (s/? (s/cat :binding-var ::variable-name
:sep #{'<-}))
:fact-type any?
:destructured-fact (s/? ::destructured)
:expressions (s/* any?))
#2018-01-0402:35cflemingThis is horribly ambiguous as written, due to the use of any?
and repeated/optional elements.#2018-01-0402:35cflemingIād like to make the first s/?
greedy, so to say if itās possible to match the :binding
part then thatās what should happen.#2018-01-0402:36cflemingThe only alternative I can think of is to do this, taking advantage of ordering precedence for alternates:
(s/or
:with-binding (s/cat :binding-var ::variable-name
:sep #{'<-}
:fact-type any?
:destructured-fact (s/? ::destructured)
:expressions (s/* any?))
:no-binding (s/cat :fact-type any?
:destructured-fact (s/? ::destructured)
:expressions (s/* any?)))
#2018-01-0402:36cflemingBut thatās kind of ugly.#2018-01-0402:38cflemingIs there any defined precedence of optional elements inside an s/cat
?#2018-01-0404:11Alex Miller (Clojure team)In short, no. The way the regex derivative stuff works is kind of doing all the options in parallel#2018-01-0502:56cflemingFWIW other regex parallel implementations allow this even though they work in lockstep with no backtracking.#2018-01-0502:56cfleminge.g. NFAs or VM approaches#2018-01-0503:23cflemingI think it would be worth specifying in the doc if ops like ?
and *
are greedy or not - itās not mentioned right now and is useful information in ambiguous cases.#2018-01-0404:18Alex Miller (Clojure team)I donāt know that I understand what youāre trying to spec well enough but the latter seems more readable to me#2018-01-0404:20Alex Miller (Clojure team)One thing Iāve found is that specāing a dsl is a pretty good way to judge how good your dsl is :)#2018-01-0406:17cflemingThis is for Clara Rules#2018-01-0406:17cflemingSo I canāt easily modify how it works.#2018-01-0413:00mikerod> One thing Iāve found is that specāing a dsl is a pretty good way to judge how good your dsl is :)
Yeah, in the case of Clara, there is a bit too much ambiguity in this case Iād say. Have to figure out a way to tighten it down a bit perhaps. Spec to the rescue.#2018-01-0416:34bbrinckI seem to recall that someone published a repository with some specs for core functions (beyond what is currently included in https://github.com/clojure/core.specs.alpha), but now I canāt find it. Anyone else remember this?#2018-01-0416:35bbrinckI realize that these specs may be incomplete or buggy, and thatās OK for my use case. This is just for toying around with instrumentation.#2018-01-0416:37bbrinckOh, nevermind, my fourth google search finally found it https://github.com/leifp/spec-play#2018-01-0416:38andre.stylianosThere's also this one: https://github.com/tirkarthi/respec
Don't know how it compares to the one you mentioned#2018-01-0416:39bbrinck@andre.stylianos Awesome, thanks!#2018-01-0419:21Alex Miller (Clojure team)the repos Iāve looked at were both mostly wrong and not very good examples of specs, so ymmv#2018-01-0419:23bbrinck@alexmiller Good to know. Iām not that concerned about accuracy for this case, but if they are not well-written specs, thatās more of an issue, since Iām using these as a temporary proxy for the real clojure.core specs#2018-01-0419:24bbrinckI should clarify: Iām not worried about completeness in the sense that if, say, some functions are missing specs for certain less common arities, thatās OK. If the specs are just buggy in general then that would cause an issue.#2018-01-0419:26Alex Miller (Clojure team)from what Iāve looked at, the specs are wrong, as in will both allow bad things and reject good things#2018-01-0419:27Alex Miller (Clojure team)and most of them spec a bunch of things that instrument wonāt work with (inline functions and functions with primitive args), which is not bad, but also not useful#2018-01-0419:27bbrinckGotcha. Yeah, for my case, rejecting good things will cause issues. Iāll take a look.#2018-01-0419:29bbrinck> bunch of things that instrument wonāt work with (inline functions and functions with primitive args)
Iām afraid I donāt understand. Can you expand on this a little?#2018-01-0419:35seancorfield@bbrinck see https://dev.clojure.org/jira/browse/CLJ-1941#2018-01-0419:39Alex Miller (Clojure team)instrument currently doesnāt work with: functions taking/returning primitives (fixable), inlines, multimethods (fixable), and protocols#2018-01-0419:39Alex Miller (Clojure team)so specāing +
will buy you nothing for example#2018-01-0419:40Alex Miller (Clojure team)but itās not harmful and does get included in docs, which might be valuable in and of itself#2018-01-0420:16bbrinck@seancorfield @alexmiller Ah, thanks. I did not know about that. If this works, Iāll be using instrumentation in CLJS, so presumably the primitive issue will be different#2018-01-0420:26Alex Miller (Clojure team)Oh yeah, Iām talking Clojure#2018-01-0420:29bbrinckWell, I didnāt mention the CLJS part before š and in any case, those specs are in CLJ, so I can see why you thought CLJ (Iāll need to copy them into a cljs file if I want to use them).#2018-01-0422:05kennyWhy is this valid?
(s/def ::cat (s/cat :a keyword? :b any?))
(s/def ::foo (s/coll-of (s/spec ::cat)))
(s/valid? ::foo {:a "a"})
=> true
#2018-01-0422:10noprompt@kenny because
(coll? {})
;; => true
#2018-01-0422:10cfleming@kenny I think because a map is a coll. Presumably s/coll-of
calls seq
on the map to get at the elements, which in this case are entries. Then s/cat
calls seq
and gets the entry key and val.#2018-01-0422:10kennyShit, that always gets me.#2018-01-0422:10nopromptš#2018-01-0422:11kennyThanks guys.#2018-01-0500:34andy.fingerhutI am guessing that if some function A included in Clojure calls function B, and I instrument B, then because of direct linking within default Clojure JAR, A will still call un-instremented version of B? And building my own Clojure with direct linking disabled is probably best way to enable A to call instrumented version of B?#2018-01-0500:36Alex Miller (Clojure team)That is all correct#2018-01-0500:36andy.fingerhutMakes sense. Thx#2018-01-0500:37Alex Miller (Clojure team)There is a build property you can set in the Clojure build (we run the tests both with and without on CI)#2018-01-0500:37Alex Miller (Clojure team)-Ddirectlinking=false#2018-01-0500:52andy.fingerhutI must be doing it wrong. I've tried "mvn -Ddirectlinking=false clean package install", and even edited build.xml to change directlinking to false in there, but instrument'd B isn't causing A-to-B call to fail as I think it should. I'll keep experimenting to see where I'm going wrong.#2018-01-0500:58bronsa@andy.fingerhut beware of inlining maybe#2018-01-0500:59andy.fingerhutA is clojure.data/diff, or actually a function/protocol implementation that it calls, that calls B = clojure.set/union, where clojure.set/union is the function I'm trying to instrument.#2018-01-0501:01andy.fingerhutUgh. I think I forgot that build.xml only affect ant builds, not mvn builds.#2018-01-0501:11seancorfieldIt's -Dclojure.compiler.direct-linking=true
#2018-01-0501:12seancorfield(or false
) ^ @andy.fingerhut#2018-01-0501:13seancorfield(or at least that's what we have in our script for production processes -- and we assume it's false
by default elsewhere)#2018-01-0501:16andy.fingerhutI believe that flag is what tells the Clojure compiler whether to direct link user Clojure programs, or not. It doesn't seem to affect whether Clojure core code is compiled with direct linking (i.e. it still seems to have direct linking between my A and B functions that are both within Clojure itself)#2018-01-0501:16andy.fingerhutI can avoid direct linking within Clojure itself by building it from source after changing true to false for directlinking inside of pom.xml#2018-01-0501:16andy.fingerhutI haven't found the right incantation to make that happen from the command line, without editing pom.xml#2018-01-0501:16seancorfieldAh, OK, misread what you were attempting then. Sorry.#2018-01-0501:18seancorfieldGiven that it's hardcoded in pom.xml
https://github.com/clojure/clojure/blob/master/pom.xml#L37 I would not expect it could be overridden at the command-line (but my mvn-fu is weak).#2018-01-0501:21Alex Miller (Clojure team)You can override it (thatās what the clojure matrix builds do). Maybe that flag is only for test compilation. Sorry, on my phone and havenāt looked at it in a couple years.#2018-01-0501:22andy.fingerhut@alexmiller No worries. Hand-editing pom.xml is doing the trick for me, and I don't need it automated for any personal reasons. I suspect you may be correct that the command line options might only be affecting compilation of Clojure tests.#2018-01-0501:23Alex Miller (Clojure team)Oh, it is just for testing - see the profile definitions in the pom#2018-01-0501:24Alex Miller (Clojure team)Activating the different profiles will change the flag#2018-01-0501:25Alex Miller (Clojure team)You change the compilation for Clojure itself in build.xml#2018-01-0501:25Alex Miller (Clojure team)In the compile-clojure target#2018-01-0501:25Alex Miller (Clojure team)There is a sysproperty there#2018-01-0501:25Alex Miller (Clojure team)I think youāll have to hard code that change#2018-01-0501:26Alex Miller (Clojure team)Tis a tangled web#2018-01-0501:34andy.fingerhutIf I get a reproducible way to build Clojure in this way, is it of any use to add it to dev wiki ?#2018-01-0501:58Alex Miller (Clojure team)Sure, although Iām not sure how wide the need is for it #2018-01-0501:59Alex Miller (Clojure team)Stu sometimes runs with just the src too, no aot at all#2018-01-0508:29romniis there any way to write a spec for a multimethod?#2018-01-0513:19Alex Miller (Clojure team)Currently, no. You can separate the dispatch function though and spec that if you like#2018-01-0515:02romnithanks, that's what i've been doing so far but i was wondering if spec
contained some syntax sugar for this#2018-01-0509:28andy.fingerhut@alexmiller I understand the need is not very wide for building Clojure without direct linking, but I took a shot at adding a short section with instructions for how to do so at the end of this wiki page. Feel free to delete it if it is not useful enough, or edit otherwise to your heart's contents: https://dev.clojure.org/display/community/Developing+Patches#2018-01-0512:36mishahow do I both have
(s/and vector? (s/cat :a int? :b (s/* string?)))
and generator for free? this exact form gives up on generating vector?
part#2018-01-0512:38mishaI do have to use gen/let
, etc. don't I?#2018-01-0512:41gfredericksideally the API would be such that you could specify that the default generator for (s/cat :a int? :b (s/* string?))
be transformed with (gen/fmap vec ___)
#2018-01-0512:41gfredericksI don't think the spec API makes that easy though#2018-01-0512:44mishaso I need to extract s/cat as dedicated spec, and use it both in s/and spec definition and generator#2018-01-0512:45mishaor is there a "shorter" way?#2018-01-0513:18Alex Miller (Clojure team)cat + vector is something that is quite difficult to do atm while retaining conform, generator, form, etc and a known gap. There are various ways to get parts of those but itās hard to get all of them. This is certainly something Iāve run into and you can see it in some of the core dsl specs. Iāve come to the conclusion that a built in s/vcat would solve all of the cases Iāve run into but I think Rich is looking for a more composable solution still#2018-01-0513:30mishas/vcat is definitely an ok solution for an application code, but smells like "giving up" in a library opieop#2018-01-0513:34Alex Miller (Clojure team)Which is one reason this is still unresolved :)#2018-01-0513:43Alex Miller (Clojure team)There is overlap here between regex ops and coll-of which both work on colls. Ideally those features could be more easily composed#2018-01-0516:21mishaIs there an "easy" way to overwrite :min-count
:max-count
of the s/coll-of
s/map-of
specs' generators "inline"? e.g. for REPLing. Or will I have to provide brand new generators for those and use them in s/with-gen
?#2018-01-0518:56Alex Miller (Clojure team)the generators will respect the min/max count you gave them#2018-01-0518:56Alex Miller (Clojure team)but you can do something like (s/with-gen (s/coll-of int? :max-count 100) #(s/gen (s/coll-of int? :max-count 3)))
#2018-01-0519:30mishaI am asking, if there is shorter way than you wrote. Many colls-of I speced as a standalone specs, and it would be cool to reuse those with something like
(s/def ::my-coll-spec (s/coll-of int? :max-count 100))
(-> ::my-coll-spec
(with-opts :min-count 5 :max-count 10)
(s/exercise))
#2018-01-0519:33misha2 main use cases for this for me, are:
- do not generate too many, because I want to eyeball it.
- do not generate too few, like (s/exercise) tends to do for first few samples (or all, if you give it small sample size (s/exercise ::my-spec 2)
)#2018-01-0519:41Alex Miller (Clojure team)in short, no. you can give exercise a generator override map though#2018-01-0519:48mishaI remember that, but it is basically the same effort stuff-typing- and copypaste-error-wise for these 2 usecases.
thank you#2018-01-0522:45seancorfieldWhen I have (s/conform (s/merge ::spec-1 ::spec-2) my-data)
this only seems to conform values in the last merged spec, not the first one. If you swap the spec order, you get conformedvalues from ::spec-1
instead but not ::spec-2
-- is that by design or is it a bug in s/merge
? /cc @alexmiller#2018-01-0522:47seancorfield(if I change s/merge
to s/and
I get conformed values from both specs)#2018-01-0522:51Alex Miller (Clojure team)By design s/merge does not flow conformed values #2018-01-0522:57seancorfieldOK. That was just surprising, given the docstring.#2018-01-0522:59seancorfieldSo I'm going to need (s/and (s/merge ::spec-1 ::spec-2) ::spec-1)
for my particular use case (so that it still generates), right @alexmiller?#2018-01-0522:59seancorfield(I know that checks ::spec-1
twice)#2018-01-0523:02Alex Miller (Clojure team)The generator for merge will produce maps that validate for both specs#2018-01-0523:03Alex Miller (Clojure team)So you shouldnāt need the and here for generation#2018-01-0523:05seancorfieldRight, but I need the s/and
to get conformance flowing through the specs.#2018-01-0523:05seancorfieldIf I want both conform-flow and generation, I need both the s/and
and s/merge
.#2018-01-0523:06seancorfieldThe s/merge
on its own doesn't flow conformance (but will generate fine).#2018-01-0608:40ikitommi@U04V70XH6 would this do both? https://github.com/metosin/spec-tools/pull/91#2018-01-0608:42ikitommispec-tools is already doing the rogue coercion stuff, so woudn't make the lib any more bad in that sense.#2018-01-0523:06seancorfieldThe s/and
on its own doesn't generate properly (but will flow conformance).#2018-01-0523:08seancorfieldThis is because we have specs that conform API parameter strings to long, double, date etc -- and I want to be able to combine them and get full conformed values across the combination and get generation across the combination š#2018-01-0523:08seancorfield(cue @alexmiller saying "Well, don't do that!" š )#2018-01-0523:09Alex Miller (Clojure team)Yeah#2018-01-0523:13Alex Miller (Clojure team)Didnāt anyone ever tell you not to do coercion?#2018-01-0611:12mishais there anything to read at length about "don't do the coercion"? It makes sense here and there, but I'd like to see the bigger picture#2018-01-0614:38Alex Miller (Clojure team)If you build coercion into your spec, particularly lossy coercion, then you have made a decision for all consumers of your spec that they have no choice over#2018-01-0617:02gfredericksis the recommendation for coercion use-cases "do it by hand outside the spec'd zone"?#2018-01-0618:40mishaso objection is not about coercion, but about complecting it with specs? Or even about lossy (in any way) specs?#2018-01-0620:28ikitommiāthey (spec consumers) have no choise overā => wouldnāt CLJ-2116 fix just that?#2018-01-0622:57Alex Miller (Clojure team)@misha right. I have no problem with coercion as a concept. :)#2018-01-0623:23gfredericksI on the other hand#2018-01-0703:33seancorfieldAs someone who uses coercions in some specs, I wouldn't support CLJ-2116. It enshrines pluggable coercion into clojure.spec
which is not the right approach IMO.#2018-01-0801:34aengelbergI'm trying to write a util that prints out a more friendly error message based on the data returned from s/explain-data
. I'm noticing that :in
contains misleading data for certain cases:
user=> (s/explain-data (s/map-of integer? integer?) {345 "a"})
{:clojure.spec.alpha/problems ({:in [345 1],
:path [1],
:pred clojure.core/integer?,
:val "a",
:via []}),
:clojure.spec.alpha/spec #<
So for a map-of
, it is giving me the key as well as 1
(to indicate the val of the map pair), and for a set, it is giving me an index into some arbitrary ordered version of that set. Neither of these would work if I tried to get-in
on those paths.
If these values were somehow prefaced with an indicator of what special case triggered them, I could work with that, but the fact that the data may or may not be misleading with no additional context makes the overall data unhelpful for writing a universal tool. Is that a bug? Or is there another way of introspecting the failed specs that I'm missing?#2018-01-0801:58Alex Miller (Clojure team)on the first case, this is a side effect of treating maps as a sequence of tuples. there is at least one ticket about this particular one with a pending patch.#2018-01-0801:58Alex Miller (Clojure team)for the set case, I donāt think anything is logged but a ticket for that would be fine#2018-01-0809:21aengelbergThanks, I'll try to find time to write up a ticket. I'll definitely check out the other ticket you mentioned for map-of
, since I'm curious what the proposed solution was, so I know what the expected behavior should be for sets.#2018-01-0816:07bbrinckI ended up writing a lot of code to disambiguate these cases in Expound. https://github.com/bhb/expound/blob/master/src/expound/paths.cljc . The ticket for map-of is https://dev.clojure.org/jira/browse/CLJ-2192 .#2018-01-0816:08bbrinckIIRC, https://github.com/athos/Pinpointer takes a different approach to the problem: it analyzed the specs directly to help understand the explain-data
(I could be mis-stating something here, @U0508956F please correct me if Iām wrong š )#2018-01-0816:09bbrinck@U0567Q30W ticket is https://dev.clojure.org/jira/browse/CLJ-2192#2018-01-0819:36aengelberg@U08EKSQMS thanks! do you have any specific thoughts on the set issue as well? are you also handling that specially from expound?#2018-01-0819:57bbrinckI havenāt traced the execution too closely, but I suspect that the set case is handled by this code https://github.com/bhb/expound/blob/master/src/expound/paths.cljc#L134-L135#2018-01-0819:58bbrinckThis could have bugs though. It took awhile to figure out an approach that would work for most :in
paths that I could find.#2018-01-0820:00bbrinckRe: sets, I would vote for any JIRA ticket that makes :in
paths non-ambiguous, because I think this is one of the biggest hurdles with building pretty-printers for spec. If you make an issue for the set case, send me the link š#2018-01-0821:10cflemingThanks @U08EKSQMS, Iāve been meaning to take a closer look at Expound actually#2018-01-0822:09bbrinckCool! Feedback is welcome and appreciated #2018-01-0810:38cfleming@aengelberg Iād like to see that too if you find it.#2018-01-0812:02mishais it overkill?
(s/nonconforming (s/or :i pos-int? :z #{0}))
#2018-01-0812:04misha- zero?
throws on strings, keywords, etc.
- just (complement neg-int?) accepts everything but neg ints: strings, maps, etc.#2018-01-0812:11mishais this in any "under-the-hood" way lighter?
(s/and int? (complement neg-int?))
#2018-01-0813:34Alex Miller (Clojure team)What about nat-int?#2018-01-0814:54mishamissed it opieop#2018-01-0820:46dadairGiven the following data: {:a {:name "a"} :b {:name "b"} :c {:name "not-c"}}
, how could I construct a spec to say "the key to a value-map must be the keywordized name within the value-map"? This spec would complain that (keyword "not-c")
does not match :c
#2018-01-0821:01Alex Miller (Clojure team)You can use any arbitrary predicate like #(= (keys %) (->> % vals (map :name) (map keyword)))
to check validity#2018-01-0821:02Alex Miller (Clojure team)but itās not going to give you that specific error#2018-01-0821:09dadairgreat thank you#2018-01-0907:11athosIn that case, I'd prefer something like (s/coll-of (fn [[k v]] (= k (keyword (:name v)))))
#2018-01-0918:30dadaircheers! I like that too#2018-01-0912:24Charles FourdrignierHello,
I would like to write a client for testing some servers' responses.
Not for production purpose, but for coding events like coderetreats (inspired by https://github.com/rchatley/extreme_startup).
I'm trying to implement this with Clojure (http-kit) and think to Clojure Spec as a validator.
I'm a beginner with Clojure and Clojure Spec and I don't know if it's a "good" idea, absolutely ridiculous or in the "Clojure's spirit".
I write two specs to demonstrate the concept.
https://gist.github.com/Charlynux/35c754f3f213012397da8ae61b97cb1c
I would appreciate any feedback about the idea, code...#2018-01-0915:12bbrinckI donāt see any problem with the idea of using spec to validate responses. Depending on how you intend to use the spec, there may be some ways you can improve it so you get better error messages when it fails.#2018-01-0915:27bbrinckFor instance, it might be useful to use a multi-spec here https://clojure.org/guides/spec#_multi_spec or maybe use s/or
so you can figure out which case matched with conform
. It depends on whether you want to just make sure the response is any type of valid response (i.e. 200 and 404 will both be OK, as long as well formed) or if you want to use spec to determine if a response was successful or not#2018-01-0915:44bbrinckIāve added an alternate idea to your gist. Also, check out https://github.com/ring-clojure/ring-spec/blob/master/src/ring/core/spec.clj#L129-L149#2018-01-0916:07Charles FourdrignierThank a lot for your ideas.#2018-01-0912:42stijnwhat can be the reason that explain is successful on an invalid conform?#2018-01-0912:55souenzzowhen I get it, i restart my repl.#2018-01-0912:44mpenetwhat's the spec?#2018-01-0912:49stijni found the 'syntax-error' in the data, so i'm going to try to narrow it down#2018-01-0912:51mpeneton a side note seems like that one still breaks#2018-01-0912:51mpenet(fn [_] :clojure.spec.alpha/invalid)
#2018-01-0912:51mpenetoh, maybe not then, it just broke in my repl, hmm#2018-01-0913:53madstapThat's why it works here @mpenet#2018-01-0915:59bhauman@alexmiller I heard Rich mention in his last talk that next version Spec would be more "programmable" or something like that. Is there anything more I can read about that? Or can I ask about specifics?#2018-01-0916:07mpenetmy guess: either it's something related to https://dev.clojure.org/jira/browse/CLJ-2112 with unforming that re-creates a new spec, or something entirely new#2018-01-0916:36bhauman@mpenet Thanks, for the pointer.#2018-01-0916:53Alex Miller (Clojure team)Nothing to read yet and not related to 2112#2018-01-1008:10mpenetWould it be considered ok to promote s/assert* to good for "public" use (atm the docstring says : "Do not call this directly, use 'assert'.")#2018-01-1008:10mpenetin cases where you want the check to always happen no matter the value of check-assert/compile-asserts#2018-01-1008:10mpenetor is there a better way? ping @alexmiller#2018-01-1008:13mpenetI guess if it would be used that way it could warrant a rename maybe#2018-01-1010:33Andreas LiljeqvistI seem to remember that I could generate with a provided example. Something like (gen/fill-in ::spec {::id "all generated will have this id, other values are generated"})
#2018-01-1010:34Andreas LiljeqvistCan't remember what it was called though... anyone?#2018-01-1011:06misha@andreas862
(s/def ::id string?)
(s/def ::foo (s/keys :req [::id]))
(s/exercise ::foo 2 {::id #(s/gen #{"a" "b"})})
;=>
;([{::id "a"} {::id "a"}]
; [{::id "b"} {::id "b"}])
?#2018-01-1011:16Andreas Liljeqvist@misha yes, thank you!#2018-01-1016:56misha(defn gen
"... Optionally an overrides map can be provided which
should map spec names or paths (vectors of keywords) to no-arg
generator-creating fns. These will be used instead of the generators at those
names/paths. Note that parent generator (in the spec or overrides
map) will supersede those of any subtrees.
parent generator will supersede those of any subtrees
part just bit me in the ass harold#2018-01-1016:57mishabut it forces me to keep generators and specs separate, which ends up to be cleaner#2018-01-1021:39pabloreI have some function (defn foo
[a]
{:pre [(s/valid? ::bar a)]}
...)
That asserts if the input conforms to a spec. It works well, but Iām having trouble to debug when it fails, I want to know what input Iām passing that doesnt conform and why.#2018-01-1021:47pabloreI just get an AssertionError#2018-01-1022:18misha@pablore I use s/explain-data
instead of s/valid
. it returns nil
if everything is ok, and a datastructure, if there are any errors.#2018-01-1022:19mishaSo you can use it in if
or when
.#2018-01-1022:21misharead through this too, may be it'll give you some ideas https://clojure.org/guides/spec#_using_spec_for_validation
afaik, specs are replacement of :pre
/`:post`-conditions#2018-01-1022:24pablores/explain works better than I expected!
I know of s/fdef but Iām writing a really complex reducing function with lot of if-cases.#2018-01-1022:28misha(s/check-asserts true)
(s/assert string? :a)
;; clojure.lang.ExceptionInfo: Spec assertion failed
;; val: :a fails predicate: :clojure.spec.alpha/unknown
note string?
and predicate: :clojure.spec.alpha/unknown
#2018-01-1101:35andy.fingerhutTrying to understand the core specs for destructuring here: https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L21-L44#2018-01-1101:36andy.fingerhutDoes s/every on a map check all key/value pairs as 2-tuples (or at least any that it does check, since I know that s/every does sampling)#2018-01-1102:47Alex Miller (Clojure team)Yes - map entries are 2-tuples#2018-01-1102:47Alex Miller (Clojure team)I wrote a blog about the destructuring spec btw#2018-01-1102:48Alex Miller (Clojure team)http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2018-01-1103:15andy.fingerhutGlad you wrote that. It is helpful for understanding.#2018-01-1114:54pabloreAm I using s/conform wrong?
(s/def ::id int?)
(s/def ::some-map
(s/keys :req [::id]))
(s/conform ::some-map {:id 1}) => s/invalid
#2018-01-1114:56ikitommitry :req-un
#2018-01-1115:01pablorethat worked! thanks#2018-01-1115:03pablorebut conform is returning {:id 1}
instead of {::id 1}
.#2018-01-1115:14Alex Miller (Clojure team)s/keys conform doesnāt modify keys. because youāre conforming a map with unqualified keys, the conformed map will have unqualified keys#2018-01-1115:15Alex Miller (Clojure team)you could also use :req
with data that looks like {::id 1}
#2018-01-1115:42Andreas LiljeqvistI have a problem with multi-spec and overriding generators#2018-01-1115:42Andreas LiljeqvistExample code at https://pastebin.com/TW886mDV#2018-01-1116:31Alex Miller (Clojure team)can you state your problem in the form of a question?#2018-01-1116:40Andreas LiljeqvistWhen using an overriding generator for the key ::obj-type
the generator seems to be ignored.#2018-01-1116:40Andreas LiljeqvistIf I do the same without multi-spec it works#2018-01-1116:44Andreas Liljeqvistupdated example with better comments https://pastebin.com/FSwMG32t#2018-01-1116:57Alex Miller (Clojure team)Isnāt the question really why the generator override doesnāt work in the first exercise?#2018-01-1117:03Andreas LiljeqvistSorry, running on low sleep - Yes#2018-01-1117:06Alex Miller (Clojure team)doesnāt seem like the generator overrides make their way to the per-method spec generators, not sure why#2018-01-1117:07Alex Miller (Clojure team)looking at the code, it certainly seems like things are done with that intent, not sure why thatās not working, feel free to file a jira#2018-01-1117:09Andreas Liljeqvistok, thanks. Will file a report tomorrow#2018-01-1117:09Alex Miller (Clojure team)the multispec generator drives from the known dispatch method values, not from the method itself, so that may also play into it#2018-01-1122:32dadairanyone know why a call to s/exercise
would throw FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:458)
?#2018-01-1122:39benalbrechti ran into the same issue a while ago. the way i understand the code, the multi-spec generator generates a value for each dispatch method (where the override map probably is respected) but then it just chooses one of these values using gen/one-of: https://github.com/clojure/spec.alpha/blob/a65fb3aceec67d1096105cab707e6ad7e5f063af/src/main/clojure/clojure/spec/alpha.clj#L939#2018-01-1122:42benalbrechtdadair: you need to add [org.clojure/test.check "0.9.0"]
to your dev dependencies#2018-01-1122:47seancorfield@dadair s/exercise
will also rely on test.check
for a spec that involves a function spec.#2018-01-1211:18Andreas Liljeqvist@alexmiller jira for multispec generator problem: https://dev.clojure.org/jira/browse/CLJ-2311#2018-01-1215:33stijnhow can you define or alter specs programmatically? I have my data model defined in an edn file and like to generate specs based on that#2018-01-1215:33stijn(let [k :my.ns/foo
values integer?]
(s/def k values))
=> user/k
(s/describe :my.ns/foo)
Exception Unable to resolve spec: :my.ns/foo clojure.spec.alpha/reg-resolve! (alpha.clj:69)
(s/describe 'user/k)
=> values
#2018-01-1215:33mpenetwith eval, or a macro#2018-01-1215:34stijnok#2018-01-1223:19kennyI'm confused why the first s/keys
call does not work and the second one does:
(def my-keys [::a])
=> #'user/my-keys
(s/keys :req my-keys)
java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol
(s/keys :req (conj my-keys ::b))
=>
#object[clojure.spec.alpha$map_spec_impl$reify__1931 0x2d5c83e "
#2018-01-1223:25kennyAnd it's not that it just compiles. The conj
is actually evaluated:
(s/valid? (s/keys :req (conj my-keys ::b))
{::a "a"
::b "b"})
=> true
(s/valid? (s/keys :req (conj my-keys ::b))
{::a "a"})
=> false
#2018-01-1223:58gklijsIt's because in the first one you suply a symbol, while in the second a list. Not really sure why though.#2018-01-1300:23kennyThe asymmetry does not make sense to me. Either both should throw or both be allowed.#2018-01-1302:11seancorfieldThe (s/keys :req (conj my-keys ::b))
is treated as (s/keys :req [conj my-keys ::b])
so it requires ::b
but the other two elements are symbols so s/keys
probably doesn't handle them as expected (it certainly does not evaluate them).#2018-01-1302:11seancorfieldHence, when you supply a map containing ::b
it validates and when it doesn't contain ::b
it is invalid.#2018-01-1302:13seancorfieldI expect that deep within the implementation of s/keys
, the (unevaluated) symbols conj
and my-keys
are processed in such a way that they fail to be treated as keys at all. @kenny#2018-01-1302:14seancorfieldAh, looking at the source of s/keys
it filters out anything that isn't a keyword: req-keys (filterv keyword? (flatten req))
req-un-specs (filterv keyword? (flatten req-un))
#2018-01-1302:16kennyThen why is this validated correctly?
(def my-keys [::a])
=> #'boot.user/my-keys
(s/def ::a int?)
=> :boot.user/a
(s/valid? (s/keys :req (conj my-keys ::b))
{::a "a"
::b "b"})
=> false
#2018-01-1302:16kenny(s/valid? (s/keys :req (conj my-keys ::b))
{::a 1
::b "b"})
=> true
#2018-01-1302:16kennyOh, duh.#2018-01-1302:16kennyI always forget the implicit checking of keys.#2018-01-1302:17seancorfieldIt looks like s/keys
tries to treat anything that isn't a keyword in the sequence supplied to :req
or :req-un
as some sort of predicate but I'm not quite sure I follow all the parts of the macro...#2018-01-1302:18kennyDoesn't Spec auto-check any keys in a map against any specs defined for those keys?#2018-01-1302:18kennyWhich would explain why
{::a "a"
::b "b"}
is invalid#2018-01-1302:19seancorfieldAh, I see what it's doing with those symbols: :pred-forms (quote
[(clojure.core/fn [%] (clojure.core/map? %))
(clojure.core/fn [%] clojure.core/conj)
(clojure.core/fn [%] my-keys)
(clojure.core/fn
[%]
(clojure.core/contains? % :user/b))]),
#2018-01-1302:19seancorfieldand when you evaluate clojure.core/conj
and my-keys
they evaluate to truthy values (not nil or false).#2018-01-1302:20seancorfield@kenny Did you read the bit where I said (conj my-keys ::b)
is not evaluated?#2018-01-1302:20kennyYes#2018-01-1302:21seancorfieldIt treats that as the key ::b
and two predicates (which are "true").#2018-01-1302:21seancorfieldSo your spec is equivalent to (s/keys :req [::b])
#2018-01-1302:21kennyRighttt#2018-01-1302:21kennyIs that from a macroexpand?#2018-01-1302:21seancorfieldYes, the above :pred-forms
snippet is#2018-01-1302:22kennyGotcha. As always, everything has a logical explanation š#2018-01-1302:22seancorfieldVery logical. Not always obvious š#2018-01-1302:22seancorfieldOK, gotta run...#2018-01-1302:23kennyThanks. Have a good night!#2018-01-1302:36seancorfield...back š#2018-01-1320:27martinklepsch(s/def ::name string?)
(s/def ::ns string?)
(s/def ::doc (s/nilable string?))
(s/def ::src string?)
(s/def ::type {:var :fn :macro})
(s/def ::line int?)
(s/def ::column int?)
(s/def ::def
(s/keys :req-un [::name ::ns ::doc ::src ::type
::line ::column]))
Trying to sample this spec but getting StackOverflowError
ā is there anything obvious Iām doing wrong?#2018-01-1320:37martinklepschNever mind, I had two specs named ::ns
š#2018-01-1321:50conanI wrote a blog post about writing specs for URLs: http://conan.is/blogging/a-spec-for-urls-in-clojure.html#2018-01-1321:59misha@martinklepsch also
(s/def ::type {:var :fn :macro}) ;;->
(s/def ::type #{:var :fn :macro})
#2018-01-1502:02gfredericksuser> (s/valid? (sorted-set 1 2 3) "you'd think this would return false")
;; throws
ClassCastException java.base/java.lang.Long cannot be cast to clojure.lang.Keyword clojure.lang.Keyword.compareTo (Keyword.java:114)
#2018-01-1502:50andy.fingerhutstack trace shows it is trying to call (s/regex? (sorted-set 1 2 3)), which raises that same exception.#2018-01-1503:02andy.fingerhutworkaround: (s/valid? (hash-set (sorted-set 1 2 3)) ...)#2018-01-1503:02gfredericksthis makes me think the answer will be "you can't use sorted sets as specs"#2018-01-1503:03andy.fingerhutI looked for a CLJ issue that I thought existed about making set lookup on sorted sets never return an exception because of exceptions thrown by the comparison function, but not sure if I am imagining it.#2018-01-1503:03gfrederickseven if it didn't fail at that point, a sorted set is not a general predicate š#2018-01-1503:04gfredericksmaking a change like that would be weird because what if you supply a buggy comparison function to sorted-set-by
, for instance? it would just swallow them#2018-01-1503:04gfredericksit could throw them on insert I guess, but would swallow on lookup#2018-01-1503:11andy.fingerhutFound several related tickets on sorted sets, but not the one I was imagining. My imagination can be vivid.#2018-01-1509:36bronsa@andy.fingerhut yeah I know which ticket youāre talking about#2018-01-1509:38bronsait was https://dev.clojure.org/jira/browse/CLJ-1242 which got refocused just on equality, but originally was about equality and lookup#2018-01-1514:04conanIs there a way to get the keys back from a map spec? i.e. the function get-keys
:
(s/def ::banana (s/keys :req [::length ::type]
:opt [::colour]))
(get-keys ::banana)
=> {:req [::length ::type]
:opt [::colour]}
I can do this by using (s/form ::banana)
but that just gives me a sequence so seems very brittle#2018-01-1515:14andre.stylianosI don't think there's a better way for now, there's an issue in Jira that seems likely to be what you want but it's still open.
https://dev.clojure.org/jira/browse/CLJ-2112#2018-01-1515:31ikitommi@U053032QC forms have been stable, so (->> (s/form ::banana) rest (apply hash-map))
while waiting the specs for specs.#2018-01-1515:37conan@U485ZRA58 yes that does look like what I want.
@U055NJ5CC yep, it certainly gets me exactly what i want, it just seems a bit wonky#2018-01-1517:00Alex Miller (Clojure team)there is likely to be more support for this in the next major update for spec#2018-01-1517:14souenzzo(let [spec (s/keys :req [:foo/bar])
_ (s/def ::req (s/coll-of keyword?))
keys-spec (s/cat :op symbol?
:args (s/keys* :opt-un [::req]))
{{:keys [req]} :args} (s/conform keys-spec (s/form spec))]
req)
@U064X3EF3, you always say to "not use conform to coerce values". In this case, we are using conform to "parse" a struct. It's a valid use?
read the topic#2018-01-1517:19Alex Miller (Clojure team)I donāt think Iāve said that. conform is designed to conform values so Iām not even sure what you mean.#2018-01-1517:21Alex Miller (Clojure team)this seems like a pretty ugly workaround (the embedded s/def seems like an obvious place where you could have issues too)#2018-01-1517:24souenzzo(it's just to fit on let
to easy copy'n'paste on repl)
I think that you say do not use conform to conform values
related to CLJ-2116
#2018-01-1517:25Alex Miller (Clojure team)do you mean ādo not use conformers to coerce valuesā which is a very different statement?#2018-01-1517:25Alex Miller (Clojure team)that seems wholly unrelated to your example if so#2018-01-1517:27souenzzoYEP. Sorry. I got lost with the words š#2018-01-1517:30Alex Miller (Clojure team)I donāt think what youāre doing here is related to coercion#2018-01-1517:30Alex Miller (Clojure team)which is independent from whether itās good :)#2018-01-1517:30Alex Miller (Clojure team)my real question is why you want to ask this question in the first place and whether you can arrange things to not ask it#2018-01-1520:32mishahttps://clojurians.slack.com/archives/C1B1BB2Q3/p1516035630000041?thread_ts=1516025043.000637&cid=C1B1BB2Q3
half life 3 confirmed kappa#2018-01-1523:04Steve PetersonHi Spec-cers! Which Clojure libraries are using Spec as a protection layer in their APIs, i.e. to validate args? Are they instrumenting API fns āby defaultā so input args are checked on each call?#2018-01-1600:06seancorfield@stevep I can tell you what we're doing in production with clojure.spec
at World Singles as far as REST APIs are concerned: we have a spec for each handler (i.e., each API endpoint) and the general pattern is roughly: (let [params (s/conform ::this-api-spec (:params req))]
(if (s/invalid? params)
(error-response (s/explain-data ::this-api-spec (:params req)))
(normal-process params)))
#2018-01-1620:44arnaud_bos(how-)do you prevent un-spec-ed keys to propagate further down the normal-process
?#2018-01-1622:15seancorfield@U1DLD0WNR In keeping with Clojure's open-for-extension approach, normal-process
will just ignore keys it doesn't care about. Similarly, if I only spec certain keys, others don't get checked -- so the constraints are open.#2018-01-1600:08seancorfieldWe have that wrapped up in a macro to reduce boilerplate but that's the essence of it. The error-response
function decodes the spec failure based on a set of heuristics and "expected" failure points so we can return documented error codes and localized messages to the client.#2018-01-1600:09seancorfieldWe only instrument
functions in our dev/test cycle, not in release code. We also have specific "tests" that run clojure.spec.test/check
on certain functions.#2018-01-1600:10seancorfieldThe argument against just relying on instrument
to check arguments is that you want better error messages than spec failures for your clients!#2018-01-1610:31vikeriCan I modify to a spec after Iāve defined it? My use case is to add the :fn
argument for an fdef
but only when I run my tests.#2018-01-1611:22conan@vikeri why can't you just add it anyway? instrument
doesn't run the :fn
and :ret
bits by default anyway#2018-01-1611:22conani use orchestra.spec.test/instrument
to do this#2018-01-1611:24vikeri@conan I also use orchestra and it runs the :fn
by default so thatās why I have that issue#2018-01-1611:25conanah, can you swap out which you use in different circumstances? so you can switch on/off the :fn
execution#2018-01-1611:25conani think you can just call clojure.spec/fdef
again fine, and it'll update the spec registry with whatever you give it#2018-01-1612:55Alex Miller (Clojure team)You can specify an alternate spec to use with instrument #2018-01-1719:30borkdudeI cannot pass a keyword as function to s/conformer
, since it will try to resolve it as a spec. e.g.: (s/conformer (fn [parsed] (:foo parsed)))
is needed right?#2018-01-1719:31borkdudeOther question: I vaguely remember something about not using spec as a string/text parser from this channel, but what exactly was the argument against it?#2018-01-1719:48seancorfieldThe main objection is that is complects parsing and validation -- and "forces" the parsing effect on all clients of the spec.#2018-01-1719:49borkdudeforces the parsing effect?#2018-01-1719:46borkdudeIf anyone wants to review my clojure.spec parser (for performance comparison with other parsers like Instaparse, Kern), please do:
https://github.com/borkdude/aoc2017/blob/master/src/day16.clj#L234#2018-01-1719:51seancorfield(what we tend to do is have a spec that accepts both the target type as well as a string that could represent the target type ... if the parse/conversion fails, the result is ::s/invalid
; if the parse/conversion succeeds or if the target type was provided, the validation is performed)#2018-01-1720:26donaldballDo you have an example you can share of this style?#2018-01-1721:13seancorfieldI've shared it before here but it'll take me a while to de-tangle the generic code from the proprietary code so don't hold your breath for me to get around to it (too busy this afternoon and I'll probably forget after that).#2018-01-1721:16seancorfieldI probably ought to just open source it at some point...#2018-01-1721:20donaldballš#2018-01-1720:59Alex Miller (Clojure team)my main objection to it is that the regex op capabilities of spec are designed to support the structure of data and only incidentally can be used as a way to describe the structure of strings. Inevitably, I suspect you will find either the capabilities or performance of āspec as string parserā to be lacking and will be disappointed in our lack of interest in addressing those. There are very good tools already in existence for creating parsers on strings and I those are the right tool for the job.#2018-01-1721:15seancorfieldAnd to be clear, what we do at World Singles is not any sort of spec-as-parser -- we just have a few, very specific specs that accept a string or a target type and, for the string, do a simple conversion. Those target types are Long, Double, Boolean, and Date. We don't attempt to do anything else via spec -- this is just for simple form or query string data.#2018-01-1721:18borkdudeI was surprised specās performance was even better than Instaparse (see link above). Handwritten parser is fastest, Kern in between.#2018-01-1721:19borkdudeBut yeah, I understand there is no interesting in optimizing spec for strings, although it does quite well#2018-01-1721:21donaldballI was also fairly delighted at how both expressive and fast spec was when validating strings as seqs of chars, at least for my silly use cases.#2018-01-1721:26aengelbergto be fair, Instaparse has enough processing overhead that parsing one or two characters at a time in a 47 KB string isn't the best (or most common) use of it š#2018-01-1721:28borkdude@aengelberg yeah, Iāve discussed this example with you before in #instaparse I believe, itās part of Advent of Code. Nothing conclusive, just very contextual.#2018-01-1722:41Alex Miller (Clojure team)I think the perf differences are going to be highly dependent on the complexity of the grammar and the size of the data so itās hard to make predictions from a single example.#2018-01-1722:53rickmoynihanAlso instaparse is a far more general purpose parsing solution, supporting both left and right recursion; Iād be surprised if that didnāt incur an overhead.#2018-01-1723:46dpsuttoni've written some specs and i put some calls to s/valid?
in comment macros in the namespaces where they are. is there a good way to take this out of a comment macro and have them run at some convenient but non-production time?#2018-01-1723:47dpsuttonclojurescript is the target if there's a compiler option to prevent spec checks on advanced compilation#2018-01-1815:39kingmobIn Clojurescript, what are people doing if they want to remove spec from production builds? Is there a lib for it? Macros wrapping fdef in the same file? Or placing all spec stuff in a dev-only namespace?#2018-01-1815:48borkdude@kingmob just curious, but why would you want to remove them?#2018-01-1815:48kingmobDownload size#2018-01-1815:49kingmobIf Iām only validating/testing with spec, I donāt want it to take up sie in the resulting Js artifact#2018-01-1815:49borkduderight. putting them in a dev only ns is what I would do then#2018-01-1815:54kingmobThx @borkdude. Thing is, I prefer fdefs
be defined by their function, so I was hoping someone had already handled this. We have some dev-time-only logging macros I could repurposeā¦ though I suspect dev-only namespaces would still be tricky, assuming itās even possible#2018-01-1815:56borkdude@kingmob this project has an example how dev only namespaces are possible (both lein and boot): https://github.com/borkdude/lein2boot#2018-01-1815:57borkdude@kingmob you could also try in #clojurescript#2018-01-1816:00dpsuttonyeah it would be nice if you could elide all spec related stuff under advanced compilation#2018-01-1816:01kingmob@borkdude Ahh, it looks like you factored out everything common into its own file and then use different :source-paths
. We do that, too, but if, e.g., I added fdef
s in animals.crud
, theyād still end up in prod builds#2018-01-1816:01kingmobWe could add it to the dev source path, but then itās separate from the definitionā¦which is fine, itās not really that big a deal#2018-01-1816:01borkdudeyeah, not getting around that when putting in a different ns#2018-01-1816:02borkdudehave you tried what difference in size you get for the production js with and without? it may not even be worth it#2018-01-1816:02kingmob@dpsutton Yeah, it would be nice. Does spec use asserts
under the hood so that the :elide-asserts
option would remove it?#2018-01-1816:04kingmob@borkdude Yeah, I have actually, itās substantial š Lemme recompile and check with source-map-explorer again#2018-01-1816:04borkdudeI believe you š#2018-01-1816:09kingmob@borkdude OK, bare-bones spec is adding 39.84 kb to the output .js. Itās not major, but this is just for spec itself, and a sprinkling of initial spec defs. Itās not extensive at all, and since none of it is used at run-time, we might as well ditch it.#2018-01-1816:11borkdudeI see there is an issue about it: https://dev.clojure.org/jira/browse/CLJS-1701#2018-01-1818:05shaun-mahoodI've been thinking about building some company/app wide specs. Say I spec something as :com.myco.app.service/id
- is there any way that I can apply that spec to some data where everything is named as :service/id
?#2018-01-1818:28seancorfield(s/def :service/id :com.myco.app.service/id)
?#2018-01-1818:28seancorfield(and then use that local "alias")#2018-01-1818:34shaun-mahoodOh I didn't think of that - thanks!#2018-01-1820:38schmeeis there something like this in spec?
(defn conform! [spec thing]
(if (s/valid? spec thing)
thing
(throw (ex-info (s/explain-str spec thing)
(s/explain-data spec thing)))))
#2018-01-1820:40seancorfieldSounds like s/assert
to me @schmee?#2018-01-1820:41seancorfieldAlso, I'd note that your function does not conform the value so it's a very misleading name...#2018-01-1820:41schmeesorry ābout that, copy-paste errorā¦#2018-01-1820:42schmeebut it sure does sound like s/assert
, thanks š#2018-01-1820:42seancorfieldPerhaps (defn conform! [spec thing]
(let [v (s/conform spec thing)]
(if (s/invalid? v)
(throw (ex-info ...))
v)))
#2018-01-1820:43seancorfield(whereas s/assert
returns the original thing
, not the conformed value)#2018-01-1820:44schmeeyeah, I copied my conform thing from another project where I needed the conformed value, now I just want to validate so s/assert
is perfect š#2018-01-1820:46seancorfieldJust bear in mind it can be turned on/off by command line options and by code (including 3rd party libraries!).#2018-01-1820:49schmeehmmā¦ using assert for config validation sounds kinda sketchy then#2018-01-1820:50schmeeahh, putting it in :pre
is what I want, forgot that it existed š#2018-01-1820:52schmeeno, that just gives an AssertionError, it doesnāt show you whats wrongā¦.#2018-01-1820:53seancorfield(and asserts can be turned on/off by calling code via *assert*
)#2018-01-1820:53uwoso, when s/keys fails on a missing required key, thereās no path information in explain-data
. Weāre attempting to dispatch on the result of explain-data, but the information we need about the missing key is in a :pred function. So is the only way to parse this data to parse the s expression used internally to describe required keys?#2018-01-1820:55seancorfield@uwo Yeah, that's ugly. We wrote a heuristic-based explain-data walker to figure that stuff out. It recognizes and picks apart the :pred
value...#2018-01-1820:56uwoyikes. off the top of your head do you know if thereās jira ticket along these lines.#2018-01-1820:57uwothanks, btw!#2018-01-1821:04seancorfieldOff the top of my head, nope, sorry.#2018-01-1821:05seancorfieldBut we needed to pick apart spec errors in general and map them back to our domain for reporting to end users... it just took us a while to flesh out the walker (it's... non-trivial).#2018-01-1821:52uwosame for us#2018-01-1821:35schmeehow do you go about spec-ing anonymous functions with side-effects? since conform
et al actually calls the function itās validating, I guess the answer is ādonāt do it?ā#2018-01-1821:40seancorfieldNot sure what situation you're talking about @schmee?#2018-01-1821:42schmeeI have a piece of code that takes an array of functions, and those functions should return a particular type of record#2018-01-1821:42schmeeI call s/valid?
on the input to this code, and if I use fspec
to check the functions in the array, it will actually call all those fns to ensure that they return the correct thing#2018-01-1821:43schmeebut those fns could grab values from a database, launch nuclear missiles or whatever, so Iām thinking I should just spec it as (s/+ fn?)
instead#2018-01-1821:44schmeeinstead of (s/+ (s/fspec :args (s/cat) :ret (partial instance? MyRecord)))
#2018-01-1821:44seancorfieldYes, best not to use fspec
for side-effecting functions-as-arguments because instrumentation uses generative testing on them.#2018-01-1821:44Alex Miller (Clojure team)Use ifn?, not fn?#2018-01-1821:45seancorfieldAn array of functions sounds more like (s/coll-of ifn?)
#2018-01-1821:46schmeenoted on ifn?
, but why s/coll-of
instead of s/+
?#2018-01-1821:48Alex Miller (Clojure team)+ is a regex op used for describing the internal structure of sequential collections (like an arg list). coll-of is for sequential collections. Either may be applicable here, depends on context.#2018-01-1821:48Alex Miller (Clojure team)In particular, what if anything is including this spec#2018-01-1821:49Alex Miller (Clojure team)If, for example, youāre including it as one arg in an args spec, using + would be bad as it would combine in the same sequential parent context#2018-01-1821:50schmeeI see, Iām using it as part of a s/keys
spec so that shouldnāt be a problem then š#2018-01-1821:51Alex Miller (Clojure team)Either is prob ok then but I would prob use coll-of#2018-01-1821:51Alex Miller (Clojure team)With a :min-count constraint#2018-01-1821:52schmeeIāll go with that, thanks for the help š#2018-01-1821:53schmeeIām a bit torn about the fspec
thing though, I think it would be much more pleasant to see (s/fspec :args (s/cat) :ret (partial instance? MyRecord)
in a doc-string instead of ifn?
, but I canāt due to generative testing#2018-01-1821:53schmeeI can just write it out in the docstring anyway of course š#2018-01-1821:54schmeeIām still new to spec so Iām kinda feeling it out, itās very easy to overdo it and use it as a type system#2018-01-1822:00seancorfieldNote that you can still run clojure.spec.test.alpha/check
on your function with the full fspec
-- independent of what the function's actual fdef
says. (right @alexmiller)#2018-01-1822:00seancorfieldOr you could instrument
it with a different fspec
too I believe?#2018-01-1822:01Alex Miller (Clojure team)yes#2018-01-1822:01seancorfieldI'd probably fdef
it with a spec that is safe to instrument
, and then check
it with a more detailed fspec
...#2018-01-1822:02seancorfield(because it might, potentially, get instrumented by code far, far away in your code base that might "forget" to override the spec for it)#2018-01-1823:55arohnerIs there a library for an s/keys
-alike but keys are strings rather than keywords, yet?#2018-01-1902:23gfredericks(s/keys :req [::foo])
or (s/keys :req ["foo"])
?#2018-01-1902:24gfredericksif the former, is it not syntactically possible to express all string keys? if the latter, how are the strings tied to the spec registry?#2018-01-1920:15pabloreDo specs replace deftype
and reify
? If not, how do I use them together?#2018-01-1920:43schmeeno. deftype
is used for performance and polymorphism, reify
is mostly used for Java interop, and spec is about specifying your data. so itās three different use-cases really š#2018-01-2209:10witekI want to spec a command vector, given (s/def ::command-name keyword?)
. Now the command-vector needs to be a vector where the first element must be the ::command-name
. All other elements are optional arguments. Any suggestions? Thank you.#2018-01-2213:32taylor(s/def ::command-name keyword?)
(s/def ::command (s/cat :name ::command-name
:args (s/* any?)))
(s/conform ::command [:foo "bar"])
#2018-01-2213:33taylorif you really need the input to be a vector then (s/def ::command
(s/and
vector?
(s/cat :name ::command-name
:args (s/* any?))))
#2018-01-2209:19jakobI have a datastructure that looks something like this:
(def my-invoice-address
{
:use-delivery-address false
:street "asdf"
:zip "123"})
If :use-delivery-address
is true, :street
and :zip
should not be there. If :use-delivery-address
is false, :street
and :zip
should be there
I find it really difficult to write a spec for this. Anyone who have any ideas?#2018-01-2209:25stathissideris@karl.jakob.lind I guess :use-delivery-address
is :req
, the others are :opt
and then you wrap your keys
with an s/and
to enforce the rest of the logic that you described#2018-01-2209:31jakobWell, the others are required if :use-delivery-address
is true. So it's not that simple#2018-01-2209:36stathissiderisyeah, but Iām proposing that this is enforced by the extra predicate that you will pass to s/and
#2018-01-2209:39jakobhm.. something like this?
(s/def ::invoice-address
(s/keys
:req-un [::use-delivery-address]
:opt-un [::street ::zip]))
where do the s/and
belong ?#2018-01-2209:44stathissiderissomething like this (Iām sure thereās a more elegant way to do it):#2018-01-2209:45stathissideris(s/def ::invoice-address
(s/and
(s/keys
:req-un [::use-delivery-address]
:opt-un [::street ::zip])
(fn [{:keys [use-deliver-address street zip]}]
(or (and (not use-deliver-address)
(not street)
(not zip))
(and use-deliver-address
street
zip)))))
#2018-01-2209:46jakobThat's at least more elegant than my attempts š Thanks! will use it!#2018-01-2209:51Jakub HolĆ½ (HolyJak)Perhaps use https://clojure.org/guides/spec#_multi_spec ?
(defmulti address-type :address/use-deliver-address)
(defmethod address-type true [_]
(s/keys :req []))
(defmethod address-type false [_]
(s/keys :req [ :address/street ...]))
@karl.jakob.lind#2018-01-2209:58jakobgood suggestion @karl.jakob.lind. will read about that!#2018-01-2216:33carkhlet's say i want to write a spec for a string that should be splitted in two, imagine i have a split function that can be called with that string and returns a pair of strings... i have a ::check-whole-thing spec and also a ::check-part spec ...how would i write that ::check-whole-thing part ?#2018-01-2216:35carkhthere are numerous examples, but these all work on clojure data structures#2018-01-2216:36carkhi think what i'm asking here is how to transform the data being specced before speccing it further#2018-01-2217:04Alex Miller (Clojure team)have you read the spec guide? https://clojure.org/guides/spec If not, maybe take a spin through that first and then refine the question.#2018-01-2217:06carkhi did ... actually found a related question to which you responded on the forum ... the conformer function looks like it fits the use case#2018-01-2217:06Alex Miller (Clojure team)that does not seem like a use case where conformers should be used to me#2018-01-2217:07Alex Miller (Clojure team)Iām confused by several aspects of your question so not sure how best to answer it#2018-01-2217:07carkhhttps://groups.google.com/forum/#!topic/clojure/9Bg_9P5o3h8#2018-01-2217:08carkhi'm conforming a string, i want to have it transformed into a vector of strings before further conforming#2018-01-2217:08carkhcould be any kind of transformation really#2018-01-2217:08Alex Miller (Clojure team)well, I would recommend not doing that#2018-01-2217:08carkhah !#2018-01-2217:08Alex Miller (Clojure team)do that in code explicitly#2018-01-2217:08Alex Miller (Clojure team)donāt use spec for it#2018-01-2217:08carkhbut semantically i'm really validating the string there#2018-01-2217:09carkhwhich is what spec is for isn't it ?#2018-01-2217:09Alex Miller (Clojure team)I think you are better off if you spec the inputs and outputs of the transformation#2018-01-2217:09Alex Miller (Clojure team)and explicitly invoke the transformation#2018-01-2217:10Alex Miller (Clojure team)and donāt use spec to actually do the transformation part#2018-01-2217:11carkhthe thing is that i'm not really interested in that transformation at all, i'm only validating it to pass the whole thing merilly along#2018-01-2217:14Alex Miller (Clojure team)if you have to do that transformation in able to pass it along, then it seems like you are interested in it after all#2018-01-2217:16carkhlet's imagine a use case for one of those html dsls ... they have this way of specifying class attributes like so [:div.app-container ...] ...that's not what i'm doing at all but that's a good example... are you saying that spec wouldn't fit to do the "destructuring" of that keyword ?#2018-01-2217:19carkhor maybe to destructure an udp packet or you know go inside a data structure and destructure it for me#2018-01-2217:20carkhthat last one was a bad example =)#2018-01-2217:23carkhwhat's the issue here ? the fact that one could not rebuild the data structure from the conformed result ?#2018-01-2217:23ghadito rephrase slightly, don't conflate spec/validation with transformation in one phase#2018-01-2217:23ghadiyou'll be perpetually unhappy#2018-01-2217:24ghadido your validation, do transformation, then do validation again if you want#2018-01-2217:24ghadithere are other gotcha with conform to look into#2018-01-2217:24ghadilike it forces conformance on all consumers of the spec#2018-01-2217:25ghadiwhich you don't want#2018-01-2217:25carkhok i need to think a little bit on this#2018-01-2217:25carkhthansk for your time the both of you#2018-01-2217:26ghadino problem. Lots of other validation libraries provide "mutative validation".... conflating transformation into its API#2018-01-2217:26ghadireally impedes reuse#2018-01-2217:27ghadiTypical example is 'validate this string into a UUID'#2018-01-2217:27carkhthat's a good example#2018-01-2217:28carkhbut now i first need to conform a big data structure, then walk it and validate parts of it#2018-01-2217:28carkhthat's a lot more code ><#2018-01-2217:28carkhanyways i'll think on it, thanks again !#2018-01-2220:58golinI'm trying to spec polymorphic maps where a particular key is used for dispatch. Is there a way to spec the dispatch key to specify the dispatch value?
(s/def ::foo
(s/keys :req [:data/type :foo/stuff]))
(s/def ::bar
(s/keys :req [:data/type :bar/stuff]))
e.g. ::foo
's :data/type
should have a value of :foo
, and ::bar
's :data/type
have a value of :bar
#2018-01-2221:20schmee@scallions I think you are looking for multi-spec: https://clojure.org/guides/spec#_multi_spec#2018-01-2221:28golinThanks, I'll look into it. That retag
argument looks interesting.#2018-01-2312:01stathissiderishereās a tricky one: I have a data structure and I use conform with a regex spec to extract information from it. If the data structure is invalid in two places, I only get the first problem reported, because the regex stops. Is there any way to get the second problem too? (I suspect itās a no)#2018-01-2313:04Alex Miller (Clojure team)No#2018-01-2313:05Alex Miller (Clojure team)To get a deeper level for something like this you would really need to write your own custom spec#2018-01-2313:43stathissideris@alexmiller ok, makes sense, thanks. Iāll see if I can circumvent the problem somehow#2018-01-2315:54vikeriIs there any way of making sure that the specs I write point to other specs? I can write (s/def ::my-spec (s/keys [:not-a-spec/key])
And :not-a-spec/key
doesnāt have to be a spec. Can I somehow enforce that it really is a spec that itās pointing to?#2018-01-2317:04bronsahttps://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2018-01-2411:44vikeri@U060FKQPN Thanks!#2018-01-2316:26vikeriAlso, if I could somehow avoid defining specs twice that would be fantastic#2018-01-2316:59stathissideriswhy do you have to define them twice?#2018-01-2410:10vikeri@stathissideris I donāt have to define them twice, but I want to avoid it happening#2018-01-2411:26misha@vikeri "them" who?#2018-01-2411:26mishado you have a specific example of what you are trying to avoid?#2018-01-2411:27vikeriI want this to throw an error/warning
(s/def ::hello int?)
(s/def ::hello string?)
#2018-01-2411:28mishathen you'll get en exception on the next ns reload in repl, would not want that, would you? )#2018-01-2411:29mishaotherwise, write a macro, which throws/warns if spec you are trying to define is already defined in specs registry#2018-01-2411:34mishadefmacro sdefonce [sym form]
`(if (contains? (s/registry) ~sym)
(throw (ex-info "NO!" {:sym ~sym :existing (s/form (get (s/registry) ~sym))}))
(s/def ~sym ~form)))
(sdefonce ::foo string?)
=> :user.specs/foo
(sdefonce ::foo string?)
clojure.lang.ExceptionInfo: NO!
data: {:existing clojure.core/string?, :sym :user.specs/foo}
#2018-01-2411:34misha@vikeri#2018-01-2411:36mishayou might check if form is the same as in registry, and it will not blow up on ns reload for unchanged specs, but still will on the ones you are iterating over (developing)#2018-01-2411:43vikeri@misha I only want to deny overrides when I run the tests to make sure I havenāt duplicated it in the code. (Loading some specs from an external lib). Yeah I was considering writing my own macro but then Iād like to override the macro so that normally it uses the normal s/def but when I run my tests it throws the error. Iāll probably go with my own macro actually.#2018-01-2411:45mishasounds like too custom use case for an out of the box solution#2018-01-2415:53pabloreIs there a way to use s/assert
and s/explain
when the assert fails?#2018-01-2416:23pabloreis it still good practice to use {:pre [(s/assert ā¦)]} on functions?#2018-01-2416:24pablorefor some reason assert is failing but explain is giving me success#2018-01-2416:27schmee@pablore just keep in mind that asserts can be toggled off#2018-01-2416:27pabloreYes but thats not the idea, I need it to fail when I give it a bad input#2018-01-2416:28schmeehereās a thing Iāve found useful many times:
(defn validate [spec thing]
(if (s/valid? spec thing)
thing
(throw (ex-info (s/explain-str spec thing)
(s/explain-data spec thing)))))
#2018-01-2416:28schmeeI use that instead of assert since it canāt be toggled off#2018-01-2416:28ghadihttps://github.com/Datomic/mbrainz-importer/blob/master/src/cognitect/xform/spec.clj#L13-L23#2018-01-2416:29ghadithere's another (similar) idea there#2018-01-2421:05mrchanceHi, I want to generate test data based on a swagger spec, but subject to some constraints. Do you think spec is a good tool for that? If yes, how would one best go about that?
If not, what would be a better way to do it?#2018-01-2422:46kingmobQuick sanity check: in Clojurescript, is cljs.spec.test.alpha working for anyone else? I keep getting compilation errors about clojure.test.check.#2018-01-2422:47kingmob(Since itās still alpha, I wasnāt sure)#2018-01-2423:04dadairIn some cases you need to explicitly import [org.clojure/test.check <ver>]
, at least on the clj side#2018-01-2423:10kingmob@dadair Thx. Is that based on https://dev.clojure.org/jira/browse/CLJS-1792#2018-01-2423:10kingmobIāll try adding it#2018-01-2423:27kingmob@dadair That did it! Thanks#2018-01-2505:43ikitommi@mrchance check out https://github.com/immoh/swagger-spec#2018-01-2508:17mrchance@ikitommi Thanks, but I don't want to generate swagger, but the data consumed by the interfaces specified in a swagger file. It doesn't look like that is possible with swagger-spec?#2018-01-2509:38ikitommioh, the endpoint data. With vanilla swagger spec, there needs to be a JSON-Schema -> Spec converter, I donāt know if such exists yet. But if you describe your endpoints with spec, you get the data generation for free.#2018-01-2510:09mrchanceYes, already doing that, unfortunately some of the other services in my project aren't written in Clojure. If one were to write such a converter, how would one go about it? I invested an afternoon because I didn't really find anything too, but it seemed really hard and messy to generate specs at runtime, due to its reliance on global mutable state š #2018-01-2510:10mrchanceMaybe <gasp> generate code </gasp> ?#2018-01-2510:12mrchance@ikitommi I saw that you had schema like maps in your spec tools, which could be handled like normal data. Would that be an option for this case?#2018-01-2510:19ikitommiSome more āfunctional specsā might be coming to core, but while waiting, yes, there are some rogue versions at https://github.com/metosin/spec-tools/blob/master/src/spec_tools/data_spec.cljc#L8-L10 and the data-specs on top of those.#2018-01-2510:23ikitommimost/all of the core specs have functional versions already in the core, but currently not documented and the apis are bit hairy. Maybe those will be polished and published as part of the public api? The forms need to be created manually but for most core predicates, itās easy to resolve those (and spec-tools does that already)#2018-01-2512:49mrchanceHm, I see. Thanks, I'll give it a go, and hope for the release of the functional api in core.#2018-01-2515:43mpenetis there something like s/cat but that doesn't do any labeling? need to match a subsequence within a sequence without too much ceremony#2018-01-2515:44mpenetit's a seq of chars (just playing), so nothing generic, I need equality matching (ex match (b a r) in (f o o b a r b a z))#2018-01-2515:45mpenetI know I can mix s/cat and sets of chars or predicates, but that's quite ugly#2018-01-2515:46mpenetI am also forced to use regex context (my question is part of a wider parsing thing)#2018-01-2515:46Alex Miller (Clojure team)if you donāt want labeling, just use s/valid?#2018-01-2515:47mpenetI dont need labeling per char, I do for the whole subseq (up one level)#2018-01-2515:47mpenetmaybe I didn't understand what you mean#2018-01-2515:49mpenetall the surounding value have a meaning that I need to retain also, I can't just check "does this seq contains this sub-seq"#2018-01-2516:01Alex Miller (Clojure team)you can use s/& to apply a custom predicate to a regex spec#2018-01-2516:02Alex Miller (Clojure team)which could be helpful here#2018-01-2516:04mpenetright, forgot about ߢ-01-2516:08mpenetthat's brilliant, I was messing with conformers but I think & might just do all this for me#2018-01-2521:26schmeeis it just me or do tools.namespace and instrument not get along?#2018-01-2522:05schmeeor does it not check things that are def
ed?#2018-01-2522:05schmeethat is, if I reload a namespace that has (def foo (some-instrumented-fn invalid-value))
, does reloading the ns throw an exception?#2018-01-2522:08Alex Miller (Clojure team)Reloading is going to remove your instrumented vars#2018-01-2522:11schmeeok, so those checks should be done through tests instead?#2018-01-2522:35Alex Miller (Clojure team)what checks?#2018-01-2522:39schmeelet me change the question: whatās the best way to check specs with clojure.test?#2018-01-2522:40Alex Miller (Clojure team)that seems pretty far from the original questions :)#2018-01-2522:40schmee(is (s/valid? ::my-spec (some-fn))
works, but it doesnāt give helpful output#2018-01-2522:40schmeeIām not very familiar with either clojure.test or spec, so Iām fumbling around a bit š#2018-01-2522:40Alex Miller (Clojure team)there used to be a pin here to an example, but maybe it has aged out#2018-01-2522:41Alex Miller (Clojure team)by ācheck specsā, Iām assuming you mean to run clojure.spec.test.alpha/check on one or more specāed functions#2018-01-2522:42schmeeactually, I want to check with s/valid?
, but use s/explain
instead of the usual error message provided by is
#2018-01-2522:43Alex Miller (Clojure team)so s/valid? is not going to be as useful in checking specāed functions as is calling check
#2018-01-2522:44Alex Miller (Clojure team)unless ::my-spec here refers to a data spec#2018-01-2522:44Alex Miller (Clojure team)I canāt really tell#2018-01-2522:45schmee::my-spec in this case is a data spec, yes#2018-01-2522:45Alex Miller (Clojure team)ok#2018-01-2522:45schmeesorry if my questions are a bit confused, itās hard to ask good questions when youāre not familiar with something#2018-01-2522:45Alex Miller (Clojure team)yep, no worries#2018-01-2522:46Alex Miller (Clojure team)I am similarly trying to re-interpret your questions :)#2018-01-2522:46Alex Miller (Clojure team)so you want to check if some data matches a spec and if not, return a message that is s/explain#2018-01-2522:46Alex Miller (Clojure team)one way would be to:#2018-01-2522:47Alex Miller (Clojure team)(is (nil? (s/explain ::spec data)))
#2018-01-2522:48Alex Miller (Clojure team)although I guess you want the explain result as a value not as a print#2018-01-2522:49Alex Miller (Clojure team)(is (not= "Success!\n" (s/explain ::spec data)))
would do that I think#2018-01-2522:49Alex Miller (Clojure team)thatās quite ugly of course :)#2018-01-2522:49schmeeexplain-data did the trick!#2018-01-2522:51Alex Miller (Clojure team)yeah, can do same idea with explain-data#2018-01-2522:51Alex Miller (Clojure team)you can supply a final custom error message to is
too#2018-01-2522:52Alex Miller (Clojure team)so something like: (is (valid? ::spec data) (s/explain-str ::spec data))
?#2018-01-2522:54schmeeI went with this: (defmacro spec-is [spec value]
`(is (nil? (s/explain-data ~spec ~value))))
#2018-01-2601:26bbrinckA shameless plug, but you can also add a message to is
. With expound you could do something like
(defmacro spec-is2 [spec value]
`(is (s/valid? ~spec ~value)
(expound/expound-str ~spec ~value)))
which will give you output like:
FAIL in (test2) (alpha_test.cljc:2152)
-- Spec failed --------------------
1
should satisfy
#{:c :b :a}
-------------------------
Detected 1 error
expected: (clojure.spec.alpha/valid? #{:c :b :a} 1)
actual: (not (clojure.spec.alpha/valid? #{:c :b :a} 1))
#2018-01-2608:37schmeehaha, I donāt mind shameless plugs when they are great! š#2018-01-2608:37schmeethatās way better, thanks for the tip š#2018-01-2522:54schmeethanks a ton for your help! š#2018-01-2522:54Alex Miller (Clojure team)np!#2018-01-2614:51trisshi all, how do I look up a functions spec at run-time? s/get-spec
doesnāt seem to to work?#2018-01-2614:53Alex Miller (Clojure team)it should if you pass it a fully-qualified symbol#2018-01-2614:53trissah thankyou. just spotted it said that in the docsā¦ an I get the fully qualified symbol for a function from that function?#2018-01-2614:54Alex Miller (Clojure team)you donāt need to get it from anywhere?#2018-01-2614:55Alex Miller (Clojure team)user=> (defn foo [] nil)
#'user/foo
user=> (s/fdef foo :args (s/cat) :ret nil?)
user/foo
user=> (s/get-spec `foo)
#object[clojure.spec.alpha$fspec_impl$reify__2451 0x8c11eee "
#2018-01-2614:56Alex Miller (Clojure team)I used back-tick there - syntax quote will resolve symbols so thatās an easy way, but I also could have done (s/get-spec 'user/foo)
#2018-01-2614:56Alex Miller (Clojure team)note that what you get back is a function spec instance, but that spec supports keyword lookup of :args
, :ret
, and :fn
#2018-01-2614:57Alex Miller (Clojure team)or you can use s/form on it directly
user=> (s/form (s/get-spec 'user/foo))
(clojure.spec.alpha/fspec :args (clojure.spec.alpha/cat) :ret clojure.core/nil? :fn nil)
#2018-01-2614:57trissah I see now. wonderful thank you Alex!#2018-01-2615:52don.dwoskeRich mentioned in a podcast (I think) that there was some sugar forthcoming which allowed a map to specify/lift the namespace for all the keys in the map (if they all had the same namespace) to the map level such that each individual key in the map could be printed / declared as un
namespaced for cleanliness? Is there news on this, or a preview of what it might look like? I'm working on something similar and an example would be great to see. This was mentioned in the context of using spec, so that's why I'm putting it here vs. another channel.#2018-01-2615:55ghadiIt's syntax sugar -- already in 1.9#2018-01-2615:55bronsa#:foo{:bar 1} = {:foo/bar 1}
#2018-01-2615:55ghadiuser=> *print-namespace-maps*
true
user=> {:foo/bar 42}
#:foo{:bar 42}
#2018-01-2615:55ghadijinx @bronsa#2018-01-2615:55ghadisame example even#2018-01-2615:56bronsathe universal example#2018-01-2615:57the2bearsSo this foo walks into a bar. Bartender says, "Hey buddy, what kind of example you trying to set?"#2018-01-2616:16bronsasimple_smile#2018-01-2616:16don.dwoskeAh, missed it.. thanks @ghadi and @bronsa#2018-01-2616:31don.dwoske...and it's been around for awhile now... I'll try not to feel dumb. https://groups.google.com/forum/#!topic/clojure-dev/8tHLCm8LpyU and https://dev.clojure.org/jira/browse/CLJ-1910#2018-01-2617:00trissok, so if I have a function `fā thats been passed in to another function and I donāt know itās fully qualified name is there a way to find it?#2018-01-2617:00bronsano#2018-01-2617:01bronsaif by function you mean var then yes#2018-01-2617:02trissah I do mean var I think - how do I look up a fully qualified name/spec for a function referenced by a variable?#2018-01-2617:02trissoh hang on `var?' just gave me a false...#2018-01-2617:09bronsamaybe if you give a code example of what youāre trying to do it might be clearer#2018-01-2617:11trissIām trying to look up if a particular function has had a spec written for it.#2018-01-2617:11trissi.e does a function have a spec specified for it.#2018-01-2617:11ghadiif you have a fully-qualified symbol, you can do this#2018-01-2617:12ghadi(get (s/registry) 'my.ns/function)
#2018-01-2617:12ghadiif you have a function, you cannot#2018-01-2617:12trissI wonāt have the fully qualified symbol when I come to do it - does this mean itās a no go?#2018-01-2617:12ghadi"So what are you actually trying to do?"#2018-01-2617:12trissah ok - this is a shame - The REPL seems to be able to look up the fully-qualified symbol for function#2018-01-2617:13triss@gihadi - it is a bit convoluted - Iām experimenting with agent based computing - and turning functions in to agents using only the information in there type signatures#2018-01-2617:14triss(well there specād signatures)#2018-01-2617:16ghadican you turn on instrumentation prior a priori?#2018-01-2617:18trissnot what I want unfortuantely - Iām doing some crazy stuff with the spec - I unpack info about each argument and tehn search a collection for it.#2018-01-2617:19trissthe only thing I canāt do is look up wether or a not a function has had a spec written for it (when I donāt have its fully wualified name)#2018-01-2617:26Alex Miller (Clojure team)If you donāt have the qualifier, then itās ambiguous what youāre referring to #2018-01-2617:26Alex Miller (Clojure team)A function of the same name could be spec ed in many nses#2018-01-2617:27Alex Miller (Clojure team)That said, the registry is just a map with either qualified symbols or qualified keywords as keys - you could search for it #2018-01-2617:32trissok - thanks Alex. So is there a way to look up a functions source namespace/name?#2018-01-2617:33Alex Miller (Clojure team)Given what info?#2018-01-2617:33trissjust the functionā¦#2018-01-2617:33trissthe REPL seems to know hwere it came from#2018-01-2617:33Alex Miller (Clojure team)Example?#2018-01-2617:34Alex Miller (Clojure team)A function instance doesnāt hold an explicit name for its source var#2018-01-2617:35trissbb-simple.core> (defn function [a] a)
#'bb-simple.core/function
bb-simple.core> (def z function)
#'bb-simple.core/z
bb-simple.core> z
#function[bb-simple.core/function]
bb-simple.core>
#2018-01-2617:35trissso I can store a function in z
- the REPL tells me where I defined it#2018-01-2617:38trisswhen all I have is z
is there a way to find that it refers to bb-simple.core/function
?#2018-01-2617:38bronsanot reliably#2018-01-2617:39Alex Miller (Clojure team)thatās not the default printer - you must be running with something addition in your repl#2018-01-2617:39trissah - itās the CIDER repl in emacs#2018-01-2617:39Alex Miller (Clojure team)the function class name is a munged version of the original function name (assuming it was from a var, can be from other places too)#2018-01-2617:40bronsawhat thatās doing is using demunge
to get from the class name to an approximation of the original var#2018-01-2617:40bronsabut that is not relibale and can only be done if the function was compiled from a defn
#2018-01-2617:40Alex Miller (Clojure team)you can demunge to recover the class name, but the munge operation is not a true function (mathematically) in that multiple inputs map to the same class name#2018-01-2617:40Alex Miller (Clojure team)thatās where the unreliability comes from#2018-01-2617:41trissoh ok this all sounds a bit yuck. Think Iāll steer clear for now. Itās a lot of extra typing for me but such islife#2018-01-2617:41Alex Miller (Clojure team)(Compiler/demunge (.getName (class z)))
#2018-01-2912:53roman01laš Given a naive example of a function spec, how to specify a custom generator such that it satisfies :fn
spec?
(s/fdef sbtrct
:args (s/cat :a int? :b int?)
:ret int?
:fn #(= (:ret %) (- (-> % :args :a) (-> % :args :b))))
#2018-01-2917:09conanwhat is it you want to generate exactly?#2018-01-2917:13conanare you trying to generate a 2-value vector containing ints that can be used with apply
as the args for sbtrct
?#2018-01-2917:14conangenerators generate values, so by providing a spec for each of your input args and for your return value, you've already provided generators (because spec provides a built-in generator for the int?
predicate)#2018-01-2917:19roman01la@U053032QC I want to generate a sequence of args/ret values such that they satisfy relation defined in :fn
spec#2018-01-2917:20conanah, ok: (s/exercise-fn `sbtrct)
#2018-01-2917:21conanthat'll generate a bunch of input values for a
and b
(using the generators provided by their specs, i.e. int?
), put them into your function, and return you a sequence of those input pairs with the calculated output#2018-01-2917:21conanif your function doesn't match your :fn
spec, it'll blow up#2018-01-2917:22conanif you want to do a larger number of tests of your function, you can do generative testing using clojure.spec.test.alpha/check
#2018-01-2917:22conanso
(stest/check `sbtrct`)
#2018-01-2917:23conanthat'll generate 1000 different inputs and check all the outputs for conformance with your :ret
spec and :fn
predicate#2018-01-2917:23conanneat, huh?#2018-01-2917:23roman01la> if you want to do a larger number of tests of your function..
thatās exactly what I need, exercise-fn
doesnāt seem to be able to generate huge sequences š#2018-01-2917:24conanyou can do (s/exercise-fn `sbtrct 100000)
#2018-01-2917:24conanoh no, looks like that blows up#2018-01-2917:24roman01layeah Iām getting integer overflow#2018-01-2917:25conandoesn't blow up so long as you don't print it#2018-01-2917:25roman01laoooooh, damn#2018-01-2917:26conanmight just be lazy though#2018-01-2917:26conanyeah it's lazy#2018-01-2917:26conanso there are numbers in there that blow up#2018-01-2917:28roman01lahm, what can be done about this?#2018-01-2917:28conando you need the ints to be very large?#2018-01-2917:28roman01lano, just a lot of them#2018-01-2917:28conanif not, you could try
(s/def ::small-int (s/int-in 0 1000000))
(s/fdef sb
:args (s/cat :a ::small-int :b ::small-int)
:ret int?
:fn #(= (:ret %) (- (-> % :args :a) (-> % :args :b))))
#2018-01-2917:29conanthat's specifying a range that the ints can be in, so you won't get huge numbers generated#2018-01-2917:29roman01laShould I use with-gen
if I donāt want to modify the original spec?#2018-01-2917:30conanwith-gen
is for when you need to create your own generator because the existing ones can't capture your set of values#2018-01-2917:31conani guess you'll have to look into what's actually causing the error#2018-01-2917:32conanthat approach above works for me though, with s/int-in
as the spec for the inputs to your function#2018-01-2917:33conani used a different name for my function though so don't just copy/paste#2018-01-2917:34roman01laworks for me as well, thanks!#2018-01-2917:39conanno worries, docs are often a bit sparse around spec, but it is alpha, so /shrug#2018-01-2918:05roman01laI ended up with (s/int-in Integer/MIN_VALUE Integer/MAX_VALUE)
#2018-01-2918:21johnDoes spec.test
in any way supersede test.check
? I couldn't find docs explaining the difference. There seems to be some overlap between the two.#2018-01-2918:39arrdemNot that I know of. spec.test
leverages test.check
extensively and seems designed to complement test.check
by enabling you to derive many generators automatically rather than specify them by hand.#2018-01-2918:43Alex Miller (Clojure team)correct. test.check is useful for writing arbitrary property-based tests and creating different kinds of custom generators#2018-01-2918:44bronsaand spec.test uses test.check internally#2018-01-2919:08johnUnderstood. Thanks#2018-01-3020:11rafaelI'm trying to pass :gen
overrides for clojure.spec.test.alpha/check
, but I get the sense my understanding of the documentation.#2018-01-3020:12rafaelI expected the following to succeed:
(defn foo [k]
(if (keyword? k) :k :nope))
(def my-kw? (partial instance? Keyword))
(s/fdef foo
:args (s/cat :a-keyword my-kw?)
:ret #{:k})
(spec.test/check `foo {:gen {`my-kw? (gen/return :kw)}})
#2018-01-3020:13rafaelI'm trying to pass in a generator for the my-kw?
spec, but it fails with the cause "Unable to construct gen at: [:a-keyword] for: my-kw?"
#2018-01-3020:15rafaelIs overriding generators on the call to check
possible and I'm getting some detail wrong, or did I just misunderstand the docs for check
and there is no way to override generators like that?#2018-01-3020:26Alex Miller (Clojure team)the values in that map should be 0-arg functions returning a generator#2018-01-3020:26Alex Miller (Clojure team)so change (gen/return :kw)
to #(gen/return :kw)
#2018-01-3020:28Alex Miller (Clojure team)although now that I back up a bit, I donāt think you can do what youāre trying to do with my-kw? either#2018-01-3020:28Alex Miller (Clojure team)you should instead register a named spec instead of my-kw?:#2018-01-3020:29Alex Miller (Clojure team)(s/def ::my-kw keyword?)
and then use ::my-kw
instead of my-kw?
and as the key in the gen map#2018-01-3020:35Alex Miller (Clojure team)Putting it all together:
user=> (require '[clojure.spec.alpha :as s] [clojure.spec.test.alpha :as stest] [clojure.spec.gen.alpha :as gen])
nil
user=> (defn foo [k] (if (keyword? k) :k :nope))
#'user/foo
user=> (s/def ::my-kw keyword?)
:user/my-kw
user=> (s/fdef foo :args (s/cat :a-keyword ::my-kw) :ret #{:k})
user/foo
user=> (stest/check `foo {:gen {::my-kw #(gen/return :kw)}})
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2451 0x6c25e6c4 "
#2018-01-3118:43rafaelThat works. I had tried the 0-arg generator function, but I was missing the fact that the spec ident had to be a keyword. Thank you!#2018-01-3022:06jqmtorhey all, I've been meddling with spec and I came across something I don't know how to represent. I want to create a spec for the s/keys
spec itself and I have this:
(s/cat :spec-kind #{`s/keys}
:keys (s/+ (s/cat :mandatory+qualified? #{:req :opt :req-un :opt-un}
:keys (s/coll-of keyword? :kind vector?))))
the problem is that the keys :req
:opt
and so on can be repeated. is there any way to avoid this? (any other unrelated suggestion is also welcome)#2018-01-3022:15Alex Miller (Clojure team)keys*
#2018-01-3022:23jqmtorgreat, thanks! missed that when looking through the api#2018-01-3022:26eriktjacobsenI'm running into the same wall described at https://groups.google.com/d/msg/clojure/i8Rz-AnCoa8/OEg04fbKBwAJ where we'd like to resolve a symbol for a multi-fn in clojurescript but apparently cannot. is this being looked into? that thread died.#2018-01-3118:29uwo@stuarthalloway Iād love to use this on a private project, though do I need to be worried about licensing? https://github.com/Datomic/mbrainz-importer/blob/5d2d90f9a35789824675a4cc86a9a433527cb41b/src/cognitect/xform/spec.clj#2018-01-3118:31uwosimple enough idea, but I did get it from you š#2018-01-3120:41Drew VerleeHas anyone seen any attempts to turn a postgres schema into a set of clojure specs so you can generate data like whats in the database? Or vise versa, fill the database with data?
In the case where your using postgres or a relational db, would that be helpful?#2018-01-3121:50eggsyntax@drewverlee at one point I did the same thing with a Datomic schema. It was probably simpler because of the natural Datomic/Clojure match, but I'd definitely think you could do the same with a relational DB. You might want to think about what you'd want the specs to capture -- type is the obvious low-hanging fruit, but there might be other stuff. And you could probably capture the set of fields present in each table with s/keys
. Possibly other stuff depending on your particular schema, but in my experience worth considering well in advance of taking on the conversion.#2018-01-3121:53Drew Verlee@eggsyntax iām thinking either this sort of thing already exists (maybe in another language) or there are really good reasons why its not useful. But maybe clojure spec opens some new doors.#2018-02-0216:55stathissiderisit doesnāt take care of traversing the schema, but you can run your data through this to get specs out of them: https://github.com/stathissideris/spec-provider#2018-02-0216:55stathissideris(my project btw, so Iām here for questions)#2018-01-3122:21Alex Miller (Clojure team)Iām pretty sure someone has probably done this already for jdbc based on your databasemetadata, and maybe thatās 90% of the work#2018-01-3122:23Alex Miller (Clojure team)donāt think Iāve seen it specifically with spec though#2018-02-0108:22andy.fingerhutIn Rich Hickey's Spec-ulation talk, he mentions the idea of wishing to make specs include some notion of side effects, for functions that have them: https://github.com/jafingerhut/jafingerhut.github.com/blob/master/transcripts/2016-dec-rich-hickey-spec-ulation.txt#L563-L569#2018-02-0108:23andy.fingerhut"A function provides its return. If you gave me what I required, I will provide to you this result. And of course I would like to broaden this discussion to include services and procedures, and things like that. So if your thing is effectful, one of the things you provide is that effect. If you call this thing with these arguments, the thing will be in the database, or I will send an email for you, or some other thing."#2018-02-0108:24andy.fingerhutDoes anyone have any ideas on how such a thing might be described in a spec-like fashion?#2018-02-0111:20mattfordSo I've been using https://github.com/ring-clojure/ring-spec to validate http responses.#2018-02-0111:21mattfordI'd like to over-ride/extend the body part of the above spec to use spec's I've written that model the JSON body I get.#2018-02-0111:21mattfordIs that a thing?#2018-02-0111:22mattfordHow'd people go about that?#2018-02-0111:55borkdudeIād write a different spec for the parsed body.#2018-02-0112:23misha@bbrinck quick expound question: is there a function, accepting vanilla spec/explain-data value and returns pretty error message as string?#2018-02-0112:31mishafound it! expound.alpha/printer-str
https://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L479#2018-02-0115:15bbrinck@misha Yep, that should work. Iāll add that to my notes about what to add to public API for the expound beta#2018-02-0119:01nwjsmithI'm having trouble getting a useful spec together for this map-entries
function I've written:
(defn map-entries
"Returns a map consisting of the result of applying f to the first entry of
the map, followed by applying f to the second entry in the map, until the map
is exhausted."
[f m]
(into {} (map f) m))
(s/fdef map-entries
:args (s/cat :f (s/fspec :args (s/cat :entry (s/tuple any? any?)) :ret (s/tuple any? any?))
:m (s/map-of any? any?))
:ret (s/map-of any? any?)
:fn #(<= (count (-> % :ret)) (count (-> % :args :m))))
#2018-02-0119:03nwjsmithIf I turn instrumentation on, and evaluate say (map/map-entries (fn [[k v]] [(name k) (dec v)]) {:a 1 :b 2 :c 3})
, I get an instrumentation exception because spec tries to run (fn [[k v]] [(name k) (dec v)])
with a generated (s/tuple any? any?)
.#2018-02-0119:06nwjsmithIf I loosen the spec on the :f
arg to just fn?
, then instrumentation is okay, but I'll lose a the "free" property tests. I guess I can just supply my own generator for check
...#2018-02-0119:08mishawhat would you gain, if you mock your f?#2018-02-0119:11nwjsmithWell if I had the following spec:
(s/fdef map-entries
:args (s/cat :f fn?
:m (s/map-of any? any?))
:ret (s/map-of any? any?)
:fn #(<= (count (-> % :ret)) (count (-> % :args :m))))
#2018-02-0119:11nwjsmithThen any function could pass instrumentation (not awesome).#2018-02-0119:12mishaI think you can at least spec f's args being a tuple, and stop there#2018-02-0119:12mattfordI have this map {"took" 1,
"timed_out" false,
"terminated_early" false,
"_shards" {"total" 1, "successful" 1, "failed" 0},
"hits" {"total" 0, "max_score" nil, "hits" []}}
can I have spec conform the strings to keywords and deal with the overloaded "hits" key somehow?#2018-02-0119:15mishaI think, strings to keywords conversion needs to happen separately. but later, you'll be able to use 2 :keys keyword specs with different namespaces#2018-02-0119:17nwjsmithMaybe I can spec the args differently here. What I'd like is the spec to be is
(s/cat :f (s/fspec :args (s/cat :entry (s/tuple <T: any?> <V: any?>))
:ret (s/tuple any? any?))
:m (s/map-of <T: any?> <V: any?>))
#2018-02-0119:17nwjsmithugh, let me format that#2018-02-0119:19misha@mattford
(def m {"took" 1,
"timed_out" false,
"terminated_early" false,
"_shards" {"total" 1, "successful" 1, "failed" 0},
"hits" {"total" 0, "max_score" nil, "hits" []}})
(s/def :foo/hits vector?)
(s/def :bar/hits (s/keys :req-un [:foo/hits]))
(s/def :bar/m (s/keys :req-un [:bar/hits]))
(->> m
(clojure.walk/keywordize-keys)
(s/explain :bar/m))
Success!
=> nil
#2018-02-0119:21misha@nwjsmith try to drop :ret (s/tuple any? any?)
for :f#2018-02-0119:22mattfordgreat ty!#2018-02-0119:22mishainto's spec might catch not-tuples from f#2018-02-0119:23mishait blows up opieop#2018-02-0119:28nwjsmith(defn map-entries
"Returns a map consisting of the result of applying f to the first entry of
the map, followed by applying f to the second entry in the map, until the map
is exhausted."
[f m]
(into {} (map f) m))
(s/fdef map-entries
:args (s/cat :f (s/fspec :args (s/cat :entry (s/tuple any? any?)))
:m (s/map-of any? any?))
:ret (s/map-of any? any?)
:fn #(<= (count (-> % :ret)) (count (-> % :args :m))))
(stest/instrument)
(map-entries (fn [[k v]] [(name k) (dec v)]) {:a 1 :b 2 :c 3})
ExceptionInfo Call to #'user/map-entries did not conform to spec:
In: [0] val: ([nil nil]) fails at: [:args :f] predicate: (apply fn)
clojure.core/ex-info (core.clj:4617)
#2018-02-0119:30nwjsmithHow does instrumentation of function arguments work? Does the function get evaluated?#2018-02-0120:26Alex Miller (Clojure team)the args spec for the function argument is used to generate inputs. the function argument function is invoked with those examples and the ret spec is validated#2018-02-0120:29Alex Miller (Clojure team)also, you should pretty much never use fn?
(if you go that route) - ifn?
is almost always what you want#2018-02-0120:30Alex Miller (Clojure team)generic higher-order functions are inherently difficult to spec in useful ways. specs are great for saying concrete things about your data. The more generic your function, the harder it is to say something specific and meaningful.#2018-02-0121:59ghadinot sure if this is a bug, but I could use a second set of eyes:#2018-02-0122:02ghadiI'm asking spec to stub
out the function TARGET while overriding the generator for ::bar, which is aliased to ::foo. The gen overriding doesn't work -- if I override ::foo (which ::bar aliases) then it works. Probably a non-minimal example if someone can help me reduce it#2018-02-0200:07Alex Miller (Clojure team)I think there is a known issue with swapping gens for aliased specs#2018-02-0200:07Alex Miller (Clojure team)On the phone so canāt easily search JIRA #2018-02-0202:35ghadihttps://dev.clojure.org/jira/browse/CLJ-2079 ^^#2018-02-0203:00Alex Miller (Clojure team)Thatās the one#2018-02-0217:14dominicmIs there a convenience function for:
(s/cat :foo.bar.baz.bosh/title :foo.bar.baz.bosh/title
:foo.bar.baz.bosh/description :foo.bar.baz.bosh/description)
#2018-02-0217:19stathissideris@dominicm catās name arguments are non-qualified#2018-02-0217:20stathissideris(s/cat :title :foo.bar.baz.bosh/title
:description :foo.bar.baz.bosh/description)
#2018-02-0217:20stathissiderisand with aliases it can be shorter#2018-02-0217:21dominicm@stathissideris But I want to return a namespaced map š#2018-02-0217:21stathissideriscat is for sequences!#2018-02-0217:21stathissiderisnot maps#2018-02-0217:21stathissiderisuse s/keys
instead#2018-02-0217:22dominicmI have a sequence of things, and I want to conveniently conform it into a namespaced map. I'm being lazy.#2018-02-0217:23stathissiderisoh!#2018-02-0217:23stathissiderisok makes sense#2018-02-0217:24dominicmI guess the manual way I go (hi ho!)#2018-02-0217:31stathissideriswould be a pretty trivial macroā¦#2018-02-0219:42mishamacro all the things!#2018-02-0221:20gklijsDon't need a macro for that, it's pretty easy with some functions to convert data based on a spec to/from a vector.#2018-02-0309:51misha@alexmiller is there any estimate on where will this be addressed? It makes generating even slightly complex data structures noticeably painful. https://dev.clojure.org/jira/browse/CLJ-2079 (Generator overrides for spec aliases are not respected)#2018-02-0312:34benalbrecht@misha @ghadi you could resolve the spec manually before passing it to the generator map:
(defn resolve-spec [spec]
(last (take-while keyword? (iterate s/get-spec spec))))
#2018-02-0313:49mishawhy? @benalbrecht#2018-02-0313:50mishado you mean "find all aliases manually and explicitly override those"?#2018-02-0313:50mishathat should work, yes#2018-02-0401:03richiardiandreaIs there a way to not report errors for an argument in a s/cat
or spec only the second position of it? The first argument in my case is a JS connection object and I am not interested in seeing it when it does not validate because very verbose#2018-02-0401:04seancorfield@richiardiandrea If you don't care about it validating, why not spec it as any?
?#2018-02-0401:04richiardiandreaYes that is what I am doing, but it is included as part of the error message#2018-02-0401:05seancorfieldOh, I see. So you want a custom error reporter instead? Have you looked at Expound?#2018-02-0401:05richiardiandreaOh right, yes I am aware of expound. Probably I can have a custom reporter if not too tough too implement#2018-02-0401:09bbrinck@richiardiandrea With expound, you can customize the function that prints out your value https://github.com/bhb/expound#configuring-the-printer#2018-02-0401:10bbrinckSee the example under āYou can even provide your own function to display the invalid value.ā#2018-02-0401:13bbrinckAlthough by default, parts of your data that is valid wonāt be shown anyway with Expound#2018-02-0401:27bbrinck@richiardiandrea Hereās an example that includes a custom printer that replaces the first arg (presumably one that is verbose) with a more succinct representation https://gist.github.com/bhb/5222914641bcdd08d07b7d930e388d89#2018-02-0404:38richiardiandreaThat is great thaaaanks, I am actually using expound
#2018-02-0405:34bbrincknp#2018-02-0414:08alex-dixonHow do you write a spec for a map that has a namespaced keyword like :db/id
?#2018-02-0414:21tayloruse a s/keys
spec, and use its :req
or :opt
to specify the individual key specs
(s/def my-spec (s/keys :req [:db/id]))
#2018-02-0414:24taylorhttps://clojure.org/guides/spec#_entity_maps#2018-02-0415:01alex-dixonAwesome. Thanks. Thought I had to use ::db/id
for some reason#2018-02-0422:45bbrinckComing soon in expound: optional human-readable error messages for predicates https://asciinema.org/a/161011#2018-02-0423:26stathissideris@bbrinck thatās really good, but would it be possible to add an error message to an existing spec? Iām thinking about the case where youāre trying to do this to a third-party spec whose code you donāt control#2018-02-0423:26stathissiderisit may be too much to ask, but I think it would be better to keep the libs off each otherās toes#2018-02-0423:28bbrinckItās an interesting idea ā¦ certainly you could add a an error message for a spec directly (I can add a function for this).#2018-02-0423:29bbrinckIf you used expound/def
on an existing function, youād probably need to overwrite the actual spec though, so as not to break the way s/def
works#2018-02-0423:29bbrinckin other words, calling expound/def
would overwrite existing spec, just like s/def
#2018-02-0423:30bbrinckAnother thing to watch out for is that this wonāt work unless the original spec author defined a spec for the predicate specifically#2018-02-0423:31bbrinckSo, for instance, you wonāt be able to add an error message for this instance of vector?
https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L16 since itās not a spec by itself#2018-02-0423:39stathissideris@bbrinck yeah, I would expect the parameter in that function to be the name of a spec, not a predicate#2018-02-0423:40stathissiderisI was looking at spec tools recently, and although I liked the functionality, I was a bit sceptical about that fact that I would have to write specs differently (using the āspec recordā I think)#2018-02-0423:41stathissiderisso thatās what struck me about your approach as well#2018-02-0423:41bbrinckYep, I could add something like (expound/add-message <qualified-keyword> <string>)
#2018-02-0423:42stathissiderisIād be very happy with that š does that get stored in a separate registry for expound?#2018-02-0423:42bbrinckYes, thatās correct#2018-02-0423:42bbrinck@stathissideris Yes, needing to write specs differently is a fair concern here. I donāt think thereās a way to bolt this functionality onto existing specs, and even if we could, Iām not sure Iād want to#2018-02-0423:43bbrinckUnless clojure.spec supported this directly, of course ā¦ š#2018-02-0423:43stathissideristhereās been talk about spec metadata for quite a while now#2018-02-0423:43bbrinckBut for now, you can write your specs in either style#2018-02-0423:43stathissiderisā¦but no official solution yet#2018-02-0423:44bbrinckand if the consumer is using spec, it works. if they are using expound, they will get the enhanced error messages#2018-02-0423:44stathissiderisyeah, I think it all composes a bit more nicely like that!#2018-02-0423:45bbrinck(for the record, Iād be happy to see something compatible go into spec, and if it did, iād use it in expound)#2018-02-0423:45bbrinckoh wait, I actually implemented this function - itās register-message
#2018-02-0423:46bbrinckMan, Iāve already forgotten what I wrote yesterday š#2018-02-0423:46bbrinckI will add it to documentation#2018-02-0423:46stathissiderislooking forward to the new functionality, I have a direct use for it when you release it, so thanks for your work!#2018-02-0423:46stathissiderishaha, thatās great, thatās the style Iām going to use#2018-02-0423:46bbrinckCool, I hope to release early this week#2018-02-0423:47bbrinckuntil then, itās in 0.4.1-SNAPSHOT
#2018-02-0423:47bbrinckbut, use at your own risk š#2018-02-0423:47stathissiderisnot in a hurry, but looking for ways to make the errors in my UI a bit more user-friendly#2018-02-0704:07bbrinckIāve released Expound 0.5.0, which includes this feature#2018-02-0704:07bbrinckhttps://github.com/bhb/expound#error-messages-for-predicates#2018-02-0707:55stathissiderisgreat, thank you very much, Iām going to give it a go in the next few days#2018-02-0913:58stathissiderisso, no register-message
in the end?#2018-02-0509:57dominicmIs there a s/cat function which doesn't create a map? I essentially want a regex op which will consume exactly 2 predicates.#2018-02-0513:15Alex Miller (Clojure team)Use s/tuple#2018-02-0513:17dominicm@U064X3EF3 That isn't a regex op, so it doesn't work for this:
(s/conform
(s/cat
:foo int?
:bar (s/tuple int? int?)
:baz int?)
[1 2 3 4])
#2018-02-0513:17dominicm(s/conform
(s/cat
:foo int?
:bar (s/cat :a int? :b int?)
:baz int?)
[1 2 3 4])
;; =>
;; {:foo 1, :bar {:a 2, :b 3}, :baz 4}
with the s/cat regex op#2018-02-0513:21Alex Miller (Clojure team)s/+ then s/& with a size constraint?#2018-02-0513:23dominicm@U064X3EF3 This works!
(s/conform
(s/cat
:foo int?
:bar (s/& (s/+ int?) #(= 2 (count %)))
:baz int?)
[1 2 3 4])
Rather cool.#2018-02-0513:24dominicmI didn't really understand s/& when I read the docstring.#2018-02-0513:49Alex Miller (Clojure team)itās like s/and but as a regex op#2018-02-0514:21dominicmThat's a good description. I prefer that #2018-02-0509:57dominicmMaybe my mistake is using something positional where I should be using a map#2018-02-0510:23mattforddo you have to conform it?#2018-02-0510:25mattfordI've instrumented and stubbed out a function that thanks to the magic of spec can now generate results. The result is generated from a multispec though, is there a way of limiting a multispec to one of the methods [so I can only return certain types of results]?#2018-02-0510:53acron@mattford add a predicate check for the dispatch type#2018-02-0510:54acron(s/cat :foo (s/and ::my/thing is-correct-thing?))
#2018-02-0510:56acronwhere ::my/thing
is your multispec#2018-02-0510:56acronthis has worked relatively successfully for me#2018-02-0510:57acronWhat would cause clojure.spec.test.alpha/spec-checking-fn/conform!
to be applied to a function that has *not* been instrumented? š¤#2018-02-0510:59mattfordFeels a bit circular having a mutlispec generate many cases only then to filter it again on the same logic each defmethod has.#2018-02-0511:01mattfordworks though...#2018-02-0511:02acronWell, the real question could be why do we want just one result type generating?#2018-02-0511:05mattfordI instrumented my general get-event
handler but I'm only interested in testing "visualisation" events in the next part of the tests..#2018-02-0516:06tcoupland@mattford check out the :gen option of instrument #2018-02-0604:01zalkyI'm assuming this is a bug?#2018-02-0605:20Alex Miller (Clojure team)I would expect it to fail in both, surprised it doesnāt in clojure #2018-02-0607:51bmaddyHmm, what am I missing here?
cljs.user> (require '[cljs.spec.alpha :as s])
nil
cljs.user> (require '[cljs.spec.gen.alpha :as gen])
nil
cljs.user> (gen/generate (s/gen int?))
#object[Error Error: Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required]
Is there somewhere I should be looking for an example of how to do this in cljs? I couldn't find anything on http://clojurescript.org or in the official spec guide.#2018-02-0608:00bmaddyOh, and here's part of my :dependencies
:
[org.clojure/clojure "1.9.0"]
[org.clojure/clojurescript "1.9.946"]
[org.clojure/test.check "0.9.0"]
#2018-02-0609:08bronsait's not throwing in clojure because asserts
are disabled by default#2018-02-0609:08bronsa>>>Can be disabled at either compile time or runtime:
If \compile-asserts\ is false at compile time, compiles to x. Defaults
to value of 'clojure.spec.compile-asserts' system property, or true if
not set.
If (check-asserts?) is false at runtime, always returns x. Defaults to
value of 'clojure.spec.check-asserts' system property, or false if not
set. You can toggle check-asserts? with (check-asserts bool)#2018-02-0615:04zalky@alexmiller: are you sure? In my example, the ::t
spec, (s/keys :req [:ns/y]))
does not specify anything about the :ns/x
key. But it seems to be applying it anyways, simply by virtue of it being defined globally. This would seem to defy the point of :req
. Maybe I'm misunderstanding how keys
was meant to be used, but I would have thought the clojure version to be correct.#2018-02-0615:05Alex Miller (Clojure team)s/keys is intended to check all registered keys in the map, regardless of whether they are listed in the spec#2018-02-0615:09zalkyah, thanks for clarifying, but then what is the point of :req
? Shouldn't you only need :opt
?#2018-02-0615:12Alex Miller (Clojure team)req defines the keys that are required (must be present)#2018-02-0615:12Alex Miller (Clojure team)req is about membership#2018-02-0615:13Alex Miller (Clojure team)so itās really checking a property of the containing map#2018-02-0615:13zalkyhmm, but if all keys are just going to be globally applied whether they are in the spec or not, wouldn't that mean all keys a implicit req, unless specified by :opt?#2018-02-0615:13Alex Miller (Clojure team)validating values according to keys is about checking properties of attributes#2018-02-0615:13Alex Miller (Clojure team)no, it does not mean that#2018-02-0615:13Alex Miller (Clojure team)theyāre not required, but if they are present, they are checked#2018-02-0615:29zalkyHmm, ok, thanks for the clarification. Is the design assumption that entity attributes should conform the same properties in all contexts? You should never have an entity attribute conform one way in one context, and another way in another context?#2018-02-0615:35Alex Miller (Clojure team)correct#2018-02-0615:35Alex Miller (Clojure team)https://clojure.org/about/spec#2018-02-0618:08zalky@alexmiller: thanks for the responses, the doc was very helpful. My use case was that I was playing with a simple FSM that brought an entity map through one or more workflows. At first spec seemed to lend itself well to defining the transitions in the FSM. For example, an entity with any number of votes is a valid entity, but once it gets a certain number of votes, it transitions to another state in the FSM. Unfortunately having one valid global definition for an attribute seems to place significant restrictions on defining multiple validity states for an entity. Am I missing an easier way to do this with spec, or is it just that spec is not the right tool for this job?#2018-02-0618:40Alex Miller (Clojure team)I find itās always useful to go back to: what is the truth about this attribute? if I see it in data, what does it mean?#2018-02-0618:40Alex Miller (Clojure team)if it has several possible kinds of values, you can use s/or to specify those alternatives - then youāre stating the truth#2018-02-0618:41Alex Miller (Clojure team)s/multi-spec can be used to choose different combinations of attributes at different points in the fsm#2018-02-0618:41Alex Miller (Clojure team)or perhaps you may want to reconsider whether itās really the same attribute at every step#2018-02-0618:41Alex Miller (Clojure team)and in that case, maybe the attribute should change#2018-02-0618:45Alex Miller (Clojure team)to be clearer, maybe there is really more than one kind of attribute#2018-02-0618:46Alex Miller (Clojure team)or another option is donāt spec that attribute - youāre expecting it to change over time#2018-02-0618:46Alex Miller (Clojure team)you can still spec functions that operate on that attribute more precisely#2018-02-0619:22zalkyYeah, I was thinking the same thing about the attribute, there are probably times where it indicates that it is really two attributes. However this isn't always the case. The simple example I gave above with the voting is probably a case where splitting would unnecessarily complicating the data model because of an assumptions in the implementation.
I think you're suggestion to not spec the attribute via the entity model, and just spec it separately using custom primitives (at least that is how interpreted your last sentence) is probably the one that makes the most sense . Unfortunately you lose some of the great higher order features of the entity model, like being able to define one transition as a merge of previous transitions and some new criteria.
One final question: do you think that allowing local bindings in keys
would break spec's entity model significantly? Something like:
(s/def ::entity
(s/keys :req [::attr]
:let [::attr #(<= 10 (count %))]))
#2018-02-0620:23Alex Miller (Clojure team)yes and you donāt need it#2018-02-0620:24Alex Miller (Clojure team)(s/and (s/keys :req [::attr]) #(<= 10 (count (::attr %))
#2018-02-0621:12zalkyThanks alex for you time (and your contributions on spec!)#2018-02-0718:55Joe LaneIf the above snippet is OT and belongs somewhere else, Iāll gladly take it there instead.#2018-02-0720:36bbrinck@lanejo01 Would it work to just remove the namespace from the keys in args
? e.g. (into {} (map (fn [[k v]] [(keyword (name k)) v])) {:foo/bar 1 :foo/baz 2}) ;; => {:bar 1, :baz 2}
#2018-02-0720:37bbrinckIf you didnāt want to remove the namespace from all keys, you could pass in a specific prefix to remove in this case, :myapp.entity.user
#2018-02-0720:41Joe LaneInteresting, yeah I could do that to just keep the unqualified keys. Im wondering if iām missing some design step though since from the jira ticket it was concluded that there wasnāt planned work for namespaced keys in records.#2018-02-0721:33Alex Miller (Clojure team)I didnāt read everything above, but have you tried using s/keys with :req-un to spec records#2018-02-0722:22Joe LaneAhh, I think thats the answer @alexmiller. I had read through that section of the spec guide probably 15 times last night but it didnāt click. This also pushes me away from desiring namespaced keywords in records. Thank you!#2018-02-0722:38seancorfield@alexmiller On that subject... given the general advice re: maps / records is usually "use maps, and only switch to records if you need extra performance", how does that sit with using namespaced maps -- since those cannot easily be switched to records?#2018-02-0722:38Alex Miller (Clojure team)it complicates it :)#2018-02-0722:38seancorfieldš but not helpful!#2018-02-0722:39Alex Miller (Clojure team)I mean if you have maps with namespaced keys, you can switch to records and change specs from :req to :req-un#2018-02-0722:39seancorfieldFair enough. The spread of namespaced maps does seem to reduce the attraction of using records further.#2018-02-0722:39Alex Miller (Clojure team)unfortunately, I agree. I seem to be in the minority, but I like records. :)#2018-02-0722:39seancorfield@alexmiller Yeah, but all of your code that manipulates those maps would need to change too -- since it will be using :foo/bar
now and would need to change to :bar
.#2018-02-0722:39Alex Miller (Clojure team)true#2018-02-0722:40Alex Miller (Clojure team)Iāve had a few conversations with RH about this#2018-02-0722:40seancorfieldI'll be interested to hear what he decides in the end š#2018-02-0722:40Alex Miller (Clojure team)records actually do have a namespace context via the package of the defining record type#2018-02-0722:41Alex Miller (Clojure team)seems like there should be a good way to leverage that#2018-02-0722:41Alex Miller (Clojure team)like field a in record my.R could use spec :my/a if reqāed in a s/keys#2018-02-0722:42Alex Miller (Clojure team)but that doesnāt really address the access difference#2018-02-0722:42Alex Miller (Clojure team)would also need to allow lookup by nsāed key in the record, which maybe would be interesting#2018-02-0722:43seancorfieldPerhaps an option on defrecord
to produce either plain keys or namespaced keys? I assume the /
could be munged to something Java would accept?#2018-02-0722:43Alex Miller (Clojure team)meh#2018-02-0722:43seancorfieldHahaha š I suspected that might be your response!#2018-02-0722:43Alex Miller (Clojure team)I think adding new access mechanisms is greatly preferable to adding options#2018-02-0722:44noisesmithitās pretty ugly when munged user=> (munge "x/y")
"x_SLASH_y"
#2018-02-0722:46Alex Miller (Clojure team)optional behavior flags either means youāre being lazy or havenāt thought about it long enough :)#2018-02-0722:48Alex Miller (Clojure team)probably better to improve how a record spec would link fields to attribute specs in a way that supports more kinds of code evolution#2018-02-0814:33justinbarclayIām learning about cljs.spec, but when I try to run the code below in a figwheel repl it times out. Does anyone know why or how I can improve my spec?
(defn split-in-half
"Splits a collection in half"
[coll]
(split-at (Math/round (/ (count coll) 2)) coll))
(spec/fdef split-in-half
:args (spec/cat :col coll?)
:ret (spec/tuple coll? coll?))
(stest/abbrev-result (first (stest/check `split-in-half))
#2018-02-0816:28mpenetExpected, merge doesn't flow conformed values. This one comes up quite often#2018-02-0816:28mpenetThere s a (closed) jira issue about this#2018-02-0921:11justinleeso from the department of This Canāt Be Happening, I speced out a reagent component with fdef and instrumented it, and now it actually calls one of the callback-handlers that is passed to the component as part of the instrumentation process. is this expected/possible? or am i misunderstanding? in the code below, the on-cancel
callback gets called 21 times if I leave the instrument call in the code (spec/def ::name string?)
(spec/def ::class string?)
(spec/def ::type string?)
(spec/def ::*value #(instance? reagent.ratom/RAtom %))
(spec/def ::on-cancel (spec/nilable (spec/fspec :args (spec/cat))))
(spec/def ::focus? boolean?)
(spec/fdef input
:args (spec/cat
:props (spec/keys
:req-un [::type ::*value]
:opt-un [::class ::focus? ::on-cancel]))
:ret any?)
(stest/instrument `input)
#2018-02-0921:14Alex Miller (Clojure team)fspec args are validated by genāing from the :args spec and invoking the function, then checking the :ret spec#2018-02-0921:15Alex Miller (Clojure team)some people have found this to be surprising :)#2018-02-0921:16Alex Miller (Clojure team)probably the easiest āfixā is to replace (spec/fspec :args (spec/cat))
here with ifn?
#2018-02-0921:17justinleethe good news is iām not crazy š#2018-02-0921:18justinleesometimes i suspect iām using spec improperly. i really just want an assertion library to catch my typos#2018-02-0921:23justinleeso is it ever possible/safe to use fspec with a side-effecting function? i donāt see how it could ever work unless you passed it a pure function#2018-02-0921:24Alex Miller (Clojure team)I would not advise it#2018-02-0921:25Alex Miller (Clojure team)passing a side-effecting function that is#2018-02-0921:25Alex Miller (Clojure team)there are ways to (for example) swap out an alternate spec when you instrument - those are optional capabilities of instrument#2018-02-0921:26Alex Miller (Clojure team)but Iām not sure youāre getting much value than bother out of having the fspec at that point#2018-02-0921:26mishaTIL about stubbing in s/instrument#2018-02-0921:26Alex Miller (Clojure team)well Iām talking about replace, not stub, but yeah#2018-02-0921:27Alex Miller (Clojure team)stubbing is great for stubbing out a remote call in instrument#2018-02-0921:28mishaif I'd commit fdef + instrument-with-stub instead of actual implementation, I wonder how quickly my teammates will notice in production? kappa#2018-02-0921:32Alex Miller (Clojure team)oh, pretty quick Iād guess#2018-02-0921:32Alex Miller (Clojure team)unless you wrote some really good generators#2018-02-0921:32mishanumber?
troll#2018-02-1018:16arohnerIām having trouble generating from a recursive spec, I keep hitting StackOverflowError. Are there any docs/guides on how to do that?#2018-02-1018:16arohnerIām doing:
(s/def ::a a?)
(s/def ::b b?)
(s/def ::c (s/with-gen c? (gen/fmap #(make-c %) (s/or :a ::a :b ::b :c ::c)))
#2018-02-1018:19arohnerIāve also tried t.c.gen/recursive-gen#2018-02-1018:25taylorhard to say because the example is missing several definitions but you could try (binding [s/*recursion-limit* 1] ...)
#2018-02-1018:30arohnerThat helped, in that gen/sample occasionally finishes, but still stackoverflows some of the time. It surprised me that in the first 10 items, it produces nested cās 5 deep, which I wouldnāt expect with a recursion limit of 1#2018-02-1018:34tayloryeah I think thereās been a JIRA ticket w/some discussion about this, canāt find it at the moment#2018-02-1018:37arohnerhttps://dev.clojure.org/jira/browse/CLJ-1978#2018-02-1018:52taylor@arohner also this discussion https://clojurians-log.clojureverse.org/clojure-spec/2016-08-17.html#inst-2016-08-17T19:58:22.002442Z#2018-02-1019:08arohnerI think I can work around it using gen/frequency#2018-02-1019:09arohneri.e. (gen/frequency [[9 (s/or :a ::a :b ::b)] [1 (s/or :a ::a :b ::b :c ::c]])
#2018-02-1019:42arohnerfrequency helps, but doesnāt completely solve the problem#2018-02-1020:20andy.fingerhutA transcript of a 2-hour talk by Rich Hickey from Dec 2016 about clojure.spec is up now: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureSpec.md#2018-02-1020:20andy.fingerhutIt doesn't go deep into the details -- more of an intro, and big picture of ways the pieces of spec can be used.#2018-02-1200:42Vincent CantinThe URL in the sticky note of this channel seems broken : https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#2018-02-1200:46Vincent CantinThe correct URL may be https://clojure.github.io/spec.alpha/ ?#2018-02-1201:29Alex Miller (Clojure team)Yes, that last url is the correct one. Earlier, spec was in clojure and the prior one was correct#2018-02-1201:55Vincent Cantinbeginner question: Can we use spec to validate the correlation between values at different places in a structure?#2018-02-1201:56tayloryes, are you thinking maps or another type of structure?#2018-02-1201:56Vincent CantinFor example, if I want to write a spec for [a b 1 5 ... 3 a b]
where a
and b
could be anything.#2018-02-1201:59Vincent Cantin... supposing that the structure is complex enough so that I would need to use s/conform
to match the location of a
and b
at different places.#2018-02-1202:00Vincent CantinI could use a classical function to check if they match at different places, but I would like to know if there is a better spec-ish way to do it. Something like declaratively defining the pattern.#2018-02-1202:03Vincent CantinFor example, something like: "For any a, b : [a b ... a b] is valid."#2018-02-1202:08taylorsomething like this?
(s/def ::my-coll
(s/* (s/alt :a-b (s/cat :a #{'a} :b #{'b})
:num number?)))
(s/conform ::my-coll ['a 'b 1 2 3 'a 'b])
#2018-02-1202:26taylorOr if you wanted to require the coll always begin and end with a b
:
(s/def ::a-b (s/cat :a #{'a} :b #{'b}))
(s/def ::my-coll (s/cat :a-b-start ::a-b
:nums (s/* number?)
:a-b-end ::a-b))
#2018-02-1202:50Vincent CantinThank you. I will try it again later tonight.#2018-02-1214:31Jakub HolĆ½ (HolyJak)Is there a way to manually conform function arguments before I call (apply myfn args)
? I.e. having (spec/fdef myfn :args (spec/cat :userid ::userid))
how do I check that the args
I have conform to that? Thanks!
(I want to use this with expound
. I have command functions invoked from a Slack bot and want to verify that the command the user typed has all the correct arguments and show a nice error if not)#2018-02-1214:34mpenetmake a separate spec of the :args and use s/valid? on it ?#2018-02-1214:35mpenetyou can then write (s/fdef myfn :args ::foo-args ... )#2018-02-1214:55mpenetwondering: why wasn't check-asserts based on a dynamic var like *compile-asserts*
? Right now it's a bit all or northing at runtime for assertion checking (for the one that are compiled at least)#2018-02-1214:57mpenetit's easy enough to work around it anyway#2018-02-1215:13Alex Miller (Clojure team)all or nothing is kind of the typical modes for assertions#2018-02-1215:14Alex Miller (Clojure team)you can also grab the :args spec of a specāed function with (-> `myfn s/get-spec :args)
#2018-02-1216:51jumarI'm wondering if I can do any better if I want to have two different specs for the same key...
I have a generic auth-provider , e.g. #:auth-provider{:type "ldap" :config {:host "abc" ...}}
This has a generic spec :
(s/def :auth-provider/config (s/map-of keyword? any?))
...
(s/def ::auth-provider-spec (s/keys :req [:auth-provider/type
:auth-provider/active?]
:opt [:auth-provider/id
:auth-provider/default-role
:auth-provider/config
:auth-provider/priority-order
:auth-provider/role-mapping]))
which is fine for some generic application code.
However, in some scenarios I want to act differently based on the type of the auth-provider.
E.g. for "ldap" I know that certain keys have to be present inside :auth-provider/config
.
I couldn't find a better solution than this:
(s/def ::ldap-provider-spec (s/and
::auth-specs/auth-provider-spec
;; this is less descriptive than using `s/keys` directly
;; but we cannot use `s/keys` because `:auth-provider/config` default
;; spec is already registered in auth-specs namespace
;; and you cannot have two different specs for the same namespaced key
#(s/valid? ::ldap-config (:auth-provider/config %))))
My problem is that since specs are global I cannot register two different specs for the same key (`auth-provider/config`).
I also tried multi-spec
but couldn't solved it either.
My solution works but it's quite opaque - the error message just says that data fails ::ldap-config
spec (not that e.g. :host
key is missing in config map)#2018-02-1218:04mishalook again at multi-spec, @jumar. it would be verbose solution, but sounds like it fits.#2018-02-1218:27jumar@U051HUZLD well, my problem was that I still couldn't define two different specs for :auth-provider/config
key. Basically, based on the value of :type
key I'd implement two multimethods - one for the :type
value "ldap" and one for :default
.
But in both of them I'd need to use something like (s/keys :req [:auth-provider/config])
- and here I'm stuck. I don't know how to define two different specs for :auth-provider/config
key.#2018-02-1218:12mishaalthough, it'd be much easier, if you'd put :auth-provider/type inside config itself#2018-02-1218:46mishaif you put type inside config itself - multispecs will be about config keys, net provider's ones#2018-02-1218:48mishaor you can just s/or
the :provider/config, but it will not keep :provider/type and or's branches "in sync"#2018-02-1221:42roklenarcichow do you guys spec a map where keys are generic and values are constant for a type? To give you an example, clojure.xml
will output something like {:tag :first_name, :attrs nil, :content ["John"]}
. How would I spec that I want my ::first-name
to have this structure? The keys
function would require me to make a spec for ::tag
but the value is constant for ::first-name
and at the same time, different in another spec, I can't globally specify ::tag
spec.#2018-02-1221:55seancorfield@roklenarcic I suspect you'll want a multi-spec for the various types of parsed maps you can get back from clojure.xml
#2018-02-1221:55seancorfield(and the multi-spec would branch on the :tag
key's value)#2018-02-1222:09roklenarcicI'll look into it.#2018-02-1222:12roklenarcicThere's something weird with the large-integer
generator. As is, it never generates a number over a 100. Even if I use the large-integer*
generator with options, the numbers are very low.
(gen/sample (gen/large-integer* {:min 1 :max 1000000}))
=> (2 2 1 4 1 1 9 1 2 15)
#2018-02-1222:20roklenarcicThe test.check
generator clojure doc states:
(def ^{:added "0.9.0"} large-integer
"Generates a platform-native integer from the full available range
(in clj, 64-bit Longs, and in cljs, numbers between -(2^53 - 1) and
(2^53 - 1)).
Use large-integer* for more control."
(large-integer* {}))
But in my experience it generates number in about a -100 +100 range instead of 2^64#2018-02-1222:56taylortry generating a larger sample#2018-02-1222:56taylor(gen/sample (gen/large-integer* {:min 1 :max 1000000}) 1000)
#2018-02-1222:58roklenarcichm... I guess I expected that generated numbers were uniformly distributed and so it would be extremely likely that a number over a 1000 was generated in 10 tries#2018-02-1223:02gfredericks@roklenarcic the large-integer generator exhibits growth just like most of the other generators#2018-02-1223:03gfredericksmeaning it respects the size
parameter#2018-02-1223:03gfredericksgen/sample
with 1 arg gives you samples using sizes 0ā9#2018-02-1223:08roklenarcicI see, thank you#2018-02-1223:13roklenarcicThe reason I was worried is that when I use integer generator to generate ids, I get a lot of non-unique small numbers (because of the small initial range), which makes it very likely for a sequence of generated entities to have non-unique IDs, which causes such-that to fail occasionally, when IDs are specced to be unique.#2018-02-1301:35gfredericksthere are generators for uniqueness, like gen/vector-distinct-by
; and when it has trouble finding different IDs, it increases the size
, which should be sufficient here#2018-02-1309:11Jakub HolĆ½ (HolyJak)Is there a way to derive a spec from a spec?
We have a checkout flow that in each step adds something to an order and a spec for the final order. I would like to have a spec for each step of the checkout process (ie the final spec with some keys either removed or changed from required to optional).
The problem is that order is not a flat map and may itself contain maps that grow across multiple steps. I want to avoid copy&paste&modify of the end spec. Any ideas? Thanks!#2018-02-1309:22gklijsYes, you can do things like (def label (s/and (s/spec string?) #(> (count %) 2) #(< (count %) 40)))
(s/def ::nl-label label)
(s/def ::label (s/keys :req [::nl-label]))
not sure if it helps, but thatās how I prevented a lot of copy-pasting#2018-02-1309:31gklijsFor specs of maps there is also merge, which seems more use-full re-reading you questionhttps://clojuredocs.org/clojure.spec.alpha/merge#2018-02-1312:24stathissiderissounds like heād need deep-merge (which doesnāt exist afaik)#2018-02-1313:30slipsetOh, and while on the topic of merge
#2018-02-1313:30slipsethttps://dev.clojure.org/jira/browse/CLJ-1981#2018-02-1313:31slipsetI'm quite disappointed that this is closed as "working as intended" with a#2018-02-1313:31slipsetdocstring fix#2018-02-1313:31slipsethttps://github.com/clojure/clojure/commit/d920ada9fab7e9b8342d28d8295a600a814c1d8a#2018-02-1313:32slipsetMaybe I just need to understand the rationale behind why this is intentional behaviour?#2018-02-1313:53vikeriAny way of overriding the generators so that different specs depend on each other? My use case is testing a function where the arguments have internal relations, i.e. argument 1 contains references to argument 2. Randomly generated arguments will not have this relation naturally.#2018-02-1314:02Alex Miller (Clojure team)independently, no. but in a shared context like an args list, yes - you can use combinators like fmap or bind to achieve this#2018-02-1410:36vikeriI see!#2018-02-1417:45firstclassfunchey @alexmiller still a little fuzzy here. Trying to build a generator that produces a common ::id
field as in (defrecord Subscriber [name id])
(defrecord Subscribed [article id])
(s/def ::name string?)
(s/def ::article uuid?)
(s/def ::id uuid?)
(s/def ::Subscriber
(s/keys :req-un [::name ::id]))
(s/def ::Subscribed
(s/keys :req-un [::article ::id]))
(defn generate-subscribers
"Mock function to generate subscribers"
[]
(gen/sample (s/gen ::Subscriber)))
(defn generate-subscribed
"Mock function to generate subscribed"
[]
(gen/sample (s/gen ::Subscribed)))
#2018-02-1417:47Alex Miller (Clojure team)sorry, donāt have time to reply today but recommend: https://www.youtube.com/watch?v=WoFkhE92fqc as an intro#2018-02-1321:39misha@slipset if you will flow conformed values through merge's "components" - you will hit false-positive invalidation.#2018-02-1321:46mishacompare:
(s/def :foo/a (s/or :s string? :i int?))
(s/def :m/one (s/keys :req [:foo/a]))
(s/def :m/two (s/keys :req [:foo/a]))
(s/def :m/merged (s/merge :m/one :m/two))
;; not flowing conformed value
(let [m {:foo/a 1}]
(and ;; not accurate, but illustrates the idea
(s/conform :m/one m)
(s/conform :m/two m))) ;; => #:foo{:a [:i 1]}
;; flowing conformed value
(let [m {:foo/a 1}]
(->> m
(s/conform :m/one)
(s/conform :m/two))) ;; => :clojure.spec.alpha/invalid
#2018-02-1321:47misha(I hope @alexmiller will call me on my BS, if it's crazy talk)#2018-02-1408:27slipset@misha ha I don't quite think this is my use case. I might very well be misunderstanding here, but you're not using :m/merged
at all#2018-02-1408:29slipsetAnd I guess what surprises me here is that the order of how the specs are passed to s/merge
is important.#2018-02-1408:30slipsetI guess you could argue that since the order of how maps are passed to clojure.core/merge
is important, it follows that the order of the specs passed to s/merge
is also important, but I have a hard time accepting that š#2018-02-1408:40misha@slipset but you do accept clojure.core/merge, don't you? :)
(merge {:a 1} {:a 2}) ;;=> {:a 2}
(merge {:a 2} {:a 1}) ;;=> {:a 1}
#2018-02-1408:40slipsetabsolutely#2018-02-1408:41mishaif you s/merge s/keys with req/opt - order should not be an issue, but it might, if you merge req-un/opt-un#2018-02-1408:42slipsetWhich I find surprising, since this is not mentioned in the doc-string, and I don't see why there should be a difference.#2018-02-1408:43mishafirst s/keys will have :foo/a, seconds ā :bar/a, which will be conformed differently#2018-02-1408:44mishathen, conformed values will be merged, and the last one will win#2018-02-1408:47misha(s/def :foo/a (s/or :i int?))
(s/def :bar/a int?)
(s/def :foo/m (s/keys :req-un [:foo/a]))
(s/def :bar/m (s/keys :req-un [:bar/a]))
(s/conform (s/merge :foo/m :bar/m) {:a 1}) ;;=> {:a 1}
(s/conform (s/merge :bar/m :foo/m) {:a 1}) ;;=> {:a [:i 1]}
#2018-02-1410:24Jakub HolĆ½ (HolyJak)If only specs were data and I could use Spectre to transform them...#2018-02-1410:28mpenet@holyjak there are plans to improve that I think#2018-02-1410:28mpenetnothing specific was mentioned but they stated the intent a few times#2018-02-1410:34Jakub HolĆ½ (HolyJak)Crazy idea: perhaps I could only have a spec for the final order and, when validating for a non-final step of the checkout, I could just filter out exceptions for paths I know this step doesnt require yet. #2018-02-1410:39mpenetmy comment was about spec composition, not about the merge oddities#2018-02-1410:39mpenetfyi#2018-02-1410:39vikeriSo, if I run a spec with 100 tests and it fails, if I input the seed number it will only also fail if I run 100 tests? The seed doesnāt seem to be for the single test that fails since if I change num-tests to 1 it will usually pass. This decreases the usefulness of seed for functions that take large data structures and take some time to run.#2018-02-1413:32taylorseeding a RNG just makes it generate a certain sequence of values, it doesnāt mean itās going to magically find the exact value in the sequence that triggered some behavior and then always generate that number first. I assume the same logic applies to test.check generators?#2018-02-1413:33tayloranyway, when the check fails it spits out a minimal failing case that you could use in a āmanualā test#2018-02-1414:38Alex Miller (Clojure team) assuming you donāt use an external source of randomness in your generators , re-using the seed from a test should reproduce the identical generator values and the identical failure#2018-02-1414:40taylortrue, I think the disconnect is expecting the failing case to be re-genād first when using the same seed, instead of at whatever position it naturally occurs w/given seed#2018-02-1414:42Alex Miller (Clojure team)right#2018-02-1413:47misha@holyjak specs are sort of data, like everything in clojure opieop #2018-02-1413:48mishaNow it just comes down to how much pain verbosity and macros you can tolerate#2018-02-1413:48mishas/form#2018-02-1415:53mishais there a builtin predicate for int in Integer/MIN_VALUE, Integer/MAX_VALUE range?#2018-02-1416:10madstap@misha I think that's what int?
is#2018-02-1416:17mgrbyte(source int?)
- is a check against type (Long/Integer/Short/Byte), not bounds.#2018-02-1416:17misha@madstap int?
includes long
s as well#2018-02-1416:24mishaguess I'll use (s/int-in 0 (inc Integer/MAX_VALUE))
#2018-02-1416:26Alex Miller (Clojure team)isnāt that nat-int?
?#2018-02-1416:26Alex Miller (Clojure team)I guess nat-int? goes to Long/MAX_VALUE#2018-02-1416:30misha(nat-int? Long/MAX_VALUE) ;;=> true
(nat-int? (inc Long/MAX_VALUE)) ;;=> java.lang.ArithmeticException: integer overflow
#2018-02-1416:32Alex Miller (Clojure team)well the error is occurring on the inc there, not the nat-int?#2018-02-1416:32mishaLong/MAX_VALUE
=> 9223372036854775807
(nat-int? 9223372036854775808) ;;=> false
#2018-02-1416:32mishayes -_-'#2018-02-1416:34Alex Miller (Clojure team)if you really want to restrict to java Integer ranges, I think I would make custom predicates and custom specs that use those predicates + appropriate generators#2018-02-1416:35Alex Miller (Clojure team)but Clojure is not going to give you that as it does not intend to support them#2018-02-1416:38mishaI am going through kafka's config documentation and writing spec for it.
Sometimes they use int, [0, ...]
, and sometimes int, [0, ... 2147483647]
explicitly#2018-02-1417:09Alex Miller (Clojure team)ah#2018-02-1420:34firstclassfuncis there anyway to force say a large-integer
generator to produce unique values?#2018-02-1420:36taylor@firstclassfunc this might apply#2018-02-1420:36gfredericksyeah you have to specify it at the higher level, where you know the scope of uniqueness#2018-02-1420:37gfredericksotherwise you'll have trouble e.g., during shrinking when all your IDs become 0#2018-02-1420:38firstclassfuncyea i just want to produce a set of integers numbered 1-100 without duplicates#2018-02-1420:39gfredericks(gen/shuffle (range 100))
? (gen/set (gen/large-integer* {:min 1 :max 100}))
?#2018-02-1420:46firstclassfuncthanks @gfredericks not quite the shape I need it yet but appreciate it!#2018-02-1420:47gfrederickssometimes another useful approach is to just remove duplicates#2018-02-1420:48firstclassfuncyea that would help I am basically trying to create a common index across records e.g. (defrecord Subscriber [name id])
(defrecord Subscribed [article id])
(s/def ::name string?)
(s/def ::article uuid?)
(s/def ::id uuid?)
(s/def ::Subscriber
(s/keys :req-un [::name ::id]))
(s/def ::Subscribed
(s/keys :req-un [::article ::id]))
(defn generate-subscribers
"Mock function to generate subscribers"
[]
(gen/sample (s/gen ::Subscriber)))
(defn generate-subscribed
"Mock function to generate subscribed"
[]
(gen/sample (s/gen ::Subscribed)))
#2018-02-1420:49firstclassfuncI thought I would just narrow the scope of the generator for ::id to do it, but that has been more challenging.#2018-02-1420:54gfredericksif your IDs are UUIDs you shouldn't have any problems actually#2018-02-1420:59firstclassfuncthe problem is that I need the set of ids
to be the same across each record to form a relationship#2018-02-1421:01gfredericksah yeah that kind of thing takes more effort#2018-02-1615:44vikeriHmm, I canāt generate positive numbers in clojurescript. (s/exercise pos?)
doesnāt work. Any pointers?#2018-02-1616:20Alex Miller (Clojure team)Pos is not specific to a particular numeric type so does not have a mapped generator#2018-02-1616:20Alex Miller (Clojure team)pos-int? does#2018-02-1616:21Alex Miller (Clojure team)Or create a composite like (s/and double? pos?) where the initial type will gen#2018-02-1705:03bmaddyIs there a way to ensure two values are the same in a structure with spec? I didn't see anything in the spec docs (but maybe I missed it). I'd like to define a spec :account/by-id
that specs data like this:
{-5 {:db/id -5 :account/title "customer five"}}
Here's what I've got so far:
(s/def :db/id (s/or :db/id int? :uuid uuid?))
(s/def :account/title string?)
(s/def ::account (s/keys :req [:db/id :account/title]))
(s/def :account/by-id (s/map-of :db/id ::account))
but gen/generate
gives me stuff like this:
{7 {:db/id -5 :account/title "customer five"}}
#2018-02-1705:04bmaddyI'd like to ensure the :db/id
is the same as the associated key in the outer map.#2018-02-1705:42seancorfield@bmaddy look at s/and
to add a constraint#2018-02-1705:44seancorfield(s/def :account/by-id (s/and (s/map-of :db/id ::account) db-id-matches))
where db-id-matches
checks that the values match#2018-02-1802:39bmaddyThanks @seancorfield, that's what I was missing!#2018-02-1712:41gfredericksthen you'll have to modify the generator too, to just copy the id from the key spot to the attribute spot#2018-02-1716:15roklenarcicI'm trying to add clojure.spec.test.alpha/check
based testing to my clojure.test
tests, but the integration doesn't seem trivial. Do I have to write my own transform of the check
map to some sensible assert or is there a lib for that out already?#2018-02-1716:23gfredericksthat's asked a lot; I don't know if anybody's published anything. but in any case doing the integration manually should be just barely nontrivial#2018-02-1716:23roklenarcicI mean I can whip up something. I have to note that one thing that is very confusing is the :failure
key in the return map#2018-02-1716:24roklenarcicit comes back set to false#2018-02-1716:24roklenarcicon a failed check#2018-02-1716:24roklenarcicthis will probably confuse a lot of people#2018-02-1716:30gfredericksyes there's a ticket about that#2018-02-1716:45roklenarcicThere seems to be a curious mix of namespaces in the returned maps as well:
To get predicate that failed you need to get:
(-> (stest/check `fnsym)
:clojure.spec.test.check/ret
:result-data
:clojure.test.check.properties/error
(.getData)
:clojure.spec.alpha/problems
(get 0)
:pred)
Goes from spec
to test.check
back to spec
keys.#2018-02-1716:46gfredericksyeah, I think spec is currently embedding data in an ExceptionInfo object because test.check didn't used to have a mechanism for adding any info to a failure#2018-02-1716:46gfredericksI thought it was being unwrapped somehow though#2018-02-1721:40mathpunkI'm trying to do spec-and-test driven development in ClojureScript, and specs are not resolving like I expect them to. This test (https://github.com/mathpunk/sherman/blob/master/test/sherman/grammar_test.cljs#L13) passes if I uncomment the specification above it, but fails as is, with the specification in another namespace (https://github.com/mathpunk/sherman/blob/master/src/sherman/grammar.cljs#L7)#2018-02-1723:00seancorfield@mathpunk Your test namespace does not require
the namespace containing the spec -- how would it know about it?#2018-02-1723:12mathpunk@seancorfield thanks! Since the name of the spec has the namespace in it, I didn't realize the test namespace still needed to require it#2018-02-1723:12seancorfieldYou need to load the namespace for the s/def
to be executed, otherwise the spec isn't defined -- the keyword is just a keyword.#2018-02-1723:13seancorfieldAlso, spec names (keywords) have no connection to code namespaces (except insofar as the ::
alias resolution works).#2018-02-1723:19mathpunkwow! ok, I thought they were somehow being 'registered' in a way that needed to match the namespaces#2018-02-1723:23seancorfields/def
and s/fdef
are the functions that perform the registration and the spec name just needs to be as globally unique within your application as it needs to be in order to not clash with any other specs.#2018-02-1723:25seancorfieldNamespace-qualified keywords have been around for a long time before spec and have never been tied to code namespaces. For example, we use :ws.web/config
as a key in Ring requests in our applications for our application configuration map -- but we don't have a ws.web
namespace.#2018-02-1723:26seancorfieldYou can also set up aliases without needing code namespaces to match: (alias 'm (create-ns 'ws.domain.member))
(alias 'mu (create-ns 'ws.domain.member.update))
(alias 'mv (create-ns 'ws.domain.member.validation))
The first one is used for specs (`::m/username` for example), the other two are just used as part of unique keys in maps.#2018-02-1723:29seancorfield(it just so happens we do have code namespaces matching those, but where we use the aliases, we don't need the functions or specs from those namespaces so we don't require
the namespaces)#2018-02-1723:29seancorfield(hope that helps @mathpunk?)#2018-02-1723:33mathpunk@seancorfield Thank you, it does#2018-02-1801:30mathpunkI'm writing my first regular expression of specs. Both these tests should pass:#2018-02-1801:32mathpunkA :sherman.grammar/expanding-term
should contain ONE or more :sherman.grammar/expanding-symbol
s, and ZERO or more :sherman.grammar/terminating-symbol
s#2018-02-1801:32mathpunkI thought this might be it:#2018-02-1801:33mathpunkAt least, I thought that would handle "This is #going# #to# #expand# maybe"#2018-02-1801:33mathpunkwhich isn't fully what I'm after but would be partway#2018-02-1801:33taylorhave you tried s/alt
#2018-02-1801:33mathpunkI diiiid but not very well#2018-02-1801:35mathpunkCome to think of it I don't know that my input data is regular#2018-02-1801:36mathpunkI'm trying to express, "A string that, if you split it at the spaces, one or more of the elements of that collection would be an ::expanding-symbol
"#2018-02-1801:37mathpunkMaybe I should write a function that tests that, and uses it as the predicate#2018-02-1801:38mathpunkI wanted to check here to see if there is a way to alt
this intent#2018-02-1801:41taylorseems like s/cat
works for your case unless Iām missing something#2018-02-1801:43tayloroh wait, you want your inputs to be actual strings?#2018-02-1801:44noisesmiths/alt could spec the result of splitting the string at spaces#2018-02-1801:44taylorin that case, I think Iād tokenize the string before specāing it#2018-02-1801:53mathpunkGetting there. I'm incorrect about what s/+
means, though:#2018-02-1802:17mathpunkI've changed my terminology a little, but I'm still failing to reject invalid input:#2018-02-1802:21mathpunkI hypothesize that the right thing to do is to make functions do more work, and specs do less. E.g., much as noisesmith suggested when they suggested tokenizing first, I could have a predicate which detects well-formed tokens, and base a spec off of that#2018-02-1802:22mathpunkI'm still working out what responsibilities belong in spec and what belong in plain ol' clojure code š#2018-02-1803:34taylor@mathpunk another option using s/&
:
(s/def ::valid-term
(s/& (s/* (s/alt :terminating ::terminating-symbol
:expanding ::expanding-symbol))
#(some (comp #{:expanding} first) %)))
(s/conform ::valid-term (tokenize "This #expands# also"))
=> [[:terminating "This"] [:expanding "#expands#"] [:terminating "also"]]
#2018-02-1803:46seancorfieldI'll channel Alex Miller and say clojure.spec
is not intended for string parsing -- there are much better tools out that for that (e.g., instaparse
).#2018-02-1803:47seancorfieldThe "regular expression" part of spec is intended for sequence processing.#2018-02-1803:48seancorfield^ @mathpunk#2018-02-1803:58mathpunkI've wondered exactly this!: whether instaparse was obsolete after spec#2018-02-1803:59aengelbergIt is not#2018-02-1803:59aengelbergCome join us#2018-02-1803:59aengelbergWe have cookies#2018-02-1803:59aengelbergAnd somewhat lackluster but existent respond rate to issues#2018-02-1804:16bmaddyI would expect clojure.spec.alpha/and
to be commutative. Is anyone else seeing this behavior?
> *clojurescript-version*
"\"1.9.946\""
> (defn f [m] (println m) (-> m (get 1) :db/id (= 1)))
"#'invest-calc.state/f"
> (s/def :account/works (s/and f (s/map-of any? (s/keys))))
":account/works"
> (s/def :account/fails (s/and (s/map-of any? (s/keys)) f))
":account/fails"
> (s/conform :account/works {1 {:db/id 1}})
{1 #:db{:id 1}}
"{1 #:db{:id [:db/id 1]}}"
> (s/conform :account/fails {1 {:db/id 1}})
{1 #:db{:id [:db/id 1]}}
":cljs.spec.alpha/invalid"
#2018-02-1804:27Alex Miller (Clojure team)s/and flows conformed results through predicates#2018-02-1804:47bmaddyHeh, I guess I should read the docs closer. Is it expected that it would operate differently in normal Clojure?
invest-calc.state> *clojure-version*
{:major 1, :minor 9, :incremental 0, :qualifier nil}
invest-calc.state> (defn f [m] (println m) (-> m (get 1) :db/id (= 1)))
#'invest-calc.state/f
invest-calc.state> (s/def :account/works (s/and f (s/map-of any? (s/keys))))
:account/works
invest-calc.state> (s/def :account/fails (s/and (s/map-of any? (s/keys)) f))
:account/fails
invest-calc.state> (s/conform :account/works {1 {:db/id 1}})
{1 #:db{:id 1}}
{1 #:db{:id 1}}
invest-calc.state> (s/conform :account/fails {1 {:db/id 1}})
{1 #:db{:id 1}}
{1 #:db{:id 1}}
#2018-02-1901:15arrdemIs there a spec around for the fn
form?#2018-02-1901:21arrdemHummmm I'm going about this wrong methinks š#2018-02-1901:27arrdemAh clojure.core/fn
is a spec ID.#2018-02-1906:32ikitommithe new spec-tools (`0.6.1`) has a version of merge that retains the already conformed values:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])
(s/def ::a (s/and (s/conformer str) string?))
(s/def ::b int?)
(s/def ::am (s/keys :opt-un [::a]))
(s/def ::bm (s/keys :opt-un [::b]))
(s/conform (s/merge ::am ::bm) {:a 1, :b 1}) ; {:a 1, :b 1}
(s/conform (s/merge ::bm ::am) {:a 1, :b 1}) ; {:a "1", :b 1}
(s/conform (st/merge ::am ::bm) {:a 1, :b 1}) ; {:a "1", :b 1}
(s/conform (st/merge ::bm ::am) {:a 1, :b 1}) ; {:a "1", :b 1}
#2018-02-1912:34igosuki@ikitommi Do you know if there is any way to conform two key aliases into one ?
problematic example :
(s/def ::user
(s/keys [(s/or ::name1 ::name2)]))
Spec verifies for me already that itās one of the other, subsequently, I donāt want to have to write (or (:name1 m) (:name2 m)) in all of my code
Is there anyway to coerce these keys into another one ?#2018-02-1912:37mpenetsyntax is wrong, I don't think you should use a ns on the or, it should read (or ::foo ::bar)#2018-02-1912:37mpenetafaik#2018-02-1912:37mpenet@igosuki I dont think you can no, you need to use eval for that, or a macro#2018-02-1912:38igosukiyeah sorry I wrote the ns by reflex, but actually am using just a plain or#2018-02-1912:41igosukiItās annoying to have a declarative syntax for validation, and then have to write custom code for something that trivial thoughā¦
something like (alias ::name1 ::name2) would be nice, equivalent :
import com.fasterxml.jackson.annotation.{JsonIgnoreProperties}
case class User(@JsonAlias(Array("name1", "name2") name))
#2018-02-1915:01mpenetit would really be handy to have specs for specs, I occasionally write (s/def foo ...) or (s/fdef ::foo ...) and it's far from obvious to trace when this happens#2018-02-1915:02mpenetyou get an Illegal arg exception in one case but the error is a bit cryptic:#2018-02-1915:02mpenet:type java.lang.IllegalArgumentException
:message Invalid match arg: /cdn-cgi/l/email-protection#2018-02-1917:06EdI'm importing something from some json and would like to use the namespaced map in the rest of the program#2018-02-1918:32misha@igosuki (s/def ::my-alias (s/nonconforming (s/or :n1 ::name1 ::name2)))
#2018-02-1918:33mishaunless I misunderstood the question#2018-02-1921:24misha@igosuki
(clojure.set/rename-keys {:name1 1 :bar 2} {:name1 :name :name2 :name})
=> {:bar 2, :name 1}
#2018-02-1921:26mishayou can do it pre- or post- spec, and your further convenience-while-working-with-data-structure is probably irrelevant to spec#2018-02-2009:31roklenarcicHm I have a spec that limits string length to be between 0 and 10 characters. As I ask generate to generate a larger sample size, it starts to throw that it failed to satisfy such-that. What's the best way to deal with that?#2018-02-2013:20gfredericksyou have to supply your own generator; (gen/map #(apply str %) (gen/vector gen/char-ascii {:max-elements 10}))
or something like that#2018-02-2013:42roklenarcictest.chuck has a cap-size
function that works great for this#2018-02-2014:03igosuki@misha faire enough, thanks for the heads up#2018-02-2017:37gfredericks@roklenarcic strictly speaking that's capping the size
parameter, not the size of the collection. But they probably happen to coincide in this case.#2018-02-2101:02richiardiandrearegarding multi-arity functions, can I spec a 4-arity fn this way?
(s/fdef select-events-sqlmap
:args (s/cat :domain :event/domain
:entity :event/entity
:key (s/alt :arity-2 nil? :arity-3-4 :event/key)
:version (s/alt :arity-2-3 nil? :arity-4 :event/version)))
It seems not to work when I pass three args#2018-02-2101:04richiardiandreaif there is a more idiomatic way I am open to it, this seems quite difficult to read#2018-02-2101:16Alex Miller (Clojure team)(s/fdef select-events-sqlmap
:args (s/cat :domain :event/domain
:entity :event/entity
:arity3-4 (s/? (s/cat :key :event/key :version (s/? :event/version)))))
#2018-02-2101:39richiardiandreaThanks a lot I will try this out!#2018-02-2117:38richiardiandreatried this, but it seems to fail when :version
is nil
, will play more with it#2018-02-2117:49Alex Miller (Clojure team)you can wrap (s/nilable <spec>)
around a spec to also allow for nil#2018-02-2117:57richiardiandreaoh nice#2018-02-2118:03richiardiandreathis is what works:
(s/fdef select-events-sqlmap
:args (s/cat :domain :event/domain
:entity :event/entity
:arity3-4 (s/? (s/cat :key (s/nilable :event/key)
:version (s/? (s/nilable :event/version))))))
#2018-02-2101:16Alex Miller (Clojure team)is one way#2018-02-2101:17Alex Miller (Clojure team)you can also s/alt at the top for the 2, 3, and 4 cases if you want a single map with the same keys conformed out#2018-02-2117:35richiardiandreas/alt
only works with keywords so does that mean that I need to break the spec in three?#2018-02-2117:49Alex Miller (Clojure team)s/alt works with any spec - keywords are used to tag the alternatives#2018-02-2108:52hawariCan I make a spec definition that depends on another spec like:
(spec/def ::start-time
(spec/and string? valid-datetime?))
(spec/def ::end-time
(spec/and string? valid-datetime?)) ;; need to validate that it's greater than start-time
(spec/def an-entity
(spec/keys :req-un [::start-time ::end-time]))
#2018-02-2109:07moxaj@hawari.rahman17 you should do that validation in the ::an-entity
spec#2018-02-2109:11hawariCan you suggest on how can that be achieved? My apologies, I've just recently started using spec, I don't fully grasp its concept yet.
Do I use spec/and
and pair the spec/keys
with another predicate which validate that start-time is greater than end-time?#2018-02-2109:12hawariIf so, how can one destructure a map from within a predicate? Does it get treated as an argument to a spec? @moxaj#2018-02-2109:15moxaj(spec/and (spec/keys ..) (fn [{:keys [start-time end-time]}] ...))
#2018-02-2109:15moxajyour predicate is just a regular function#2018-02-2109:16hawariAh great, I'll try it right away, thank you very much for your suggestion @moxaj!#2018-02-2113:55yogidevbearHi everyone. I'm a spec n00b so please have patience with my question here. Can/should I use spec for validating user input passed into my functions? For example, if I have a function that should accept positive integer values only and those value might need to be within a particular range, would spec work for this (and possibly what might the general structure of code look like)?#2018-02-2114:03taylorI wrote some examples that might help https://taylorwood.github.io/2017/10/15/fspec.html#2018-02-2114:04yogidevbearThanks Talyor :+1: I'll check that out now#2018-02-2114:52Alex Miller (Clojure team)also check out s/int-in for your range spec#2018-02-2114:02OlicalConsidering it makes a very very good parser for all sorts of data structures, that'd probably be a fine use case. Since the result is in itself more data, you could add more constraints in the future then build your error messages from that output.#2018-02-2114:02OlicalAlso, by user input do you mean an actual form on a page, or as an API?#2018-02-2114:03yogidevbearAt the moment, user input from a REPL.#2018-02-2114:04yogidevbearvia running lein repl and the calling the functions from the REPL#2018-02-2114:05OlicalAh, so you're talking runtime, not dev time, which is a big difference. At dev time you can just define your function specs and the user can decide to enable instrumentation (in my opinion). At runtime you'll be able to run your inputs through conform and respond with an informative error if it doesn't fit.#2018-02-2114:05OlicalAh, so you're talking runtime, not dev time, which is a big difference. At dev time you can just define your function specs and the user can decide to enable instrumentation (in my opinion). At runtime you'll be able to run your inputs through conform and respond with an informative error if it doesn't fit.#2018-02-2114:07yogidevbearCan you explain what you mean by
>the user can decide to enable instrumentation#2018-02-2114:08OlicalSure! So, if you define specs for all of your functions with s/fdef, nothing actually happens at runtime. It's just sort of there, like documentation.#2018-02-2114:08OlicalThe user, or developer, can decide to instrument a namespace / all namespaces so that these fdef specs are checked.#2018-02-2114:09OlicalIn normal spec, this only means the arguments being passed to functions are checked, but you can use something like Orchestra to check the return values too.#2018-02-2114:10OlicalYou may find https://github.com/bhb/expound interesting, it makes spec errors human readable at a glance. And here's https://github.com/jeaye/orchestra#2018-02-2114:10yogidevbearCool, thanks Oliver š#2018-02-2114:11OlicalNo problem. I think you'll want to use https://clojuredocs.org/clojure.spec.alpha/conform on your data and a spec that checks your int is in range. For which you can use https://clojuredocs.org/clojure.spec.alpha/int-in-range_q#2018-02-2114:12yogidevbearPerfection#2018-02-2114:19Olicaluser=> (s/conform #(s/int-in-range? 5 10 %) 2)
:clojure.spec.alpha/invalid
user=> (s/explain #(s/int-in-range? 5 10 %) 2)
val: 2 fails predicate: :clojure.spec.alpha/unknown
#2018-02-2114:21OlicalYou can use (s/def ::my-num #(s/int-in-range? 5 10 %))
then (s/explain ::my-num 2)
gives you a more descriptive error too. Not sure how much spec you know though, so sorry if I'm explaining too much š
#2018-02-2114:23yogidevbearAt present, I know about 2 hours worth š#2018-02-2114:23yogidevbearSo all the explanation is very welcome#2018-02-2114:27yogidevbearAnd if I'm using clojure 1.9.0, do I simply add this to my ns?
(ns my-ns.core
(:require [clojure.spec.alpha :as s]))
#2018-02-2114:29OlicalIt should be present anyway, it's included as part of Clojure. No need to depend on it afaik#2018-02-2114:29OlicalBut yes, that require line is fine.#2018-02-2114:30OlicalAnd you can use cljs.spec.alpha in cljs#2018-02-2114:31taylor> Clojure 1.9 depends on this library and provides it to users of Clojure. Thus, the recommended way to use this library is to add a dependency on the latest version of Clojure 1.9, rather than including it directly#2018-02-2114:31yogidevbearYeah, it seemed to be throwing some error on my spec function without needing to depend so looks like 1.9.0 includes it by defaut#2018-02-2114:39yogidevbearI think I've got a spec working without throwing an error when I run lein repl
. Excuse the trivial function example:
(defn my-fn
"An example function. Takes two arguments and makes a vector from them"
[a b]
(into [] (list a b)))
(s/fdef my-fn
:args (s/cat :a #(s/int-in-range? 1 11 %)
:b #(< 10 %)))
#2018-02-2114:40yogidevbearDoes that seem correct?#2018-02-2114:41OlicalLooks right to me, yep!#2018-02-2114:41OlicalYou can put the fdef before the defn if you want to btw#2018-02-2114:41OlicalAnd to get clojure to check this you have to use spec tools instrument or something like orchestra#2018-02-2114:42OlicalSo these are more of a developer tool thing, if you want to check things at runtime that you present to the user, you probably want s/conform and s/explain#2018-02-2114:42yogidevbearSo my next question then... if I run (my-fn 1 2)
within the REPL, it works instead of throwing an error#2018-02-2114:43yogidevbearIs this where the spec tooling you're referring to would come into play?#2018-02-2114:43OlicalYep, this is where you have to instrument#2018-02-2114:43OlicalYou don't want spec checking EVERY function call in prod#2018-02-2114:43OlicalSo it's opt in#2018-02-2114:43yogidevbearAh, that makes a lot of sense š#2018-02-2114:45OlicalCheck out orchestra though, it has some really neat tooling https://github.com/jeaye/orchestra#defn-spec#2018-02-2115:52yogidevbear@U38J3881W any ideas why I might be getting Could not locate orchestra/core__init.class or orchestra/core.clj on classpath
when running lein repl
?
I have [orchestra "0.2.0"]
in my dependencies and (:require [orchestra.core :refer [defn-spec]])
in my ns declaration#2018-02-2115:55OlicalOoh, odd. Maybe it's a doc thing? I suppose it's not found in [orchestra.spec.test :as st]
?#2018-02-2115:55OlicalAlso, maybe the docs are wrong in another way, try [orchestra "2017.11.12-1"]
, that one seems to be for newer versions of Clojure.#2018-02-2115:55yogidevbearHmm, not sure. They don't seem to include orchestra.spec.test in their defn-spec example on that readme#2018-02-2115:56yogidevbearIsn't that version the cljs version?#2018-02-2115:56Olical0.2.0 says it's for "1.9.0 >= Clojure < 1.9.0-alpha16"#2018-02-2115:56yogidevbearI'm on 1.9.0#2018-02-2115:56yogidevbearLet me try the other version and see#2018-02-2115:57OlicalYeah, I think you want the other one, it's for CLJS and CLJ š#2018-02-2115:57Olical0.2.0 seems to be legacy for an older style of spec.#2018-02-2115:58yogidevbearThanks, that seems to have fixed it#2018-02-2116:00OlicalNice! I hope it's all working well. Tried it with expound yet? Definitely makes the errors more palatable.#2018-02-2116:01yogidevbearNot yet#2018-02-2116:44viestihum#2018-02-2117:40bfabryfollowing on from hawari's example above, what's the best way to attach a generator to the compound spec that generates end-time by adding some random amount to start-time. assume the spec/keys call involves a lot of keys all the things that I'm coming up with have a lot of boilerplate, but I probably just don't understand test.check very well#2018-02-2117:43gfrederickswhat do you have so far?#2018-02-2117:56bfabrywell I think I just ran into some unrelated cljs macro problems#2018-02-2117:56bfabrybut ignoring those#2018-02-2117:57bfabrycljs.user=> (s/def ::call' (s/keys :req-un [::direction ::duration ::agent_id ::group_id ::id ::start_time ::end_time]))
cljs.user=> (def call-gen #(gen/let [e (s/gen ::call')] (assoc e :end_time (+ (:start_time e) (rand-int 3600)))))
(s/def ::call (s/and (s/with-gen ::call' call-gen) #(> (:end_time %) (:start_time %))))
#2018-02-2118:47ghadihttps://dev.clojure.org/jira/browse/CLJ-2197
I just ran into this :(((#2018-02-2119:13johanatanhi, does spec have support for heterogeneous maps? in particular i want to validate a map like this: {:a-special-kwd [:some :elements] "string" ::some-structure "string2" ::some-structure ... }
#2018-02-2119:52Alex Miller (Clojure team)http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2018-02-2119:52Alex Miller (Clojure team)hybrid map example#2018-02-2119:16johanatani suppose i could do: #(s/and (valid-special? %1) (s/valid? (s/map-of string? ::some-structure) (dissoc %1 :a-special-kwd)))
?#2018-02-2119:16johanatanis there a better way?#2018-02-2119:23johanatani suppose i was trying to avoid having lambdas which explicitly call valid?
though (if there is a way)#2018-02-2119:35aaron51Hello! How do we check return values using fdef
and stest/instrument
? I see you can add the :ret
key, but it doesn't seem to fail when it should.
Also can stest/instrument
check all fdef
'd functions (instead of having to list them all)?#2018-02-2119:36taylorstest/instrument
doesnāt check :ret
specs, only :args
AFAIK#2018-02-2119:36johanatanstest/check
does however#2018-02-2119:36taylorand calling (stest/instrument)
with no args should instrument everything thatās been loaded#2018-02-2119:37taylorhttps://clojure.org/guides/spec#_combining_code_check_code_and_code_instrument_code#2018-02-2119:40aaron51Any way to check ret
without auto generating tests with check
?
We're unit testing and trying to use specs on the return value#2018-02-2119:41taylorI think someone has written some utility code for doing that, canāt remember which lib itās in#2018-02-2119:41taylorhttps://github.com/jeaye/orchestra#2018-02-2119:41taylorpretty sure thatās it āļø#2018-02-2119:42aaron51We were looking for a way to continue using defn
, instead of defn-spec
, because defn-spec
confuses Cursive#2018-02-2119:44taylorI donāt think youāre required to use defn-spec
with orchestra#2018-02-2119:45taylori.e. just using orchestraās instrument
seems like it will do :ret
checking on plainā ol defn
s and s/fdef
s (hopefully?)#2018-02-2119:47aaron51oh wow, that's exactly the answer I needed. We were fighting with Orchestra and Cursive for a while#2018-02-2119:47aaron51thanks!#2018-02-2119:48taylorFWIW (and it probably wouldnāt help you here) but Cursive has a feature where you can tell it to treat custom macros like the core/known macros, assuming the custom macro follows one of those known patterns#2018-02-2123:11aaron51oh, we tried that. It partially worked, but then jumping between tests and implementation broke. Seems like the feature we want but it isn't quite fully baked yet#2018-02-2123:11aaron51thanks again#2018-02-2119:39gfredericks@bfabry ignoring the unrelated issues with using rand-int
, I think the boilerplate you have is necessary given the current API
maybe a helpful thing would be a variant of s/and
that lets you modify the default generator of the first argument#2018-02-2119:40bfabry@johanatan I think you're looking for s/keys#2018-02-2119:40johanatan@bfabry no, that won't work. the "strings" (string1, string2) are dynamic/unknown#2018-02-2119:40bfabry@gfredericks what are the issues with using rant-int and what should I be doing instead?#2018-02-2119:41bfabryoh sorry I didn't realise you meant heterogeneous keys as well#2018-02-2119:41johanatanyes, heterogeneous#2018-02-2119:48gfredericksusing your own randomness instead of a generator means you lose growth, reproducibility and shrinking; one way to address that is to add a temporary key to ::call'
that maps to something that generates the sort of nonnegative integer you want, and then remove that key in ::call
#2018-02-2119:53bfabrycan I just use duration (s/gen (s/int-in 1 3600))
in the gen/let and then use that?#2018-02-2119:55gfredericksoh yeah that's a good idea#2018-02-2119:56bfabryok cool, slowly understanding#2018-02-2120:00gfrederickscould be helpful: https://www.youtube.com/watch?v=F4VZPxLZUdA&t=1625s#2018-02-2122:30bfabrythanks!#2018-02-2312:01otfromdumb question of the day: can you add a fsepc to a defmethod?
(I remember reading somewhere you couldn't but I think I'm misremembering and google has been failing me)#2018-02-2312:47mpenetYou can't. you can spec the dispatch fn or/and create separate defns that will be called by the multimethod impl#2018-02-2313:27otfrom@mpenet thx!#2018-02-2313:28otfromI hadn't thought about specing the dispatch function but it is obvious now that you point it out#2018-02-2313:29mpenet@alexmiller pointed that out the other day, didn't think of it either before then š#2018-02-2313:29Alex Miller (Clojure team)That covers the args side at least#2018-02-2313:29otfromman, he should write a book#2018-02-2313:30otfromoh, sorry didn't see you there @alexmiller;-)#2018-02-2313:31otfrom@alexmiller I was a bit surprised yesterday and today w/instrumenting an fspec. It looks like it calls gen/check on the way in which I wasn't expecting (21 times if I'm reading correctly)#2018-02-2313:32otfromwe got caught out b/c we had a spec that was wider than the code that was being called#2018-02-2313:32Alex Miller (Clojure team)Yeah, you are not the first#2018-02-2313:32otfromš#2018-02-2319:16jjttjjIs leaning on spec's conform/unform to do translations to and from some outside api an intended and/or acceptable use case? IE if the api gives dates and integers as strings and i need to convert them to java dates and real ints when the come in, and convert them back to strings when they go back out to the api, and there are a lot of varied translations like this in a nested structure. I'm thinking I spec all the incoming string data with conformers, then additionally spec out all my own representations as normal. Is there a reason I'm missing not to do this?#2018-02-2319:58Alex Miller (Clojure team)This recent thread is probably a good starting point https://groups.google.com/d/msg/clojure/Tdb3ksDeVnU/LAdniiZNAgAJ#2018-02-2320:08tayloragree that cautioning against āabusingā conformers should be documented#2018-02-2320:31seancorfieldI would have sent a PR for the site repo -- but conformer
isn't even mentioned anywhere there. So I think it would have to be a docstring update for the function itself? @alexmiller thoughts?#2018-02-2321:03Alex Miller (Clojure team)Itās intentionally not mentioned as anti documentation :)#2018-02-2321:04Alex Miller (Clojure team)but maybe it should be more explicitly unmentioned :)#2018-02-2320:02jjttjjthank you#2018-02-2408:54Peter WilkinsHi - I'm learning spec. I've googled and doc'ed but can't find a clue how to spec the sum of all numbers in a collection (a map) in this case (s/def ::percentage (s/and double? #(>= % 0) #(<= % 100)))
(s/def ::scores (s/map-of keyword? ::percentage))
(s/valid? ::percentage 50.05)
(s/valid? ::scores {:a 70.49 :b 20.21 :c 9.29})
I'd like to confirm that the sum of ::scores = 100#2018-02-2409:54robert-stuttaford#(= 100 (apply + (vals %)))
surely?#2018-02-2410:04yogidevbearDoes anyone here have an open source repo where they're using spec? I feel like I'm missing something fundamental about spec and I'm hoping that looking at a few good examples of it being used within a project might help solidify some of the core concepts for me.#2018-02-2410:24jannis@yogidevbear We wrote https://github.com/functionalfoundry/entitydb some time ago. Starting from workflo.entitydb.core/empty-db
, most functions and input/output data have specs to allow random testing and integrity checking.#2018-02-2410:26jannisSomething less complex perhaps: specs for the Om Next query language: https://github.com/functionalfoundry/macros/blob/master/src/main/workflo/macros/specs/om_query.cljc#2018-02-2410:55Peter Wilkins@robert-stuttaford sorry I worded the question badly. I can't figure out how to compose a spec using map-of and #(= 100 (apply + (vals %)))
. (s/def ::scores (s/and (s/map-of keyword? ::percentage) #(= 100 (apply + (vals %)))))
doesn't work. Also, wouldn't we want to be the kind of community to support and encourage beginners, surely?#2018-02-2412:48luskwater@poppetew (defn approx=
"Return a fn that checks that a value is approximately `n` (within `e`)"
[n e]
#(-> (apply + (vals %))
(- n)
(Math/abs)
(< e)))
(s/def ::percentage (s/and double? #(>= % 0) #(<= % 100)))
(s/def ::scores
(s/and (s/map-of keyword? ::percentage)
(approx= 100 0.1)))
(s/def ::scores-restrictive
(s/and (s/map-of keyword? ::percentage)
(approx= 100 0.001)))
(s/valid? ::percentage 50.05)
(def the-map {:a 70.49 :b 20.21 :c 9.29})
(apply + (vals the-map))
(s/valid? ::scores the-map)
(s/valid? ::scores-restrictive the-map)
#2018-02-2412:52luskwaterActually, @poppetew, your (s/and ,,, #(= 100 (apply + (vals %))))
would work, if you were using integers. Thatās why we needed the approx=
fn up there.#2018-02-2412:53luskwaterYour e
may vary, of course. š#2018-02-2413:00luskwater(def the-map {:a 70.49, :b 20.21, :c 9.29})
=> #'cognitect.transcriptor.t_1/the-map
(apply + (vals the-map))
=> 99.98999999999998
(s/valid? :cognitect.transcriptor.t_1/scores the-map)
=> true
(s/valid? :cognitect.transcriptor.t_1/scores-restrictive the-map)
=> false
#2018-02-2417:03robert-stuttaford@poppetew i apologise if my rapidly typed response wasnāt suitably welcoming for you - i figured something would be better than nothing. i recognise now that it could have come across as condescending - not my intent! looks like @luskwater provided a far more considered response, which has taught me something!#2018-02-2417:03robert-stuttaford@poppetew i am incredibly supportive of beginners š#2018-02-2417:38luskwaterWouldn't have had an answer for @poppetew so quickly, had it not been for @robert-stuttaford and his initial suggestion. Collaboration across timezones takes a little time itself. #2018-02-2514:39Petrus TheronHaving a hard time with Spec performance on runaway regular expressions for a simple lexer/parser to lex a string into tokens before I parse it. I'm trying to parse a string into tokens so I can put it through a second "compiler" pass:
(s/def ::plexpr*
(s/+
(s/alt
:whitespace (s/+ #{\space \tab \formfeed \backspace})
:line-sep (s/+ #{\newline \return \formfeed}) ;(s/+ line-separator?)
:atom (s/+ (set "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890")))))
(s/def ::plexpr
(s/and
(s/conformer seq)
::plexpr*))
(s/conform ::plexpr "drive home fast") ;; => (fast), [[:atom [\d \r \i \v \e]] [[:whitespace [\space]] [:atom [\h \o \m \e]] [:whitespace [\space]] [:atom [\f \a \s \t]]]]
(s/conform ::plexpr "drive home fast but now it gets slow") ;; => extremely slow
I'm not using any optional s/? regexes, so I don't understand why this would cause any kind of exponential search#2018-02-2518:59seancorfield@petrus clojure.spec
is not intended for string parsing -- you know that, right?#2018-02-2518:59seancorfieldThe regexes are for "parsing" sequences, not strings. You probably want to look at Instaparse which is designed for parsing strings.#2018-02-2607:46Petrus TheronThanks - a string is a sequence, no? š I can switch to instaparse for tokenising, but I'm worried about the length of data sequences I'll be able to handle with Spec#2018-02-2617:54seancorfieldA string can be coerced to a sequence š I saw @U0516PHE3 mention that your spec is an ambiguous grammar and that might be causing the exponential slow down. As for data sequences, I would expect most folks would use coll-of
or every
and the elements would be self-contained specs (so, no ambiguity).#2018-02-2619:54Petrus TheronHow can I make my spec unambiguous? If I split my string into a word tokens, can I still use spec and coll-of/every to parse the substrings? Or do I need an instaparse step before I send it to spec?#2018-02-2620:04seancorfieldRight now "aa"
can parse as "atom (a)" "atom (a)" or "atom (aa)" so you could specify that the overall sequence is "atom" then a sequence of "whitespace" and "atom" perhaps? (assuming you trim leading/trailing whitespace before parsing?)#2018-02-2620:05seancorfieldIf you require whitespace separating the atoms, then "aa"
can only parse as "atom (aa)"#2018-02-2620:07seancorfieldYou'll want to make sure whitespace and line endings don't overlap too (so there's no ambiguity there) and if you care about the difference have your overall spec clearly indicate line / sep / line structure and line as a spec indicate atom / whitespace / atom structure. Make sense?#2018-02-2519:02gfredericksit's still a surprising slowdown, I would say#2018-02-2519:02gfredericks(and I did say it, in fact)#2018-02-2519:08seancorfieldI'm kinda surprised even the 15-element string is handled "fast". That would be long for an argument list so I'd say it was outside the intended design space?#2018-02-2519:11gfredericksthe spec regexes are only meant for arglists?#2018-02-2519:11seancorfieldProcessing "drive"
takes 3ms, "drive home"
takes 72ms, "drive home fast"
takes almost 1 second. That doesn't feel "fast" to me.#2018-02-2519:12gfredericksI coincidentally just wrote code that uses spec to parse the lines of a 17277-line file; it took 0.6s#2018-02-2519:13seancorfield@gfredericks Not just arglists but I don't think folks are spec'ing long sequences in data...? And data is more likely to get spec'd with coll-of
or every
I think?#2018-02-2519:13ghadiI wonder why that example appears to show exponential behavior. What are the bounds on parsing with derivatives?#2018-02-2519:13gfredericksI'm using regexes because I actually want the parsing capabilities, and I think this would be too hard with instaparse#2018-02-2519:13ghadiThere was an updated paper Matt Might released that improved performance.#2018-02-2519:14ghadiEven if that example above isn't a bug -- it's a valuable example for spec's test suite.#2018-02-2519:14ghadi(... but it's probably a bug)#2018-02-2519:16gfredericksyeah I can't imagine any good reason for something that simple to take a long time#2018-02-2519:16seancorfieldI got an OOM error trying to parse that long string š#2018-02-2519:16ghadiIs the performance still bad if you feed it the seq directly and remove the s/and & the conformer?#2018-02-2519:22seancorfieldAdding each new word seems to double the time it takes. I think there's a clue in how it conforms -- for one word you get a sequence of one element which is a sequence for the atom: (time (s/conform ::plexpr "one"))
=> [[:atom ["o" "n" "e"]]]
(in just a ms or two), but (time (s/conform ::plexpr "one two"))
=> [[:atom ["o" "n" "e"]] [[:whitespace [" "]] [:atom ["t" "w" "o"]]]]
-- note: two elements, with the second containing the whitespace and the next word. Adding more words still produces a two-element sequence with the second one containing all the extra elements.#2018-02-2519:34aengelbergYou could change the set predicates to functions that print some debug information. That would give you an idea of what things the spec engine is unnecessarily repeating.#2018-02-2519:38aengelbergI think that parser is ambiguous, because a string of "aa" could match [:atom ["a"]] [:atom ["a"]]
or [:atom ["a" "a"]]
. That could explain the exponential growth.#2018-02-2519:57gfredericksooh good point#2018-02-2523:59devnHow does one access a spec within the registry's req/req-un/opt/opt-un info?#2018-02-2523:59devnI'd like to peek at a spec's list of req/req-un/opt/opt-un#2018-02-2600:00devnare s/describe
or s/form
what I want to use here, or am I missing something obvious?#2018-02-2600:01devnI'd like to peek at req
and opt
to pass them into a pull query#2018-02-2600:30seancorfield@devn https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/registry ?#2018-02-2600:51devn@seancorfield that'll get me the spec objects by name, but not in a way that's easy to pluck out the req/opt#2018-02-2600:52devni have a feeling im just missing something obvious, maybe in how i define the :req to begin with. Is it possible for me to use a spec as the arg to :req
I wonder#2018-02-2600:53seancorfieldYeah, right now you'd have to get the form for each spec and then walk them.#2018-02-2600:53seancorfieldSince the keys
part could be embedded deep inside a spec...#2018-02-2600:53gfredericksdoesn't somebody mention a yet-to-exist spec spec every time this comes up?#2018-02-2600:54devnsomething loosely like:
(s/def ::the-keys [::a ::b])
(s/def ::the-map :req ::the-keys)
#2018-02-2600:55seancorfieldAlex says they're looking at ways to make it easier to build specs programmatically but it's not there yet.#2018-02-2600:55devnerr, well, when it's a set, I can at least do (s/form ::the-keys)
and get back #{::a ::b}
, it's just that I can't use it in the s/keys
form#2018-02-2600:56seancorfieldclojure.spec
is a wonderful concrete example of how macros don't compose š#2018-02-2601:02devnim not complaining. i think there's probably a solution im missing here, let me try to be a bit more clear on what i want to do.
I have a spec which says that ::a
and ::b
are required.
I need some value holder that can be used both in the spec definition: (s/keys :req ...)
and a pull query (d/q '{:find [(pull ?blah ?the-qualified-keys) :in [$ ?the-qualified-keys]] ...} db the-qualified-keys)
#2018-02-2601:05devnerr, i could relax that second part and say I'm not sure if that's possible. if it isn't, what kind of clever solutions are there? I suppose I could generate the spec and a corresponding def, for instance, so i don't have to manually keep modifications to either place consistent.#2018-02-2601:11seancorfieldWhat we've done at work is to define a spec with keys, build any more complex specs on top of that, then get the spec form itself, parse out the :req [...]
part and pull the keys from that to use elsewhere. So the base spec was the "system of record" and we derived other stuff from it.#2018-02-2601:11seancorfieldIt's kind of icky but it works, given what we have available right now. It seems like the path of least resistance.#2018-02-2601:13devnmaybe (s/describe)
is good enough for what i want to do here, the structure of it just felt a little wonky to me: (keys :req [:foo/a :foo/b])
, so i need to do something like (apply hash-map (rest (s/describe ::the-map)))
or whatever#2018-02-2601:19seancorfieldHmm, we use s/form
... what is s/describe
?#2018-02-2601:19devnroughly the same deal#2018-02-2601:19seancorfieldAh, an "abbreviated" form as data...#2018-02-2601:20devnform: (clojure.spec.alpha/keys :req [:foo/a :foo/b])
describe: (keys :req [:foo/a :foo/b])
#2018-02-2601:21devni'm surprised it's not a map, but perhaps there are Reasonsā¢ why that I don't understand#2018-02-2601:21gfredericksgetting a map isn't hard though: (apply hash-map (rest thing))
#2018-02-2601:31devnfair point#2018-02-2601:31devnit gave me that "is this a hint that i shouldn't be using it like this?" feeling#2018-02-2601:37gfredericksprobably just a hint that it's not done yet#2018-02-2613:26mpenet@alexmiller any details/insight about how could spec meta data look like in the future?#2018-02-2613:39Alex Miller (Clojure team)No#2018-02-2613:26mpenetanother question while I am at it: was it ever considered to implement (s/def ::foo ::bar) via clojure.core/derive
? I would personally find it quite interesting with that form, you can leverage ancestors/isa? etc#2018-02-2613:43Alex Miller (Clojure team)Not sure if Rich ever considered that.#2018-02-2613:44mpenetdoes that sound like something desirable in your opinion?#2018-02-2613:46Alex Miller (Clojure team)Donāt know. The example above I would consider aliasing, not a hierarchy. The notion of derivation with respect to s/keys (containership) is interesting though for additive aggregations.#2018-02-2613:46mpenetwe use that on a small lib to add metadata to specs, we then can just say spec ::x derives from another and add useful little utils on top of that that can optionally fetch all the metadata from the ancestors and merge them for instance.#2018-02-2613:46mpenetyes exactly why I am asking#2018-02-2620:00ghadiI filed https://dev.clojure.org/jira/browse/CLJ-2327 just to capture that slowdown#2018-02-2620:09Alex Miller (Clojure team)I think probably replacing the big set with a char int numeric check would probably help a lot#2018-02-2620:12ghadiI don't think so @alexmiller -- it's OOMing which means some extra retention#2018-02-2620:13ghadiI think we could remove the chars entirely and have a smaller example#2018-02-2620:23Alex Miller (Clojure team)if you replace the :atom branch with something like (defn atom-char? [c] (let [i (int c)] (or (<= 48 i 57) (<= 65 i 90) (<= 97 i 122))))
, I think youāll find it doesnāt have the issue#2018-02-2620:24ghaditrying it...#2018-02-2620:26ghadiit does not help at all @alexmiller#2018-02-2620:27Alex Miller (Clojure team)helped for me#2018-02-2620:27ghadidid you (s/+ atom-char?)#2018-02-2620:27ghadimine is still running#2018-02-2620:27Alex Miller (Clojure team)oh, maybe I just did atom-char?#2018-02-2620:27ghadipretty sure it's the s/alt and s/+ combo#2018-02-2620:28Alex Miller (Clojure team)not (s/+ atom-char?)#2018-02-2620:28Alex Miller (Clojure team)yeah, prob#2018-02-2712:33borkdudeI have written a macro to get rid of some boilerplate. Iām using a spec to validate arguments passed to a generated function.
(s/merge ::util/job-args
(s/keys :req-un
[::importers
::database
::rabbitmq
::stats]))
But I still have to destructure the args
, where something could go wrong if you make a typo.
(let [{:keys [report importers database rabbitmq stats]}
args]
)
Is there a good solution for this, e.g. key destructuring using spec?#2018-02-2712:41gfredericksI'd probably use plumbing/safe-get
#2018-02-2712:42borkdudeyeah, I thought about writing that before I wrote the specsā¦ might be just as effective#2018-02-2712:42borkdudebut nice to have it in a lib#2018-02-2715:51mishawell, it has nothing to do with spec opieop#2018-02-2716:33borkdudebut it does what I want. š#2018-02-2716:37gfrederickstheoretically it overlaps with spec because the spec could tell you whether you're accessing a key that can be ex-spec-ted to be there#2018-02-2716:39stathissideris@borkdude Iāve written this https://github.com/stathissideris/spectacles but it doesnāt cover destructuring#2018-02-2716:40stathissiderisit has get, get-in, assoc, assoc-in, update, update-in#2018-02-2716:41stathissiderisyou can also compose the lenses#2018-02-2716:45misha@gfredericks should it, say, throw, if you are asking for key not in spec?#2018-02-2716:45gfredericks@misha that sort of thing, yeah#2018-02-2716:46mishathen it'd be nice app-level let-speced
macro#2018-02-2716:46stathissideristhatās what spectacles does! (more or less)#2018-02-2716:47stathissiderisif someone wants to extend it with let-speced
pull requests are very welcome#2018-02-2716:47mishaI think you are missing binding part of destructuring, @stathissideris#2018-02-2716:48mishathis is where the money is#2018-02-2716:48stathissiderisdefinitely, but there is quite a bit of code there to introspect the specs etc#2018-02-2716:49stathissiderisso itās a good base to build on, thatās what Iām saying š#2018-02-2716:49mishaI think the most common and useful case would be specced-destructuring for maps (s/keys)#2018-02-2716:50mishabecause most of conformed things end up being maps anyway opieop#2018-02-2716:51stathissideristhe main reason I didnāt do it was the syntax, couldnāt come up with a nice way to express it#2018-02-2716:51mishaand seq-destructuring does not need any names from input value anyway#2018-02-2716:51stathissiderisbecause the spec name has to be mentioned#2018-02-2716:51mishatrue#2018-02-2716:52stathissiderisitās already a bit awkward with get-in
#2018-02-2716:52mishaI'd love it to have let-like form#2018-02-2716:54stathissideris@misha can you come with an example of what it would look like using it?#2018-02-2716:58mishanot yet, thinking of options.
what I know, is:
- binding has to be a core feature, to avoid repetition
- should look "familiar", so plugins, like #cursive could understand it (most probably like let
)
- should be natural to use, because following does not really looks good:
(let [_ (spec-bind spec m [foo bar baz])]
...)
#2018-02-2716:59stathissiderishow about:
(lens/let [{::logging [::level ::destination ::type]} foo]
(prn "level is" level)
(prn "type is" type))
#2018-02-2716:59mishabut that is second best of my ideas (but I dont have top1 yet :) )#2018-02-2716:59stathissiderisinstead of :keys
you give the name of the spec#2018-02-2716:59stathissideris(havenāt worked out the implications of the syntax yet)#2018-02-2717:00mishathis is better, although preserving all destructuring features would be neat#2018-02-2717:03misha(not saying you can't have both with this design, just need to think it all through)#2018-02-2717:03stathissiderisyeah, the ideal situation would be for spectacles to handle the āspecialā parts and pass on anything not related to spec to clojure.core/let
#2018-02-2717:04mishaup next:
how to bind keys from the same map using different specs?
2 lines? or 2 specs in destructuring form?#2018-02-2717:05stathissideris(lens/let [{::storage [::url ::port ::password]} foo
{::logging [::level ::destination ::type]} foo]
(prn "level is" level)
(prn "type is" type))
#2018-02-2717:05mishaselect-keys should just work for this (passing the rest to core/let)#2018-02-2717:06stathissiderishmm, it will throw if you try to destructure a key that is not speced, but not when the map contains extra keys#2018-02-2717:07mishaa week ago we were discussing adding &
to the map destructuring, so I spent some time looking at clojure.core/destructure#2018-02-2717:09borkdudeactually my problem was like this:
my-ns=> (s/def ::job-args (s/keys :req-un [::report ::scheduler]))
:my-ns/job-args
my-ns=> (lens/get {:scheduler 1} ::job-args :scheduler)
1
my-ns=> (lens/get {:foo 1} ::job-args :scheduler) ;;=> nil
So the input map should conform to the spec, which I can validate with e.g. s/assert
, s/valid?
etc.#2018-02-2717:10borkdudebut you can also get the keyword wrong of course, this is what spectacles
checks#2018-02-2717:10stathissideris@borkdude are you saying that the last call should throw instead of returning nil?#2018-02-2717:10mishaI think the value proposition here would be, "(say) throw, if trying to bind value under key not in spec"#2018-02-2717:11borkdudeboth things can go wrong: your input map or your selection in the map#2018-02-2717:12mishaif we are still on let-like
macro topic: I'd just check if the key is supposed to be in map according to spec,
because validation can be expensive, and I'd like to be able to separate the two#2018-02-2717:13mishaalthough it starts to sound like static types opieop#2018-02-2717:13borkdudesafe-get
is the most effective in my case here: I just get an exception and prints out the keys that were in the map and the key I used. I can figure out which one was wrong#2018-02-2717:14borkdudeI guess you can mess up specs themselves too, nothing protects you from typosā¦ typos all the way down#2018-02-2717:15mishafrom a broader perspective, destructuring and select-keys - are a bit orthogonal to each other, and both would need to be addressed individually#2018-02-2717:15stathissiderisāØ :thunder_cloud_and_rain:#2018-02-2717:15stathissideris@misha select-keys would be a good (and easy) addition to spectacles#2018-02-2717:16borkdudeI just discussed select-keys with someone else. After you use select keys you can still mess up selecting from the result of select keys#2018-02-2717:16stathissiderisnot if youāre using lens/get
! š#2018-02-2717:16misha@borkdude this is why I want let
:)#2018-02-2717:17mishaI much more often (destructure) bind keys, than select-keys and then bind#2018-02-2717:18stathissiderisIāve raised this, https://github.com/stathissideris/spectacles/issues/6#2018-02-2717:18stathissideriswould be good to work out a good syntax for this#2018-02-2717:19mishaif collision with :my-ns/keys can be solved - it is the best syntax there could be, already#2018-02-2717:20mpenetPersonally I would just add a :spec key in the map destructuring form and limit it at that: {:as foo :keys [..] : spec ::foo}#2018-02-2717:20stathissiderisjust donāt ever call your specs ::keys
š#2018-02-2717:21mishaanother data point, i usually don't use keys in :keys, but symbols
(let [{:my.ns/keys [foo bar]} m])
#2018-02-2717:21stathissideris@mpenet that does sound a bit less collision prone, thank you!#2018-02-2717:21mpenetWould work for seq destructuring too#2018-02-2717:21mpenet[..... :as foo :spec ::foo]#2018-02-2717:21misha@mpenet you then miss on opportunity to destructure by multiple specs in the same map :)#2018-02-2717:22mpenetYes true, trade-offs #2018-02-2717:22stathissideris@mpenet Iāve added your idea to the issue#2018-02-2717:22stathissideriswith attribution!#2018-02-2717:22mishayou probably don't need spec-destructuring for seqs#2018-02-2717:23mpenetProlly not#2018-02-2717:23mishaunless you are willing to support spec form walking for nested specs like s/or#2018-02-2717:23mpenetSpec walking is gross#2018-02-2717:23mishathen OMG you'd solve "damn, I forgot which keys I used in s/or for this"#2018-02-2717:23mpenetToo alpha to play this game imho#2018-02-2717:25mpenetNecessary evil for now#2018-02-2717:25misha2 years strong alpha kappa#2018-02-2717:25mpenetI am hoping we ll get something better for introspection/composition #2018-02-2717:26stathissiderisspectacles relies on looking at forms to figure out the keys#2018-02-2717:26stathissiderisbut yeah it feels risky#2018-02-2717:27mishaat this point I'd like to have compositional spec-walking#2018-02-2717:27mishait seems like everyone does it over and over again, for spectacles, for expound, etc.#2018-02-2717:29mpenetYup, spec-tools, speculate, etc been there done that too and trying to avoid doing it from now on;)#2018-02-2717:31stathissiderisspec-based seq destructuring could check that
(1) the spec is in fact a coll-of, a cat or a tuple
(2) for finite length specs, youāre not attempting to destructure more elements than what is defined in the spec#2018-02-2717:31stathissiderisnot sure how usefulā¦#2018-02-2717:39stathissiderissorry Iām not sure what that emoticon means!#2018-02-2717:39stathissideris@misha thanks for the comment#2018-02-2717:40mishaI went for "not really useful"#2018-02-2717:41stathissiderisyeah maybe not really worth it#2018-02-2717:56misha#2018-02-2717:58mishaoh wait, is this fixed in spec already, sweet#2018-02-2804:27bbrinckIs there a way to provide a seed to s/exercise
or gen/sample
? I have some code that uses these functions and Iād like to test the result, so forcing a specific seed would keep my tests reliable.#2018-02-2804:27bbrinck(or, if there is another test.check
function that can generate values based on a fix seed, I could use this)#2018-02-2810:27taylor@bbrinck looking at test.check there seems to be a way to do this by tweaking the sample-seq
function that gen/sample
wraps:
(defn sample-seq
"Return a sequence of realized values from `generator`."
([generator] (sample-seq generator 200 nil))
([generator max-size] (sample-seq generator max-size nil))
([generator max-size seed]
(core/let [r (if seed
(random/make-random seed)
(random/make-random))
size-seq (gen/make-size-range-seq max-size)]
(core/map #(rose/root (gen/call-gen generator %1 %2))
(gen/lazy-random-states r)
size-seq))))
(def my-gen (s/gen (s/map-of string? int?)))
(take 10 (sample-seq my-gen 200 0x42))
the only practical difference being the sample-seq
in test.check doesnāt take seeds. sample
is just calling this function with a default sample size. consider this answer extremely non-authoritative, but it does always generate the same samples for a given seed š#2018-02-2811:49gfredericksalso gen/generate
has a seed arg in the master
version I believe#2018-02-2811:50gfredericksit wasn't there originally because sample
is meant to be a dev-only function
which raises the question of what you're doing with it that it needs to be tested š#2018-02-2812:40misha@gfredericks eyeballing data structure generated with the same seed is easier while developing spec for it. Presumably, only subset of data structure, related with spec change, would change on subsequent generate calls#2018-02-2812:42gfredericksyeah that makes sense; I think I misread what @bbrink said originally also, I had thought he must be calling sample
in non-test code#2018-02-2812:43mishaI think you were not wrong :)#2018-02-2812:44mishabut exposing seed argument would benefit dev-time use case.#2018-02-2812:46gfredericksyep; for now gen/generate
is pretty similar#2018-02-2813:15taylornice, I was only looking into the 0.9.0 code#2018-02-2815:11gfredericksI'm like two years late at making a new release#2018-02-2816:55mishaand a book!#2018-02-2816:59gfredericksand a guide#2018-02-2817:00misha#2018-02-2817:00gfrederickssecond edition of the book#2018-02-2817:00mishalambo license plate#2018-02-2817:00gfrederickscompany offering enterprise support#2018-02-2814:33bbrinck@taylor @gfredericks thanks for the ideas. Yes, my original question about was using s/exercise
(with random seed) in non-test code, but then controlling the seed for tests (to make them reliable). Although, on further consideration, I guess it would be fine to control the seed in non-test code as well. The use case is, given a spec, to print out example data.#2018-02-2814:34bbrinckIn this case, the randomness of examples isnāt desirable#2018-02-2814:36taylorI also did this to auto-gen some API docs, but I was too lazy to try using a fixed seed#2018-02-2814:43gfredericksYeah, test.check generators are awkward in a few ways for the example use case#2018-02-2819:10jjttjjwhat if I want to spec something as being an ordered list of "params" or a map of the same params with names attached, like this:
(def s (s/or :arglist (s/cat :mylib/my-id (s/? :mylib/my-id)
:mylib/my-map :mylib/my-map
:x string?
:y boolean?
:z boolean?)
:argmap (s/keys :req [:mylib/my-map]
:opt [:mylib/my-id]
:req-un [:mylib/x :mylib/y :mylib/z])))
(s/conform s {:mylib/my-map {:a "b"}, :x "asdf", :y true, :z false})
;;=> [:argmap {:mylib/my-map {:a "b"}, :x "asdf", :y true, :z false}]
(s/conform s [{:a "b"} "asdf" true, false])
;;=>[:arglist {:mylib/my-map {:a "b"}, :x "asdf", :y true, :z false}]
I have many different things that need to be spec'd like this. Is this worth trying to write some hacky function to make easier to type out? Or is this probably just a bad idea and I should stick to being explicit about it?#2018-02-2819:15gfredericksthe :arglist
version is unfortunate because it dictates the order of the keys#2018-02-2819:21jjttjjnot sure I get what you mean. the idea is i can figure out what i'm getting either via the order of the values, or by names being associated with the values#2018-02-2819:22gfredericksoh I'm sorry#2018-02-2819:22gfredericksI misread the first half as being like (f :a1 x :a2 y ...)
#2018-02-2819:23gfredericksdisregard#2018-02-2819:23jjttjjno worries š#2018-02-2819:37Alex Miller (Clojure team)s/keys*
can be used for your arglist I think#2018-02-2819:37Alex Miller (Clojure team)itās like keys, but in any order, in a sequential structure#2018-02-2819:45gfredericksI don't think that kind of signature is even being used, that was my misunderstanding#2018-02-2820:00jjttjjyeah. it's more like "if x is a map, validate it with s/keys. if it's sequential, validate the conformed map after calling s/cat on it"#2018-02-2820:01mishais s/keys*
considered being public api?#2018-02-2820:02jjttjjI believe it is, it's how you're supposed to validate optional keyword arguments i think#2018-02-2820:03jjttjjlike (json->clj x :keywordize true)
or something. it's mentioned in the spec guide#2018-02-2820:12Alex Miller (Clojure team)yes, the *
here is supposed to be evocative of s/*, not as a marker for something implementation-ish#2018-02-2820:13Alex Miller (Clojure team)you can still use s/keys* here in with a wrapper s/spec so you get a layer of nesting#2018-02-2820:15Alex Miller (Clojure team)(def s (s/or :arglist (s/spec (s/keys* ...stuff...))
:argmap (s/keys ...stuff...)))
#2018-02-2820:16Alex Miller (Clojure team)or maybe Iām still not understanding the intent#2018-02-2820:17Alex Miller (Clojure team)I guess looking at your examples above, Iām not#2018-02-2820:19Alex Miller (Clojure team)for the arglist, I think you can just (s/and (s/cat ...) (s/keys ...))
? where the s/keys bit is the same spec as :argmap, presuming so, Iād pull out the :argmap spec into itās own thing to reuse it#2018-03-0101:41clumsyjediI must be missing something very simple about composing specs#2018-03-0101:41clumsyjedi(s/def ::nums (s/alt :pos pos? :neg neg?))
(s/def ::int int?)
(s/def ::all (s/and ::nums ::int))
(s/valid? ::all 5)
;; => false
(s/explain ::all 5)
;; => val: [:pos 5] fails spec: :myns/int predicate: int?
#2018-03-0101:44clumsyjediso the ::nums spec uses s/conform
and changes the value being validated. So ::int
is trying to validate [:pos 5]
and failing.#2018-03-0101:44clumsyjediHow do I avoid this though? I just want to chain validations together, each validating the original value I pass in#2018-03-0101:45gfredericksI feel like I ran into this recently but can't find any evidence of that#2018-03-0101:46gfredericksI think there's an s/unform
or something like that that could be a workaround#2018-03-0101:46gfredericksbut that seems like it shouldn't be necessary#2018-03-0101:46gfredericksI know this has been talked about a lot, just don't remember any conclusions#2018-03-0102:02clumsyjedithanks @gfredericks - It seems like for my actual work scenario I can use s/merge
instead of s/and
.#2018-03-0102:03clumsyjediI'm curious what the recommended approach is for the example I pasted though#2018-03-0102:03clumsyjediI googled around for unform, didn't find much#2018-03-0103:17Alex Miller (Clojure team)Itās best to start s/and with the pred that conforms to itself, particularly a type predicate like int? that will also generate#2018-03-0103:17Alex Miller (Clojure team)Youāre probably thinking of the undocumented s/nonconforming wrapper#2018-03-0103:18Alex Miller (Clojure team)Still on the fence about that one#2018-03-0103:19Alex Miller (Clojure team)However Iād just do (s/and ::int ::nums) here#2018-03-0103:21Alex Miller (Clojure team)And s/or is better than s/alt here (although you wonāt see any difference in behavior until you combine ::nums with another regex spec#2018-03-0114:39mishaI use s/nonconforming
quite often (not sure how to feel about this though)#2018-03-0115:57stathissiderishereās a tricky situation: I have a very large spec to describe my very large config. The config contains values in different nested maps that need to be consistent with each other (some have to co-occur, some refer to each other so IDs need to be consistent etc), so in order to validate the config I have extra predicates on the top-level spec to check for consistency. That was great, until I tried generating a config, and of course it wasnāt valid (because itās very unlikely to get consistency by chance). So I started writing overwriter functions that would be used via fmap
in the generator in order to re-align the different parts of the randomly generated config. But it does feel a bit like an uphill struggle. Am I missing something?#2018-03-0116:15Alex Miller (Clojure team)generating large structures with semantic constraints is (inherently) hard#2018-03-0116:16Alex Miller (Clojure team)an alternate approach is to supply a custom generator at the point of use (with stest/check for example) that picks from example configs#2018-03-0116:16Alex Miller (Clojure team)by using a generator on a set of enumerated examples#2018-03-0116:34stathissideris@alexmiller thanks for the insight. I donāt like the idea of giving up the variability that I get from using an actual generator instead of example configs (which I assume youāre implying would he āhandā-written). Youāre right, itās inherently hardā¦ Iāll see if I can refactor a bit to decomplect the constraints between distant parts of the structure, and also see if I can express the ones that cannot be avoided in a more fluent way.#2018-03-0116:45Alex Miller (Clojure team)the general approach if you are using custom generators is to first build a generator for a simpler model, then use gen/bind to create a generator that produces the appropriate structure#2018-03-0116:49stathissiderisIāve done this before, but I think itās not so applicable in this case, this one has cases where you get a bunch of IDs generated for part of the structure (say metrics
collection), and then in another part (for example filter-metric
) you need to refer to one of the existing metrics, so the overwriter just replaces the generated filter-metric
with one from the IDs in the metrics
collection#2018-03-0116:51stathissiderisyouād think that a flag on one of the metrics (something like :filter?
) would be betterā¦ Iām beginning to think the same š#2018-03-0117:11cap10morganI have a string I want to write a spec for, but it is composed of previously-spec'd string A, followed by a /
, and then previously-spec'd string B. They are GitHub "user/repo"
strings. If I already have a :github/user
spec and a :github/repo
spec, how would I write a spec for the "user/repo"
strings that composes those first two specs?#2018-03-0117:13Alex Miller (Clojure team)spec is not great for doing composition of string specs like this - I suspect composing the patterns and creating a spec from that will probably end up better#2018-03-0117:13cap10morganš Hey Alex! OK, thanks. I figured it was either something like that or I was missing something really simple. I'll go down the route you suggest. Thanks!#2018-03-0117:14Alex Miller (Clojure team)itās possible, I just donāt know that youād be happy with the spec, the generated values, or the conformed values once you were done#2018-03-0117:14cap10morganI see#2018-03-0118:49misha@stathissideris generator is "just a function", so you can generate a set of ids upfront, and inject those in sub-generators. it will be verbose and might not be pretty, but sometimes you got to do what you got to do...#2018-03-0118:50misha(or change config format, if you control it)#2018-03-0118:50ghadithat's what I do ^#2018-03-0118:51ghadigenerate the stuff that you want to "align" first -- then pass them down#2018-03-0118:51mishaanother option, is to generate sub-structure and explicitly overwrite id. this saves you some code, and sub-generators customizations#2018-03-0118:53mishaofc, "thicker" dependency is ā tighter your generators would be coupled, and "just finish it with assoc-in's" would not be enough#2018-03-0217:08jimbobI donāt really understand why Cloujre spec is any better than simple pre conditions for simple validation. example: (defn mk-protobuf
[{:keys [reward-value available event-uri event-name event-at schema revision
hostname fake environment shopper-uri service-agent shopper-name reward-uri reward-name action-at]
:or {service-agent service-agent
hostname (hostname)
environment (str env)
event-at (c/to-epoch (t/now))
schema (str schema)}}]
:pre [(string? event-uri)
(string? reward-uri)
(string? shopper-uri)
(string? reward-value)
(instance? Boolean available)]
wonāt these preconditions be sufficient run time validation? what willl i get from spec that i cant get from these?#2018-03-0217:09Alex Miller (Clojure team)docs, generated examples, automated generative testing#2018-03-0217:13jimbobi suppose so, but is not these pre conditions and or conditions documentation enough?#2018-03-0217:13jimboband much fewer lines of code#2018-03-0217:13bbrinck@ben.borders whatās the error message if you pass in something incorrect? What if one of the args was not just a string, but, say, a vector of strings?#2018-03-0217:13jimbob(assuming we are not doing generative testing)#2018-03-0217:18jimbobits not terrible#2018-03-0217:18jimbobex: Exception in thread "async-dispatch-12" java.lang.AssertionError: Assert failed: (integer? event-uri)
#2018-03-0217:18jimbobbut yes not as good as spec errors.#2018-03-0217:19bbrinckI think the different may be starker if you pass in nested data e.g. a vector of maps that must contain keys#2018-03-0217:19bbrinckbut for simpler predicates like string?
(as in your example), the differences in error messages wonāt be as significant#2018-03-0217:20jimbobsure, but then couldnt i just use clojure.test/is ?#2018-03-0217:38nhaI donāt recall what exactly but this ended up causing trouble for me down the line (maybe it was nested is
? I donāt remember now)#2018-03-0217:20jimboband get things like ;FAIL in
#2018-03-0217:22bbrinckYes, although it becomes a little less clear for nested data e.g. a vector of strings#2018-03-0217:22bbrinckor vector of maps of specific keys#2018-03-0217:27jimbobi see#2018-03-0217:28jimbobthanks for your responses#2018-03-0217:29bbrinck@ben.borders np. hereās an example of error message with is
vs explain
(using expound) https://gist.github.com/bhb/44d68a7ab878187b68d4fd48579d591b#2018-03-0217:29bbrinckI copied and pasted from my REPL session and omitted my mistakes, so hopefully I didnāt miss anything important š#2018-03-0217:32jimbobso then to get the same desired runtime validation, would it be advised to place s/valid? in the :pre block of the function? or are there better ways to do this?#2018-03-0217:32jimbobiāll probably end up exploring this.. it does indeed seem to be much better self documenting#2018-03-0217:33bbrinckthere are a few options. You can use fdef
, or you could use a s/assert
#2018-03-0217:34jimbobmm alright#2018-03-0217:34bbrinckfor s/assert
, keep in mind youāll need to run (s/check-asserts true)
#2018-03-0217:35bbrinckyou could also use s/explain-str
and raise an exception#2018-03-0217:35bbrinck(if the result is not literally "Success!\n"
)#2018-03-0217:43jimbobgotcha. and for speccing the example map above, would i have to create an s/def for every key in the map? for example some of the keys in the map should contain the same type and structure of data, so it would be nice to reuse something like (s/def ::non-empty-string not-empty string?)
for multiple keys instead of (s/def thing1 ::nonempty-string) (s/def thing2 ::Nonempty-string
#2018-03-0217:49Alex Miller (Clojure team)yes, you will need one for every key#2018-03-0217:49Alex Miller (Clojure team)that puts it in the spec registry which is used by s/keys#2018-03-0217:49jimbobok, gotcha thanks alex#2018-03-0217:50Alex Miller (Clojure team)if you have many repetitive such things, macros can help#2018-03-0217:50jimbobright, cool#2018-03-0217:50Alex Miller (Clojure team)maybe eventually weāll provide something to def multiple things at a time#2018-03-0311:55pfeodrippeWoww, I saw the problem, s/and
enters at the land of predicate instead of staying at the regex as said by the great @alexmiller at https://groups.google.com/forum/#!topic/clojure/S3UaS_4FRqY. Gonna use s/&
#2018-03-0502:50seancorfield@mpcarolin You can use back tick you expand a symbol's name: (-> (test/check `my-index-of) (test/summarize-results))
will probably work.#2018-03-0502:52mpcarolin@seancorfield Hey, thanks! I did not know that existed. It will save me a lot of redundant typing.#2018-03-0506:15tianshuhow to write a spec allows only the specified keysļ¼ is there a convenient way?#2018-03-0506:17tianshuI want to use spec to constraint the schema of edn which used as a configuration file. I want useless keys are not allowed.#2018-03-0509:29stathissideris@doglooksgood there is no official way, you use s/and
with s/keys
and another predicate that constrains the keys to be in a specified set, but then you are listing the keys twice. Some people have written macros for this sort of thing#2018-03-0509:30stathissiderisbut spec is like that by design, probably the decision that has caused the most controversy about it#2018-03-0509:31tianshuso I should write something like s/keys
, okay#2018-03-0510:53taylorhereās an example https://github.com/gfredericks/schpec/blob/b2d80cff29861925e7e3407ef3e5de25a90fa7cc/src/com/gfredericks/schpec.clj#L13#2018-03-0511:28tianshuThanks!#2018-03-0514:24Andreas Liljeqvistis https://github.com/clojure/spec.alpha the main repository for spec?#2018-03-0514:26Andreas LiljeqvistAsking because it is like 3 months since the last commit#2018-03-0514:31bronsayes#2018-03-0514:31bronsaclojure repos usually see activity in bursts not continuous#2018-03-0514:32bronsaapart from initial development#2018-03-0514:37Andreas Liljeqvistok, thanks. Looking to fix a bug#2018-03-0514:44Andreas LiljeqvistNow I just have to figure out how it is built, I expected a .deps.edn,project.clj, build.boot, makefile....#2018-03-0514:51Alex Miller (Clojure team)Itās maven - you can test with mvn clean test
etc#2018-03-0514:53Andreas Liljeqvistgot it to run with mvn clojure:compile
#2018-03-0514:53Andreas Liljeqvistgoing with mvn clean test
since I want to fix a problem#2018-03-0514:53Andreas Liljeqvistthansk#2018-03-0516:55nnboskoIs there a way to make a function return an error if parameters don't conform to their given spec?#2018-03-0517:37mpcarolinI think what youāre looking for is spec.test/instrument. Once you execute it, it will tell you if any :args specs are violated for a given function. #2018-03-0517:56mpcarolinE.g (stest/instrument āfoo)#2018-03-0517:59mpcarolinBut if you want it at runtime without instrument, you can also use :pre condition for a function with s/valid? #2018-03-0516:57nnboskoThe function's given spec, anyways#2018-03-0517:17gfredericksdoes clojure 1.9 come with specs for clojure core functions somehow?#2018-03-0517:20taylorthere must be some coverage in there, hereās a spec error for invalid let
:
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs.alpha/bindings at: [:args :bindings :init-expr] predicate: any?, Insufficient input
#:clojure.spec.alpha{:problems [{:path [:args :bindings :init-expr], :reason "Insufficient input", :pred clojure.core/any?, :val (), :via [:clojure.core.specs.alpha/bindings :clojure.core.specs.alpha/bindings], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x482c71e3 "
#2018-03-0517:23gfrederickswell that's a macro, which is a pretty different use case (in particular, it's checked at compilation; function specs would only get involved if a user intentionally instrumented them)#2018-03-0517:18gfredericks(or are there official specs somewhere else?)#2018-03-0517:29gfredericksI guess they'd probably be in core.specs.alpha if anywhere, and I see those are basically just a handful of macros#2018-03-0517:59bronsathatās all there is available#2018-03-0519:22gfrederickssurely somebody somewhere is assembling a non-official collection of such things?#2018-03-0520:13bbrinckThe two I know of are https://github.com/leifp/spec-play and https://github.com/tirkarthi/respec . I have used neither but @alexmiller has pointed out errors in those specs#2018-03-0520:13bbrinckBut they might be sufficient for some perf testing#2018-03-0520:13gfredericks:+1: thanks#2018-03-0520:15Alex Miller (Clojure team)Theyāre not good#2018-03-0520:16bbrinckIf we had a large enough test suite across enough projects, could we infer āgood enoughā specs for core functions with https://github.com/stathissideris/spec-provider ? š¤#2018-03-0520:17bbrinckI understand that for general usage, āgood enoughā is not really a concept, or weāll get confusing false positives and false negatives š#2018-03-0520:18bbrinckBut in my case, āgood enoughā specs are useful so I can see if error messages can be improved#2018-03-0608:13stathissideris@U08EKSQMS spec inference for functions is still work in progress in spec-provider (but not too far off). I suspect it wouldnāt be able to give you anything very useful if you ran it on map
though (which I think is very tricky to write a spec for anyway)#2018-03-0519:22gfredericksI wonder how much it would slow down a test to instrument all of clojure.core š#2018-03-0520:16Alex Miller (Clojure team)A lot :)#2018-03-0520:16gfredericksdo you think some aspect of this will be useful for some things at some point?#2018-03-0520:17Alex Miller (Clojure team)I hope so #2018-03-0520:18Alex Miller (Clojure team)I have worked with portions of core speced and itās really useful at the repl#2018-03-0520:18gfredericksdo you think there's a reasonable place to draw the line that maximizes utility and speed?#2018-03-0520:18Alex Miller (Clojure team)I think itās mostly not useful in a test suite (if the suite already runs)#2018-03-0520:19Alex Miller (Clojure team)I donāt feel ready to answer that yet#2018-03-0520:20gfredericks:+1:#2018-03-0519:25cap10morganI have a utility fn that can operate on any map, so its :args
spec is just (s/cat :v (s/keys))
. But when I do generative testing with that, I get a bunch of empty maps (which makes some sense I suppose). What's a good way to make that generate a bunch of random maps instead (ideally w/ an emphasis on namespaced keyword keys)?#2018-03-0519:32cap10morganShould I just use map?
instead of s/keys
?#2018-03-0519:34gfrederickssounds plausible
I'm of the opinion that virtually all uses of maps fall into two nonoverlapping categories, roughly corresponding to s/keys
and s/map-of
#2018-03-0519:34gfredericksmap?
seems to not commit to either#2018-03-0519:59cap10morganyeah, map?
seems better#2018-03-0520:22bbrinck@alexmiller Are specs for the core library something that contributors could help with? In other words, are they blocked on dev/reviewer time, or are they more fundamentally blocked on any upcoming changes in spec?#2018-03-0520:24Alex Miller (Clojure team)They are really blocked on Rich review time. have written some of them but donāt have them public anywhere#2018-03-0520:24Alex Miller (Clojure team)They tend to raise many questions with issues of taste and judgement#2018-03-0520:25bbrinckI see, that indeed seems like it would require quite a bit of time to review#2018-03-0520:25bbrinckand change accordingly#2018-03-0520:26Alex Miller (Clojure team)Eric Normand actually was working on some of this at one point and I gave him some ideas of things to do with it#2018-03-0520:26bbrinckHe was working on official specs? Or on his own set?#2018-03-0520:28bbrinckI understand the issues of taste will take time to resolve, but to the extent that the official specs need testing, Iām sure the community could provide feedback there#2018-03-0520:28Alex Miller (Clojure team)He wanted to help and so I told him what would be helpful #2018-03-0520:28bbrinckGotcha#2018-03-0520:29bbrinckPerhaps correctness is much easier than making idiomatic specs that conform/generate cleanly š#2018-03-0520:29gfredericksI doubt correctness is easy either#2018-03-0520:29seancorfieldAnd there's also the issue that some functions are extremely hard to spec correctly...#2018-03-0520:30bbrinckCorrect, although thatās one place where the community could help - install specs, run tests, see what breaks#2018-03-0520:30Alex Miller (Clojure team)I donāt remember now what that was but I think I asked Eric to help in categorizing what was out there. I think he did but it was a while ago#2018-03-0520:31Alex Miller (Clojure team)My experience in testing core specs is that itās extremely difficult to test more than one at a time#2018-03-0520:35bbrinckHm, thatās a good point. Since we can see the list of isntrumentable symbols, a project could loop over them, instrument each one in isolation, and run tests#2018-03-0520:35bbrinckthat way, if there was a failure, it would be isolated from other specs.#2018-03-0520:36bbrinckagain, if it was useful to get broader validation of specs by running across a large variety of project#2018-03-0522:54ScotHi, is there a good way to spec anonymous functions? I am trying to use fspec
, but I keep getting Cannot read property for_all_STAR_ of undefined
#2018-03-0523:39seancorfield@scot-brown Do you have org.clojure/test.check as a dependency?#2018-03-0523:41seancorfieldIn order to check whether the function conforms to the spec, it is tested used a generative approach, so you need test.check in your project for that (`clojure.spec` will try to load it dynamically and you'll get some odd-looking failures if it isn't available)#2018-03-0611:20roklenarcicHow would I create a BigDecimal generator?#2018-03-0612:05tayloryou could use fmap
to coerce some other number generator to BigDecimal
but you wouldnāt get the full range of values (and you might not want that?)#2018-03-0614:18gfredericksyeah it's hard to give a good answer without knowing what kind of distribution you want; but (gen/let [factor gen/large-integer, log gen/int] (* factor (... 10 to the logth power ...)))
wouldn't be bad#2018-03-0614:23gfredericksthere's probably a method on BigDecimal that would do the scaling a bit more naturally#2018-03-0700:07roklenarcicthanks#2018-03-0701:14taylorHaha I just read that the largest possible big decimal value can consume about 8GB RAM#2018-03-0701:33gfredericksI'm disappointed that there's a largest#2018-03-0619:02cap10morganIs there a good workaround for keeping NaN
out of your spec-generated test data? Filtering at the top level doesn't keep them out of nested data structures, and they break all kinds of equality tests when they show up.#2018-03-0619:04seancorfield@cap10morgan I thought the double-in
generator had options to suppress that?#2018-03-0619:04cap10morganI'm working with any?
#2018-03-0619:05seancorfieldAh, then you'll get "anything" š#2018-03-0619:07cap10morganyep. there just doesn't seem to be a way to define any-but-nan
. my code handles NaNs just fine, it's the specs that barf on them (since the equality tests in my predicates choke on them).#2018-03-0619:07cap10morganmakes spec'ing fns that can handle any value tricky#2018-03-0620:24cap10morganI have tested it locally and it seems to solve the problem.#2018-03-0621:09gfredericksthis is a sticky issue because merely removing NaN from everywhere isn't obviously the best thing, since NaN is an important edge case to test in a lot of situations#2018-03-0621:09gfredericksit's obviously hard to opt-out of at the moment#2018-03-0621:17cap10morganAgreed but the concept of removing it from the any?
generator seemed at least tacitly endorsed by the core team.#2018-03-0621:18cap10morganIf they instead requested a way to toggle it, Iād be happy to work on that too.#2018-03-0621:19noisesmithpedantically, if you accept any value, you better do the right thing if the value is NaN right?#2018-03-0621:21cap10morganYes. And the code does. Only the spec has a problem. And it is not easy to work around.#2018-03-0621:21cap10morgan(In my use case)#2018-03-0621:22cap10morganA NaN at any level of nesting blows up equality checks in predicates.#2018-03-0621:23dpsuttonand you said your code gracefully handles that, right?#2018-03-0621:23dpsutton> do the right thing if the value is NaN right?#2018-03-0621:25cap10morganYes, the code handles it just fine.#2018-03-0621:25cap10morganItās not doing any equality checking.#2018-03-0621:30cap10morganI think my ideal solution might be a spec-specific equality function that handled NaN
, but I don't know where the core team stands on that.#2018-03-0621:32Alex Miller (Clojure team)Just driving by here - would much rather prevent the NaN from occurring in the first place. Custom equality has never gone well imho :)#2018-03-0621:33cap10morgan@alexmiller That's fair. š Does that patch ^^^ seem OK to attach to the JIRA ticket?#2018-03-0621:36Alex Miller (Clojure team)No time to look atm but itās ok either way :)#2018-03-0621:36cap10morganOK, attaching it then.#2018-03-0623:06gfredericksthe downside to that is that the list of generators that gen/any
is composed of is now in two places#2018-03-0623:08cap10morgan@gfredericks Not sure I follow. Where are the 2 places?#2018-03-0623:10gfredericksDidn't you paste & tweak that code from test.check?#2018-03-0623:13cap10morganyes, but it is the one and only code for the any?
generator now. and test.check itself largely repeats that one-of
form between the any
and any-printable
generators.#2018-03-0623:14gfredericksYes, I was pointing out that if test.check added anything to its gen/any, that wouldn't propogate to spec anymore#2018-03-0623:14cap10morganIf test.check defined the vector separately I definitely would have just re-used it though. Not sure there's a good way to re-use it as is.#2018-03-0623:14gfredericksProbably not#2018-03-0623:15gfredericksEven if test.check changed you couldn't take advantage of that without introducing a stricter version requirement between the two. I'm not sure if that's acceptable or not#2018-03-0623:18Alex Miller (Clojure team)Not#2018-03-0700:06gfredericksthat was my guess#2018-03-0715:36Andreas Liljeqvistis there any function like (complete ::somespec {::provided-value 2})
that will generate keys that aren't provided?#2018-03-0715:37Andreas LiljeqvistI could provide overrides like {::provided-value #(gen/return 2)}
#2018-03-0716:33Andreas LiljeqvistI have written such a function, just interested to know if it exists somewhere in spec#2018-03-0717:11jimbobis there any way to have fdef functions be runtime validated?#2018-03-0717:11jimbobOr are these function specs only for documentation / testing and generation#2018-03-0717:11tayloryou can instrument
the functions to check the :args
specs#2018-03-0717:12taylorand thereās another lib called Orchestra that gives you a version of instrument
that checks :args
and :ret
(and maybe :fn
?)#2018-03-0717:12taylor@ben.borders https://clojure.org/guides/spec#_instrumentation#2018-03-0717:13jimbobbrilliant. exactly what i was looking for, thanks a lot @taylor!#2018-03-0721:59aengelbergIs this a bug?
user=> (s/def ::a integer?)
:user/a
user=> (s/explain (s/merge (s/keys :req [::a]) (s/keys :req [::b])) {::a "a" ::b 1})
In: [:user/a] val: "a" fails spec: :user/a at: [:user/a] predicate: integer?
In: [:user/a] val: "a" fails spec: :user/a at: [:user/a] predicate: integer?
nil
#2018-03-0722:00aengelbergI would expect only one message to get spit out, not two. But since s/merge
calls each of its sub-maps' validators, it gets the error message twice.#2018-03-0722:35misha@aengelberg each s/keys
validates keys for presence, and every key of a known spec (even not mentioned ones) for conformance#2018-03-0722:37misha(s/def ::a integer?)
(s/def ::m (s/keys :req [::b]))
(s/explain ::m {::a "a"})
In: [:user/a] val: "a" fails spec: :user/a at: [:user/a] predicate: integer?
val: #:user{:a "a"} fails spec: :user/m predicate: (contains? % :user/b)
=> nil
#2018-03-0722:38mishaand s/merge does not reduce issues from each subspec#2018-03-0811:29Andreas LiljeqvistIf I found a lot of problems with indentation in spec source, what is the best chance of doing a successful pull-request?#2018-03-0813:05souenzzohttps://github.com/clojure/core.specs.alpha/blob/master/CONTRIBUTING.md#2018-03-0814:03Andreas LiljeqvistThanks, actually I am registered as a contributor.
More like if the changes would be accepted in any form.
Perhaps they don't consider it as a problem#2018-03-0814:43tbaldridgeI doubt it, itās really hard to make a good argument about indentation#2018-03-0814:43tbaldridgeGot an example though?#2018-03-0816:48Andreas Liljeqvisthttps://pastebin.com/6w3Rs1ai#2018-03-0816:48Andreas LiljeqvistLine 8 and 9 seems strange#2018-03-0816:55Andreas LiljeqvistPerhaps this also: https://github.com/clojure/spec.alpha/blob/739c1af56dae621aedf1bb282025a0d676eff713/src/main/clojure/clojure/spec/alpha.clj#L754#2018-03-0913:00tbaldridgeon the second one I don't see a problem at all#2018-03-0913:01tbaldridgeto be frank, indentation doesn't matter much in Clojure, so I doubt these sort of patches would ever make it through.#2018-03-0913:01tbaldridgeThe style guides that do exist for Clojure are more opinions than actual rules, so it's fairly common to see changes in indent styles across projects.#2018-03-0913:43Andreas LiljeqvistDisclaimer, I know I cannot possible "win" this#2018-03-0913:43Andreas LiljeqvistThe problem is that a lot of tools force indentation, parinfer for example. So diffs becomes problematic#2018-03-0913:44Andreas LiljeqvistThis is kind of a problem with the tools, but I figure that Clojure would be better of with a standard like pep8#2018-03-0913:46Andreas LiljeqvistMostly because indentation really doesn't matter that much. But a standard that enables linters and automatic formatters is valuable according to me#2018-03-0914:34Andreas Liljeqvisthttps://github.com/bbatsov/clojure-style-guide and usage of parinfer could eventually become a defacto standard#2018-03-0918:30souenzzoI highly recommend you to watch this video
https://www.youtube.com/watch?v=1YCkOo5Y4Oo#2018-03-0811:35mpenetClose to none#2018-03-0912:47guyHi people, whats the best way to generate a not blank string?#2018-03-0912:48gfredericksyou can wrap the generator in gen/not-empty
unless you meant something different by "blank"#2018-03-0912:48guyok thanks!#2018-03-0915:40Andreas LiljeqvistI have made a fix for CLJ-2311 - an issue that was preventing me from providing an override for the dispatch-fn(key) in a multimethod#2018-03-0915:40Andreas Liljeqvisthttps://dev.clojure.org/jira/browse/CLJ-2311#2018-03-0915:41Andreas Liljeqvistplease test it and leave comments#2018-03-0915:42Andreas Liljeqvistsource here: https://github.com/bonega/spec.alpha/tree/multi-spec-override#2018-03-0917:26gfredericksspec doesn't have anything analogous to schema completers, is that correct?#2018-03-0917:27gfrederickshas anybody used spec somehow to fill in incomplete test data?#2018-03-0921:25misha@gfredericks (merge (ffirst (s/exercise :foo/bar)) incomplete)
? :)#2018-03-0922:22gfredericksthat works at one level, yes#2018-03-0922:22gfredericksschema does it recursively#2018-03-1002:33mpcarolinHi everybody. Iāve been learning spec lately, and I think I have a decent grasp of its mechanics. What I donāt fully grasp is the real life effective use of spec. When should we be using spec? Should I be speccing as many of my functions and as much of my data as I can? Or just the borders? Or just what is publicly accessible?#2018-03-1003:32Alex Miller (Clojure team)there is no one right answer to that question#2018-03-1003:32Alex Miller (Clojure team)I donāt think you should or need to instrument everything#2018-03-1003:33Alex Miller (Clojure team)specāing at the borders is particularly helpful, but any function with sufficiently complex inputs or outputs is worth considering#2018-03-1003:33Alex Miller (Clojure team)instrumentation is worth using at dev time, check is worth doing at test time, and validation is (maybe) worth doing at runtime#2018-03-1004:49mpcarolinThanks for the response. Iāll keep that in mind!#2018-03-1021:14borkdudeweāre considering putting some (s/assert ...)
at the beginnings of some functions, which can be compiled away with the right system property#2018-03-1021:15borkdudeinteresting thought on spec: https://twitter.com/pjstadig/status/972434922794373121#2018-03-1021:20Alex Miller (Clojure team)Well, disagree#2018-03-1108:37mishaboy, I hope that failing to read doc from db due to validation error about extra unspecified kv ā is not "the clojure way"#2018-03-1109:06borkdudeI think his argument was more about representing specs in data rather than code#2018-03-1110:29ikitommiboth open & closed specs/schemas have valid use cases. Some comments from the GraphQL world: https://github.com/facebook/graphql/issues/127#2018-03-1111:12mishaThe defaults matter.#2018-03-1111:22borkdudeWe have a poor manās graphQL in which we can express *: {:a {:filter/default {:b true}}}
applied to {:a {:some-key {:b 1 :c 2}}}
gives you {:a {:some-key {:b 1}}}
#2018-03-1120:43borkdudeIdea: https://github.com/bhauman/rebel-readline/issues/135#2018-03-1120:44borkdudeAs of now itās kind of hard to relate function arguments to the specs for them. Maybe this is what the above tweet was also about: creating a different naming scheme#2018-03-1214:08misha@borkdude what do you mean by ārelateā exactly?#2018-03-1214:10mishaThe only coupling/relation you should have is order.#2018-03-1214:36oconnIām working on writing a spec for a model that uses multi-spec
. Is it possible to use unqualified keywords in a multi-spec? I found this thread https://www.mail-archive.com/clojure@googlegroups.com/msg99107.html and tried altering the example in the docs with no success.#2018-03-1214:43bbrinck> Is it possible to use unqualified keywords in a multi-spec?
@oconn Can you explain this a little bit more? Iām not sure I understand the question.#2018-03-1214:44taylor@oconn itās possible, and the example in the multi-spec
doc string uses an unqualified keyword#2018-03-1214:44taylorhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/multi-spec#2018-03-1214:46oconnOh nice, yeah not sure what Iām doing wrong here.. So hereās the spec#2018-03-1214:49taylorthat explain
returns āSuccess!ā for me#2018-03-1214:50taylorhaving trouble inferring what the variant-type "Text"
spec is intending to do though#2018-03-1214:51oconnI stripped out a lot of other keys, there are other types as well.#2018-03-1214:52bbrinck@oconn Can you expand a bit on what is happening when you call explain
and what you expect to happen?#2018-03-1214:55bbrinckI also see "Success!"
printed, but Iām guessing thatās not what you expect (?)#2018-03-1214:56bbrinck(minor thing, but your defmethod
appears to have a missing paren at the end)#2018-03-1214:56oconnYeah, not seeing that but I think itās because I stripped out some keys before pasting here.#2018-03-1214:57oconnOk, now Iām getting Success!. Sorry for the trouble#2018-03-1214:57bbrincknp, good luck!#2018-03-1219:23eggsyntaxI'm currently using gen/generate
to make some sample data for myself, including fairly nested structures which use s/*
at multiple levels. It'd be nice during dev to have the produced data be fairly short so it doesn't take up a whole repl screen. Wasn't there some way to tell it to keep those short? I vaguely thought there was. From the documentation it seems like it would be s/*recursion-limit*
, but if I do
(binding [s/*recursion-limit* 1] (sgen/generate (s/gen ::my-spec)))
I still get large numbers of items on the s/*
specs.#2018-03-1219:26eggsyntaxOh, I see why it's not *recursion-limit*
, I think; it's not actually recurring many layers deep, just creating a long sequence at each layer.#2018-03-1219:28eggsyntaxBut it'd be really nice to have an equivalent way to force shorter sequences. Am I missing something?#2018-03-1219:30mishaI'm afraid you are not missing anything, @eggsyntax#2018-03-1219:31eggsyntaxAh well. I guess I can walk it and take 3
or something.#2018-03-1219:31mishayou can provide overrides, which wrap original seq specs with just items count upper limit#2018-03-1219:31mishayeah ~#2018-03-1219:32eggsyntaxOh, so something like (s/with-gen (s/* ...) (take 3 ...))
#2018-03-1219:32mishaI'd love to be proven wrong on this one, though : )#2018-03-1219:33eggsyntaxIt'd make a terrific addition to a future version of spec, to have s/*sequence-limit*
alongside s/*recursion-limit*
.#2018-03-1219:34mishaI doubt it is a good idea to have a single global count limit. I'd rather had a way to override :min-count/:max-count opts for maps and seqs specs#2018-03-1219:35eggsyntaxFrom my POV, that'd be nice too -- but for the case of just quickly generating a piece of data in the REPL, it'd be lovely to just bind a global limit.#2018-03-1219:37eggsyntaxHmm, test.check
does have :min-elements
and :max-elements
on a number of its generators, I see....#2018-03-1219:37mishalike an extra count args here:
(with-gen* [_ gfn] (every-impl form pred opts gfn))
#2018-03-1219:37mishahttps://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1318#2018-03-1219:38eggsyntaxThanks @misha š#2018-03-1220:59trisswhy are all the facilities for defining specs macros? Iām tempted to use spec for pattern matching but the syntax is too heavyā¦#2018-03-1221:29Alex Miller (Clojure team)theyāre macros to capture the form for explanations#2018-03-1221:29Alex Miller (Clojure team)that said, some of that is likely to change in the next rev#2018-03-1221:57pavani@alexmiller How can I validate a schema like this using Spec?
{:billing_contact
{
(schema/optional-key :street_address) schema/Str
(schema/optional-key :city) schema/Str
(schema/optional-key :region) schema/Str
(schema/optional-key :postal_code) schema/Str}}
I have the following spec but if ::billing_contact
has keys other than the optional keys, it considers the map to be valid. I just want to check if the map has either those specified keys in opt or no keys.
(s/def ::street_address string?)
(s/def ::city string?)
(s/def ::region string?)
(s/def ::postal_code string?)
(s/def ::billing_contact (s/keys :opt [::street_address ::city ::region ::postal_code]))
(s/def ::changes (s/keys :req [::billing_contact]))
#2018-03-1222:06Alex Miller (Clojure team)You can s/and with an every? predicate that checks every key is a known key set#2018-03-1222:16pavani@alexmiller Thank you for that. I am not sure how exactly I could use that here. Could you please exhibit that with my example?#2018-03-1303:19bbrinck@U7MMSSQKC Perhaps something like (s/def ::billing_contact (s/and (s/keys :opt [::street_address ::city ::region ::postal_code]) #(every? {::street_address ::city ::region ::postal_code} %)))
?#2018-03-1303:30bbrinckWhoops, I forgot the call to keys
. Would this work: (s/def ::billing_contact (s/and (s/keys :opt [::street_address ::city ::region ::postal_code]) #(every? #{::city ::street_address ::region ::postal_code} (keys %))))
#2018-03-1322:10pavaniThanks a lot @U08EKSQMS#2018-03-1303:36rahulr92Hi, can someone help me understand the difference between spec/and
and spec/merge
?
The example of merge given in the Clojure Spec guide https://clojure.org/guides/spec (using :animal/common and :animal/dog) seems to work even if we use and
instead of merge
.#2018-03-1304:17seancorfield@rahulr92 s/merge
will merge two s/keys
specs (and, if I recall correctly, does not flow the conformed value); s/and
is for combining a general spec with a general predicate (or spec) -- and it does flow the conformed value.#2018-03-1304:21seancorfieldAlso, if you look at the docstrings, the differences should be reasonably clear https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/merge#2018-03-1304:22seancorfieldhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/and -- that specifies that conformed values are propagated.#2018-03-1304:25seancorfieldIn the merge
docs, you'll see another difference: it generates maps that conform to all the merged specs. With and
, the first spec is used as the generator, and then the other specs are used as predicates to filter the generated values (which may not satisfy those predicates and so overall generation of values can fail).#2018-03-1304:25seancorfieldHope that helps @rahulr92?#2018-03-1304:40rahulr92@seancorfield Thanks for the help. I understood the difference in generated value. However I still do not fully understand the difference while using these for validation.
Could you please elaborated on what you mean by the āflowā of conformed value?
> āUnlike āandā, merge can generate maps satisfying the union of the predicates.
Doesnāt and
also perform union i.e. it passes validation only if all the specs in it pass?#2018-03-1304:52seancorfield@rahulr92 Here's an example showing a difference boot.user=> (require '[clojure.spec.alpha :as s])
nil
boot.user=> (s/def ::a (s/or :i int? :s string?))
:boot.user/a
boot.user=> (s/def ::b (s/or :b boolean? :s string?))
:boot.user/b
boot.user=> (s/def ::merged (s/merge (s/keys :req [::a]) (s/keys :req [::b])))
:boot.user/merged
boot.user=> (s/def ::anded (s/and (s/keys :req [::a]) (s/keys :req [::b])))
:boot.user/anded
boot.user=> (s/conform ::merged {::a 1 ::b "x"})
#:boot.user{:a [:i 1], :b [:s "x"]}
boot.user=> (s/conform ::anded {::a 1 ::b "x"})
:clojure.spec.alpha/invalid
boot.user=>
#2018-03-1304:55seancorfieldIf you look at the result of conforming that map against (s/keys :req [::a])
you'll see why it fails with s/and
but works with s/merge
: boot.user=> (s/conform (s/keys :req [::a]) {::a 1 ::b "x"})
#:boot.user{:a [:i 1], :b [:s "x"]}
boot.user=>
#2018-03-1304:58seancorfieldand finally, regarding generation: boot.user=> (s/exercise ::merged)
([#:boot.user{:a 0, :b false} #:boot.user{:a [:i 0], :b [:b false]}] [#:boot.user{:a "", :b "N"} #:boot.user{:a [:s ""], :b [:s "N"]}] [#:boot.user{:a "u", :b "N"} #:boot.user{:a [:s "u"], :b [:s "N"]}] [#:boot.user{:a "", :b false} #:boot.user{:a [:s ""], :b [:b false]}] [#:boot.user{:a "8", :b true} #:boot.user{:a [:s "8"], :b [:b true]}] [#:boot.user{:a "u", :b "k"} #:boot.user{:a [:s "u"], :b [:s "k"]}] [#:boot.user{:a 2, :b "IzO"} #:boot.user{:a [:i 2], :b [:s "IzO"]}] [#:boot.user{:a 1, :b "8s0"} #:boot.user{:a [:i 1], :b [:s "8s0"]}] [#:boot.user{:a 0, :b "QsU1J"} #:boot.user{:a [:i 0], :b [:s "QsU1J"]}] [#:boot.user{:a 11, :b true} #:boot.user{:a [:i 11], :b [:b true]}])
boot.user=> (s/exercise ::anded)
clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries.
#2018-03-1304:58seancorfield(which is no surprise because the ::anded
spec won't accept anything)#2018-03-1304:58seancorfieldDoes that help @rahulr92?#2018-03-1305:09rahulr92@seancorfield Great explanation. This really helps. Thanks a lot!#2018-03-1316:50tbaldridgeI'm not sure if anyone else has had this issue before, but I have a really strange spec problem that has shown up twice: I have a spec that defines a or:#2018-03-1316:51tbaldridge(s/def ::val (s/or :int integer? :symbol :simple-symbol?))
#2018-03-1316:51tbaldridgeAnd then I put that inside something else:
(s/def ::coll-of-stuff (s/coll-of ::val))
#2018-03-1316:53tbaldridgeBut when I try to validate some data, explain fails and tells me:
(s/explain ::coll-of-stuff '[x 1 2])
"value [:int 1] failed to conform to integer? or symbol?"#2018-03-1316:54tbaldridgeThe problem is that something is trying to validate the conformed value? What's strange is I can't seem to reproduce this in a smaller situation. But so far it's always been with s/or
and every time it tries to validate the conformed value#2018-03-1316:57guyhow are you doing the symbols out of interest?#2018-03-1316:58tbaldridgedoing the symbols?#2018-03-1316:58guy(s/explain-data ::vals ['s 1])
#2018-03-1316:58guylike that?#2018-03-1316:58tbaldridgethat was part of the example here, I'm actually using keywords#2018-03-1316:58guyThe symbols in your data you are trying i mean sorry#2018-03-1316:59guySorry iām a little confused#2018-03-1316:59tbaldridgeSo here's the actual error message:
In: [:ast/select-attrs 0] val: [:attr :bar] fails spec: :ast/select-attr at: [:ast/select :ast/select-attrs :attr] predicate: simple-keyword?
In: [:ast/select-attrs 0] val: [:attr :bar] fails spec: :ast/select-attr at: [:ast/select :ast/select-attrs :child] predicate: map?
nil
#2018-03-1316:59tbaldridge(namespace names changed for security reasons#2018-03-1317:02tbaldridgeAnd the relevant specs:
(s/def ::select-attr (s/or
:attr simple-keyword?
:child (s/keys :req [::from ::select-attrs ::logical-attr]
:opt [::where])))
(s/def ::select-attrs (s/coll-of ::select-attr))
#2018-03-1317:02tbaldridgeSo what I don't understand is why it's saying [:attr :bar]
is failing to conform to simple-keyword?
#2018-03-1317:02tbaldridgeIt's like or
is trying to validate after it conforms the value or something.#2018-03-1317:03jgdaveyWouldnāt [:attr :bar]
be a (s/coll-of simple-keyword?)
, rather than a simple-keyword?
?#2018-03-1317:05tbaldridgethis is interesting, at the top level, there's a multi-spec
. If I remove that it seems to work fine.#2018-03-1317:06jgdaveyOh, I see, youāre saying that [:attr :bar]
is what youād get after conforming (s/conform ::select-attr :bar)
#2018-03-1317:06tbaldridge@jgdavey that's what's strange, the data I'm passing in looks like this [:foo :bar]. Hence the path in the error being [:ast/select-attrs 0] so it gets into the select-attrs
path, and then the first item doesn't validate because [:attr :bar] is not a simple-keyword.#2018-03-1317:06tbaldridgeright#2018-03-1317:07jgdaveyDoes using s/every
instead of s/coll-of
yield the same results?#2018-03-1317:08tbaldridgewow, yep, I changed my ::select-op spec to this:#2018-03-1317:08tbaldridge(s/def ::select-op (s/and
(fn [x]
(println "VALIDATE " x)
x)
(s/keys :req [::select-attrs ::from ::where])))
#2018-03-1317:09tbaldridgeThat's the top level spec. If I run it through multi-spec the values it tries to conform are [:attr :bar]. If I run it without multi-spec, the values are :bar
#2018-03-1317:09tbaldridgeSo multi-spec is doing something#2018-03-1317:15tbaldridgelet's see if I can get a minimal case now#2018-03-1317:27cap10morgan@alexmiller Does https://dev.clojure.org/jira/browse/CLJ-2054 need to be marked as "ready for testing?"#2018-03-1319:04Alex Miller (Clojure team)it is already marked thusly#2018-03-1320:25cap10morganah, great. thanks!#2018-03-1317:38tbaldridgewait...does:
(s/and ::foo
::bar)
pass the conformed value from ::foo to ::bar?#2018-03-1317:38tbaldridgeguess so: " Returns a spec that returns the conformed value. Successive
conformed values propagate through rest of predicates."#2018-03-1317:42tbaldridgeboom that's it, @jgdavey I used s/and
and that flows conformed values through the predicates (rather mis-leading name). Replaced it with s/merge
and it works fine#2018-03-1317:43jgdaveyInteresting. Thatās probably a subtly worth calling out in docs or something.#2018-03-1318:02tbaldridgeyeah, funny thing is that the docs are fairly clear, both s/and and s/merge mention it. I wonder if the behavior there changed somewhat recently#2018-03-1318:05tbaldridgelol, nope, it's been like that since the start#2018-03-1318:24mpenetI guess both could have options to control how/if the flow of conformed values is done#2018-03-1318:26mpenetI tripped on the behavior of s/merge when I started with spec, so the other way around. I am not alone either#2018-03-1319:00tbaldridgeYeah, I think part of it is that names are a tad overloaded in English. Merge makes sense to me since I'd expect (s/merge ...) to and things together. But it seems like a better name for s/and would be s/flow
or something like that.#2018-03-1319:01tbaldridgeMaybe we should rename s/merge
to s/smoosh
š#2018-03-1319:01Alex Miller (Clojure team)we considered having s/and
(non-flowing) and s/and->
(flowing) at one point#2018-03-1319:01tbaldridges/&
works the same way#2018-03-1319:01Alex Miller (Clojure team)unfortunately the current s/and
does the latter :(#2018-03-1319:02tbaldridgebut in the end, I didn't read the docs, so I'll slap my own hand this time#2018-03-1319:02Alex Miller (Clojure team)yes, s/and and s/& flow. s/merge does not#2018-03-1319:02Alex Miller (Clojure team)s/merge is conceptually pushing the value through all specs āin parallelā#2018-03-1319:03Alex Miller (Clojure team)so flowing doesnāt make sense (except conforming returns the conformed result of the last spec which is hard to reconcile with that view)#2018-03-1319:06tbaldridgeSo what's the common use-case for flowing specs? I think I've only ever used them with s/conformer, which I tend to avoid these days#2018-03-1319:12Alex Miller (Clojure team)if you flow through a regex youāll get structure which you can then easily apply predicates to#2018-03-1320:02bronsanaming vars &
is not the best idea#2018-03-1320:15Alex Miller (Clojure team)it was originally called andre :)#2018-03-1320:19mishaflowing is handy when you have s/keys spec checking map's keys and vals independently, and then you apply predicate checking inter-kv-dependencies.#2018-03-1320:20mishaalthough, thinking about it a moment longer ā this use case is sort of orthogonal to flowing :D#2018-03-1320:28tbaldridge'Is it pronounced "an-dray" or "and-re"?'#2018-03-1320:28tbaldridgeWould be the new "a-sosh" or "a-sock". š#2018-03-1320:49Alex Miller (Clojure team)the former#2018-03-1401:32gfredericksif it was big you could call it andre the giant macro#2018-03-1401:33gfredericks@bronsa what's wrong with &
?#2018-03-1408:31bronsasyntax quote doesn't qualify it#2018-03-1411:56gfrederickswhy on earth would that be true?#2018-03-1412:12bronsabecause otherwise youād have to `(fn [a# ~'& b])
#2018-03-1412:24gfredericksaw dang
I figured it somehow related to &env
and &form
; forgot about destructuring#2018-03-1412:25gfredericksso this only affects users who are refer
ing s/&
#2018-03-1412:25bronsayeah#2018-03-1412:25bronsaor macros defined in the same ns#2018-03-1412:25bronsabut tbh the same happens when naming a var def
#2018-03-1412:25bronsawhich I also donāt like#2018-03-1412:26gfredericksthat's a different underlying cause, right?#2018-03-1412:26bronsano same reason#2018-03-1412:27gfredericksI guess I figured special formedness was checked in one place and &
ness somewhere else; are there any other exceptions?#2018-03-1412:28bronsatheyāre all in one place#2018-03-1412:28bronsauser=> (keys Compiler/specials)
(& monitor-exit case* try reify* finally loop* do letfn* if clojure.core/import* new deftype* let* fn* recur set! . var quote catch throw monitor-enter def)
#2018-03-1412:28bronsa&
is considered a special form by the reader#2018-03-1412:33gfrederickswelp.#2018-03-1413:47mikerodInsightful#2018-03-1422:28dpsuttoncan you spec cljs.core/+
?#2018-03-1422:31dpsuttonand i realize that's a seemingly silly question but we've got js/big floating around which allows for arbitrary precision decimal arithmetic but the built in operators do not behave well. So i'd like to spec it and use it to help verify that no js/Big's are leaking into any arithmetic anywhere#2018-03-1500:45gfredericksI don't think you could instrument it (which I assume is the point) at runtime, since it's normally inlined#2018-03-1500:46gfredericksI'm curious how easily compilation could be modified to avoid the inlining
does cljs spec do runtime instrumentation at all? I don't even know that#2018-03-1505:11didibusHow does one go about adding functionality to spec? I tried to extend s/keys so that it also did a contains check, basically making it a closed-key spec. I did so by wrapping s/keys in a separate macro which takes the same input then s/keys and returns an anded spec of keys and one which will validate that no extra keys are present. But composing macros like that isn't great. And so now I'm trying to extend it even more, so that my closed-keys macro can be used in a s/merge style. Anyways, bottom line is, the experiment has been tedious. So I was thinking, maybe instead of building on top of s/keys, I can just build a whole new spec, but how do you do that?#2018-03-1505:13didibusI also found it frustrated all the specs were macros, which some of them were plain old functions that took clojure data. Like s/keys could have taken a map, or a vector of vectors, instead of a custom macro DSL.#2018-03-1505:57ikitommi@didibus current version of spec doesnāt really support extensions. But you can reify clojure.spec.alpha/Spec
and copy-paste current spec internals to make something similar.#2018-03-1506:00ikitommiI think s/keys
could be rewritten as a function, as all keys need to be referenced via keyword. Something like: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/data_spec.cljc#L39-L68.#2018-03-1506:01ikitommi.. actually the closed keys would be easy to implement on top of that via extra :closed?
key or similar. hmm.#2018-03-1513:44guyIf you were writing an fdef for a simple function that takes a collection of strings (at least one) could you use (s/+ string?)
instead of (s/coll-of string?)
for the :args
.#2018-03-1513:45guy(s/fdef my-fn
:args (s/cat :item (s/+ string?))
or
:args (s/cat :item (s/coll-of string?))
..)
#2018-03-1515:59madstapThose are different though, (s/cat :item (s/spec (s/+ string?)))
works the same as the latter.#2018-03-1516:16guy@U0J9LVB6G ah thanks!#2018-03-1516:22guy@U0J9LVB6G what was the difference out of interest? What does s/spec do to it?#2018-03-1516:23madstaps/spec
makes it nested#2018-03-1516:23madstap(s/conform (s/cat :item (s/+ string?)) ["a" "b"])
(s/conform (s/cat :item (s/spec (s/+ string?))) [["a" "b"]])
(s/conform (s/cat :item (s/coll-of string?)) [["a" "b"]])
#2018-03-1516:24guyahhhhh right got it thanks#2018-03-1514:07Alex Miller (Clojure team)absolutely#2018-03-1514:07Alex Miller (Clojure team)I would even say thatās preferred (although you wonāt really see much practical difference)#2018-03-1514:14guyok great thanks#2018-03-1514:26bbrinck@guy keep in mind coll-of
also has :min-count
arg which might be useful here#2018-03-1514:26bbrinck(although more verbose than +
)#2018-03-1514:26guyah yes thats right thank you!#2018-03-1514:33Alex Miller (Clojure team)generally when specāing function signatures, youāre specāing syntax, and spec regex ops are the best match so I usually try to stick to those in fdef args#2018-03-1514:33guyok so in general favour spec regex ops then?#2018-03-1514:40Alex Miller (Clojure team)when specāing function args, favor regex op specs#2018-03-1514:41Alex Miller (Clojure team)when specāing other kinds of data, I favor collection specs#2018-03-1514:42guyCan you give me an example of the latter sorry#2018-03-1514:42guyDo you mean if you were creating a spec for some sample data perhaps?#2018-03-1514:49Alex Miller (Clojure team)yeah#2018-03-1514:50guythanks for the help!#2018-03-1515:39borkdudeI have a spec:
(s/keys :req-un [::title
::content]
:opt-un [::icon
::widget-class
::settings
::collapsed?
::dropdowns
::controls
::preview?
::tabs
::collapse-opts
::help])
Now I want to have a warning for myself when someone passes something that has more keys than I expected. Can I use my spec for this or do I have to duplicate the keys?#2018-03-1515:40borkdudeIām refactoring this component, so other arguments are suspicious#2018-03-1515:44seancorfield@borkdude You can extract the list of keys from the spec form itself, and write a predicate that checks those are the only keys. Stu Halloway posted an example on the mailing list a while back I think... Probably in a Gist somewhere...#2018-03-1515:46taylorhttps://github.com/gfredericks/schpec/blob/b2d80cff29861925e7e3407ef3e5de25a90fa7cc/src/com/gfredericks/schpec.clj#L13-L35 hereās a macro to make closed keys specs and hereās Stuās gist which I think was more meant for finding āforgottenā key specs https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2018-03-1515:47seancorfield@taylor Ah, yes. I misremembered. Stu's code was to find any keys in a spec that did not have definitions.#2018-03-1515:48borkdudeHey @taylor, great answers on Stackoverflow btw š#2018-03-1515:49seancorfieldBut this code from Stu's Gist is similar to what you'd need (if you don't want to go with shpec
): https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#file-missing_keys_specs-clj-L31-L33#2018-03-1515:50borkdude@taylor Oh thatās great. I think I will adopt Schpec#2018-03-1516:05borkdudeI want to use schpec in clojurescript, but now I also have to bring in test.check.. ok#2018-03-1516:07borkdudeHmm, I already have thatā¦ maybe I should upgrade. I get errors about clojure/spec/gen/alpha
#2018-03-1516:09gfredericksWhat sort of errors?#2018-03-1516:10borkdudeā¢ public/js/app.js
WARNING: Use of undeclared Var com.gfredericks.schpec/limit-keys at line 53 src-cljs/dre/components/widget.cljs <-- this is my namespace
WARNING: No such namespace: clojure.spec.gen.alpha, could not locate clojure/spec/gen/alpha.cljs, clojure/spec/gen/alpha.cljc, or JavaScript source providing "clojure.spec.gen.alpha" at line 53 src-cljs/dre/components/widget.cljs
WARNING: Use of undeclared Var clojure.spec.gen.alpha/fmap at line 53 src-cljs/dre/components/widget.cljs
#2018-03-1516:11borkdudemy ns:
(ns dre.components.widget
(:require
[clojure.spec.alpha :as s]
#_[com.gfredericks.schpec :refer-macros [excl-keys]]
[dre.page-util :refer [open value-of in-iframe?]]
[dre.react-util :refer [Collapse DropdownButton MenuItem]]
[reagent.core :as r]
[taoensso.timbre :refer-macros [info warn debug]])
(:require-macros
[com.gfredericks.schpec :refer [excl-keys]]))
#2018-03-1516:11borkdudeI first tried the one with #_
but then I get:
No such namespace: com.gfredericks.schpec, could not locate com/gfredericks/schpec.cljs, com/gfredericks/schpec.cljc, or JavaScript source providing ācom.gfredericks.schpecā in file src-cljs/dre/components/widget.cljs
#2018-03-1516:19gfredericksDoes clojure.spec.gen.alpha exist in cljs?#2018-03-1516:19borkdudehm, it appears not no#2018-03-1516:20gfredericksInteresting.#2018-03-1516:20borkdudeah, cljs.spec.gen.alpha
does exist#2018-03-1516:21borkdudehave you tried this in clojurescript?#2018-03-1516:23borkdudeprobably limit-keys
should be in a cljc
file?#2018-03-1516:23borkdudeand some other adaptations#2018-03-1516:24Alex Miller (Clojure team)cljs should rewrite the clojure to cljs one automatically, but you would need it in a cljc file#2018-03-1516:24Alex Miller (Clojure team)iirc#2018-03-1516:25borkdude@gfredericks you could also put everything in a .cljc file with the help of macrovichā¦ just wrap every macro in macros/deftime
#2018-03-1516:26gfredericks@borkdude I have not tried it in cljs; schpec hasn't gotten a lot of attention. I had the silly idea that I could just make a generic bucket to put things in and then people would put things in it and use it, but I think people prefer to make libraries with more specific purposes#2018-03-1516:26gfredericksbut I'm happy to incorporate any improvements you have, like making sure it works in cljs#2018-03-1516:28borkdudeIāll make an attempt#2018-03-1516:30mgrbytehi, I'm wanting to make a model in spec of my data where the values for 2 keys in my map are dependant on another key's value. I've watched Stu's blog post about using gen/bind
and gen/fmap
; are there any other examples someone could point me to?#2018-03-1516:51anmonteiro@alexmiller weāre very confused about this behavior, wondering if it has come up before:
user> (s/conform #{11 false :foo} false)
:clojure.spec.alpha/invalid
#2018-03-1516:51anmonteirothe behavior seems to be in this condition: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L875-L877#2018-03-1516:51Alex Miller (Clojure team)you should never use logically false values in literal sets#2018-03-1516:52Alex Miller (Clojure team)(and this applies to more than spec - some
is another case)#2018-03-1516:52anmonteiroyeah I guess that really messes things up#2018-03-1516:52Alex Miller (Clojure team)use false?
as a predicate#2018-03-1516:52anmonteirothanks#2018-03-1516:52Alex Miller (Clojure team)or nil?
#2018-03-1516:52Alex Miller (Clojure team)or s/nilable
#2018-03-1516:53Alex Miller (Clojure team)this particular case does not typically come up in general use very often in my experience#2018-03-1516:54anmonteirowhat you said actually makes total sense, it just got me puzzled here for a minute#2018-03-1516:54anmonteiroI guess weāll always need to return logical true
for valid?
to work#2018-03-1516:57Alex Miller (Clojure team)yes#2018-03-1517:03borkdude@gfredericks This is how far I got, but I still get errors:
https://github.com/borkdude/schpec/blob/master/src/com/gfredericks/schpec.cljc#2018-03-1517:03borkdudeThe errors being:
WARNING: No such namespace: clojure.spec.gen.alpha, could not locate clojure/spec/gen/alpha.cljs, clojure/spec/gen/alpha.cljc, or JavaScript source providing "clojure.spec.gen.alpha" at line 54 src-cljs/dre/components/widget.cljs
WARNING: Use of undeclared Var clojure.spec.gen.alpha/fmap at line 54 src-cljs/dre/components/widget.cljs
when I call the macro excl-keys
#2018-03-1517:05borkdudeIāll try to change the require into:
(:require [clojure.spec.alpha :as s]
[clojure.spec.gen.alpha :as sg]
[clojure.set :as set]
#?(:clj [net.cgrand.macrovich :as macros]))
which I think should ājust workā ā¢ in cljs as well#2018-03-1517:06borkdudehmm same errors#2018-03-1517:08borkdudeI give up for nowā¦ short on time#2018-03-1517:40gfredericksMy apologies.
cljc and macros can be a pretty gnarly combination#2018-03-1518:28borkdudeNo problem man. Iām sure it could work with a small change.#2018-03-1518:28borkdudeI also ran into this issue with cljc, macros and clojurescript: https://dev.clojure.org/jira/browse/CLJS-2636#2018-03-1520:26pabloreI have a spec like this: (s/def (s/map-of ::id ::person)
where ::id
must be a property of ::person
. Could anyone help me making the spec with s/with-gen
so I can generate this map for testing?#2018-03-1520:29guy(s/def :entity.person/id string?)
(s/def :entity/person (s/keys :req [:entity.person.id])
#2018-03-1520:29guylike that do you mean?#2018-03-1520:29guywhat do you mean by property of person?#2018-03-1520:30pabloreyes#2018-03-1520:30pabloreperson is defined in terms of ::id#2018-03-1520:31pabloreso when I generate a map of ids and persons, the map of id-persons must have sense#2018-03-1520:41pabloreIndexed collections are structures I've come to make a lot of times doing clojure.
Ie, I rather do (get-in state [:people id :name])
than have to find the vector index every time.#2018-03-1520:34bbrinck@seancorfield @borkdude keep in mind that any attempt to analyze specs wonāt work for multi-specs in CLJS, at least AFAICT#2018-03-1520:35bbrinck(which limits our ability to do things like look for undefined specs, or to automatically add checkers for missing values)#2018-03-1520:42Alex Miller (Clojure team)@pablore I donāt have time to give you a full answer, but I would start with a generator that creates a collection of ::person (which will have ::idās in it). then gen/fmap over that generator to construct the map from the ids in the generated person entities#2018-03-1520:43Alex Miller (Clojure team)something like (gen/fmap (fn [ps] (zipmap (map ::id ps) ps)) #(s/gen (s/coll-of ::person)))
#2018-03-1520:59pabloreThis almost worked! Just has to replace ::id with :id, because (s/keys) generates with keys with single colons#2018-03-1521:01pablore(defn gen-indexed
[spec idx]
(gen/fmap (fn [xs] (zipmap (map idx xs) xs))
(s/gen spec)))
#2018-03-1521:01pablorethis would be a more generic version#2018-03-1521:59dottedmagHow do I write a spec for [[1]]
?#2018-03-1522:00dottedmagObviously my data is more complicated than that ā it's an number of vectors enclosed one in other. I'm trying to use spec for validation, so it's an external data type, not something I can change.#2018-03-1522:00Alex Miller (Clojure team)#{[[1]]}
#2018-03-1522:00Alex Miller (Clojure team):)#2018-03-1522:01Alex Miller (Clojure team)seriously though, do you know how many levels of vectors?#2018-03-1522:01dottedmagTwo levels, it's a parsed CSV.#2018-03-1522:02dottedmagNot a "nice" uniform CSV, a complicated one. It's got a multiline header and a multiline footer, and empty strings delimiting data sections.#2018-03-1522:03Alex Miller (Clojure team)well, spec the inner levels, then spec the outer levels out of those :)#2018-03-1522:04Alex Miller (Clojure team)(s/def ::line (s/coll-of string? :kind vector?))#2018-03-1522:04dottedmagThat's what I tried. s/cat
on outer level, then s/cat
on inner level, and first predicate in inner level tried to match the whole line, not the first element of an array.#2018-03-1522:06dottedmagLet me shorten it a bit#2018-03-1522:09dottedmag(s/def ::header-fields (partial = ["Account Number"]))
(s/def ::account-number (s/and string?
(partial re-matches #"^[0-9]{16}$")))
(s/def ::header-info (s/cat ::account-number ::account-number))
(s/def ::delimiter (partial = [""]))
(s/def ::header (s/cat ::header-fields ::header-fields
::header-info ::header-info))
(s/def ::statement (s/cat ::header ::header
::delimiter ::delimiter))
#2018-03-1522:10dottedmagThis ought to match [["Account Number"] ["1234123412341234"] [""]]
, but apparently the specs for sequences are flattened.#2018-03-1522:10dottedmagBecause (s/explain ::statement data)
In: [1] val: ["1234123412341234"] fails spec: :bacc.foo/account-number at: [:bacc.foo/header :bacc.foo/header-info :bacc.foo/account-number] predicate: string?
#2018-03-1522:13dottedmagAh, if I add (s/and (s/coll-of string? :kind vector) ...)
to the ::header-info
then in succeeds.#2018-03-1522:13dottedmag@alexmiller Thanks#2018-03-1605:14ikitommiToyed more with spec coercion, with idea that coercion functions could be defined in spec meta-data so that the specs could remain self-contained . Currently requires spec-tools to work.
(require '[spec-tools.core :as st])
(require '[spec-tools.conform :as conform])
(def spec
(st/spec
{:spec string?
:description "a string spec"
::conform/json #(str %2 "-json")
::conform/string #(str %2 "-string")}))
(st/conform spec "kikka")
; "kikka"
(st/conform spec "kikka" st/json-conforming)
; "kikka-json"
(st/conform spec "kikka" st/string-conforming)
; "kikka-string"
#2018-03-1618:42jjfineAnyone know why my function doesnt-work
throws a java.lang.AssertionError
:
(s/def ::bar (fn [x] true))
(defn return-true [_] true)
(defn doesnt-work [something & bar]
{:pre [(s/assert ::bar bar)]}
true)
(defn works [something & bar]
{:pre [(return-true bar)]}
true)
(deftest failing-test
(is (works 1))
(is (thrown? AssertionError (doesnt-work 1))))
This didn't seems spec-specific at first but this test shows I'm probably misunderstanding something about s/def#2018-03-1618:47taylor@jjfine I think the problem is assert
is returning its input if itās valid, and bar
is nil
, and :pre
is interpreting nil
as an assertion failure#2018-03-1618:47taylortry {:pre [(s/valid? ::bar bar)]}
#2018-03-1618:47jjfineoof#2018-03-1618:48jjfinethanks#2018-03-1718:36lgesslerhi all, I have an xml schema describing a data format I want to translate into a clojure spec. i should be able to translate it by hand, but i was wondering if anyone knew of an easier way? the closest thing I found takes the XSD and then turns it into some kind of clojure schema, but it's not a spec and it didn't seem to produce the right output for my XSD https://github.com/kolov/xelery#2018-03-1914:39mgrbyteI have 3 specs describing a possible "identity" for an entity. I then have an "identity" spec along the lines of (s/def ::identity (s/or :domain/id1 ::id1 :domain/id2 ::id2 :domain/id3 ::id3)
. one of the specs can't use the built-in generator, how would I go about providing a custom generator that works for s/or
? I've managed to write some custom generators before for maps, s/and specs, but can't get my head around how to start writing one for this s/or case, and google hasn't yielded anything so far. Does anyone have any pointers pls?#2018-03-1914:48mgrbyteI'm guessing I need to figure out how to use a combination of gen/tuple and gen/one-of š¤#2018-03-1915:04bbrinck@mgrbyte Can you call with-gen
inside the or
?#2018-03-1915:04bbrinck(require '[cljs.spec.alpha :as s])
(require '[clojure.test.check])
(s/def ::id1 string?)
(s/def ::id2 pos-int?)
(s/def ::identity (s/or :id1 ::id1 :id2 ::id2))
(map first (s/exercise ::identity)) ;; => (1 2 "" 2 "" 2 "F91Q6p" "o6" "oet" "DE7")
(s/def ::identity (s/or :id1 (s/with-gen ::id1 (fn [] (s/gen #{"foo" "bar"})))
:id2 ::id2))
(map first (s/exercise ::identity)) ;; => (1 "bar" 2 2 2 15 31 "bar" "bar" 75)
#2018-03-1915:04bbrinckWould that work for your case?#2018-03-1916:16mgrbyte@bbrinck Trying to keep generators in tests only, since I'm using minter.strgen/test.check stuff to help make the gen for "::id1" .
I've managed to get somewhere near what I need with (gen/one-of [gen1 gen2 gen3])
, but as mentioned in the spec guide, need it's a good idea to generate s/conform
like example for any s/or like specs: "For r specs that have a conformed value different than the original value (anything using s/or, s/cat, s/alt, etc) it can be useful to see a set of generated samples plus the result of conforming that sample data."#2018-03-1916:17mgrbyteSo I'll need to interleave the specs and generators. but think that will do it#2018-03-1916:31bbrinck@mgrbyte I havenāt tried it out, but I believe you can use (require '[clojure.spec.gen.alpha :as gen])
and that will be OK in dev and test#2018-03-1916:32bbrinck(it lazy loads test.check
when generator is called, not defined, but presumably you wonāt be invoking generators outside of tests)#2018-03-1916:33bbrinckMight simplify your implementation, although I donāt know if you want to keep generators in test for dependency reasons, or for other reasons like attaching different generators at different tiems#2018-03-1916:37ikitommiHi. Did someone make a stab at spec bijections? Any lessons learned? Bumped into a actual need: need to uncoerce path-parameters in reverse-routing into string based either on the value type or on the defined spec.#2018-03-1916:38ikitommiWith spec, I guess we could put both the encode & decode functions into spec metadata, something like:
(st/spec
{:spec keyword?
:description "a bijecting keyword"
::encode/string #(name %2)
::decode/string #(keyword %2)})
#2018-03-1916:57gfredericksst/spec
? does that exist?#2018-03-1916:58gfredericksI've been talking about bijections a lot; I was thinking of setting up a generic library for composing bijections, that's not spec-specific#2018-03-1916:58gfredericksif you have ideas about how to tie them with spec, that would be interesting#2018-03-1916:59ikitommist/spec
is from spec-tools, until spec meta-data appears.#2018-03-1916:59gfredericksthat's a 3rd-party library?#2018-03-1916:59ikitommihttps://github.com/metosin/spec-tools#2018-03-1917:00ikitommidoes already the encode/coercion, could do the other way too.#2018-03-1917:01gfredericksmy starting point is assuming that the most common use for bijections would be where you have some internal canonical representation of something, and you want to specify how to transform it to/from other "contexts", which could be things like json, jdbc, cvs, etc.#2018-03-1917:01ikitommiGeneric library/foundation would be nice, as this is needed with Schema too.#2018-03-1917:02gfredericksso when deciding how to integrate that with spec, one question that would come up would be whether it's worth automating the process of transforming the specs themselves so that you get context-specific versons of the specs, or merely to automate the process of writing the functions that take values in each direction#2018-03-1917:02ikitommiyes, the name of the (`string` in ::encode/string
) is the context name.#2018-03-1917:03gfredericksthe downside of actually generating specs for each context is that it's more complicated and probably more work for the programmer
the upside is I think you can get more features, like validating closer to the edge, and giving validation errors based on that representation rather than the internal one#2018-03-1917:04ikitommiI tried the context-versions of spec, but coudnāt get it working.#2018-03-1917:04ikitommifailed with the qualified keys with s/keys
.#2018-03-1917:05gfrederickss/keys is a big barrier; especially if you want to convert the keys to strings like in a json context#2018-03-1917:05ikitommi(s/def ::id keyword?)
(s/keys :req [::id])
#2018-03-1917:06ikitommiif you have that, you canāt have a different ::id
as the full key names are exposed outwards.#2018-03-1917:07ikitommiwhy would the validation be closer to the edge?#2018-03-1917:07gfredericksyeah you'd need a new namespace for each context probably#2018-03-1917:08ikitommiā¦ or a different registry.#2018-03-1917:08gfrederickse.g., if I write a public JSON API and biject my internal spec to a JSON version of that spec, I can theoretically validate the user input using the json spec, before trying to transform/coerce the data to the internal representation#2018-03-1917:08gfredericksyou can also then provide validation errors to the user in a way that's specific to the json representation, rather than the less helpful internal representation#2018-03-1917:09gfrederickshere's what I came up with a few months ago; I did a proof of concept of integrating it with some code at work
https://gist.github.com/gfredericks/358c39478f104281a6364f446f5b3c6b#2018-03-1917:09ikitommioh, true that. Would mean multiple walks on the same model, but in most cases, the perf impact would not matter.#2018-03-1917:11ikitomminice draft!#2018-03-1917:11ikitommiare there bijections in other languages?#2018-03-1917:12ikitommicould read some theory/reasoning to understand this.#2018-03-1917:28gfredericksI've not seen it, other than I've speculated you could typecheck them in a dependently typed language#2018-03-1917:28gfredericksthere is of course the relevant set theory stuff: https://en.wikipedia.org/wiki/Bijection,_injection_and_surjection#2018-03-1917:30gfredericksI've also wondered about separating the bijective parts from surjective parts (the times when you want to drop information, e.g., the order of keys in a query string)#2018-03-1917:30gfredericksif the surjections are formalized, you can use them to create generators of all the variant representations of something#2018-03-1918:22stathissiderissuper-interesting stuff, I think I should do some reading on this#2018-03-1918:30ikitommithanks for the pointer! In my test, the transforming functions take two arguments: the spec and the value. This allows the transformation to extract information from the spec to do the transformation.#2018-03-1918:31ikitommifor example, stripping out non-defined keys from keys-spec needs the set of keys defined:
(defn strip-extra-keys [{:keys [keys]} x]
(if (and keys (map? x))
(select-keys x keys)
x))
#2018-03-1918:42gfredericksI'd always approached that by building the function at compile-time#2018-03-1918:42gfredericksI should also point out this old effort in case it hasn't been seen: https://github.com/gfredericks/schema-bijections#2018-03-1918:43gfredericksI used that for real code at my old job, and it worked pretty good but made the code difficult to read#2018-03-1919:09gfredericksValidating before converting isn't any extra walks compared to validating after converting I don't think#2018-03-1916:59Drew VerleeKnowing full well this doesnāt make sense. How would you write a spec for a collection of maps where each maps keys were
1) unique
2) represents a value themselves
e.g
[data, data, data, ā¦]
where data is something like {5 6}
#2018-03-1917:04Alex Miller (Clojure team)well you would start with (s/coll-of (s/map-of int? int?))
#2018-03-1917:05Alex Miller (Clojure team)each individual maps keys are (by definition) unique, but I assume you mean all of the keys over all the maps are unique#2018-03-1917:05Alex Miller (Clojure team)not sure what #2 means#2018-03-1917:06Alex Miller (Clojure team)for checking the uniqueness constraint, s/and another predicate that checks whatever property you need - something like #(apply distinct? (mapcat keys %))#2018-03-1917:22Drew Verlee> each individual maps keys are (by definition) unique, but I assume you mean all of the keys over all the maps are unique
correct#2018-03-1918:38ghadiyou can't have duplicate keys in a map @drewverlee#2018-03-1918:39ghadioh I didn't see the collection of maps requirement. yeah, s/and
is your friend here#2018-03-1918:52roklenarcicI'm trying to write my own Spec protocol implementation. Is there somewhere where Spec and Specize protocols' contracts are explained?#2018-03-1919:05Alex Miller (Clojure team)no, because they are likely to change#2018-03-1919:08misha@mgrbyte you can override generator just for ::id1 during exercise time
(s/def ::id1 string?)
(s/def ::id2 int?)
(s/def ::id (s/or :id1 ::id1 :id2 ::id2))
(map first (s/exercise ::id 10 {::id1 #(s/gen #{"foo"})}))
;;=> (0 -1 1 "foo" 0 -7 "foo" -1 "foo" 0)
#2018-03-1919:09borkdudeI have:
(s/def ::key
(s/and keyword?
#(= "widget"
(namespace %))))
(s/fdef widget
:args
(s/cat :opts
(s/keys
:req-un [::title
::content]
:opt-un [::key
::icon
::widget-class
::settings
::collapsed?
::dropdowns
::controls
::preview?
::tabs
::collapse-opts
::help])))
but ::key
does not seem to get checked when itās provided#2018-03-1919:12taylorthis is working for me locally, w/a dummy, instrumented widget
function#2018-03-1919:13taylorIn: [0 :key] val: :foo fails spec: : at: [:args :opts :key] predicate: (= "widget" (namespace %))
#2018-03-1919:14borkdudehmm yeahā¦ for me too when I isolate this#2018-03-1919:16borkdudeworks now!#2018-03-1919:16borkdudeI guess some reloading glitch#2018-03-1921:18cch1Is there an idiomatic way of expressing a spec for āanythingā other than (constantly true)
?#2018-03-1921:19cch1Iām trying to define a spec for a value that must ultimately be usable as a key in an associative structure.#2018-03-1921:19cch1In Clojure, I think that is pretty much everything.#2018-03-1921:20bronsaany?
is what you want#2018-03-1923:18camachomI'm trying to pass a vector to s/keys
, but it keeps failing. Works great if I just write it out though. Something like this: (def required
[:foo/city :foo/country :foo/street])
(spec/def :foo/form (spec/keys :req required
:opt []))
#2018-03-1923:18camachomany ideas as to what i'm doing wrong?#2018-03-1923:28alexisvincentHi guys, got a bit of a modelling problem. How do you build up specs with context dependent information.
E.g. Say youāre building a space for card details, say a map of the keys :card/number
:card/expiry
:card/name
:card/cvv
.
Now we want this to be a nice composable spec called :card/details
. But in some contexts :card/cvv
is required and in others it isnt. So The :card/details
spec is really a partial spec with requiredness of :card/cvv
dependent of context.
We might then say make two specās :card/details
and :card/full-details
. But this isnāt really scalable. It means whenever we have branching up the composition tree we need to rename things for all cases, giving an exponential explosion. Is there any way to build āpartialā specs where specs higher up the composition tree provide more or overriding data for specs they depend on.
Similar to dependent types.#2018-03-2000:11bbrinckThis feature still requires a lot of work, but soon(ish), Expound will be able to provide examples of valid clojure code https://gist.github.com/bhb/f637ef589ef3ac3d2ca5a883fafc2c12#2018-03-2000:12bbrinckHowever, this adds a dependency on test.check to use expound at dev time. Would you prefer to have this be a hard dependency OR would you prefer expound to disable this feature if you donāt have test.check available?#2018-03-2000:24seancorfield@mcama200 spec/keys
is a macro so it takes literal code forms, not variables.#2018-03-2000:25seancorfieldWhat you can do instead is to use the literal vector in s/keys
and then call s/form
(I think?) on the spec itself and walk that to get the list of required keys back out of a spec definition.#2018-03-2000:35camachomawesome, thanks for the help!#2018-03-2000:29seancorfield@alexisvincent The short answer is: specs are currently pretty static so you have to have multiple specs -- or specify that :card/cvv
is :opt
in the base spec and then wrap it in another spec that uses s/and
and a predicate that "requires" that key be present.#2018-03-2000:30seancorfield(s/def :card/full-details (s/and :card/details #(contains? % :card/cvv)))
something like that (untested)#2018-03-2009:12borkdudeI guess spec could eat its own dog food?
boot.user=> (s/fdef foo (s/cat :a1 string?))
java.lang.IllegalArgumentException: No value supplied for key: (s/cat :a1 string?)
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: No value supplied for key: (s/cat :a1 string?), compiling:(null:1:1)
#2018-03-2013:12Alex Miller (Clojure team)Actually it canāt, unless you enjoy infinite recursion#2018-03-2010:47mpenetbeen there done that: my personal favorite is (s/def foo any?) or (s/fdef ::foo :args (s/cat))#2018-03-2010:48mpenetboth silently fail, which makes it even better#2018-03-2010:57alexisvincent@seancorfield Thanks for the answer, the s/and
is neat, and will help reduce branching factor, but I donāt think itās a long term behaviour. I can use functions that return specs, but loose out on specās repository. Do you know if dynamism will be added to named specs in the future, or if it even needs to be?#2018-03-2011:06borkdudeI have the current spec (simplified):
(s/fdef widget
:args
(s/cat :opts
(s/keys
:req-un [::title
::content]
:opt-un [
::init-collapsed?
::fixed?])))
How do I say init-collapsed?
and fixed?
are mutually exclusive?#2018-03-2011:07mpenetbut they can be both absent as well?#2018-03-2011:07borkdudeyes#2018-03-2011:08borkdudeif you say the widget must be fixed, init-collapsed? can not be true, because the widget wonāt be visible at all that way#2018-03-2011:09borkdudeI could just emit a warning outside of spec, but maybe thereās a nice solution#2018-03-2011:09borkdudeI also donāt want to introduce nesting in the arguments#2018-03-2011:09borkdudeI could maybe use s/conformer
?#2018-03-2011:10mpenetI guess you need a separate pred in s/and since opt-un will not accept (or ...) etc#2018-03-2011:11borkdudeyes,
(fn [m]
(not (and (:init-collapsed? m)
(:fixed? m))))
works with s/and
thanks#2018-03-2011:14mpenetI imagine a multispec could do it too, but that's prolly too much complexity for such as simple check#2018-03-2014:44acronjava.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {}
#2018-03-2014:44acronMost depressing error ever.#2018-03-2014:50Alex Miller (Clojure team)agreed :)#2018-03-2014:51Alex Miller (Clojure team)unfortunately, the current linkage between spec and test.check prevents good error reporting on where this is happening. there are plans to make this better, but it requires changes in both projects.#2018-03-2015:26gfredericks@alexmiller I'm curious how what you just described relates to an earlier comment that spec can't enscricten its version requirements of test.check#2018-03-2015:26Alex Miller (Clojure team)enscricten = ?#2018-03-2015:28danielnealmake more scrict#2018-03-2015:29Alex Miller (Clojure team)and scrict = ?#2018-03-2015:33danielnealopposite of pecmissive#2018-03-2017:59misha@alexisvincent define all your fieldsā specs, and then have several s/keys specs for different combinations of those fields. Then you might want to have the card spec, which is s/or around those s/keys#2018-03-2018:05mishaRemember, you can have keys with different namespaces in the same map/object, so donāt paint yourself into corner with unified namespace ā:cardā when some attributes might be better named, like :amex.card/number, because itāll have different spec and generator.#2018-03-2100:45gfredericks@alexmiller sorry, meaning that spec would, e.g., require test.check >= 0.10.0#2018-03-2100:45gfredericksyou made a comment a few weeks back that implied that wouldn't be acceptable#2018-03-2100:45alexisvincent@misha š Thanks. Im just concerned about the name explosion when composing up the tree#2018-03-2100:53Alex Miller (Clojure team)@gfredericks I wouldnāt make a range version but we could certainly bump the requirement #2018-03-2100:53gfredericksthe difference between those two things is whether a max version is specified?#2018-03-2100:53Alex Miller (Clojure team)I just donāt like version ranges#2018-03-2100:54seancorfieldNo one likes version ranges š#2018-03-2100:54Alex Miller (Clojure team)The only people that like version ranges are people that havenāt used them :)#2018-03-2100:58gfredericks:+1: thanks#2018-03-2101:00kahunamooreI'm looking to do some visualization of specs and by default would think using the reader would be the most straight forward way to consume/read the specs. Is there a better way? A library perhaps that produces a higher level representation? Thoughts?#2018-03-2102:13andy.fingerhut@kahunamoore I have not used this tool, and not sure if it is what you are looking for, but it sounds at least related: https://github.com/jebberjeb/specviz#2018-03-2102:14andy.fingerhutLearned about it from Stuart Halloway's talk "REPL driven development": https://github.com/matthiasn/talk-transcripts/blob/master/Halloway_Stuart/REPLDrivenDevelopment.md#2018-03-2103:15seancorfieldOh, I didn't know there was a transcript of that! Cool!#2018-03-2111:24tayloryesterday I helped a coworker spec some financial calc functions, then we immediately found some bugs by check
ing them š#2018-03-2111:26tayloraside: will the default generator for number?
predicate generate all/most numeric types (like ratios)?#2018-03-2111:29taylordoesnāt seem like it generates ratios#2018-03-2111:49borkdudeWould it make sense if Spec could aid in producing human readable errors?
https://gist.github.com/borkdude/89799a764a89af5c0be1a5745be58b4d#2018-03-2111:57ikitommi@borkdude related: https://dev.clojure.org/jira/browse/CLJ-2115#2018-03-2114:00hkjelsthe above obviously does not work#2018-03-2114:00hkjelsI could use this and digg into the meta, but Iād prefer not to#2018-03-2114:05Alex Miller (Clojure team)what problem are you trying to solve?#2018-03-2114:14hkjelsI output form-controllers based on the type of a specification. So for simple spec definitions like (spec/def ::placeholder string?)
I can output a textfield because I know itās a string with (type the-def)
, but for definitions that use with-gen
; type
returns nil
#2018-03-2114:24bbrinck@hkjels Can you talk more about how you determine the type in the simple case? Are you inspecting the spec itself, or an example generated value?#2018-03-2114:26bbrinckAlso, that generator above doesnāt generate valid values: (fn? nil)
is false
#2018-03-2114:27bbrinckIf you want a āmaybe-fnā, perhaps s/nilable
would help?#2018-03-2114:27Alex Miller (Clojure team)heās returning (constantly nil)
, which is a function#2018-03-2114:27bbrinckah#2018-03-2114:27bbrinckdoh#2018-03-2114:28Alex Miller (Clojure team)I feel like you should still be driving from the spec, not the generator#2018-03-2114:29hkjelsyeah, not sure how though#2018-03-2114:29Alex Miller (Clojure team)(you should also be using ifn?
, not fn?
btw)#2018-03-2114:29bbrinckAgreed, @hkjels if I were trying to generate forms based on the spec, Iād look at introspecting the specs using s/form
. Obviously itās be hard with all possible specs#2018-03-2114:29bbrinckbut if you have a subset of forms in your app, you could reasonably inspect them and generate the cases that interest you#2018-03-2114:30Alex Miller (Clojure team)well in this case, s/form doesnāt currently work on predicates#2018-03-2114:30Alex Miller (Clojure team)(pending patch to fix that)#2018-03-2114:30hkjelsahh, nice#2018-03-2114:30bbrinckBut s/form
returns something sensible for ::placeholder
, yes?#2018-03-2114:30bbrinckAnd ::maybe-fn
#2018-03-2114:31bbrinckYou could parse those (probably using specs!!) and then build cases based off that#2018-03-2114:31Alex Miller (Clojure team)you could use spec specs for that :) (CLJ-2112)#2018-03-2114:31bbrinckftw#2018-03-2114:32bbrinckš#2018-03-2114:34hkjelsš#2018-03-2114:34hkjelshere you see what Iām trying to achieve for all kinds of elements#2018-03-2114:35hkjelsit currently works for buttons, since thereās no atoms etc in use#2018-03-2114:36hkjelsOhh, shoot. I got to run#2018-03-2117:30kahunamoore@andy.fingerhut Thanks! I remember reading/hearing about it but Google failed me - too many spec tutorials, docs and other misses.#2018-03-2119:37hkjelsohh, that nonconforming
is just some cruft left there from trial and error#2018-03-2123:43gfredericks@hkjels would (spec/* (s/alt string? ::hiccup))
work?#2018-03-2123:52ghadiNeed to label the branches @gfredericks ^#2018-03-2123:52gfredericksyeah that sounds truthful#2018-03-2207:21misha@hkjels just (s/def ::hiccup (s/tuple #{:div :span} string?))
?#2018-03-2207:42andy.fingerhutSo Stuart Halloway gave a talk on Clojure spec at Strange Loop 2016 that I am transcribing now (mostly done), where he said this: 'So one of the things that is fun about developing with spec is: as soon as you have made a spec, and it can even be a very rudimentary one, you can then say "give me example data" or "give me example functions that do this".'#2018-03-2207:42andy.fingerhutIf you want to see it in context of the full text, you can search for that quote here: https://github.com/jafingerhut/talk-transcripts/blob/add-halloway-agility-robustness-clojure-spec-transcript/Halloway_Stuart/AgilityRobustnessClojureSpec.md#2018-03-2207:43andy.fingerhutThe part that I am wondering how to do is the part where he says "give me example functions that do this". Does anyone know if that is possible, and if so, has examples of doing that?#2018-03-2212:44Alex Miller (Clojure team)He was probably referring to the stub capabilities of instrument#2018-03-2211:44gfredericksfspec
s can be generated I think#2018-03-2211:48taylor(def my-fn (-> (s/fspec :args (s/cat :x int?) :ret string?)
(s/gen)
(gen/sample 1)
(first)))
(my-fn 1)
=> "8VRbFJ59e2K2Ic79F8CX8HB1"
(my-fn 1)
=> "WSKN70JT5kZG"
#2018-03-2211:49tayloris it just making a function that ignores its inputs and using the :ret
spec to generate return values?#2018-03-2211:50taylor> fspecs can generate functions that validate the arguments and fabricate a return value compliant with the :ret spec, ignoring the :fn spec if present.#2018-03-2211:51guyWell you could try doing (my-fn "string")
and see if it complains#2018-03-2211:51guyBecause you are giving it an int right?#2018-03-2211:51taylorit does validate the inputs, I asked before I read the docstring š#2018-03-2211:51guyah#2018-03-2211:52guyš#2018-03-2211:59gfredericksyes the return is unrelated to the input I think#2018-03-2211:59gfredericksI made a ticket many decades ago to discuss whether it should at least be a pure function#2018-03-2211:59gfredericksit's also currently independent of the test.check seed I think#2018-03-2212:00guyYou can use orchestra to make it related with the :fn
i think?#2018-03-2212:03guy
#2018-03-2212:21taylorI think that only applies to instrumentation#2018-03-2212:55danielcomptonI think this is a really dumb question, but how can I detect which branches were taken/captured in a s/cat
?#2018-03-2212:56gfredericksdoesn't the conformed value tell you that?#2018-03-2212:57danielcomptonyeah I can see it in the data, I was just wondering if there was a function you could call to get back which parts matched#2018-03-2212:58danielcomptonthe context is that I'm trying to parse defn
forms and reassemble them with tracing wrapped around parts of the function#2018-03-2212:58danielcomptonand the function specs have quite nested data that is a little bit annoying to pull out#2018-03-2213:23danielcomptonSo I end up with this disgusting code:
(defmacro fn-traced
[& definition]
(let [conformed (s/conform ::ms/fn-args definition)
name (:name conformed)
bs (:bs conformed)
arity-1? (= (nth bs 0) :arity-1)
args+body (nth bs 1)]
(if arity-1?
(if (= :body (nth (:body args+body) 0))
(if name
`(fn ~name ~(:args (:args args+body))
#2018-03-2213:24danielcomptonwas wondering if there's a pattern for doing this in a cleaner way#2018-03-2213:26mpenetsmall critique, seems like the if name
bits could be done inside the (fn ...
part to avoid repeating the whole fn expression#2018-03-2213:26mpenetboth times#2018-03-2213:26mpenetor I might be missing something#2018-03-2213:27danielcomptonbut if there is no name then I'll end up with (fn nil [] ...)
#2018-03-2213:27mpenetoh right, I didn't pay attention#2018-03-2213:28danielcomptonAm I possibly approaching this whole thing in the wrong direction?#2018-03-2213:28danielcomptonFeels pretty dirty#2018-03-2213:44Andreas Liljeqvist@alexmiller anything missing for https://dev.clojure.org/jira/browse/CLJ-2311 ?#2018-03-2214:11danielcomptonI'll be able to clean this up a lot, but here's how it turned out:
(defmacro fn-traced
[& definition]
(let [conformed (s/conform ::ms/fn-args definition)
name (:name conformed)
bs (:bs conformed)
arity-1? (= (nth bs 0) :arity-1)
args+body (nth bs 1)]
(if arity-1?
(if (= :body (nth (:body args+body) 0))
(if name
`(fn ~name ~(:args (:args args+body))
#2018-03-2214:16danielcompton@mpenet here's how to not repeat the name section twice:
`(fn
#2018-03-2214:16danielcomptonWrap a single element in a collection and then unsplice it#2018-03-2214:16mpenetright#2018-03-2214:16danielcomptonš#2018-03-2214:35Andreas Liljeqvistwhat would be the most idiomatic way of expressing xor for two keys in a map?#2018-03-2214:38Alex Miller (Clojure team)it is possible to actually define xor and use it in :req, but I do not guarantee that that will work forever#2018-03-2214:38Alex Miller (Clojure team)separately you can s/and around the map to enforce a predicate#2018-03-2214:50danielcomptonThat's a bit better:
(defn fn-body [args+body]
(if (= :body (nth (:body args+body) 0))
`(~(or (:args (:args args+body)) [])
#2018-03-2319:20bbrinckApologies if this doesnāt work in your case, but would this approach work? Weāve used it successfully to insert timers around function bodies http://blog.klipse.tech/clojure/2016/10/10/defn-args.html#2018-03-2319:23bbrinckThe trick is to modify the conformed value and then call unform to go back to valid clojure code#2018-03-2319:42bbrinckEven if I deleted the docstrings, examples, and workaround for the spec bug, I think my code would longer than your solution, but hereās an example of the approach described in that link https://gist.github.com/bhb/128bf97619e83541a8adda7094bc370d#2018-03-2320:56danielcomptonThanks, thatās a very clever way to do it#2018-03-2215:56borkdudeBy mistake, but it surprised me:
(s/def :dropdowns/options (s/coll-of :dropdowns/options))
(s/valid? :dropdowns/options []) ;; true, ok, I can get that
(s/valid? :dropdowns/options [[]]) ;; true, uuh...
#2018-03-2215:58guyš¤#2018-03-2215:58Alex Miller (Clojure team)coll-of doesnāt limit cardinality by default#2018-03-2215:58Alex Miller (Clojure team)0 size is valid#2018-03-2215:58borkdudeyes, I get that#2018-03-2215:58Alex Miller (Clojure team)Use :count, :min-count etc if needed#2018-03-2215:59guywhat is your spec for :dropdown/option
? suprised it allowed [[]]
#2018-03-2215:59borkdudeno, you see, itās recursive#2018-03-2215:59guyoh right#2018-03-2215:59guySorry missed that#2018-03-2216:00borkdudeI missed that too, but then I was surprised about this behavior, because spec will allow arbitrary nesting of empty colls this way#2018-03-2216:00borkdudeI donāt know if set theory is happy with this#2018-03-2216:01borkdudethe way to justify: an empty coll is a valid :foo
, a collection of empy collections is a collection of valid :foo
, hence an arbitrarily nested empty coll is a valid :foo
#2018-03-2216:02borkdudeso actually this is a spec for arbitrarily nested empty colls#2018-03-2216:05mpenetif you re thinking about monoid instances, it's perfectly valid imho#2018-03-2216:06borkdudewell if you think about types, a [a]
isnāt the same as [[a]]
#2018-03-2216:06mpenetyeah but the mempty of [a] is []#2018-03-2216:06mpenetbut yes, no type annotation to validate that [] is indeed potentially a coll of foo#2018-03-2216:06mpenetbut other than that it's ok#2018-03-2216:07borkdudeI guess#2018-03-2216:08borkdudeitās an unexpected way to describe all possible sets of empty sets#2018-03-2216:08borkdude(s/valid? ::empty-nested [[] []])
is also true of course#2018-03-2216:08mpenetspec cannot go that far. same problem with (s/def ::first-name string?) (s/def ::last-name string?)#2018-03-2216:09mpenetusing ::first-name instead of ::last-name in an fdef args signature will be silent, in haskell it would scream at you#2018-03-2216:09mpenetsame shape, not necessary same meaning#2018-03-2216:10borkdudeyes, I guess that makes sense. itās enabled things too#2018-03-2216:10mpenetno such thing as newtype or some kind of wrapper for validation#2018-03-2216:11mpenetyes, it's all trade-offs I guess#2018-03-2216:13borkdudeHow would you write this:
(s/def :dropdown/option (s/keys :req-un [:dropdown-option/id
:dropdown-option/label]))
dropdown-option/id
or dropdown/option-id
. You can only namespace one level#2018-03-2216:14guy>dropdown-option/id
Says to me, thats the id of the dropdown option.
>dropdown/option-id
Says to me, thats the option-id of the dropdown#2018-03-2216:14mpenetI use a little helper and create a new namespace for it#2018-03-2216:14mpenethttps://github.com/mpenet/spex/blob/master/src/clj/qbits/spex.clj#L7-L10#2018-03-2216:15mpenet(rel-ns 'dropdown.option)#2018-03-2216:15mpenetthen ::dropdown.option/id#2018-03-2216:15mpenetThat's what I found to be the least horrible so far.#2018-03-2216:15borkdudeI could just use the dot anyway right? without creating a namespace?#2018-03-2216:16guyYeah#2018-03-2216:16mpenetyou can use a keyword with a dot yes#2018-03-2216:16borkdudeok, Iāll do that then#2018-03-2221:50roman01laIs it possible that both s/conform
and s/valid
are failing, but s/explain
returns Success!
?#2018-03-2221:51roman01laNot doing anything fancy. Clojure 1.9.0#2018-03-2221:53roman01laok, nevermind, forgot to reload ns#2018-03-2223:52Alex Miller (Clojure team)any case of that is a bug#2018-03-2301:22pabloreAre there any differences in using spec in clojure and clojurescript? Theres a cljs.spec.alpha
namespace#2018-03-2301:23pabloreCan I define specs in a .cljc file and use them in both clojure and clojurescript?#2018-03-2301:30seancorfieldI believe that ClojureScript is supposed to do an automatic mapping from clojure.*
namespaces to cljs.*
namespaces so I would expect you could just write .cljc
files with a require
of clojure.spec.alpha
and it should just work...#2018-03-2301:30seancorfield(caveat: I don't do anything with cljs so I might be talking nonsense)#2018-03-2301:34benzapI believe that is correct, I just pull in clojure.spec.alpha, and it works in clj, cljs, and cljc#2018-03-2302:36Alex Miller (Clojure team)thatās the idea#2018-03-2316:29borkdudeIs there a way to not have this conform in the two options but just pass through as it is?
(s/def :dropdown.option/id (s/or :s string?
:k keyword?))
#2018-03-2316:29borkdudeso without the :s
or :k
#2018-03-2316:29taylorcould you s/and
it with a conformer that discards the tag?#2018-03-2316:30borkdudeI am using s/and
already I run into this in the conformer š#2018-03-2316:30borkdudebut I can just say (fn [id] (or (keyword? id) (string? id)))
of course#2018-03-2316:32taylor(s/def ::s-or-n
(s/and (s/or :s string? :n number?)
(s/conformer second)))
(s/conform ::s-or-n "hey")
=> "hey"
was what I was suggesting but not sure if thatās a terrible idea or not#2018-03-2316:37Alex Miller (Clojure team)You can wrap s/nonconforming around the or#2018-03-2316:37borkdudeawesome#2018-03-2316:38Alex Miller (Clojure team)Currently thatās undocumented#2018-03-2316:38Alex Miller (Clojure team)I think we will ultimately either doc it or make a nonconforming or#2018-03-2316:38borkdudehmm, not in clojurescript yet probably?#2018-03-2316:38borkdudeoh it is#2018-03-2316:40borkdudeworks great, thanks#2018-03-2611:37borkdudeis it possible to instrument an anonymous function?#2018-03-2611:41mpenetfspec?#2018-03-2611:42borkdudehow? say I have a re-frame event:
(reg-event-fx ::foo
(fn [...] ...))
#2018-03-2611:43borkdudeI could pull the fn
out and name it, but that gets tedious#2018-03-2611:44mpenetI think the approach would be to spec reg-event-fx and have the second arg speced via fspec#2018-03-2611:44mpenetotherwise you can/need to add s/asserts if you want to have it "contained" at the anonymous function level#2018-03-2611:46borkdudeyou canāt do that because the function reg-event-fx receives is a different one every time. I want to be specific about the concrete function, not the function argument of reg-event-fx#2018-03-2611:46borkdudeI think for now itās easiest to name it then#2018-03-2612:03borkdudehmm:
cljs.user=> (def param-keys [:query/query
#_=> :query/source
#_=> :vocab/filter
#_=> :dict/guid
#_=> :dict/labels?])
#'cljs.user/param-keys
cljs.user=> (s/def ::params
#_=> (s/keys :opt
#_=> param-keys))
^
param-keys is not ISeqable at line 2
#2018-03-2612:04borkdudeit only expects me to pass in literals? thatās disappointing#2018-03-2612:04mpenetyou need to use eval to pass params-keys, s/keys is a macro#2018-03-2612:04mpenetyeah#2018-03-2612:05mpenetHopefully it something improved in the next spec iteration, it's a common problem#2018-03-2612:06mpenets/merge (or s/and) can sometimes help in these case too#2018-03-2612:07borkdudehow?#2018-03-2612:08mpenetto compose multiple sets of s/keys#2018-03-2612:09mpenethence the "sometimes" it does not always work#2018-03-2612:15borkdudeIs this a valid way of saying I want either :dict/guid or :query/query + these optional keys?
(s/def ::params
(s/merge (s/or :guid (s/keys :req [:dict/guid])
:query (s/keys :req [:query/query]))
(s/keys :opt
[:query/source
:vocab/filter
:dict/labels?])))
#2018-03-2612:19Alex Miller (Clojure team)(s/def ::params
(s/keys
:req [(or :dict/guid :query/query)]
:opt [:query/source :vocab/filter :dict/labels?]))
#2018-03-2612:20borkduderegular or?#2018-03-2612:21borkdudebtw it seems to work#2018-03-2612:22borkdudethanks. is there also a way to get exclusive or?#2018-03-2612:22borkdudeand is this usage of or
undocumented?#2018-03-2612:23borkdudeI could also handle this via an extra s/and
+ predicate#2018-03-2612:25borkdude(s/def ::params
(s/and
(s/keys
:req [(or :dict/guid :query/query)]
:opt [:query/source :vocab/filter :dict/labels?])
(fn [m]
(not (and (:dict/guid m)
(:query/query m))))))
#2018-03-2612:33Alex Miller (Clojure team)if you implement xor as a function and use there, it will work. but I make no guarantees that that will continue to work in the future.#2018-03-2612:41borkdudethis is undocumented right#2018-03-2612:53Alex Miller (Clojure team)yes and not guaranteed to work in the future#2018-03-2614:15borkdudeAre there parts that we can rely on, since spec is still in alpha?#2018-03-2614:17borkdudeDonāt mind breakage btw if itās for the better#2018-03-2614:19Alex Miller (Clojure team)generally, itās better to rely on the documented usage than the undocumented usage#2018-03-2614:32borkdudeIām emitting a warning in a s/and
conformer function.
(fn [{:keys [opts]}]
(if-let
[unexpected
(seq
(apply dissoc opts expected-keys))]
(do (warn "Unexpected options" unexpected "in widget" (or (:id opts)
(:title opts)))
false)
true))
I see this message twice. Is this because spec does something like āif not valid run it again for the explanationā?#2018-03-2614:35borkdudeThe error that spec returns itself is a bit unsatisfying here, thatās why I want to print extra info#2018-03-2614:38Alex Miller (Clojure team)there are no guarantees on the number of times a conformer function might be run (in a regex with branches, it may be run many times). Iām not sure why itās run more than once in this particular case, but itās possible that your explanation is correct.#2018-03-2614:38borkdudeok no problem at all, just wanted to know#2018-03-2614:39borkdudemaybe I could write this spec a little better, so spec provides what keys are unexpected though#2018-03-2614:39borkdudebut this goes against the default of spec where extra keys are just allowed#2018-03-2614:42borkdude(clojure.test/is (empty? [1 2 3]))
FAIL in () (NO_SOURCE_FILE:37)
expected: (empty? [1 2 3])
actual: (not (empty? [1 2 3]))
#2018-03-2707:31borkdudeI think it doesnāt know how to generate a ratom from a predicate.#2018-03-2708:20hkjelsThat springs another question though. How do I express that?#2018-03-2707:33borkdudeMaybe you should try making ::ratom
with itās own with-gen so that this with-gen also determines the contents of the ratom.#2018-03-2708:01hkjelsahh, ofcourse. Thank you @borkdude#2018-03-2708:20hkjelsfails with the same message#2018-03-2708:32borkdude@hkjels I made a small example in JVM Clojure which should be straightforward to port to cljs:
(s/def ::ratom
(s/with-gen #(instance? clojure.lang.IAtom %)
#(gen/fmap atom (gen/string-alphanumeric))))
(gen/sample (s/gen ::ratom))
;; (#atom["" 0x6ff87e8] #atom["" 0x710f246c] #atom["7" 0x43b11d93] #atom["N" 0x6fd3f101] #atom["k6" 0x15e9ebd4] #atom["xFdxB" 0x1dcd8141] #atom["" 0x56db5aee] #atom["J7" 0x26f0f2cd] #atom["67Pzk94" 0x7c1d0a8c] #atom["e" 0xdab4bc7])
#2018-03-2708:45hkjels(set? (deref %))
is what causes it to fail. Iāll digg a little further#2018-03-2708:50borkdude@hkjels
(s/def ::set-ratom
(s/with-gen (s/and #(instance? clojure.lang.IAtom %)
(fn [a]
(set? @a)))
#(gen/fmap atom (gen/fmap set (gen/string-alphanumeric)))))
#2018-03-2708:59borkdude@hkjels You can check with gen/sample
#2018-03-2709:01borkdude(s/def ::id string?)
(s/def ::value string?)
(s/def ::label string?)
(s/def ::item (s/keys :req-un [::id ::value] :opt-un [::label]))
(s/def ::items
(s/coll-of ::item :kind? set))
(take 2 (gen/sample (s/gen ::items))) ([{:id "", :value ""} {:id "", :value "", :label ""} {:id "", :value ""} {:id "", :value ""} {:id "", :value ""}] [{:id "", :value "", :label ""} {:id "", :value "6"} {:id "E", :value "2", :label ""} {:id "", :value "", :label "w"}])
Indeed, the generator doesnāt return sets#2018-03-2709:02borkdudeDonāt know if this is a bug or by design. Alex probably knows#2018-03-2709:08borkdude@hkjels This is a workaround. Not pretty, but it works:
(s/def ::items
(s/with-gen
(s/coll-of ::item :kind? set)
#(gen/fmap set (s/gen (s/coll-of ::item)))))
Iām not sure if thereās a better way as Iāve not used this part of spec much yet.#2018-03-2709:11hkjelshey, that works#2018-03-2709:11hkjelshmm, weird that (s/coll-of ::item :kind? set)
does not return a set#2018-03-2709:11hkjelsthat must be a bug right?#2018-03-2709:12borkdudeDonāt know. Iām sure Alex will reply later today#2018-03-2709:13hkjelsThank you!#2018-03-2709:59minimal@hkjels use a predicate like everything else: (s/coll-of ::item :kind set?)
#2018-03-2710:00hkjelsahh, ok#2018-03-2710:00minimaland itās :kind
, not :kind?:
#2018-03-2710:00hkjelsahh, that explains why it didnāt barf#2018-03-2710:00hkjelsš#2018-03-2710:01minimalyeah, would be nice if it told you#2018-03-2710:10borkdudeah I wasnāt awake enough to spot those typos š#2018-03-2714:39borkdudeI like it how spec lets you specify things gradually or as fine-grained as you want#2018-03-2719:34manutter51My google-fu is failing me: I want to write a spec for a function that takes an atom (actually a Reagent atom). Is there a way to do that?#2018-03-2720:20borkdude@manutter51 Maybe this helps: https://clojurians.slack.com/archives/C1B1BB2Q3/p1522140603000191#2018-03-2720:22borkdude@manutter51 replace the exact type with something from here: https://github.com/reagent-project/reagent/blob/master/src/reagent/ratom.cljs#L121#2018-03-2720:22borkdudebut IAtom will also do, but not specific to Reagent#2018-03-2720:24bbrinck(s/def :test/ratom? #(instance? reagent.ratom/RAtom %))
(s/valid? :test/ratom? 1) ; false
(s/valid? :test/ratom? (r/atom nil)) ; true
#2018-03-2720:24manutter51Thanks much, Iāll see if I can get that to work. The tricky part is this all has to work in CLJS, but I want to match both regular atoms and Reagent atoms so I can pass in test dummies#2018-03-2720:24borkdudethen IAtom seems the right abstraction#2018-03-2720:24bbrinckArguably, testing anything more than the class isnāt very useful, but YMMV#2018-03-2720:25bbrinck(I mean, testing the interface may be useful too, but the contents, maybe not so much)#2018-03-2720:25borkdudeyes, you can just as well create ratom dummies, so itās a good question why you want to do that#2018-03-2720:26bbrinckThe advice I remember is that spec is mostly useful for testing values. Testing references is likely less useful#2018-03-2720:27bbrincksince testing the reference at any point in time doesnāt say much about what it will become at any point in the future#2018-03-2720:28bbrinckit might be more useful to spec the functions that mutate that reference, or, say, use spec in a validator#2018-03-2720:28manutter51Yeah, thatās a good point, I could just use ratoms everywhere#2018-03-2720:28manutter51Thatās probably the best solution ā tks much#2018-03-2802:56leontalbotHello! I am looking for a way to give free pass empty string values when provided as optional#2018-03-2802:56leontalbote.g.#2018-03-2802:57leontalbot(s/def ::item (s/keys :req-un [::id] :opt-un [::url]))
#2018-03-2802:58leontalbotI could check for url, but if empty string, since it is in opt-un, let it be valid#2018-03-2802:58leontalbot(but if was in :req-un, then should apply full ::url spec)#2018-03-2802:59leontalbotis there a way to achieve that?#2018-03-2802:59leontalbotIt does work for nil but Iād like it to work for āā too#2018-03-2802:59leontalbotthanks!#2018-03-2803:14seancorfield@leontalbot I think what I'd do there is use :opt/url
as the :opt-un
key and define it as a spec like (s/def :opt/url (s/or :url ::url :empty empty?))
which will allow it to be the full ::url
string spec or nil
or ""
#2018-03-2803:15seancorfield(or however you want empty values to be spec'd... maybe #{nil ""}
)#2018-03-2803:19leontalbotcool! Thanks @seancorfield. Is this good too to you:#2018-03-2803:19leontalbot(defn valid-map?
ādissoc keys with empty vals to let optional keys pass, then validate...ā
[spec m]
(s/valid? spec (apply dissoc m (for [[k v] m :when (#{āā nil} v)] k))))
#2018-03-2803:20leontalbot(valid-map? ::my-spec-map my-map)
#2018-03-2803:21leontalbotprobably less general#2018-03-2803:34seancorfield@leontalbot I wouldn't use that approach because now you have a custom function, not just a spec. You can't conform
or explain
with that approach.#2018-03-2803:35seancorfieldIt also wouldn't work for specs that have keys whose values can be nil
or ""
but are still required.#2018-03-2803:37seancorfieldFWIW, we have exactly this situation at work and the approach I suggested is basically what we do.#2018-03-2803:38leontalbotThank you so much @seancorfield!#2018-03-2803:40seancorfieldSpec is pretty flexible and powerful -- and the ability to use multiple qualified keywords for "similar" unqualified keys
specs lets you deal with a lot of context.#2018-03-2803:41seancorfieldAnother aspect to consider is having different versions of specs at different "layers" in your application. For example, we have API-level specs (which deal with strings mostly) and we have domain-level specs (and we have a few DB-level specs as well).#2018-03-2803:41leontalbotmakes sense#2018-03-2803:41leontalbotand one last question:#2018-03-2803:42leontalbotwhy :opt/url
and not :url/opt
?#2018-03-2803:45leontalbot@seancorfield ^#2018-03-2803:45leontalbotthanks!#2018-03-2803:46leontalbotYeah I guess this is to map this hierarchy:#2018-03-2803:46leontalbot(s/keys :req-un [::id] :opt-un [::url]))
#2018-03-2803:46leontalbotyeahā¦#2018-03-2804:45seancorfieldYeah, if the unqualified key is :url
then the options are :<something>/url
#2018-03-2813:46leontalbotWanted to get end-user error message from spec. Useful for webform field validation.#2018-03-2813:46leontalbotFound Phrase library.#2018-03-2813:47leontalbotSeemed a bit overkill though. As I just wanted is attach an error text to a spec.#2018-03-2813:47leontalbotThen I saw https://github.com/metosin/spec-tools#2018-03-2813:48leontalbot(s/explain (st/spec pos-int? {:reason āpositiveā}) -1)
; val: -1 fails predicate: pos-int?, positive
(s/explain-data (st/spec pos-int? {:reason āpositiveā}) -1)
; #:clojure.spec.alpha{:problems [{:path [], :pred pos-int?, :val -1, :via [], :in [], :reason āpositiveā}]}
#2018-03-2813:49leontalbotMaybe use the :reason field accessible with explain?#2018-03-2813:49leontalbotWanted to know what you would do for form validation, thanks!#2018-03-2813:55guytheres https://github.com/bhb/expound as well which might be helpful#2018-03-2814:09leontalbotYes looked at this. Not sure if we can āextractā only the error string#2018-03-2814:17leontalbothttps://github.com/bhb/expound/issues/77#2018-03-2814:37bbrinck@leontalbot Would this work?
(expound/def :user/name string? "should be a valid name")
(defn msg [spec val]
(if (s/valid? spec val)
nil
(expound/error-message spec)))
(msg :user/name "John") ; => nil
(msg :user/name :John) ; => "should be a valid name"
#2018-03-2814:42leontalbotHey! Thanks for answering @bbrinck! Fantastic! Thanks!#2018-03-2814:42bbrincknp, let me know if you have any other questions about expound#2018-03-2814:43leontalbotOk! Nice lib btw!#2018-03-2814:43bbrinckthanks! š#2018-03-2817:43borkdude@bbrinck Sometimes I get an error message from expound when it complains about not being able to render an error (exact error message I donāt have handy). Wouldnāt it be better in that case to print the vanilla spec error instead of only the expound error?#2018-03-2817:55bbrinck@borkdude Is this a bug in expound? Or a case where your spec has conformers, perhaps?#2018-03-2817:55borkdudethe latter#2018-03-2817:56bbrinckGotcha. Yeah, in that case, it might very well make sense to just print the default error. Let me think about that. Thanks for the idea!#2018-03-2817:57bbrinckAlthough if youāre using conformers, the vanilla spec error may not be very helpful either š#2018-03-2817:57bbrinckBut still better than nothing, for sure#2018-03-2817:57bbrinckhttps://github.com/bhb/expound/issues/78#2018-03-2817:58borkdudeUsually when I run into this, I turn off expound, re-run the code, inspect the error, fix it, and enable expound again.#2018-03-2817:58bbrinckAgreed, thatās a pain. Iāll fix it.#2018-03-2818:08seancorfield@bbrinck FWIW, that was why we tried and then stopped using Expound -- we have several conformers in our specs.#2018-03-2818:18bbrinck@seancorfield Would it work in your case to just fallback to s/explain
?#2018-03-2818:19bbrinckDepending on how many conformers you have, I guess at some point you would rarely see an expound error#2018-03-2818:20bbrinckSo the fix @borkdude suggested is a good idea, but itās mostly useful for projects that have a relatively small number of conformers compared to the total usage of spec#2018-03-2818:20bbrinck@seancorfield FWIW, apparently pinpointer works with conformed values#2018-03-2818:25seancorfield@bbrinck Well, the main cases where we wanted Expound's better messages were all conformer-specs, so we just fell back to explain
ourselves š#2018-03-2818:25seancorfieldI'll take a look at pinpointer -- I hadn't heard of that.#2018-03-2820:40lilactownhas anyone used spec-tools to generate swagger objects?#2018-03-2820:40lilactownI'm having trouble with getting the response schema's to generate correctly#2018-03-2903:47ikitommi@lilactown one of the authors here and just integrated that to reitit routing library and worked ok. Porting Schema=>Swagger into the same model, which will cause small changes to spec-side to get unified tranformation model for both. How can I help?#2018-03-2903:52ikitommiStill need to backport support for json schema refs for recursive references, also will change the apis a bit. e.g. tranformations return both the transformed swagger schemas and the named schema definitions they refer to.#2018-03-2914:17lilactownI was trying to use the swagger-spec
fn with an endpoint that had a response map like in the README: :responses {200 {::swagger/spec ::user
:description "Found it!"}
404 {:description "Ohnoes."}}
#2018-03-2914:18lilactownhowever, this was getting converted to something like :x-spec-tools.swagger.core/spec :user/user
#2018-03-2914:19lilactowninstead of expanding the ::user
spec into the :schema
key#2018-03-2918:23ikitommi@lilactown sorry, thread got buried. You should raise the ::swagger/responses
one level up. So, ::swagger/responses {200 {:schema ::user}}
#2018-03-2918:24ikitommishould add specs to the input so it would tell what happened.#2018-03-2918:41ikitommialso, I think there isn't ::swagger/spec
defined, see https://github.com/metosin/spec-tools/blob/master/src/spec_tools/swagger/core.cljc#L103-L131#2018-03-2918:55lilactownyeah, I ended up using ::swagger/responses
instead#2018-03-2918:55lilactownbut I was basing it off of this example in the README#2018-03-2918:55lilactownhttps://github.com/metosin/spec-tools#full-example#2018-03-2918:55lilactownwhich AFAICT does not work#2018-03-2920:39ikitommiOh, good catch! I was too blind to see that it's not correct. Could you write an issue, need to fix either the code or the sample in README.#2018-03-2916:38jimbobhow do you guys organize your specs? Are all of your specs in a top level spec directory? or do you also mix and match spec assertions and definitions in core namespaces#2018-03-2917:45taylorI usually fdef
next to the function, but put other specs in their own namespaces#2018-03-2916:38jimbobany tips?#2018-03-2917:44borkdudeWhen I write an fdef I place it above the function#2018-03-2918:56seancorfield@ben.borders It depends. If I need code to work with pre-1.9 Clojure, I put all the specs in a separate namespace. Otherwise, if I'm spec'ing data I tend to put the specs in their own namespace (along with predicates and closely related helper functions), else if I'm spec'ing functions I tend to put those directly above the function they are for.#2018-03-2919:15borkdude@ben.borders Just moved the fdef + related specs to a widget/spec.cljs namespace because it became too large š#2018-03-3006:41robert-stuttafordlikely a common question, and also likely not something spec is suited for, but iām curious even so - i have a map with start and end dates. is there a spec pattern for declaring a relationship between those two values, i.e. one must be larger than the other? one structure that occurs is spec/fdef
ās :ret
, but iām wondering if perhaps the spec api has something else like this?#2018-03-3009:55borkdude@robert-stuttaford you can use s/and
+ a conformer function#2018-03-3009:56borkdudeSomething like
(s/def ::dates
(s/and
(s/keys :req-un [::start ::end])
(fn [{:keys [start end]}]
(< start end))))
where <
is the comparison function of your choice.#2018-03-3010:08robert-stuttafordwonderful, thank you! that seems ridiculously simple, in hindsight. like, āhow did i not see thisā simple#2018-03-3010:10robert-stuttaford@borkdude i guess iād have to write my own generator too then#2018-03-3010:11borkdudeDonāt know if the default generator generates enough samples where the conformer can strip away the invalid ones. Not as efficient as writing your own, but it could work#2018-03-3010:15borkdudeitās about 50% chance for each sample#2018-03-3010:17robert-stuttafordright#2018-03-3010:22borkdudewhen you write an fdef in another namespace than the function, require the namespace where the function lives? or donāt and just fully qualify the symbol in fdef? hmm#2018-03-3010:23borkdudeRight now I have an init namespace that just requires them all, so no cyclic dependencies and I can just fully qualify in fdef#2018-03-3010:24robert-stuttafordwhatās causing you to want to keep the fdef separate, @borkdude?#2018-03-3010:24borkdudebecause itās more than 100 lines#2018-03-3010:26dominicmI feel like I remember there being some kind of way to add conformers to specs "locally"? Is that a correct memory?#2018-03-3010:26robert-stuttafordah š Qualifies fn-sym with resolve, or using *ns* if no resolution found.
seems to suggest that fully-qualified is fine, and probably better for discoverability#2018-03-3010:28borkdudeI have an interesting problem:
(s/def ::selection
(s/nilable
(s/and
(s/keys :req-un [:selection/options
:selection/id]
:opt-un [:selection/default-option
:selection/type])
;; default option must be one of the option ids
(fn [dropdown]
(if-let [opt (:default-option dropdown)]
(contains?
(set (map :id (:options dropdown)))
opt)
true)))))
(s/def ::dropdown ::selection)
I want the id in dropdown to be optionalā¦ Maybe I should make an extra spec without the required id and then make the id in selection required with s/and
?#2018-03-3010:31borkdudeLike this:
(s/def ::selection
(s/nilable
(s/and ::selection*
(s/keys :req-un [:selection/id]))))
(s/def ::selection*
(s/nilable
(s/and
(s/keys :req-un [:selection/options]
:opt-un [:selection/default-option
:selection/type])
;; default option must be one of the option ids
(fn [dropdown]
(if-let [opt (:default-option dropdown)]
(contains?
(set (map :id (:options dropdown)))
opt)
true)))))
(s/def ::dropdown ::selection*)
#2018-03-3010:34borkdudeSeems to work#2018-03-3011:51robert-stuttafordwhatās the blessed method for checking whether a spec is registered? s/spec? seems to be for something else#2018-03-3011:52robert-stuttafordaha s/get-spec
#2018-03-3011:53borkdude@robert-stuttaford brute force method: use a println in a conformer š#2018-03-3011:53borkdudeor just make an obvious mistake and if no exception, then no š#2018-03-3011:54robert-stuttaford-grin- its for datomic attrs. the code using the spec canāt assume a spec is registered; it has to check first before it attempts to use it to validate.#2018-03-3011:54robert-stuttafordget-spec
works#2018-03-3011:59robert-stuttafordgrr, iām having to wrap my defmulti with a normal defn so that i can instrument it. otherwise defmethods defined after instrumentation fail#2018-03-3011:59borkdudeFunny that s/spec?
doesnāt return a boolean#2018-03-3012:00borkdudedonāt all new fdefs after instrumentation fail?#2018-03-3012:00borkdudein the sense that they arenāt instrumented yet#2018-03-3013:01Alex Miller (Clojure team)Yes, although I wouldnāt call that a fail#2018-03-3013:05borkdudemore appropriate: not yet in effect#2018-03-3014:33gfredericksargument validation in libraries: going forward, should it be done entirely with s/fdef
, meaning no validation happens unless the user instruments the functions?#2018-03-3014:35borkdudes/assert is also an option I guess which can be compiled away#2018-03-3014:35borkdudebut for arguments s/fdef is nicer#2018-03-3014:37gfredericksthe downside is that things are a lot more GIGO for users who don't think to instrument
e.g., as a user, can I easily instrument all the spec'd functions in all my libraries without having to know which ones have specs?
is that too much? wouldn't it be more efficient to instrument only the functions I'm calling directly?#2018-03-3014:38borkdudeyou can instrument all fdef-ed functions with stest/instrument?#2018-03-3014:38borkdudebut it has to be after you load their namespaces#2018-03-3014:38borkdudemaybe adding it to reloaded.repl/reset
will be a common thing#2018-03-3014:39borkdudebut I get what you mean now. so you want to go only one level deep#2018-03-3014:39borkdudeno transitive fdef checking#2018-03-3014:40gfredericksthat'd be nice, since you probably have a large tree of libraries and don't want to slow down your dev by testing all the interactions between them#2018-03-3014:41borkdudeWhat overhead are we talking about? I donāt mind a couple of milliseconds more during dev#2018-03-3014:41gfrederickstotally depends on the libraries and what they're doing#2018-03-3014:42gfredericksin the extreme case, if specs get added to most of the clojure.core
functions, instrumenting those will result in milliyears instead of milliseconds#2018-03-3014:45borkdudegood question#2018-03-3020:20hlshipHere's a question. I have a value that I want to ensure is a keyword, string, or symbol AND that it's string conforms to a particular regexp. Example here:
https://github.com/walmartlabs/lacinia/blob/05940c7f7819fd88bc4e50c860b8d9854c3fa0b2/src/com/walmartlabs/lacinia/schema.clj#L306#2018-03-3020:21Alex Miller (Clojure team)thatās not a question :)#2018-03-3020:22hlshipI'm working on it ...#2018-03-3020:22Alex Miller (Clojure team):)#2018-03-3020:21hlshipI've been down this path before, and what I've found is that the next term in the s/and
gets the conformed value from the s/or
, a tuple of (say), [:keyword :frob]
.#2018-03-3020:23hlshipThat's been fine so far EXCEPT as I'm switching to using Expound, the use of a conformer here is a problem:
(s/explain ::schema/enum-value "this-and-that")
clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound
clojure.lang.Compiler$CompilerException: clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound {:form "this-and-that", :val [:string "this-and-that"], :in [], :in' []}, compiling:(/Users/hlship/workspaces/github/lacinia/src/com/walmartlabs/lacinia/expound.clj:50:3)
#2018-03-3020:23Alex Miller (Clojure team)this issue is actually discussed in the backchat#2018-03-3020:24hlshipRecently? Got a link?#2018-03-3020:24Alex Miller (Clojure team)2 days ago in this room - just scroll up till you see the expound stuff#2018-03-3020:29hlshipThat discussion wasn't helpful, if its the right one. I think they're hitting the same problem and want Expound to print it differently. I want to modify my spec to not trip over this scenario. s/nonconforming may work!#2018-03-3020:24hlshipSo my question is, how can I achieve the kind of spec I want in a way that avoids the use of a conformer in the middle.#2018-03-3020:24Alex Miller (Clojure team)another option is to wrap s/nonconforming around s/or#2018-03-3020:25Alex Miller (Clojure team)then you get just the value without the tag#2018-03-3020:25Alex Miller (Clojure team)when you conform that is#2018-03-3020:25Alex Miller (Clojure team)currently thatās an undocumented function but I think itās likely we will either keep it or add a nonconforming variant of s/or#2018-03-3020:27seancorfieldAnd in the backchat, one suggestion was to look at pinpointer
instead of Expound
.#2018-03-3020:28bbrinck@hlship I will likely be adding a āfallbackā feature to expound soonish where you will see the vanilla spec error in this case#2018-03-3020:29bbrinckNote that will help if you only occasionally use conformers, but not if you have lots of them. Definitely check out pinpointer š#2018-03-3020:29hlshipAgain, I'm quite willing to modify my spec to bypass this problem.#2018-03-3020:29bbrinckAh, sorry, I missed that in the thread. Yes, thatād be the best approach! š#2018-03-3020:31bbrinckIāve seen people run into other issues with conformers so itās probably best to avoid e.g. https://groups.google.com/forum/#!searchin/clojure/conformer%7Csort:date/clojure/Tdb3ksDeVnU/uU0NT4x6AwAJ#2018-03-3020:33bbrinckFWIW, I tried to support conformers in expound but itās really tricky if youāre just looking at the explain-data
#2018-03-3020:34hlships/nonconforming
looks to be just what I want:
(s/explain ::schema/enum-value "this-and-that")
-- Spec failed --------------------
"this-and-that"
must be a valid GraphQL identifier: contain only letters, numbers, and underscores
BTW why are explicit messages not indented by Expound? Should I file an issue?#2018-03-3020:35bbrinckCan you modify that to show what youād prefer?#2018-03-3020:36bbrinck(Iām always happy to get bug reports too š if thatās easier to discuss the options)#2018-03-3020:37bbrinckIāll say this - off the top of my head, I think itās working as I intended, but Iām always interested in improving the layout of error messages if itās not clear#2018-03-3020:40hlshipHere's a better example:
(s/explain ::schema/resolve {})
-- Spec failed --------------------
{}
should satisfy
fn?
or
implement the com.walmartlabs.lacina.resolve/FieldResolver protocol
The final line should be indented the same as the fn?
line, don't you think?
;; is passed and should return.
(s/def ::resolve (s/or :function ::resolver-fn
:protocol ::resolver-type))
(s/def ::resolver-fn fn?)
(s/def ::resolver-type #(satisfies? resolve/FieldResolver %))
#2018-03-3020:52bbrinckYeah, when itās part of the āorā it looks weird. I guess I was thinking that since āshouldā starts on left, custom messages would be the same.#2018-03-3021:05bbrinckThe reason itās on the left is that it should be in the same spot as āshouldā like so:
(s/def :example/temp #{:hot :cold})
(expound/expound :example/temp 1)
;;-- Spec failed --------------------
;;
;; 1
;;
;;should be one of: :cold, :hot
(expound/def :example/name string? "should be a string")
(expound/expound :example/name 1)
;;-- Spec failed --------------------
;;
;; 1
;;
;;should be a string
#2018-03-3021:05bbrinckbut I agree that when itās part of a long āsatisfy..orā block, it looks bad. Iāll make a bug#2018-03-3021:07bbrinck@hlship https://github.com/bhb/expound/issues/80 thanks for letting me know!#2018-03-3021:46hlshipAnother expound question (time for its own channel?) :
user=> (require [clojure.spec.alpha :as s])
nil
user=> (s/explain keyword? 3)
val: 3 fails predicate: :clojure.spec.alpha/unknown
nil
user=> (require '[expound.alpha :as expound])
nil
user=> (alter-var-root #'s/*explain-out* (constantly expound/printer))
#object[expound.alpha$printer 0x3c64e2a2 "
this works fine when I use set!
, but not when I use alter-var-root!
. Any ideas?#2018-03-3021:50bbrinckIāve always used set!
myself, so Iām not sure. Can you talk a little bit more about your use case and neither set!
nor binding
are a good fit here?#2018-03-3021:50hlshipWell, at application startup, might want to alter-var-root, so that any later threads will use the Expound printer, not the default one.#2018-03-3021:58bbrinckThanks for that. The short answer is I donāt know unfortunately - this seems to affect any printer, not just expound. Try: (alter-var-root #'s/*explain-out* (constantly (fn [ed] "Hello!")))
. It may be a result of the way the REPL is set up - does it reproduce if you put the alter-var-root
in the main
of an application (as opposed to doing it in the REPL context)?#2018-03-3022:00hlshipI suspect it's because clojure.spec is AOTed. I hit something similar with redefining stuff in clojure.test and the hack was to do a RT/loadResourceScript:
https://github.com/AvisoNovate/pretty/blob/db4e7677f74d8efb149db3e9ba5974fa9c84b6a0/src/io/aviso/repl.clj#L72#2018-03-3022:03bbrinckAh, gotcha. I certainly understand your use case and I suspect others may run into the same thing. If you figure it out, let me know and Iāll update the docs. I have a note about using alter-var-root
in non-REPL context, which IIRC, works fine, but if your scenario is at the REPL, then no dice#2018-03-3022:07hlshipbinding
should work, as that will carry over into most started threads, including core.async threads.#2018-03-3023:32Alex Miller (Clojure team)I think the difference is that the repl binds explain-out so alter-var-root changes the root but the repl binding is the one being seen#2018-03-3023:33Alex Miller (Clojure team)So you have to set! at the repl#2018-03-3115:34gabrielehow can i write a spec to validate a map like this: {[:type1 "args"] {:key-type1 value} [:type2 "args"] {:key-type2 value} ...(type-n)}
#2018-03-3115:35gabrielewhere the value is dependent on the first element of the list that composes the key#2018-03-3115:39borkdude@gabriele.carrettoni First of all I would try to get the source of that data in a more workable format#2018-03-3115:40borkdudeI usually try to go from dynamic keys to something more regular#2018-03-3115:41gabriele@borkdude i see, i wanted to use specs to do that#2018-03-3115:41borkdudeE.g.
(def data [{:type 1
:args āargsā
:value value}
{:type 2
:args āargsā
:value value}])
#2018-03-3115:43borkdudeAnd then you can spec that after transforming the awkwardly shaped data and verify if your normalization worked#2018-03-3115:44gabrielethanks, i'll do that#2018-03-3115:47borkdude@gabriele.carrettoni
E.g.:
(def data {[:type1 "args"] {:key-type1 1}})
(defn transform-data
[data]
(for [[k v] data]
(let [[type-key args] k
other-key (keyword (str "key-"
(name type-key)))
value (get v other-key)]
{:type type-key
:args args
:value value})))
(transform-data data) ;;=> ({:type :type1, :args "args", :value 1})
#2018-04-0220:07aviš hi all!
Iām having some trouble with a function spec with a :fn
predicateā¦ and my Web-searching has turned up nothing.
Iām using stest/check
like so:
(-> (stest/check 'fc4c.core/shrink) first stest/abbrev-result)
wherein fc4c.core/shrink
is my function that has an attached spec.
I always get a failure, butā¦ if I grab the value of :val
from the result, and then manually pass it to my predicate, like so: (pred val)
then I always get true
.
So am I missing something? I thought that if the predicate returns true, then the value conformsā¦ Iām confused!
Thanks!#2018-04-0220:13seancorfield@aviflax We'll need to see a bit more code in order to help you...#2018-04-0220:15aviyeah? Iām happy to share more code, but I think Iām trying to just confirm an invariant about stest/check
and function specsā¦ that if the :fn
predicate returns true
then the ātest caseā will be considered to have succeededā¦ does that make sense? Or am I thinking about this wrong?#2018-04-0220:16taylorare you sure itās failing due to the :fn
spec? could it be failing on the :ret
spec?#2018-04-0220:17aviIām fairly sure but the output of check
failures is somewhat overwhelming, even when abbreviated with abbrev-result
ā¦ Iāll double-check right now#2018-04-0220:19aviHereās a chunk of the output:
:failure
{:clojure.spec.alpha/problems
[{:path [:fn],
:pred
(clojure.core/fn
#2018-04-0220:19avithe function spec is here: https://github.com/FundingCircle/fc4c/blob/tests-yay/src/fc4c/core.clj#L76#2018-04-0220:25taylor#2018-04-0220:27aviThanks for the help!
Iām a little confusedā¦ I donāt see that. If I dig the value of :ret
out of the value of :val
ā I see a map.#2018-04-0220:27aviSo it looks like shrink
returned a map when check
called it?#2018-04-0220:28tayloractually disregard, I copy/pasted your code poorly#2018-04-0220:28aviah ok no worries! happens to us all!#2018-04-0220:40taylorI was just able to successfully check
the function when I replaced the :args
and :ret
spec:
:args (s/cat :in map?)
:ret (s/nilable map?)
#2018-04-0220:40taylorsince the shrink
function is more general/doesnāt need to know about your :diagram
spec#2018-04-0220:41taylorbutā¦ I did try running your :fn
spec function against that sample and it does return true
, so not sure whatās up with that#2018-04-0220:45avithat makes senseā¦ thanks for the help!#2018-04-0220:48seancorfield@aviflax I notice you're post-walking the tree -- are you aware of the bug in postwalk where MapEntry elements are not preserved?#2018-04-0220:48avino, I wasnāt aware of that#2018-04-0220:48aviI guess I can search the Web to find more info on that#2018-04-0220:49seancorfieldhttps://dev.clojure.org/jira/browse/CLJ-2031#2018-04-0220:49taylorthat doesnāt seem to be the big issue here#2018-04-0220:49aviBut Iām not sure how that helps me understand the ācontractā of fn
predicatesā¦ my predicate is returning true
so I just donāt understand why a failure is being reported#2018-04-0220:50taylormaybe dumb question, but have you tried restarting your REPL if youāve been working on this for a while?#2018-04-0220:50avinot dumb!#2018-04-0220:50aviyes, more than a few times#2018-04-0220:50aviš¤ maybe thereās a problem with the name shrink
ā¦ maybe ā¦ thatās somehow confusing check
since it internally does shrinking (seems like a long shot)#2018-04-0220:51seancorfieldWhy not test it with a much simpler :fn
predicate that is (constantly true)
and see if it still fails?#2018-04-0220:51avihm ok good idea Iāll try that!#2018-04-0220:52seancorfieldIf that works (passes) then build the predicate up, piece by piece until you get a failure.#2018-04-0220:52taylorhis :fn
spec predicate does return true
if you manually call it with the map from the check
problem output, which is odd#2018-04-0220:53aviyeah thatās precisely what Iām confused by#2018-04-0220:53taylorbut I agree w/trying to (re)build up from the simplest working case until it breaks#2018-04-0220:54avibut yeah, since (as I just learned) replacing the predicate with (constantly true)
does lead to the check
call succeedingā¦ Iāll try rebuilding the predicate to try to debug at what point find a small case wherein I can trigger the confusing behavior#2018-04-0220:54tayloranother thing to keep in mind is that it check
s out fine if you donāt use his custom keys
spec for in/out#2018-04-0220:55aviyeahā¦ my input and output specs (and thus the generated args) are deeply nested collsā¦ not sure why thatād matter but maybe it does#2018-04-0221:15avianyone know whether/how to use stest/check
and have it skip shrinking?#2018-04-0221:22aviah never mind, I can work with it š#2018-04-0221:45avihmm well Iāve been able to get the predicate to always work as expected by changing (= in-vals ret-vals)
to (= (set in-vals) (set ret-vals))
ā it appears that the two colls had the same values but in different orders. Iām confused as to why I wasnāt seeing this earlier. Iām very unclear on what Iām doing differently now š ā¦ PEBKAC ā¦ sorry to have bothered everyone#2018-04-0221:57seancorfieldThat's cool that you've figured out the problem!#2018-04-0221:58avithanks!#2018-04-0221:58aviyes!#2018-04-0221:59avibut also a bit disorienting that I suddenly canāt reproduce my confusing tangentā¦#2018-04-0222:00avimy only idea is that earlier I was copy-and-pasting the values as strings and then pasting them into the repl prompt, versus using e.g. (-> *1 :key :key)
to get the in-memory values insteadā¦ I donāt know why that would matter though.#2018-04-0423:30flyboarderhello everyone, is there a version of multi-spec that uses an outside piece of data to determine the shape of the data? Instead of the current multi-spec which uses a piece of the data to figure out the shape of the rest of it?#2018-04-0501:52Alex Miller (Clojure team)No#2018-04-0501:54Alex Miller (Clojure team)And generally that implies you are prob doing something you shouldnāt as you then have a spec that is impure.#2018-04-0614:39flyboarderCould you explain an impure spec? I am trying to generate a spec from some nested data#2018-04-0516:07Charles FourdrignierIf I want to specā data from a CSV file, what would be the best strategy ?
Should I describe it as a tuple ? Or should I convert it to a map then spec the map itself ?
(I donāt find any article or blog post on this subject.)#2018-04-0516:32Alex Miller (Clojure team)a csv file is inherently a collection of collections of strings#2018-04-0516:32Alex Miller (Clojure team)and thus very uninteresting to spec and unlikely to tell you much#2018-04-0516:33Alex Miller (Clojure team)I suspect from an app perspective, it is far more interesting to convert to maps (coercing data if needed) and to spec the result, which is the data your app actually uses#2018-04-0516:41Charles FourdrignierThanks !
Tuples was my first intuition, but some reading changes my mind and your response finish to convince me.#2018-04-0520:53Drew VerleeI have some unqualified keywords that conceptually map to some specs i have created. Whats the suggested way to convert a unqualified keyword to a qualified one so i can exercise it (generate data) or conversely, use a generator on a unqualified keyword.
I feel i could do this my pulling things apart and putting them back together with strings but im guessing im missing an obvious exisiting way to handle this:
(str *ns* "/" :unqalified)
#2018-04-0521:25manderson@drewverlee any reason that you couldn't just add namespaces to your unqualified keywords?
If you do want to go back and forth, the best way I can think of is as follows (perhaps someone knows of something better):
(def +qualified+
(keyword (-> *ns* ns-name str) (name :unqualified)))
=> #'user/+qualified+
+qualified+
=> :user/unqualified
(-> +qualified+ name keyword)
=> :unqualified
#2018-04-0613:09Drew Verleeno reason, just wonder if thats the right approach.#2018-04-0614:11mandersonSpec is opinionated about using namespaced keywords (see the rationale doc on this: https://clojure.org/about/spec#_global_namespaced_names_are_more_important), so in general, if you are using spec, I've found that it is best to follow this pattern.#2018-04-0613:01kardanAnyone who can point on where to read on how structure specs? I started to spec in com.example.user but then it was odd define the user spec using namespaced keys. Would really want to to type (s/valid? :: #::{:email ā<mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>ā :first-name āBillā :last-name āKarlsonā}) . So Iām guessing I should read up š#2018-04-0712:56leongrapenthin@kardan it has become commonplace to use unsegmented namespaces for domain specific entities like :user/email#2018-04-0712:57leongrapenthinBut I'd also love if someone wrote a state of the art summary because there is still lots of experimentation going on#2018-04-0712:59leongrapenthin@kardan consider that in your example you usually don't come up with a single user spec#2018-04-0713:00leongrapenthinin practice you end up creating different specs with keys in the usernamespace depending on what you need#2018-04-0713:01leongrapenthinfor example an api that changes a user may say all user keys are optional#2018-04-0713:01leongrapenthinan api that creates a user may have some required user namespaced keys#2018-04-0713:02leongrapenthinthis flexibility is easily missed when you are looking to create "the" ::user spec#2018-04-0713:04leongrapenthinthe great thing is that you can compose and mix the keys freely in the context you need them whereas in classical type systems you need inheritance or whatever to reuse them.#2018-04-0717:25ikitommiunsegmented namespaces are not safe, :user/id
might mean different things in different domains. If you need to integrate the two domains, you'll have conflictings specs.#2018-04-0719:05seancorfield@ikitommi It all depends on the scope of the data. If you're writing an application, not a library, and you have data whose scope is just that application, and that data is used entirely internally in that application, then something like :user/email
might be fine. If that needs to flow through other code for additional processing, then you might want a more unique prefix. If data is truly isolated to a single namespace for processing (and is entirely opaque to everything else), the ::email
would make more sense. If the data does lie somewhere in between, a "reasonably" unique prefix should be sufficient. Using namespace aliases makes working with namespaced maps a lot less painful -- and you can decide whether those should be real (code-based) namespaces or just arbitrary namespace prefixes independent of the alias. /cc @kardan#2018-04-0719:31mathpunkI'm starting a wrapper for an interesting Java library, for which I have domain knowledge of the problem they're working on. I am interested in adding specs based on invariants that I know so as to make it easier to use this library. Advice on how to approach such a goal is most welcome.#2018-04-0914:39bbrinckPerhaps this post would be helpful? http://blog.cognitect.com/blog/2017/6/19/improving-on-types-specing-a-java-library#2018-04-1118:55mathpunk@U08EKSQMS This is exactly the kind of thing I was looking for, thank you!#2018-04-1118:58bbrincknp, good luck!#2018-04-0812:09leongrapenthin@seancorfield what do you think about keys like :git/sha used in ctd#2018-04-0812:10leongrapenthin@alexmiller would be interested how you decided that, too#2018-04-0812:20Alex Miller (Clojure team)Sorry, whatās the question?#2018-04-0819:14seancorfield@leongrapenthin when Alex talked about that (on the list? in #tools-deps here?) he said that's a human written format so it needs to be clear but not too verbose. A while different set of considerations compared to code. #2018-04-0819:50leongrapenthin@alexmiller I'm interested in a rule when its ok to pick unqualified/unsegmented namespaces. "Lib no;app yes" seems what has emerged mostly, but in tools.deps you claim for example "git". When should a lib author choose to do this and how can he not be worried about conflicts?#2018-04-0819:51leongrapenthinOr should we say unqualifies nses in libs are reserved for core libs#2018-04-0819:52leongrapenthin@seancorfield thanks, that makes sense from a practical standpoint. I still couldn't do it in my libs without worrying about conflicts ))#2018-04-0901:04Oliver GeorgeI have an "on the fly spec generation in clojurescript" question.#2018-04-0901:05Oliver GeorgeI'd like to generate and register new specs but clojure-spec has a macro based api and clojurescript doesn't have eval.#2018-04-0901:06Oliver GeorgeCan someone point me at a sane approach please.#2018-04-0901:07Oliver GeorgeIt's possible a macro is the right way to do this but I get super confused about when macros evaluate in clojurescript.#2018-04-0901:08Oliver GeorgeMy spec generation looks at existing registered specs. That means my macro needs to consider the correct registry (presumably there's a CLJ one at compile time and a CLJS one at runtime).#2018-04-0901:08Oliver GeorgePretty sure I'm lots in the weeds.#2018-04-1006:17aaron51looking to auto-generate an HTML form from spec/schema for non-programmers (not API documentation, with validations). is there a simpler alternative to swagger out there, that works with compojure.api.sweet?#2018-04-1119:13donaldballWhatās the least wrong way of tightening a spec in the registry in the context of another spec?
For example, Iām writing specs for config maps in general, and for config maps for production. The general spec for e.g. :config.mode
might be #{:app :lib}
but when it appears in my :config.prod
spec, Iād like to further constrain it to be simply #{:lib}
.
s/and
with predicates is a perfectly adequate way to do this, but I seem to recall a better way.#2018-04-1119:47Alex Miller (Clojure team)s/and is what I would say#2018-04-1120:09donaldballIt makes generating values unlikely, but this is also a case where I really donāt care about that#2018-04-1212:49Andreas LiljeqvistHow would I spec a map like {:type akeyword :args dependsonkey}
?#2018-04-1212:54Alex Miller (Clojure team)s/multi-spec is designed to handle cases where you choose the spec based on the data (here the :type)#2018-04-1212:54Alex Miller (Clojure team)So that may be a good match here#2018-04-1212:54Andreas LiljeqvistI can use a multimethod to spec the map depending on type (multimethod methodname :akeyword [m] (s/keys [:args]))
#2018-04-1212:55Alex Miller (Clojure team)Yes, thatās how s/multi-spec works#2018-04-1212:55Andreas LiljeqvistBut I can't see at the moment how I would specify that :args
should have a different spec for that match#2018-04-1212:56Andreas LiljeqvistLike that it should accept string?
if :type
is :a
, but int?
if :type
is :b
#2018-04-1213:02Alex Miller (Clojure team)Is :args actually unnamespaced?#2018-04-1213:02Andreas Liljeqvistnah, everything should be namespaced#2018-04-1213:05Alex Miller (Clojure team)When unsure how to spec something, itās best to always return to how to represent the truth of your actual data. In this case youāre saying that you have one attribute that can have a variety of different structures#2018-04-1213:06Alex Miller (Clojure team)So you need to capture that in the spec#2018-04-1213:06Alex Miller (Clojure team)The spec for that attribute is A or B or C#2018-04-1213:08Alex Miller (Clojure team)You then have a separate constraint that says that a particular value of :type should co-occur with a particular form of :args and you should capture that constraint as a separate predicate #2018-04-1213:09Alex Miller (Clojure team)So you would model the attribute with s/or, the map with s/keys, and the constraint by s/and-ing the map and the constraint#2018-04-1213:23Andreas LiljeqvistThank you. s/and-ing is always powerful, only problem is that we have to provide a custom-gen.#2018-04-1213:23Andreas LiljeqvistBut usually I have to write custom-gens anyway...#2018-04-1214:52guyWhat would you do for speccing [& args]
?#2018-04-1214:53Alex Miller (Clojure team)(s/cat :args (s/* any?))
#2018-04-1214:53guyah perfect thanks#2018-04-1214:53Alex Miller (Clojure team)unless you have other knowledge about args#2018-04-1217:32kennyIs it possible to have a multi-spec
dispatch on the first value of a vector and return a Spec for the rest
of the vector? i.e. take [:my-vec 1 "2"]
. The multi-spec
would dispatch on :my-vec
and each defmethod
would return a spec for (vec (rest [:my-vec 1 "2"]))
- [1 "2"]
.#2018-04-1217:36Alex Miller (Clojure team)no, but you could dispatch on the first value of a vector and return a spec for the whole vector#2018-04-1217:37kennyYeah... It's just the Spec for the first part of the vector is uninteresting -- it's always going to be any?
. This makes the return value for the defmethod
s very repetitive.#2018-04-1217:39kenny(defmethod event-vec :my-vec
[_]
(s/cat :x any? :a int? :b string?)
^^^^^^^
)
That part will always be the same.#2018-04-1217:45Alex Miller (Clojure team)if only there was a way to remove boilerplate syntaxā¦.#2018-04-1217:45Alex Miller (Clojure team)oh wait, macros! :)#2018-04-1217:49kennyThat's also possible. The problem there is that once I move that to a macro, I need to move all functions that register a method for that spec to be macros.#2018-04-1217:54dadairWould x always be any? You could have x be say #{:my-vec}
so the spec is more specific to the event spec you are returning?#2018-04-1217:54dadairMore specific for tests around that event#2018-04-1217:54kennyYes, always any?
.#2018-04-1217:55dadairbut for that specific defmethod isntā x :my-vec?#2018-04-1217:55kennyYes, but that's already guaranteed because the multimethod is called.#2018-04-1217:57kennyThe API consists of a lot of functions that look like this:
(defn reg-my-thing
[id spec other-stuff]
(defmethod my-multimethod id
[_]
(s/cat :x any? :rest spec))
;; do other stuff
)
In order to do what you're saying I'd need make most of the API macros. That isn't the end of the world but it does make the code base a lot messier to do what seems like such a simple operation.#2018-04-1217:57Alex Miller (Clojure team)@kenny re āThe problem there is that once I move that to a macro, I need to move all functions that register a method for that spec to be macros. ā - why?#2018-04-1217:57kennyBecause the above code will not work.#2018-04-1217:58kennycat
needs a form, not a symbol.#2018-04-1217:58kenny(defmacro reg-my-thing
[id spec other-stuff]
`(defmethod my-multimethod id
[_]
(s/cat :x any? :rest ~spec))
;; do other stuff
)
#2018-04-1217:59Alex Miller (Clojure team)I donāt think that macro is correct, but it can be fixed#2018-04-1217:59kennyIt's not - more of psuedo code.#2018-04-1217:59kennyEssentially in order to program with spec, everything needs to be a macro.#2018-04-1218:01Alex Miller (Clojure team)this is a macro already, itās just not the right macro#2018-04-1218:01kennyNot sure I understand what you mean.#2018-04-1218:06kennyMy API is defined a bunch of functions that are passed a spec for the (vec (rest [:my-vec 1 "2"]))
. The functions all do global registration sort of thing (akin to defmethod
). Each of these functions needs to register a spec for the whole vector [:my-vec 1 "2"]
. I could construct that spec at the macro level based on the spec they passed in, but that'd mean my whole API needs to be defined at the macro level.#2018-04-1218:07kenny... because this doesn't work š
(defn reg-my-thing
[id spec other-stuff]
(defmethod my-multimethod id
[_]
(s/cat :x any? :rest spec))
^^^^
;; do other stuff
)
#2018-04-1218:15Alex Miller (Clojure team)something like this works:
(require '[clojure.spec.alpha :as s])
(defmulti v first)
(defmethod v :hi [_]
(s/cat :o #{:hi} :p #{:there}))
(s/def ::v (s/multi-spec v (fn [val tag] val)))
(s/valid? ::v [:hi :there])
(defmacro defvspec
[op tail-spec]
`(defmethod v ~op [_#] (s/cat :op #{~op} :rest ~tail-spec)))
(defvspec :a (s/cat :x int?))
(s/valid? ::v [:a 100])
#2018-04-1218:15Alex Miller (Clojure team)another option is to register your specs with s/def and then refer to them by their keyword name#2018-04-1218:16kennyYes. Except that requires a breaking change to the API.#2018-04-1218:16Alex Miller (Clojure team)which gets you out of caring about the form#2018-04-1218:16kennyreg-my-thing
is passed a Spec in its arguments.#2018-04-1218:16Alex Miller (Clojure team)if you have the spec instance, you can also have the macro invoke s/form to get back the form#2018-04-1218:17Alex Miller (Clojure team)in that case I donāt know that you even need a macro#2018-04-1218:18kenny(defn reg-my-thing
[id spec other-stuff]
(defmethod my-multimethod id
[_]
(s/cat :x any? :rest (s/form spec)))
^^^^
;; do other stuff
)
?#2018-04-1218:18kennyWon't that create a mess of the error messages?#2018-04-1218:26Alex Miller (Clojure team)(defn defvspec2
[op tail-spec]
(defmethod v op [_] (eval `(s/cat :op any? :rest ~(s/form tail-spec)))))
#2018-04-1218:28Alex Miller (Clojure team)(s/valid? ::v [:b 10]) ;; true
(s/conform ::v [:b 10]) ;; {:op :b, :rest {:y 10}}
(s/explain ::v [:b nil])
;; In: [1] val: nil fails spec: :user/v at: [:b :rest :y] predicate: int?
#2018-04-1218:29kennyInteresting.#2018-04-1218:29Alex Miller (Clojure team)really the same thing youāre doing with a macro#2018-04-1218:29kennyThis is a CLJS project so I'm not sure about the eval
usage.#2018-04-1218:30Alex Miller (Clojure team)oh sure, throw that in at the end :)#2018-04-1218:32kennyShould've mentioned that in the beginning š¬ It's essentially adding Spec to re-frame, thus the reg-*
API.#2018-04-1218:32Alex Miller (Clojure team)well then, I donāt know :)#2018-04-1218:33Alex Miller (Clojure team)I donāt understand the constraints in cljs as well. There are some changes coming to spec that will help with stuff like this too but Iām not sure when or how they will play out in cljs.#2018-04-1218:34kennyI'm guessing everything will need to be done at the macro level. I think the only constraint is the lack of eval
.#2018-04-1223:31mvIf I have a list of lists, is it possible to write a spec that enforces that no two sublists start with the same value?#2018-04-1223:35seancorfield@mv Sure, if you can write a predicate that tests for that, you can use that predicate in a spec (or even as a spec).#2018-04-1223:35mvSo just a standard function?#2018-04-1223:36seancorfieldYup.#2018-04-1223:36mvAnd Iām assuming s/valid?
? Iām new to spec#2018-04-1223:37seancorfieldPresumably you already have a spec for "list of lists"?#2018-04-1223:38mvNot yet, but there is a spec being applied for the individual elements with s/*
#2018-04-1223:39seancorfieldOK, so when you have your spec for list of lists, then you just s/and
that spec with your predicate and that's your complete spec.#2018-04-1223:39mvCool#2018-04-1223:39seancorfield(s/def ::list-list-spec (s/and (s/coll-of (s/coll-of ::sublist-element-spec)) sublists-have-unique-prefix))
#2018-04-1223:39seancorfield(or something like that)#2018-04-1223:40mvNeat!#2018-04-1223:41seancorfieldBear in mind you may not be able to generate data from that spec (you might, but generation may produce sublists with identical first elements quite often so the check on the generated data might fail).#2018-04-1223:41seancorfieldIf that's important, you'll need to write a custom generator.#2018-04-1223:41seancorfieldBut if you're just getting started with spec, you may not need that. Yet š#2018-04-1223:49mvYea I donāt think I need that yet, this is more to enforce a bug wonāt come back#2018-04-1322:00hmaurerIs it possible to use clojure multi-specs with derived keywords (through derive
) and ensure that āchild specsā are a superset of parent specs?#2018-04-1400:16hmaurerhttps://github.com/clojure/spec.alpha#2018-04-1400:16hmaureris clojure spec still in alpha?#2018-04-1400:25seancorfieldYes @hmaurer#2018-04-1400:25seancorfieldBut lots of people are using it heavily in production.#2018-04-1411:06hmaurer@seancorfield is it still in development? it seems ike there hasnāt been a single commit to the source code in 6 months?#2018-04-1411:22gklijs@hmaurer apparently Rich has some changes planned. There are some things which not yet work smoothly.#2018-04-1413:46matanlooks like docs are still pre-1.9
how do I use spec in 1.9? any good 1.9 getting started?
https://clojure.org/guides/spec#2018-04-1413:47matando I really need to
(:require [clojure.spec.alpha :as s])
?#2018-04-1413:57Alex Miller (Clojure team)yes#2018-04-1413:57Alex Miller (Clojure team)where do you see pre-1.9 docs?#2018-04-1413:58Alex Miller (Clojure team)docs are at https://clojure.github.io/spec.alpha/index.html#2018-04-1414:06matanhttps://clojure.org/guides/spec#2018-04-1414:11Alex Miller (Clojure team)why do think thatās pre-1.9?#2018-04-1414:11Alex Miller (Clojure team)I wrote the guide btw#2018-04-1414:16matanokay, so the namespace remains alpha
, got it š#2018-04-1414:17matangood guide BTW! š#2018-04-1414:18matanbut don't need to explicitly require
it anymore with clojure 1.9 do we?#2018-04-1414:18Alex Miller (Clojure team)you do#2018-04-1414:18Alex Miller (Clojure team)require just loads the code into the clojure runtime, as you need to do with any namespace#2018-04-1414:20matanright!#2018-04-1414:20matanit's not part of core that's why#2018-04-1421:18seancorfield@hmaurer That says it's very stable in its 'alpha' state! š#2018-04-1421:20seancorfieldI gather that some changes to make specs easier to work with programmatically are on the roadmap @gklijs but it's been great for us to use as-is in our production code. We're very happy with it. And we were using it early on in the 1.9 prerelease cycle, before it became .alpha
and was moved out of core (so, yeah, there were some code changes needed along the way -- but overall Clojure doesn't require many of those over the years).#2018-04-1422:54gklijsThat's fine and even if there are changes it will probably not change the main use. However I use it to store and retrieve data with the option to use a backward compatible spec to get the full data back, and I'm a bit scared changes might break it.#2018-04-1423:45mathpunkOh, also --- if I'm incorporating generative tests into a library, is the idea that I run stest/check
inside a deftest
?#2018-04-1501:03Alex Miller (Clojure team)as the doc string for s/fdef says: ā :args A regex spec for the function arguments as they were a list to be
passed to apply - in this way, a single spec can handle functions with
multiple aritiesā#2018-04-1502:37seancorfield@mathpunk :args
should always be an s/cat
with each argument specified.#2018-04-1502:38seancorfieldIn your case :args (s/cat :v :simplexity.simplex/natural-vertices)
if it has one argument, v
#2018-04-1502:39mathpunkI've been meditating on what @alexmiller said and, i just realized, i think i figured "well a simple string is a 'regex' that matches very few things so, one argument is just a spec for the argument"#2018-04-1502:39mathpunknow, i think i see it#2018-04-1502:39seancorfieldAlso (:args %)
in your :fn
spec will be the collection of argument values, so you'll likely want (:v (:args %))
-- to match the :v
in the :args
spec#2018-04-1502:40seancorfieldRead the Spec guide about spec'ing functions again to see how it uses s/cat
#2018-04-1520:37mathpunkWhen I run something like "(test/check `simplex {:clojure.spec.test.alpha/opts {:num-tests 1}})" it looks like it's still running 1000 tests. š¤#2018-04-1521:07mathpunkMind you, that might not be my problem --- I think I'm tucking way too much into my spec for a simplex, when I should be instead specifying the functions that act on a simplex (i.e. the ones named in the Simplex protocol). I'm a little fuzzy on how to specify functions that should be true in /every/ implementation, though...#2018-04-1521:55mathpunkhmm, I'm also a little unclear on how to run a generative test on a function from outside of the namespace it's defined#2018-04-1522:18seancorfield@mathpunk Just specify the fully-qualified name of the function to test/check
.#2018-04-1608:52hkjelsI have a lot of cases in my code where I destructure the input of a function using a spec and then extract the keys using a long vector thatās basically the exact same as the spec itself. Is there a way to just use a spec to destructure keys?#2018-04-1608:52hkjelssomething like {:keys ::component/params}
#2018-04-1608:57hkjelsin other languages you have like a splat
or explode
that would give you all of the keys as vars in the current scope. I get that it would be a bad idea, but with a spec your still explicit#2018-04-1615:05borkdude@hkjels This is exactly what I also asked for recently (some weeks back)#2018-04-1615:05borkdude@seancorfield wrote a macro but I wanted it to do without writing my own let macro#2018-04-1707:33hkjelsyeah, this seems like something that should be natively supported#2018-04-1615:25bbssI wrote a macro that uses the registry to return a vector with the keys, and use that in combination with C-c C-v C-w
eval and replace in the editor. Of course there is some potential erosion as you'd need to maintain it in multiple places. But at least it's explicit.#2018-04-1618:31mathpunkI'm still working out where to put specs in my project. I'm coming from test-driven development so it seems natural to put fdef
s in test namespaces: define what the function should do, separate from the implementation. But then again, does that mean that code that calls the function won't have loaded those specs and will be missing out on the benefits? I'm curious how people are organizing their projects.#2018-04-1621:47bbrinckI tend to put the fdef
right before the defn
. That provides the most documentation benefits IMO and also anyone who loads my code is guaranteed to get the specs if they want to instrument them#2018-04-1621:48bbrinckAs you noted, if you put them in test namespaces, then clients will need to load both namespaces in order to instrument, which I think would be surprising#2018-04-1621:48bbrinckYMMV#2018-04-1618:37leongrapenthin@mathpunk use :clojure.spec.test.check/opts#2018-04-1618:37leongrapenthinFor some reason they missed to alpha it#2018-04-1618:39mathpunkAh ha! Thanks so much#2018-04-1700:10d._.bI have a few transforms that are related in my application.
{:aH "abc"} ;;phase-1
{:aH :what-x-means} ;;phase-2
{:a-h :what-x-needs-to-be-for-an-external-app} ;;phase-3
What would be cool is if I could somehow link all of these specs together, and control generation. So, an initial spec for phase 1 would refer to phase 2, and phase 2's spec would refer to phase 3.#2018-04-1700:16d._.bmaybe this is more of a controlled generation question?#2018-04-1700:17d._.bmaybe conform/unform is what im looking for here?#2018-04-1700:20d._.bwhere a named conformer would be the transform between two phases, so I could (s/def ::phase-1->phase-2 (s/conformer ...))
(s/conform ::phase-1->phase-2 {:aH "abc"})
#2018-04-1700:55bbrinck@d._.b Hm, hard to say without seeing the code, but I suspect itād be easier to just write transform functions with a transformation library and use the specs to validate each transform is working#2018-04-1700:57bbrinckGenerally speaking, using a conformer to transform data can cause issues https://groups.google.com/d/msg/clojure/Tdb3ksDeVnU/LAdniiZNAgAJ#2018-04-1701:27d._.b@bbrinck yes, this makes sense to me. it is fine for me to write the transform functions and validate the transform, but for some of these specs, the difference is very small. For instance, "person": {:name "bob"} => {:name (get {"bob" :fun-bob} "bob")} => {:nick-name "fun-bob"}. It would be nice if I could link the specs as an ordering, and then choose which phase I want to generate.#2018-04-1701:28d._.bfrom an initial state or spec#2018-04-1701:38bbrinck@d._.b Yep, I can see how this would be convenient. I canāt speak for spec maintainers, but from what Iāve read so far, they have consistently indicated they want to stay away from transformation (and leave this to other libraries - both libraries that are spec-aware or those that are spec-agnostic)#2018-04-1701:39bbrinckI can certainly see the use, but this canāt be done in spec today and I doubt it will be added š . https://github.com/nathanmarz/specter might be a good fit here#2018-04-1702:29andy.fingerhutNot sure if it is something targeted at what you are hoping to do, but late in the Google group discussion linked above is a mention of the project spec-coerce, which the author says "instead of making conforms that do coercion, it uses the spec and a separated registry to conform the value (similar to what spec does to find the generators for a given spec)". https://github.com/wilkerlucio/spec-coerce#2018-04-1704:17Alex Miller (Clojure team)+1 spec-coerce#2018-04-1705:22ikitommi@d._.b tested spec-driven data expansion some time ago, via conforming. Not sure if thatās an good idea, but hereās an example: https://www.metosin.fi/blog/clojure-spec-as-a-runtime-transformation-engine/#data-macros#2018-04-1705:43Alex Miller (Clojure team)itās not a good idea#2018-04-1706:06ikitommiwould it be a good idea to add support for spec coercion in the core? or support for generic walking with coercion as one application for walking.#2018-04-1711:58Alex Miller (Clojure team)Generic walking yes, coercion maybe#2018-04-1712:41ikitommiI renamed CLJ-2251 to āGeneric Spec Walking for clojure.specā. Is something like that coming? Would you be interested in a demo / patch of that?#2018-04-1712:53Alex Miller (Clojure team)Itās something weāve talked about but not particularly looking for a patch. #2018-04-1706:19d._.bThanks for those recommendations. #2018-04-1706:27d._.bWhen an entity like āfooā exists in numerous, ordered forms, it feels superfluous to name intermediate specs, and edges on ātypesā. I donāt want that extra, unnecessary complexity in my programs, so Iām looking for a way to define a āfooā as a succession of small transformations, where the phases (minor transforms) are named loosely, abstractly. One place to define all of the successions of miniature transforms is better than enshrining stronger, named entities. The names wind up feeling forced.#2018-04-1706:29d._.b:thing/upcased-name feels like extra nonsense if itās a short-lived intermediate.#2018-04-1706:33d._.bCoalescing specs into logical, ordered groupings feels valuable if only to avoid the naming of tiny-delta intermediates.#2018-04-1707:11mathpunkI think I'm getting the hang of test/check and custom generators, but I'm getting this oddball response from one of my fdef's: https://github.com/mathpunk/simplexity/blob/complexes/src/simplexity/core.clj#L70#2018-04-1707:13mathpunkNB: Every simplex is a complex, but not every complex is a simplex.#2018-04-1714:39bbrinckThe problem seems to be with your :fn
spec on dim
. The reason the examples work e.g. (dim (complex []))
is that they donāt run the :fn
code#2018-04-1714:39bbrinckI deleted the :fn
spec on dim
and now (test/check
dim)` returns with no errors#2018-04-1714:41bbrinckIt looks like the issue with the :fn
is that :ret
does not contain the value, it contains the conformed value e.g. not -1
, but rather [:empty -1]
#2018-04-1714:52bbrinckI think this will avoid the exceptions, although the definition of dim
still fails the :fn
test in some cases:
(s/fdef dim
:args (s/cat :complex ::complex)
:ret (s/or :nonempty nat-int?
:empty #(= -1 %))
:fn #(and (number? (-> % :ret last))
(= (size (-> % :args :complex))
(+ (-> % :ret last) 1))))
#2018-04-1714:52bbrinckFWIW, if you want to test out your fn
spec outside of check try https://github.com/jeaye/orchestra#2018-04-1717:34mathpunk@bbrinck Ah ha! I forgot about conformance.... aaaagain. And orchestra looks useful, much obliged#2018-04-1707:18mathpunkWhat's really baking my noodle is, there are a few things I can think of that might just be plain broke in the way I'm writing this, but, it would be broke with MUCH smaller examples than I seem to be getting as examples of failing results#2018-04-1720:09mathpunkI'm trying to define different specs for different implementations of a protocol, which I don't know if it's even possible. This code passes my example tests, but when I try and test/check
any of the functions, I just get an empty sequence: https://github.com/mathpunk/simplexity/blob/complexes/src/simplexity/simplex.clj#2018-04-1720:12mathpunkThe problem I'm trying to solve is: A simplex is a special case of a complex. It would be nice to construct a simplex by going (simplex [0 1 2])
and a general complex by going (complex [[0 1 2] [2 3] [4 5 6]])
#2018-04-1720:12mathpunkThat is, construct a simplex from a collection of vertices, and a complex from a collection of (maximal) simplexes#2018-04-1720:13mathpunkI'm open to other ways of accomplishing this goal#2018-04-1720:27bbrinckI havenāt tried it myself, but the advice Iāve heard is that protocols should be internal implementation details and you should spec wrapper functions, which become the public-facing API.#2018-04-1720:27bbrinckI could be misremembering the advice š#2018-04-1720:29bbrinckIn this case, youād have a spec for the wrapper function. Iām not sure that itād need to be different though - if complex
is the general case, could the work in terms of complex
? Iām not sure#2018-04-1720:30bbrinckYou could certainly create a generator that creates either simplex or complex values though, and use that#2018-04-1720:38bbrinckTry wrapping dim
in another function e.g. (defn dim1 [x] (dim x))
, then write spec for that. test/check
works for me on dim1
(poorly named, of course š )#2018-04-1720:28Alex Miller (Clojure team)You canāt spec protocol functions#2018-04-1721:07kennyWhy is the predicate function in the below error message :clojure.spec.alpha/unknown
?
(s/assert nat-int? -1)
ExceptionInfo Spec assertion failed
val: -1 fails predicate: :clojure.spec.alpha/unknown
clojure.core/ex-info (core.clj:4739)
#2018-04-1721:09kennyI need to wrap with s/spec
?
(s/assert (s/spec nat-int?) -1)
ExceptionInfo Spec assertion failed
val: -1 fails predicate: nat-int?
clojure.core/ex-info (core.clj:4739)
That seems asymmetrical with the rest of Spec's API -- you can pass a Spec kw, predicate, or Spec obj.#2018-04-1721:13Alex Miller (Clojure team)itās a bug - there is a pending patch for it#2018-04-1721:13Alex Miller (Clojure team)wrapping with s/spec bypasses it#2018-04-1721:14Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2068#2018-04-1802:22jumblemuddleIs it possible for me to have a spec: s/cat
regex where latter parts of the regex use predicates that depend on the conform of prior parts?#2018-04-1802:29jumblemuddleI guess the alternative would be to have a second spec to run after the first one, so it can depend on the output of the first one.#2018-04-1816:54cap10morganIf you had some spec'd code where correctness was far more important than performance, a) would it make sense to turn on instrumentation of every fn at production runtime and b) if so, how would one go about doing that?#2018-04-1817:10seancorfield@cap10morgan instrument
accepts a sequence of symbols to instrument as I recall. You'd probably want to just instrument
the functions in specific namespaces (your own) to avoid turning on instrumentation for all libraries functions in your app.#2018-04-1817:11seancorfield(you can "instrument everything" by calling instrument
with no arguments but you wouldn't know what functions got instrumented so you'd have no idea about performance impact)#2018-04-1817:12Alex Miller (Clojure team)you would know as it returns a collection of all symbols instrumented#2018-04-1817:12seancorfieldI meant, ahead of time.#2018-04-1817:12Alex Miller (Clojure team)you do have to ensure the specs have been loaded (by requiring them) before calling instrument#2018-04-1817:13seancorfieldBut, yeah, you could potentially get a very long list of instrumented functions that you weren't expecting š#2018-04-1919:09xiongtxIs there an easy way to modify an existing s/keys
spec?
I often find myself needing to take an existing spec and update a few keys. Itād be nice if I could just replace a key instead of copying everything.#2018-04-1919:50seancorfieldMaybe s/merge
would work for you @xiongtx? I'm not quite sure how it handles merging over existing keys tho'#2018-04-1919:51xiongtxI managed to get around it by redefining the specs that differ#2018-04-1923:04kennyIs it possible to instrument initial function calls? i.e. I have my dev namespace where I enable instrumentation:
(ns dev.user
(:require
[clojure.spec.test.alpha :as st]
[my-project.util]))
(st/instrument)
But by the time I load my dev.user
namespace, I have already ran some functions:
(ns my-project.util
(:require
[clojure.spec.alpha :as s]))
(defn reg-event
[id]
;; do side effecting stuff
nil)
(s/fdef reg-event
:args (s/cat :id keyword?)
:ret nil?)
(reg-event "my-event")
The call to reg-event
is not instrumented at this point so I do not get a instrumentation error that my call to reg-event
is incorrect.#2018-04-2001:40dadairYou could just (st/instrument reg-event)
after itās fdef
?#2018-04-2003:10kennyTrue but that doesn't scale well and requires special considerations in production.#2018-04-2019:12kennyI wrote a small library that addresses the above problem -- instrumenting initial/global function calls. It takes a slightly different approach than fdef
. During development it immediately wraps function calls with assertions on a function's args and ret value. Thoughts on this idea? š https://github.com/Provisdom/defn-spec#2018-04-2019:35franquitoNeat! Looks pretty similar to orchestra.#2018-04-2020:15kennyAh you're right. I did not realize orchestra did more than enable :ret
checking for instrumentation. This approach is still different than orchestra's defn-spec
. Orchestra's defn-spec
also doesn't have great IDE support.#2018-04-2020:29kenny@U3UFFB420 Added an excerpt in the README addressing this: https://github.com/Provisdom/defn-spec#how-is-this-different-from-orchestra.#2018-04-2103:00franquito@U083D6HK9! I pretty much skimmed the README at work. Now I gave it a read and see more clearly the purpose of the lib (BTW, that's a well written README). My thoughts are: I liked it. You just don't care anymore about instrument.#2018-04-2105:03kennyThank you. And yeah, I came to a similar conclusion. #2018-04-2305:34steveb8nI have a problem with a recursive spec in a cljc file. itās the ::tree (and ::node) spec here https://github.com/stevebuik/Stu/blob/master/src/cljc/viz/core.cljc#2018-04-2305:35steveb8nworks fine within this project but when invoked via an installed dep in another project, this spec fails to load#2018-04-2305:35steveb8nanyone seen anything like this before?#2018-04-2305:36steveb8nIāve tried using (s/spec (s/* ::node)) but no change in behaviour#2018-04-2315:15thomassilly question.. but how do I spec a string which has to have a certain value?#2018-04-2315:15tayloryou can use a set as a predicate#2018-04-2315:15thomassomething like this: (s/def :mqtt/protocol-name "MQTT")
but that doesn't quite work#2018-04-2315:15thomasat least not for the generators.#2018-04-2315:15taylor(s/def ::my-spec #{"MQTT"})
#2018-04-2315:16thomas:+1:#2018-04-2315:17thomasthank you, that did the trick of course!#2018-04-2317:09kkruitIs there a way to compare the :pred response from s/problem to the s/def record?#2018-04-2317:10kkruiti'm trying (=(:pred (first (s/problem issue))) :test.spec/equal?)#2018-04-2317:12kkruitwhen i debug the value of pred is the value of the s/def#2018-04-2317:12kkruitbut i'm getting false#2018-04-2317:22kkruitsomething like that#2018-04-2317:23kkruitis there a way to compare functions like that?#2018-04-2317:45kkruitoh, nevermind, it's in :via, thanks š
#2018-04-2319:05richiardiandreaI have never noticed this, but basically the output of clojure spec does not expand nested specs:
user=> (doc my-ns.core/test-fn)
-------------------------
my-ns.core/test-fn
([n])
Spec
args: (cat :n :test-ns)
ret: any?
nil
Is there a way to see what the :test-ns
spec actually is?#2018-04-2319:08Alex Miller (Clojure team)you can invoke doc
on spec names too#2018-04-2319:08Alex Miller (Clojure team)I would expect that to be a qualified keyword above though#2018-04-2319:09Alex Miller (Clojure team)not sure if you altered the output#2018-04-2319:09Alex Miller (Clojure team)user=> (doc :clojure.core.specs.alpha/args+body)
-------------------------
:clojure.core.specs.alpha/args+body
Spec
(cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))
#2018-04-2319:09Alex Miller (Clojure team)for example#2018-04-2319:12richiardiandreado I need to namespace it? I have a global :test-ns
only#2018-04-2319:13Alex Miller (Clojure team)spec names are expected to be namespaced#2018-04-2319:13Alex Miller (Clojure team)maybe weāre talking about different things#2018-04-2319:13richiardiandreaoh right#2018-04-2319:14Alex Miller (Clojure team)spec names should be qualified, as in :foo/bar#2018-04-2319:14richiardiandreano you are right my bad#2018-04-2319:18richiardiandreaperfect thanks it works fine š#2018-04-2409:36ouvasamHi,
With clojure.spec, Is there a way to get all the errors with s/and instead of having only the first one ?
e.g.
``
(s/explain (s/and int? even? #(> % 1000)) 9)
;; val: 9 fails predicate: even?
``
Is it possible to also have the last predicate error ? (> % 1000)
Many Thanks#2018-04-2412:12Alex Miller (Clojure team)In this case, no. Because s/and flows values through if an earlier one fails it canāt run a later one#2018-04-2412:51ouvasamThanks. Is there another way to get all the errors ?#2018-04-2413:44Alex Miller (Clojure team)not without breaking apart the s/and into pieces and running each individually#2018-04-2417:23xiongtxIs there a way to express the idea that a spec is any subset of some provided set, and to have generator behave accordingly?#2018-04-2417:49misha@xiongtx something like:
(s/def :foo/item #{:a :b :c})
(s/def :foo/items (s/coll-of :foo/item :kind set?))
or just
(s/coll-of #{:a :b :c} :kind set?)
#2018-04-2417:56xiongtxš#2018-04-2609:11borkdudeGary talked about an interesting issue recently. What if you instrument all and there are loads of fdefs for core functions. This will create a lot of validation overhead. Anyone experienced this becoming an issue?#2018-04-2611:54Alex Miller (Clojure team)Currently, no core functions are specāed (just macros, which are done at compile time not runtime, although admittedly those are often pretty close together in repl dev). Having too much instrumented definitely can create a noticeable effect.#2018-04-2609:46Oliver GeorgeThere's definitely overhead. By rights it shouldn't matter so much at dev time which is the common use case for instrumenting.#2018-04-2612:36gfredericksit'll matter if specs get added to clojure core fns; I can easily imagine a 10000x slowdown depending on what you're doing#2018-04-2609:54Rose MolinaGood morning,
I recently started using spec and I wanted to know the reason behind why s/and
conforms values and propagates them.
I hadn't read the spec guide (https://clojure.org/guides/spec#_spec_ing_functions) in full before I found the first example of this in our code base and I found it hard to read.#2018-04-2611:52Alex Miller (Clojure team)conforming tells you not just whether something validated (like valid?
but how it validated - indicating which alternatives are taken and how value components were parsed). Flowing conformed values allows s/and
to also flow conform info out to the user from internal components (which is also done in most other conforms)#2018-04-2613:03dominicmIs it possible to "redirect" a failure. For example, if a list of numbers must be sequential, this validation can only be performed on the "whole", but the failure is on a particular index.#2018-04-2613:06Alex Miller (Clojure team)not without implementing your own spec#2018-04-2614:20dominicm@alexmiller is that api private currently?#2018-04-2614:20Alex Miller (Clojure team)itās public but undocumented and subject to violent change#2018-04-2614:21Alex Miller (Clojure team)as in, I expect it to change, and possibly even be completely replaced.#2018-04-2614:21dominicmI'll wait on that, knowing it isn't impossible is good.#2018-04-2614:21Alex Miller (Clojure team)things like spec-tools are using it extensively#2018-04-2614:21dominicmIt was one frustration I'd had, e.g. for password validation data.#2018-04-2614:21dominicmYeah, I wasn't touching it because I know it will break some day.#2018-04-2615:23ikitommithe Spec
protocol will be replaced? Is it possible to comment or discuss on the upcoming changes, from 3rd party library maintainer perspective?#2018-04-2615:31Alex Miller (Clojure team)no, I donāt know yet what they will be#2018-04-2615:31Alex Miller (Clojure team)it may be replaced, it may not be replaced, it may change, I have no idea yet#2018-04-2615:32Alex Miller (Clojure team)this is the alpha part of spec.alpha#2018-04-2615:32Alex Miller (Clojure team)my advice right now would be - donāt use that#2018-04-2615:34Alex Miller (Clojure team)I expect changes to the main user api of spec to be minimal or at least well thought about#2018-04-2615:38ikitommiok. thanks. there is a alpha disclaimer too on spec-tools, but just checked, it has 13k downloads, so someone is using that already. hopefully there is a way to port the features to new spec apis - or remove stuff that is implemented by the new core directly.#2018-04-2615:41Alex Miller (Clojure team)I have not been shy about telling spec-tools and others what I said above#2018-04-2615:41Alex Miller (Clojure team)spec-tools is almost certain to be broken by future spec changes#2018-04-2615:42Alex Miller (Clojure team)I donāt agree with most of what that library is doing#2018-04-2921:27devurandomHi! I would like to spec a map where the values have to conform to some spec, but the fields can be arbitrary. How do I do that?#2018-04-2921:30Alex Miller (Clojure team)Use s/map-of and any?#2018-04-2922:07devurandomThanks, that works! This was the first time with Spec for me and the original code I modified was using cat
, which was confusing me.#2018-04-2922:14devurandom@alexmiller s/cat
also matches maps, is that correct? So I need to get the ordering right to make s/map-of
match first / instead of s/cat
?#2018-04-2922:15devurandomI came up with this, which appears to work (and passes tests), but I'll have to read more about spec to be sure: https://github.com/devurandom/venia/commit/31d0a15a28f4a20c9355f7d452f7b2b430f2ff16#diff-c1f178afedb93b23e615114ff149605d#2018-04-3006:13jimmyhi guys, how can I instrument spec in another namespace?#2018-04-3008:40gnl(st/instrument `namespace.something/function)#2018-04-3009:13jimmy@clojurians.net thanks for you help. I have tried it and it returns empty []
. If I have the fdef in the same ns, and I run (st/instrument)
it works#2018-04-3009:20gnlJust tested it to make sure I'm not missing something, but it works for me. Have you made sure that the external namespace the fdef is in is :required?#2018-04-3009:37jimmyyes it does. I feel like there is something missing as well, but I havenāt figured it out yet, it should work like your said.#2018-04-3009:44gnl@nxqd Can you post the head of your fdef, as in (s/fdef <name> ...)
?#2018-04-3009:45guyI found if your fdef doesnāt work correctly you canāt instrument it#2018-04-3009:45guyif it gives that error saying it cant be satisfied after 100 tries or something, it wonāt be able to be instrumented#2018-04-3009:46jimmy@guy if I put this spec in the same ns, it works.#2018-04-3009:46guyinteresting#2018-04-3009:46jimmymy purpose is to reuse the same fdef button ns for different functions#2018-04-3009:46guyare you doing it across clj/cljs?#2018-04-3009:47jimmyyes, but for now, I test in in clj repl, so itās only java for now.#2018-04-3009:47guykk#2018-04-3009:47gnlAm I being blind here or is there no (defn button ....)?#2018-04-3009:47guyi think he had ;;button fn#2018-04-3009:47jimmy@clojurians.net itās defined in another namespace.#2018-04-3009:48gnlah#2018-04-3009:48guydont you have to import it?#2018-04-3009:48guysome-ns/button#2018-04-3009:48gnlwell then you have to (s/fdef some-ns/button)#2018-04-3009:48guy(s/fdef some-ns/button
:args (s/cat :opts (s/keys :opt-un [:button/type :button/state :button/size
::class])))
#2018-04-3009:48guyyeah#2018-04-3009:48guywhat gnl said š#2018-04-3009:48jimmyok I see#2018-04-3009:48jimmythanks š#2018-04-3009:49guyDid that fix it?#2018-04-3009:49jimmyit does. Thanks guys#2018-04-3009:49guyexcellent!#2018-04-3009:49guyit was all gnl really š#2018-04-3009:49guyAlso thats a super common mistake to make so donāt worry about it#2018-04-3009:50guyI had the same issue before#2018-04-3009:50guyand spent an hour or two being confused#2018-04-3009:50guyš#2018-04-3009:50jimmyyeah, most of the time I just define it in the same ns š so I didnāt notice this.#2018-04-3009:50guyš#2018-04-3009:50jimmythanks a lot guys#2018-04-3015:06deansherThis is a question about clojure.spec.alpha as an example code base, rather than about using it. In the following code, it seems to me as though alpha.clj is working very hard to avoid multimethods. Does anyone have insight into why?
(defprotocol Specize
(specize* [_] [_ form]))
(extend-protocol Specize
clojure.lang.Keyword
(specize* ([k] (specize* (reg-resolve! k)))
([k _] (specize* (reg-resolve! k))))
clojure.lang.Symbol
(specize* ([s] (specize* (reg-resolve! s)))
([s _] (specize* (reg-resolve! s))))
Object
(specize* ([o] (spec-impl ::unknown o nil nil))
([o form] (spec-impl form o nil nil))))
(defn- specize
([s] (c/or (spec? s) (specize* s)))
([s form] (c/or (spec? s) (specize* s form))))
#2018-04-3015:29Alex Miller (Clojure team)protocols are faster#2018-04-3015:52deansherThanks, Alex. I wondered if it was as simple as that.#2018-04-3016:35ghadihttps://clojurians.slack.com/archives/C03S1KBA2/p1525106043000733#2018-04-3016:35ghadi@deansherthompson ^#2018-04-3016:35ghadiit's not just performance.#2018-04-3016:53Alex Miller (Clojure team)in this particular case, the protocol has one method, so thatās not a concern#2018-04-3017:10deansherHmm, Iām thinking of it quite differently. The Specize protocol is allowing keywords, symbols, and (to a degree) objects to participate in the multiple-method Spec protocol, instead of using multi-methods. For example:
(defn conform
"Given a spec and a value, returns :clojure.spec.alpha/invalid
if value does not match spec, else the (possibly destructured) value."
[spec x]
(conform* (specize spec) x))
(defn unform
"Given a spec and a value created by or compliant with a call to
'conform' with the same spec, returns a value with all conform
destructuring undone."
[spec x]
(unform* (specize spec) x))
#2018-04-3016:38jimbobis there a good spec function that will return the req-unāed keys for a spec defined map?#2018-04-3016:40jimbobexample. i have a record blah {:a a :c c :e e }
and a spec def (s/def ::blah-fact
(s/keys ::req-un [::a ::e])
whats the best way to call a function to get back #{:a :e}
?#2018-04-3016:42ghadi@ben.borders Here's one way:
user=> (s/def ::mymap (s/keys :req-un [::boo ::bar]))
:user/mymap
user=> (-> (apply hash-map (-> ::mymap s/form rest)) :req-un)
[:user/boo :user/bar]
#2018-04-3016:42ghadi(s/form ::mymap) == '(clojure.spec.alpha/keys :req-un [:user/boo :user/bar])
#2018-04-3016:53eoliphantHi, Iām working on a system of clojure based services, that passes messages based on clojure maps around. So Im thinking I can just create a common module of specs (or at least a common as they need to be), but I have one other potential wrinkle. We basically need to support ātenantsā. So I might have the same attribute, that means the same thing throughout the system (s/def :account/number ; like "A123")
so far so good. But for another tenant maybe :account/numbers should start with āBā.. Is a good way to handle this ?#2018-04-3016:53Alex Miller (Clojure team)note that that wonāt work if :req-un contains and
or or
@ghadi @ben.borders#2018-04-3016:59jimbobcool that works. thanks!#2018-04-3021:35ghadiHas anyone experienced... "Overspeccing"?#2018-04-3021:52seancorfield@ghadi You mean, someone trying to spec "everything"? I've seen some folks (new to spec) go down that path...#2018-04-3021:54ghadiMaybe not as an intent, but definitely as an effect#2018-04-3021:54ghadie.g. spec the wire boundary, spec post-transformation, spec the output (of whatever)#2018-04-3021:54ghadithen you end up with hundreds of specs#2018-04-3021:55Alex Miller (Clojure team)itās useful to spec core data structures, boundaries, functions amenable to property testing#2018-04-3021:56Alex Miller (Clojure team)but itās totally ok to not spec stuff, or use any? or ifn? etc to under-spec things that are still in process or too annoying to spec#2018-05-0110:24dominicmThis possibly stems from not knowing when to not apply specs, causing an excess of inappropriate specs. It's easy to assume spec = types, ergo spec all the things [meme]#2018-05-0110:25dominicmI personally struggle at times to know when to, and not to apply certain tools that Clojure provides, and the entire ecosystem even less. e.g. there are still regulars queries about protocols vs multimethods, until recently I reached for protocols more often than I did multi-methods, for no real reason.#2018-05-0111:15mpenetspecs "could" be used everywhere in theory, but in practice gen gets in the way (even via instrumentation)#2018-05-0111:17mpenetthen if you want to s/check everything you're in for some waiting. not really usable in practice that way either. s/check requires to be more selective#2018-05-0111:25mpenetI personally wish instrumentation would never trigger gen, or at least that we would have the option to disable it without having to dumb down the spec (ex: not having to resort to ifn? for predicates)#2018-05-0116:39seancorfieldWe tend to spec arguments to APIs, some data structures around the DB (where we can extract the lists of fields to save/update from the specs), core domain data structures, and a handful of functions that interact with those, either across module boundaries, or for certain "critical" domain functions. We try hard not to overuse them because we want to avoid brittleness.#2018-05-0117:59Drew VerleeWhats the ideal way to spec a nested structure? e.g
{:x {:y {:a "hi" :b "bye"}}}
All the examples in the docs are fairly flat data structures. Also most examples assume the key is a key and not a unique value.
Which in most cases, it should be, in this case i was modeling a graph and iām not sure its reasonable/easy to do it that way.
In a related vein. I wonder how easy it is to do a depth first search with datalolg and datascript. Which would be another way to go about doing this.#2018-05-0118:37Drew VerleeYea. After some more thought, its not the nesting thats the issue. Its the fact that my keys are unique in this example. Iāll have to think about if i want to lose some terseness in order to label things more.#2018-05-0119:03tbaldridgegraphs are going to be hard to spec completely, you'll need a "any?" in a few places.#2018-05-0119:03tbaldridgespec will consume stack as it traverses the graph, so if you manage to create a really deep graph, or a cyclic graph, you'll have problems validating it#2018-05-0214:28trissso how do I stop stest/check
from generating NaN
s all the time?#2018-05-0214:29trissthe spec that fails is: (s/and number? #(<= 0 % 100))
#2018-05-0214:33Alex Miller (Clojure team)do you really need number?
? or is something narrower like int?
appropriate?#2018-05-0214:34Alex Miller (Clojure team)it is a known issue right now that NaNs cause issues for certain generators and something we intend to fix#2018-05-0214:35trissah ok. Iāll see if int-in works out for me. thanks.#2018-05-0214:37Alex Miller (Clojure team)#(not (Double/isNaN %))
pred may be useful to filter too#2018-05-0214:37guySounds silly but does the order matter inside the s/and#2018-05-0214:37guyin the NaN example#2018-05-0214:37guywould you have to have #(not (Double/isNaN %))
second?#2018-05-0214:38guythen #(<= 0 % 100)
last?#2018-05-0215:07Alex Miller (Clojure team)yes, order matters as the first spec generates, then each subsequent one filters, so the order you suggest is probably good#2018-05-0215:11guyok thanks!#2018-05-0214:44eoliphanthi I have , I guess a āphlisophicalā lol question about spec. Im working on a system that uses datomic, etc. So, Iād gone though and created a bunch of specs, now Iām building out datomic schemas, using my specs to name my attributes, etc. So of course, some of this isnāt especially DRY. Iām just wondering about creating my own (or using umlaut, etc) lightweight DSL that in turn genās datomic schemas, specs, etc. Is this theārightā way or is starting from specs ābetterā, etc. Just wondering#2018-05-0820:09kenny@eoliphant We wrote a library to do this called Spectomic https://github.com/Provisdom/spectomic and have been using it successfully for about a year now.#2018-05-0215:08Alex Miller (Clojure team)some people have done this to various degrees#2018-05-0215:08Alex Miller (Clojure team)hard for me to say if they found that to be a win in the long run or not#2018-05-0215:40robert-stuttaford@eoliphant Datomic schema is not the same as a spec. they do similar things - link behaviour to a name, but they are not the same thing (they create different behaviour). conflate them only if you are confident that in doing so, users of your new combined API understand the consequences of this clearly. semantically, they are different. i find i prefer a little extra typing, and having decoupled systems that use very well documented APIs.#2018-05-0215:41robert-stuttafordmy hobby project https://github.com/robert-stuttaford/bridge keeps them separate, and i donāt think it actually costs anything to do that#2018-05-0215:43robert-stuttafordso i guess the question is, are you actually Repeating Yourself? š#2018-05-0215:50trissSo Iāve got nice s/fdef
ās for my functions. Is it possible to use those in a test?#2018-05-0215:50trissI know (-> (stest/enumerate-namespace 'fca.slipnet.math)
(stest/check))
#2018-05-0215:51robert-stuttaford@triss yes: https://github.com/robert-stuttaford/bridge/blob/master/test/bridge/test/util.clj#L8-L27#2018-05-0215:51trissbut when I run tests from CIDER it tells me there a no tests defined.#2018-05-0215:51robert-stuttafordoh, thatās a different question š#2018-05-0215:51trissš#2018-05-0215:55trissSo stest/check
returns a :clojure.test.check.clojure-test/trial
how do I tell clojure.test
to look out for it?#2018-05-0216:25eoliphantyeah thatās kind of the way I was leaning @robert-stuttaford#2018-05-0216:45javiHi everyone, new to the channel.
Does anyone know of any articles on best practices /process to spec a pre-existing codebase?
I know spec, but so far haven't used it much and I have a "not-small" cljs codebase that i want to start spec-ing, but not sure where/how to start...#2018-05-0216:46guylike this you mean?
https://clojure.org/guides/spec#2018-05-0216:46guyoh sorry right larger codebases sorry no#2018-05-0216:49javithanks!!#2018-05-0216:45Alex Miller (Clojure team)start by writing specs :)#2018-05-0216:45Alex Miller (Clojure team)if you have a map that is used a lot, write specs for the attributes#2018-05-0216:46Alex Miller (Clojure team)then write an s/fdef for a function that takes that map#2018-05-0216:46Alex Miller (Clojure team)if you want to know where to start, pick the most common/stable data you have#2018-05-0216:46Alex Miller (Clojure team)work outward from there#2018-05-0216:47Alex Miller (Clojure team)or pick the thing you trust the least and add constraints around it by making specs and using stest/instrument#2018-05-0216:47Alex Miller (Clojure team)if you get an instrument failure, youāve learned something (either your data does things you didnāt anticipate, or something is wrong)#2018-05-0216:48Alex Miller (Clojure team)if the former, fix your spec. if the latter, fix your code.#2018-05-0216:48Alex Miller (Clojure team)if you have data transformation functions, write specs for the inputs and outputs and use stest/check#2018-05-0216:48Alex Miller (Clojure team)these are ideas, not a plan#2018-05-0216:48javi> or pick the thing you trust the least and add constraints around it by making specs and using stest/instrument
ha ha, good one š i think i ll start there. thanks!#2018-05-0216:52robert-stuttafordone very useful practice is to spec anything before you make changes. we have code from clojure 1.6 days that we do this to, and it really helps deal with stuff like regression testing, safe refactoring, etc#2018-05-0220:18djtangoHey folks, probably a noob question but am really struggling to find an answer via search (probably because of my lack of terminology):
If I define a spec using a non auto-resolved keyword (s/def :my/spec any?)
how do I use it if I've required that namespace in a separate namespace? I can't get a call to (s/valid? :my/spec 1)
to resolve nor does (s/valid? :: 1)
#2018-05-0220:42Alex Miller (Clojure team)the first should work#2018-05-0220:43Alex Miller (Clojure team)the latter is not valid#2018-05-0220:43Alex Miller (Clojure team)not a valid keyword that is#2018-05-0220:45Alex Miller (Clojure team)user=> (ns foo (:require [clojure.spec.alpha :as s]))
nil
foo=> (s/def :foo/thing int?)
:foo/thing
foo=> (ns bar (:require [clojure.spec.alpha :as s] foo))
nil
bar=> (s/valid? :foo/thing 10)
true
#2018-05-0222:20djtangoorz it works now. I think I must have had something else wrong in the file - thanks @alexmiller#2018-05-0222:39Alex Miller (Clojure team)Cool#2018-05-0307:56pyrLet me post my twitter question here as well š
> Clojure library writers, what's your position on spec asserts? Leave it to library consumers to set, leaving it to calling code to> choose whether to assert or not, or enforcing it from the library to ensure malformed payloads always throw?#2018-05-0311:45joost-diepenmaatdo you mean using spec.alpha/assert
?#2018-05-0311:48joost-diepenmaatprobably wouldnāt use that in a library, Iād rely on instrument during dev/test. if youāve got data coming in that must be validated (say user input or stuff coming in over a network) then you do an unconditional (when-not (s/valid ..) (throw ā¦))
#2018-05-0311:48joost-diepenmaatnote that assert
can be turned off by changing *compile-asserts*
#2018-05-0311:48joost-diepenmaatand you wouldnāt want that for stuf that must be checked.#2018-05-0400:52jstewWhat do I use if I just want to ensure that a fn returns a string or something? (s/fdef ::string-fn
:args (s/cat)
:ret string?)
(s/explain ::string-fn #(identity "string"))
#2018-05-0400:52jstewIt wants me to add test.check as a dependency#2018-05-0401:27Alex Miller (Clojure team)I think youāre mixing syntax here and thatās leading you down a path trying to gen anonymous functions with check#2018-05-0401:28Alex Miller (Clojure team)(s/def ::string-fn (s/fspec :args (s/cat) ret string?))
would be better for this#2018-05-0401:29Alex Miller (Clojure team)When function specs are validated they use the generator for the anonymous function to invoke the function being validated and verify the outputs. Using that generator requires test.check#2018-05-0410:38jstewAh hah. That makes sense. Thank you, Alex.#2018-05-0522:56madstapWhat does it mean to call s/fdef
with a keyword instead of a symbol?#2018-05-0600:23Alex Miller (Clojure team)You shouldnāt do that#2018-05-0600:23Alex Miller (Clojure team)But itās similar to calling s/def with an s/fspec#2018-05-0622:18gfrederickshttps://github.com/clojure/spec.alpha/blob/a65fb3aceec67d1096105cab707e6ad7e5f063af/src/main/clojure/clojure/spec/test/alpha.clj#L286
There's a thing here where spec is relying, for no clear reason, on what I'd like to say is a quasi-buggy implementation detail of test.check that I'd like to change.
So I'm wondering if actually changing that is realistic, given the alphas and the whatnot. It would mean that the next release of test.check is not compatible with the current releases of spec.alpha. And it's hard to think of how to keep that from being a confusing pain to users who upgrade in the "wrong" order. š#2018-05-0711:45Alex Miller (Clojure team)I would greatly prefer this to change in a growth compatible way (new fn if necessary)#2018-05-0711:49gfredericksthere are enough moving parts that I hadn't thought about something like that, but I might be able to make it work; thanks#2018-05-0702:09alexandergunnarsonSo, reached what seems to me to be a bug:
(gen/sample (s/gen (s/and (s/? string?) string?)))
-> ([""] [""] ["4"] [""] ["Ii5"] ["fye"] ["x4oa"] ["6i"] ["2LH7cR"] ["Y"])
But it seems that this should fail just as (s/and number? nil?)
. [""]
, for instance, is not a string?
.#2018-05-0711:40Alex Miller (Clojure team)s/? matches collections but conforms to a value and s/and flows conformed values so I think this is the expected behavior#2018-05-0711:43Alex Miller (Clojure team)Using s/? as a top level regex op often leads to confusing behavior - this is tricky. Usually it gets nested in s/cat or something#2018-05-0713:27alexandergunnarsonThanks! Yeah I found that it works with s/exercise
so that makes sense at least#2018-05-0713:35alex-dixonShould I expect spec validation errors to be thrown if I call a function that has a spec registered with s/fdef in CLJS?#2018-05-0713:38alex-dixon(defn foo [a b] a)
=> #'elp.components.dialogs/foo
(s/fdef foo :args (s/cat :attributes map?
:children vector?))
=> elp.components.dialogs/foo
(foo 1 2)
=> 1
I have check-asserts onā¦working with specs defined with s/def and {:pre (s/valid? ā¦.#2018-05-0713:38Alex Miller (Clojure team)have you instrumented it?#2018-05-0713:39Alex Miller (Clojure team)with instrument
?#2018-05-0713:40alex-dixonI was confused about that as well. An example I saw used s/instrument from the spec namespaceā¦.maybe it moved? Showing as undeclared var#2018-05-0713:40Alex Miller (Clojure team)it moved long long ago#2018-05-0713:41Alex Miller (Clojure team)itās in the clojure.spec.test.alpha namespace (in clojure) and parallel in cljs#2018-05-0713:41Alex Miller (Clojure team)although I think you can just require that same ns in cljs and it will automatically load the right one#2018-05-0713:41Alex Miller (Clojure team)https://clojure.org/guides/spec may be of help#2018-05-0713:47alex-dixonThanks. Sorryā¦lazy this morning#2018-05-0713:51alex-dixonIām on cljs 1.9.946 (unfortunately). Tried googling and canāt discern whether spec.test.alpha is a separate library#2018-05-0713:52alex-dixonQuestion after that would be how to instrument all functions thereās a fdef for#2018-05-0713:56Alex Miller (Clojure team)clojure.spec.test.alpha is a namespace and is included in clojurescript#2018-05-0713:56Alex Miller (Clojure team)(well the cljs equivalent)#2018-05-0713:58Alex Miller (Clojure team)https://cljs.github.io/api/cljs.spec.test.alpha/#instrument#2018-05-0713:59Alex Miller (Clojure team)several arities - no args = instrument all registered fdefs, or with a single symbol or with a collection of symbols#2018-05-0713:59Alex Miller (Clojure team)instrument is covered in the guide above and should work the same way in clojure and clojurescript#2018-05-0714:03alex-dixonThanks @alexmiller#2018-05-0714:04Alex Miller (Clojure team)oh, one caveat is that you will likely need to include org.clojure/test.check 0.9.0 as a dep#2018-05-0714:06alex-dixonAh hah. Thank you š#2018-05-0714:07Alex Miller (Clojure team)ClojureScript 1.10.238
cljs.user=> (defn foo [a b] a)
#'cljs.user/foo
cljs.user=> (require '[clojure.spec.test.alpha :as stest])
cljs.user=> (require '[clojure.spec.alpha :as s])
cljs.user=> (s/fdef foo :args (s/cat :attributes map?
:children vector?))
cljs.user/foo
cljs.user=> (stest/instrument `foo)
[cljs.user/foo]
cljs.user=> (foo 1 2)
#error {:message "Call to #'cljs.user/foo did not conform to spec:\nIn: [0] val: 1 fails at: [:args :attributes] predicate: map?\n:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha1385]\n:cljs.spec.alpha/value (1 2)\n:cljs.spec.alpha/args (1 2)\n:cljs.spec.alpha/failure :instrument\n", :data #:cljs.spec.alpha{:problems [{:path [:args :attributes], :pred cljs.core/map?, :val 1, :via [], :in [0]}], :spec #object[cljs.spec.alpha.t_cljs$spec$alpha1385], :value (1 2), :args (1 2), :failure :instrument}}
Error: Call to #'cljs.user/foo did not conform to spec:
In: [0] val: 1 fails at: [:args :attributes] predicate: map?
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha1385]
:cljs.spec.alpha/value (1 2)
:cljs.spec.alpha/args (1 2)
:cljs.spec.alpha/failure :instrument
#2018-05-0714:17alex-dixonThanks @alexmiller. Dev setup for cljs with spec is now:
(defn dev-setup []
(when config/debug?
(stest/instrument)
(s/check-asserts true)
(set! s/*explain-out* expound/printer)
(println "dev")))
#2018-05-0714:43pfeodrippeHi, guys, we'are having problems spec'ing protocols, we're implementing the protocols with records, so instead of wrap manually the protocol methods with functions, we're defining a defrecord* macro of the kind defmacro defrecord*
[name fields pname & opts+body]
(let [[{:keys [method-parser],
:or {method-parser rest-symbol}}
body]
(parse-opts opts+body)]
`(do (defrecord ~name
~fields
~pname
#2018-05-0714:44pfeodrippeIs it a good approach for now?#2018-05-0715:01Alex Miller (Clojure team)I think itās a bad idea to wrap protocol functions (as you then destroy many of the benefits of protocol functions)#2018-05-0716:45bronsawhy? that's the advice I've always seen given (even before spec was a thing) : use protocol methods for polymorphism and a wrapping function for validation + varargs + else#2018-05-0716:45bronsaunless you're talking about a different "wrapping"#2018-05-0716:49Alex Miller (Clojure team)I wouldnāt wrap it purely to use spec#2018-05-0716:50Alex Miller (Clojure team)there are some good reasons to wrap a call to a protocol but you donāt necessarily need one#2018-05-0715:02Alex Miller (Clojure team)rather, I would recommend just not specāing protocol functions#2018-05-0716:35pfeodrippeOk @alexmiller, thanks#2018-05-0717:39dominicm@alexmiller I'm curious, why not spec protocol functions?#2018-05-0720:09hiredmanhttps://dev.clojure.org/jira/browse/CLJ-1941?focusedCommentId=43063&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-43063#2018-05-0718:10Alex Miller (Clojure team)you canāt#2018-05-0718:11Alex Miller (Clojure team)or rather you canāt for the purposes of instrument#2018-05-0718:11Alex Miller (Clojure team)I suppose you could spec them regardless for doc (and maybe check)? I havenāt tried it.#2018-05-0721:18dominicm@alexmiller I thought you were suggesting it would be a bad idea even if it could be done. I wasn't missing any reason it would be a bad idea š#2018-05-0919:03plinsany suggestions where to find (libs maybe?) specs for strings contaning datetime in the ISO 8601 format#2018-05-0919:13ghadiuse java.time @plins#2018-05-0919:14plinsnot sure if i expressed my self properly, but id like to validate if a strings conforms to the iso, so
(s/valid? ::iso-spec "2018-05-06T13:14Z")
shoud evaluate to true
#2018-05-0919:14ghadi#(try (java.time.OffsetDateTime/parse %) (catch java.time.DateTimeParseException e false))#2018-05-0919:15plinsok! thanks#2018-05-0919:15ghadii wouldn't pull in any of the clj date time libraries, they all have serious defects compared to java.time#2018-05-0921:07seancorfield@ghadi That's a bit of a sweeping statement. Could you provide a bit more specificity?#2018-05-0921:08seancorfield(we are using clojure.java-time
which is a thin wrapper around Java Time that automates the chains of conversions between (Java) types)#2018-05-0921:09seancorfieldThe clj-time
readme already tells people not to use it if they're on Java 8 or later -- and recommends Java Time instead, optionally using clojure.java-time
as a wrapper.#2018-05-0921:11ghadiI wouldn't even use a wrapper#2018-05-0921:12seancorfieldFair enough. I find Java Time's raw classes to be a bit ugly if you're doing anything more than basic stuff. I find clojure.java-time
's wrapper much more pleasant for more complex date/time arithmetic/manipulation.#2018-05-0921:21ghadiclojure.java-time pulls in clj-tuple and potemkin, yuck#2018-05-0921:21ghadi"white gloves" for interop; mostly field accessors#2018-05-0921:23ghadiso much error-prone wrapper code... looking under the covers#2018-05-0921:23ghadisorry I stand by my statement#2018-05-0921:24ghadishow me a clojure time library and I'll show you a simpler way to do it with java.time#2018-05-0921:24seancorfieldI don't see it pulling in potemkin. clj-tuple has no dependencies.#2018-05-0921:25seancorfieldMy teammate is also a fan of direct Java interop rather than wrappers, but some interop code can definitely be made cleaner with wrappers (in this particular context, as we've seen in code reviews during pull requests).#2018-05-0921:26mgIt doesnāt pull in potemkin, it copies the relevant code to java-time.potemkin.*#2018-05-0921:26seancorfieldYeah, and I'll take that little bit of potemkin code but I agree that I wouldn't want the whole thing pulled in...#2018-05-0921:27ghadithe dep issue is minor at best#2018-05-0921:27ghadithese libraries add nothing to your understanding of the primitives, and introduce new names for everything#2018-05-0921:27ghadinames were very carefully chosen in java.time#2018-05-1000:53gfrederickswhat are the issues with clj-time? just the parallel "adds no value over joda"?#2018-05-1000:53gfredericksI've been using it for years and haven't noticed anything#2018-05-1002:15ghadijoda itself had serious design issues that the authors rectified as it became java.time#2018-05-1002:15ghadithere are a bunch of articles about the api differences of joda vs java.time#2018-05-1002:29gfrederickscoolthx#2018-05-1002:31seancorfield@U0GN0S72R FWIW, the readme of clj-time
points to Joda Time's recommendation to migrate to Java Time and that links to several articles I think.#2018-05-1002:31gfredericks:+1:#2018-05-0921:12seancorfield(esp. if you're unfortunately enough to have to convert back and forth with java.util.Date
and/or any of the SQL date/time types!)#2018-05-1010:55trissis there a quick way to flush the slipnet library thing?#2018-05-1017:07devnWhat's the rationale for having a spec registry?#2018-05-1017:18Alex Miller (Clojure team)https://clojure.org/about/spec#2018-05-1017:18devnThe Data spec registration and Function spec registration sections in https://clojure.org/about/spec get at it#2018-05-1017:18devnguess I was wondering if you'd add any additional color#2018-05-1017:19Alex Miller (Clojure team)not a lot to add beyond that#2018-05-1017:21Alex Miller (Clojure team)there was a desire to not keep shoving stuff into var meta#2018-05-1017:21devn@alexmiller I wondered "Vars themselves are a kind of registry, no?"#2018-05-1017:21Alex Miller (Clojure team)or vars#2018-05-1017:21devngotcha#2018-05-1017:22Alex Miller (Clojure team)with good names, we are not limited to a single āvarā registry#2018-05-1017:22Alex Miller (Clojure team)if you use vars, you need namespaces to hold them#2018-05-1017:23Alex Miller (Clojure team)with a separate registry, itās perfectly ok to make a spec named :a.b.c.d.e.f.g/foo without having a related namespace#2018-05-1017:24Alex Miller (Clojure team)Rich has some future ideas on managing names and aliases independently from namespaces#2018-05-1017:36devnthis pleases me greatly#2018-05-1017:37devnwe have a spec aliases macro for calling alias
+ create-ns
that we use, and while it works, it doesn't have good mouth feel#2018-05-1017:38Alex Miller (Clojure team)yeah, thatās a hack. we have aims to do better.#2018-05-1023:05Oliver GeorgeThis hurts most in clojurescript. Do you think it will be supported in the new approach?#2018-05-1100:23Alex Miller (Clojure team)Probably but I donāt know what it is yet :)#2018-05-1102:52Oliver GeorgeCan't say fairer than that. I look forward to specs becoming less verbose and quicker to write.#2018-05-1017:38devnthere's also the potential for the "oops, someone clobbered a spec name and didn't know about it" situation#2018-05-1119:12cap10morganAnyone know why the default generator for (s/map-of keyword? string?)
would take 10+ seconds to generate 1,000 samples?#2018-05-1119:15ghadiit's generating large maps. set (s/map-of .... :gen-max 2)
to constrain sizes#2018-05-1119:16cap10morganAh, cool. I hadn't looked for that option in the spec fn itself. Thanks @ghadi.#2018-05-1119:16ghadi(doc s/every)
will show some others#2018-05-1122:28ambroiseI have this spec
(s/def ::foo (s/keys :req-un [::zipcode
::state
::bar
::baz]
And Iād like to force zipcode and state to be consistent (i have a zipcode->state
function so if they are both present, they need to match)
How can I write my ::zipcode
and ::state
specs?#2018-05-1122:38Alex Miller (Clojure team)(s/def ::foo' (s/and ::foo (fn [{:keys [state zipcode]}] (= state (zipcode->state zipcode)))
#2018-05-1122:39Alex Miller (Clojure team)wrap your ::foo in another spec that combines it with a custom predicate to ensure the zip code constraint#2018-05-1122:39ambroiseawesome, thanks!#2018-05-1212:25dottedmagI have a spec: (s/def ::statement (s/cat ::header ::header
::delimiter ::delimiter
::txs-header ::txs-header
::txs ::txs))
-- is it how it is supposed to look like, or am I missing some shorthand?#2018-05-1212:35dottedmag::header
, ::delimiter
etc are declared using s/cat
themselves, so I suppose s/tuple
can't be used here.#2018-05-1212:37dottedmagMaybe there is a simpler way to spec this data format? The input is a CSV file parsed into a vector of vectors. Header is a number of lines, more-or-less fixed, but some are optional and may be omitted. Delimiter is a single fixed line. Txs headers is another set of more-or-less fixed lines, and txs is the rest of the document.#2018-05-1213:16Alex Miller (Clojure team)s/tuple might be better#2018-05-1213:17Alex Miller (Clojure team)For vector of fixed position fields#2018-05-1213:18dottedmagToo bad they are mostly, but not completely, fixed. Banks are, well, banks.#2018-05-1213:22ghadiI would turn the vec of vecs into a vec of maps, then spec with s/keys#2018-05-1213:29dottedmagParsing individual lines is not so bad ā they can be handled by s/tuple
. Parsing the sequence of lines is worse.#2018-05-1213:29dottedmagI'll leave it with s/cat
for now.#2018-05-1318:16andy.fingerhutI saw this message by you, Alex, on a Reddit discussion: https://www.reddit.com/r/Clojure/comments/8i3hh0/what_is_the_future_of_clojure_in_the_industry/dysixcn/#2018-05-1318:17andy.fingerhutWould adding the links you gave for Cognitect blog articles on spec, and the videos, to one or both of the spec rationale and spec guide articles be of interest?#2018-05-1318:24Alex Miller (Clojure team)Nah, most of it is also in the guide#2018-05-1318:24andy.fingerhutI have read about some operation in spec doing 'sampling' in its validation/checking of a value against the spec, for better performance (but also, obviously, less precise checking). Is there something in the spec documentation I can look for to remind me where this occurs?#2018-05-1318:25Alex Miller (Clojure team)s/every and s/every-kv#2018-05-1318:30andy.fingerhutThanks. I see those in the spec guide, and good to see the doc strings for s/every and s/every-kv mention that partial checking property.#2018-05-1405:46Oliver GeorgeBit of a thought bubble but how about using spec's generative features to do "type checking" like behaviour in the IDE: https://gist.github.com/olivergeorge/584b54fe0b1d4c6ce3c7a44ee8c29095#2018-05-1406:12gklijsShould be doable at least with cursive I think, it already has a check for having the correct arity.#2018-05-1413:46jmayaalvwhatās the best way to get the keys descriptions for a s/keys :req
) form? calling (s/describe)
returns a (keys)
but not sure how can it be used.#2018-05-1414:09jmayaalvgot it. managed to do what i wanted this way:
(s/def :clecto/person (s/keys :req [:clecto.person/first-name :clecto.person/last-name]))
(apply hash-map (rest (s/form :clecto/person)))
user> {:req [:clecto.person/first-name :clecto.person/last-name]}
#2018-05-1423:08jimbobHow can i check against a collection that only certain values of maps in the collection are unique? i.e. that the collection of maps is valid iff all maps in the collection have distinct values for at least one of two fields.
ex: [{:unique-field "abc" :a 1 {unique-field2 "abc2"} {:unique-field "abc" :a 3 {unique-field2 "abc2"}]
does not pass validation but
[{:unique-field "abc1" :a 1 {unique-field2 "abc2"} {:unique-field "abc" :a 3 {unique-field2 "abc2"}]
does#2018-05-1504:21nenadalmyour datastructures are not valid (parenthesis do not match)#2018-05-1509:33guyI think you would write a spec that for those values would have to be unique#2018-05-1509:34guy(s/def :a/unique-field #{"unique 1" "unique2"})
#2018-05-1509:34guyso then the spec for that map would include that field and that spec above and would only allow those values#2018-05-1509:35guyOr do you mean the maps are only valid when, regardless of the values, they have to all be unique?#2018-05-1515:43jimbobthe first one yes.#2018-05-1515:55jimbobso mostly just curious if there are ways to spec collections of things, but spec them in terms of the collection, not in terms of the individual items in the collection (so whether or not only certain fields for the items are unique, or that summing fields for all items is within a certain range, etc)#2018-05-1516:03ghadicustom predicates with s/and is the common mechanism for doing this#2018-05-1520:24lilactownso I'm trying to do some work where I validate a JSON blob against a swagger spec. I'm thinking about trying to translate the swagger spec into clojure.spec and do it that way#2018-05-1520:24lilactownbasically the opposite of what spec-tools does with it's swagger stuff#2018-05-1520:25lilactownhowever, it seems like there's an assumption that clojure.spec's are registered globally with a name and everything. I would be generating these specs dynamically, on the fly#2018-05-1520:25lilactownis this a good idea?#2018-05-1520:25hiredmanany predicate is a spec#2018-05-1520:26hiredmanso if you have a function that given a swagger spec and some data returns true or false, then you just partial that with the swagger spec and you have a spec#2018-05-1520:26hiredmanit isn't a very nice spec, it won't generate and you won't get vary granular error messages, but it is a spec#2018-05-1520:28lilactownokay. I was thinking of trying to use clojure.spec to do some of the heavy lifting around e.g. checking whether a map conforms to a set of expected values#2018-05-1520:29lilactownso like x-allOf: [
{
type: "object",
properties: {
personId: {
type: "integer",
format: "int64"
},
familyMemberId: {
type: "integer",
format: "int64"
}
},
required: [
"personId",
"familyMemberId"
]
},
gets turned into (s/def ::personId integer?)
(s/def ::familyMemberId integer?)
(s/def ::my-obj (s/keys :req-un [::personId ::familyMemberId]))
#2018-05-1520:30lilactownbut obvi this is made more difficult by the fact that this would need to be done for arbitrary number of swagger specs at runtime, so I would have to make sure I'm not clobbering things in the registry...#2018-05-1520:32guyout of interest why couldnāt you model the data in spec? would it really be dynamic?#2018-05-1520:33guyIām probably using the wrong definition of dynamic, but i thought you would know what sort of data you would be getting? so you could potentially spec out the shape of the data?#2018-05-1520:36lilactownI'm writing some code that you give it a URL to a swagger.json, a URL path, example request and an example response#2018-05-1520:36lilactownand it validates that the URL path, request and response are valid based on the swagger.json#2018-05-1520:36guyohhh i see#2018-05-1520:37guyso you can give it basically anything, and then you use the swagger to create a spec#2018-05-1520:37lilactownthat's the idea. not sure it's a good one#2018-05-1520:37guyit sounds pretty cool š#2018-05-1520:37guyi see ur worries about the registry now#2018-05-1520:37guyi guess you could test it by creating multiple unique specs and seeing how many you could create#2018-05-1520:40lilactownthe whole thing seems gross. I wish you didn't have to register them š#2018-05-1520:45seancorfield@lilactown Would it be easier/better to have something that generated spec source code from a swagger URL? After all, you only want to generate the spec once per API end point, but you'll want to validate data against that spec repeatedly...#2018-05-1520:49hiredmanit seems like it would be a better idea to use some swagger validator to validate the swagger, for a swagger validation service#2018-05-1520:52hiredmanI think the direction you see in spec-utils (spec -> swagger) makes sense because the idea is, you can consolidate different ways of specifying data (database schemas, swagger, json schema, etc) as spec specs, and then generate (or using spec to make sure you data matches the contract) those other things from spec as needed. But it doesn't seem like turning swagger in to specs, if all you care about is validating arbitrary swagger specs against arbitrary data, makes a lot of sense#2018-05-1520:59lilactownyeah I guess I'm looking for a swagger validator, and I decided to try and build my own using clojure.spec? and that sounds like it's not a good idea#2018-05-1521:02lilactownI'm actually struggling to find such a thing that exists, which is why I'm trying to build one in the first place#2018-05-1521:03guyIf you are doing it for fun, why not give it ago and see how it goes#2018-05-1521:03guyYouāll defo learn something#2018-05-1521:03lilactowneh it's not exactly for fun š
#2018-05-1521:04lilactownit happens to be fun but also, sprint ends in a week and a half#2018-05-1521:04guyah well thats a different matter, if ur doing it for work, i might try and find a different solution#2018-05-1521:11hiredmangoogling java swagger validator and jvm swagger validator both seem to turn up good leads#2018-05-1521:26ikitommiHere's one implementation: https://github.com/metosin/ring-swagger/blob/master/src/ring/swagger/validator.clj#2018-05-1521:58lilactown@U055NJ5CC scjsv is exactly what i'm looking for! š thanks!#2018-05-1609:54troglotitusing (s/def ::my-spec ..)
- is not a hard requirement. Everytime your keys are not about namespaces in your application - i think itās better to qualify by another namespace like (s/def :swagger.employee-schema/employee ..)
#2018-05-1802:28vladis it possible to evaluate spec description?#2018-05-1802:28vlad(def big-integer 1000)
(s/def ::small-integer {:spec (fn [n] (n < big-integer)) :description (str "Smaller than " big-integer)})
#2018-05-1802:28vlad(s/explain-data ::small-integer 5000)
#2018-05-1802:28vladproduces#2018-05-1802:29vlad#:clojure.spec.alpha{:problems [{:path [],
:pred {:spec (fn [n] (n < big-integer)),
:description (str "Smaller than " big-integer)},
:val 5000,
:via [:xxx.spec/small-integer],
:in []}],
:spec :xxx.spec/small-integer,
:value 5000}
#2018-05-1802:29vladI woud expect Smaller than 1000
in the description instead#2018-05-1802:43vladthe following one works, but I want to see this in the explanation#2018-05-1802:43vlad(-> (s/form ::small-integer) :description eval)
#2018-05-1803:02Alex Miller (Clojure team)this is not supported#2018-05-1805:27mpenetIt's easy to hack a (simple) meta registry to do that without abusing the internals of clj.spec.#2018-05-1805:28mpenetI did that and a few other horrors here if you re looking for inspiration https://github.com/mpenet/spex#2018-05-1805:28mpenetLook for spex/with-doc in the readme#2018-05-1807:52mpenetgist of it:
(-> (s/def ::foo any?)
(spex/with-doc "That's a foo"))
(spex/doc ::foo) => "That's a foo"
#2018-05-1813:55sundarjthere's a coll-of
but no corresponding one-of
; i can go from a scalar spec to a collection spec, but i don't think i can go from a collection spec to an element spec. is that intentional?
i'm wondering because i'm currently speccing a text editor, and it's going to have a collection of working documents as well an active document, which would naturally be one of the working documents#2018-05-1813:59mpenetseems like you just need a set#2018-05-1813:59mpenetor I am misunderstanding#2018-05-1814:01sundarji presently have this: (s/def :world.sometimes.nota.editor/working-notes
(s/coll-of :world.sometimes.nota.app/note
:kind vector?
:distinct true))
(s/def :world.sometimes.nota.editor/active-note
:world.sometimes.nota.app/note)
(s/def :world.sometimes.nota.app/editor
(s/keys :req [:world.sometimes.nota.editor/working-notes
:world.sometimes.nota.editor/active-note]))
#2018-05-1814:01sundarjbut it feels wrong because active-note
isn't just a random note, it's only ever going to be one of the working notes#2018-05-1814:05Alex Miller (Clojure team)this would be easier if the working-notes was a set (not sure why it isnāt) but you could s/and an additional check into editor that working-notes contains active-note#2018-05-1814:07sundarjit's not a set because the active-note is going to be pulled from the working-notes, and sets would be the wrong data structure for that, right? sets are just about membership, as far as i know#2018-05-1814:08sundarji hadn't thought about putting an s/and on editor, that makes a lot of sense. thanks š#2018-05-1814:17Alex Miller (Clojure team)youāre declaring in working-notes that its elements are distinct. thatās the same guarantee sets give you but they can check membership in (near) constant time rather than via a linear search#2018-05-1814:20sundarjyeah, i know. i had the :kind as set? before, but then i realised you couldn't pull arbitrary members out of a set; there's going to be a UI that lists all the working notes, and then you click on one to open it. i think a vector is a better choice given that use-case. i suppose i could have a vector and a set, but then i'd have to keep them in sync#2018-05-1814:22Alex Miller (Clojure team)what do you mean by āyou couldnāt pull arbitrary members out of a setā ? it seems like you can do exactly that#2018-05-1814:22Alex Miller (Clojure team)do you mean you need to retain ordering?#2018-05-1814:29sundarji meant that to pull an item out of a set you need the item itself; i thought you couldn't iterate over a set, but i realise i was mistaken. in any case, i think i do need to retain ordering#2018-05-1814:47Alex Miller (Clojure team)and thatās a good reason to use a vector, not trying to necessarily change your mind, just probing#2018-05-1814:54sundarji understand; i appreciate the questions. it did make me more fully analyse why i wanted it to be a vector, so thanks š#2018-05-2015:28karol.adamiecin clojurescript/javascript when i get json on the wire with UUID as a string, how should i spec that? uuid? passes even with empty string, (but has a correct genereator). I want the spec to be used as a tool for dev time to quickly catch misbehaving APIās. Also want to generate sample data for unit tests. (ie javascript harness reads a file generated by simple clojurescript spec gen and runs a suite of tests for each generated input. i could swap / regenerate entry fixtures as i see fitā¦)#2018-05-2105:41nenadalmany predicate can be used to spec value: https://clojure.org/guides/spec#_predicates#2018-05-2105:42nenadalmYou can write custom generator if existing ones are not good enough for you: https://clojure.org/guides/spec#_custom_generators#2018-05-2213:07troglotitHey! I want to spec for five characters long numbers: I could use regexes #(re-matches #"\d{5}" %)
, but I as far as I can understand that could be done in spec. I looked up a bit, but what I found is only s/* s/+ s/?
that kind of operators. So, is there that kind of spec? I know about s/cat
- but that seem too overkill for when I just want to repeat my regex#2018-05-2213:09ghadiwhile they may work for characters of sequences, I wouldn't use s/* s/+ etc. for that#2018-05-2213:10ghadiif you're parsing strings, a normal regex is more appropriate#2018-05-2213:10troglotitOh, I found (s/every :count)
- seems like what I wanted#2018-05-2213:11ghadilike i said, it's a bad idea#2018-05-2213:12ghadithe regex predicate you have up there is the better approach#2018-05-2213:13troglotitI want to have more composable regexes and spec seem suited for that, compared to regular regexes#2018-05-2213:16Alex Miller (Clojure team)spec is not designed to be a good string regex parser and is unlikely to perform or work well for that use case as actual regex or a parser#2018-05-2213:17Alex Miller (Clojure team)you might find something like https://github.com/cgrand/regex interesting#2018-05-2213:18Alex Miller (Clojure team)I want to say there is something else like this out there for composable regexes too#2018-05-2214:44bbrinck@troglotit The comment thread on this reddit post goes into some of the issues you may run into when using spec for parsing strings https://www.reddit.com/r/Clojure/comments/7vwpu4/parsing_with_clojurespec/#2018-05-2306:59flowthingI have something like this:
(spec/def ::groups
(spec/every-kv ::id ::group))
(spec/def ::students
(spec/every-kv ::group/id (spec/every-kv ::id ::student)))
(spec/def ::root
(spec/keys :req [::groups ::students]))
Is there a way for me to specify that when I do (gen/generate (spec/gen ::root))
that every group ID for every student has an equivalent group ID under the ::groups
key?#2018-05-2307:00flowthingIn other words, I'd like every group ID in ::students
to have a corresponding entry in ::groups
. I know how to spec that, but I'm wondering what the best way to write a generator like that is.#2018-05-2307:01flowthingI could always just post-process the generated map, but for more complicated maps, that becomes a bit difficult.#2018-05-2307:26andy.fingerhutI've not used this before, but have heard that test.check has something called bind
that lets you generate one value randomly, e.g. in your case perhaps a set of group IDs, and then use that set to generate other things, e.g. your ::groups map and also the ::students map, which could be generated to have exactly the same set of group IDs in both.#2018-05-2307:27andy.fingerhutNot sure if the test.check examples of bind
on this doc page are enough to get you going or not: https://clojure.github.io/test.check/generator-examples.html#2018-05-2310:36gfredericks@flowthing you can generate the groups first, and then (using bind) generate students where their group-id is from (gen/elements (map :id groups))
#2018-05-2310:47flowthingMany thanks for the tips! I'll give gen/bind
a go.#2018-05-2310:47gfredericks@flowthing gen/let
can do the same thing, and you may find it more intuitive#2018-05-2310:48flowthingYeah, I've tried gen/let
actually, but I haven't yet come up with a clean solution. Nonetheless, it's good to know I'm heading in the right direction.#2018-05-2315:18lwhortonhey peoples, iām not really sure the term/noun for what iām trying to do with spec, so itās hard to googleā¦ I want to define a spec where an id field thatās present in two places is guaranteed to be the same:
{"id" {:id "id" :other-stuff true}}
#2018-05-2315:18lwhortonIām assuming I have to write a custom generator to enforce this?#2018-05-2315:24guyI think you would have to have a predicate that for that map, it checks that key and the :id value is the same.
And i believe yes, you would have to have a custom generator for it too.#2018-05-2315:24guyLike i donāt know how you would do it with s/keys
for example#2018-05-2315:25guybut saying that, there might be another way#2018-05-2315:25Alex Miller (Clojure team)yes and yes#2018-05-2315:26lwhortonyea iām not sure how to handle it either, i might just scrap this gen test entirely.. like how do you s/map-of ::id ::some-spec
with also ::some-spec (s/keys :req [::id])
, then write a generator that builds out two(?) specs. maybe i have to merge the map-of?#2018-05-2315:26guyYou could probably use this.
https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/spec
To create a spec for it#2018-05-2315:27guy(s/def ::some-spec (s/spec #{200 202 400}
:gen (fn [] (gen/return (gen/generate (s/gen #{200 400 500}))))))
like something like this maybe but catered for making your map#2018-05-2315:27guywhere u can supply a spec or a predicate function which would check both id key and value#2018-05-2315:28lwhortonah, thatās interesting#2018-05-2315:28guyyeah let me find an example soz#2018-05-2316:12rickmoynihanSo Iāve written some specs in an existing project with a clojure.test
test suite.
Does anyone have any recommendations on incorporating generative tests using clojure.spec.test.alpha/check
with clojure.test
?
Obviously I could do:
(t/is (:result (first (clojure.spec.test.alpha/check `foo))
But that will mask the errors.#2018-05-2316:13rickmoynihanA while back I had a similar issue and tried hooking into the clojure.test
assert macros but it was never very satisfactory#2018-05-2316:14rickmoynihanis there a lein
test runner for fdef
s?#2018-05-2316:15rickmoynihanactually just remembered Iāve been here beforeā¦ and have already implemented a lein alias in this project for running specs using instrument etcā¦#2018-05-2316:19lwhortoni wrote a little helper for your first problem..
(defn passes?
"A light helper function to evaluate a generative/property test so that
clojure.test spits out the failing result (if any) for easier debugging."
[spec]
(let [res (spec)]
(t/is (nil? (:shrunk res)))
(t/is (true? (:result res)))))
#2018-05-2316:20lwhortoniām not sure if thatās too implementation-specific with checking shrunk, but it works for me and i didnt have to dig around docs for a long time#2018-05-2316:22rickmoynihan@lwhorton what are the arguments?#2018-05-2316:23rickmoynihanobviously itās a functionā¦ but I guess I should ask how do you use it?#2018-05-2316:23lwhortonah, itās just like you would use testing
or is/are
:
(defspec my-spec my-prop-test)
(deftest some-test
(testing "some generative test should pass"
(passes? my-spec))))
#2018-05-2316:24lwhortonerm, rather āļø#2018-05-2316:25rickmoynihanwhat is defspec
?#2018-05-2316:25lwhorton[clojure.test.check.clojure-test :refer-macros [defspec]]
#2018-05-2316:26rickmoynihanahh ok ā thatās not strictly spec though is it?#2018-05-2316:27rickmoynihanIsnāt that a property test defined with test.check
? Rather than through s/fdef
etc#2018-05-2316:29rickmoynihanI appreciate spec builds on test-check and that theyāre compatible to some degree; but will that pick up an fdef
:fn
spec etc#2018-05-2316:29lwhortonyea thereās a few moving pieces, the spec itself is defined using clojure.test.check.properties
and a generator that uses spec/gen
#2018-05-2316:30lwhortonthis was pretty helpful for me: https://github.com/clojure/test.check#examples#2018-05-2316:30lwhortonparticularly:
(defspec first-element-is-min-after-sorting ;; the name of the test
100 ;; the number of iterations for test.check to test
(prop/for-all [v (gen/not-empty (gen/vector gen/int))]
(= (apply min v)
(first (sort v)))))
#2018-05-2316:31rickmoynihanyes Iāve seen that before; but Iām not sure itās what I need ā e.g. if you had a spec defined like this from the clojure docs:
(s/fdef ranged-rand
:args (s/and (s/cat :start int? :end int?)
#(< (:start %) (:end %)))
:ret int?
:fn (s/and #(>= (:ret %) (-> % :args :start))
#(< (:ret %) (-> % :args :end))))
#2018-05-2316:32lwhortonooph, iām not sure man sorry š ā¦ im just getting into the gen testing myself#2018-05-2316:33rickmoynihanthat spec is already def
ed so to speak; as itās metadata is registered already with the system.#2018-05-2316:33rickmoynihan@lwhorton: š no worries; itās subtle stuff#2018-05-2316:34rickmoynihanI have used defspec
beforeā¦ and tbh it seems like itās a better way to integrate with clojure.test
right now#2018-05-2316:36rickmoynihan@lwhorton in the interests of sharing I have something like this that works as a different entry point i.e. run via a lein
alias:#2018-05-2316:38rickmoynihanit uses st/instrument
to enable all the fdef
specs and run them through st/check
#2018-05-2516:18kvltHey all, I'm seeing something I didn't expect. Could anyone explain this?
(distinct (remove coll?(gen/sample (s/gen any?) 1000)))
=> (nil)
Why are there no numbers, strings, ... etc?#2018-05-2516:33Alex Miller (Clojure team)b/c any? doesnāt have a very good generator right now#2018-05-2516:34Alex Miller (Clojure team)there is a ticket for this#2018-05-2516:40kvltThanks Alex#2018-05-2517:32jplazaHi all! Iām getting an assert exception when using and
, or
or any combination of both inside (s/keys)
#2018-05-2517:33jplaza(s/def ::invoice :req-un [::sequence ::uuid ::properties (or ::live ::environment)])
is this a bug?#2018-05-2517:34jplazaThis is the exception I get:
Assert failed: all keys must be namespace-qualified keywords
#2018-05-2517:35nenadalm@jplaza were in the code are your s/keys
? I don't see that.#2018-05-2517:37jplazaSorry, a typo while simplifying my code. This is the correct snippet
(s/def ::invoice (s/keys :req-un [::sequence ::uuid ::properties (or ::live ::environment)]))
#2018-05-2517:38nenadalmI don't think you tried to run the code above...#2018-05-2517:42jplazaYou are right that simple example is working.#2018-05-2517:42jplazaBut my actual code isnāt. It works if I remove the and
and or
s though#2018-05-2517:52jplazaFound the error, I was stupidly trying to use or
in :opt-un
vector :face_palm:#2018-05-2610:31karol.adamiec@jplaza can you share ::uuid spec? i have issues with that one, as uuid? of āā is trueā¦.#2018-05-2619:59jumarNot sure how uuid?
can return true for empty string:
(uuid? "")
;;=> false
;; clojure.core/uuid
(defn uuid?
"Return true if x is a java.util.UUID"
{:added "1.9"}
[x] (instance? java.util.UUID x))
#2018-05-2622:21karol.adamiecin clojurescript š#2018-05-2615:37jplaza@karol.adamiec we are using compact uuids (https://github.com/tonsky/compact-uuids) so we are using a predicate#2018-05-2622:22karol.adamiecthanks#2018-05-2802:31alex-dixonIs there a way to access why a spec passed?#2018-05-2802:38gfredericksisn't that what conforming is?#2018-05-2803:02alex-dixonNot sure. The situation that prompted that question is a multi spec that matched on a particular :type
value of a map and spec associated with that value. I can tell that it passes and get the original map back, just curious if thereās a way to access explain information when it passes. May be interesting stuff that could be done with it but seems only to get generated upon failure#2018-05-2813:45Alex Miller (Clojure team)explain info is only being built up when using explain, so no?#2018-05-2901:36callumI'm finding myself having to duplicate spec definitions in order to override generators and control the number of generated values. e.g.
(s/def ::name string?)
(s/def ::a (s/coll-of ::name))
(s/def ::b (s/keys :req-un [::a]))
(s/def ::c (s/keys :req-un [::a
::b]))
;; only generate a single element for ::a
(gen/generate (s/gen ::c {::a #(s/gen (s/coll-of ::name :min-count 1 :max-count 1))}))
#2018-05-2901:36callumIs there a way to avoid this duplication?#2018-05-2903:47bbss(s/def ::select
(s/cat :plan-name #{'select}
:thing-to-select #{"SCV" "SupplyDepot"}
:at-least (s/cat :at-least-kw #{:at-least}
:at-least-number (s/int-in 1 50))
:and-do-form (s/? (s/cat :and-do-kw #{:and-do}
:additional-do ::select))))
(gen/sample (s/gen ::select) 1)
Why does this stackoverflow? I'd expect the recursive spec to hit a "0 arity branch" of the s/? before it runs out of stack#2018-05-2904:01bbssI notice https://clojuredocs.org/clojure.spec.alpha/*recursion-limit* doesn't mention s/? so maybe it's not intended to be used recursively.#2018-05-2905:49bbssEnded up accepting I can't generate it, but still validate which will be enough for now.
(s/def ::select
(s/cat :plan-name #{'select}
:thing-to-select #{"SCV" "SupplyDepot"}
:at-least (s/cat :at-least-kw #{:at-least}
:at-least-number (s/int-in 1 50))
:and-do-form (s/with-gen (s/? (s/cat :and-do-kw #{:and-do}
:additional-do (s/spec (s/and list?
::select))))
#(gen/return ()))))
#2018-05-2908:40cvic@bbss https://gist.github.com/bhauman/2dca87815dfd92b3ff596bdc1e56c964#file-compiler-options-schema-clj-L30#2018-05-2908:45bbss@victor.cleja you're saying a duplicate key is the issue?#2018-05-2918:03bhaumanjust released a spec library#2018-05-2918:03bhaumanhttps://twitter.com/bhauman/status/1001523240529936384#2018-05-3007:00raymcdermottIām trying to find a nicer way to report simple errors based on clojureās core specsā¦.#2018-05-3007:00raymcdermottfor example if a user types in (defn foo 1)
#2018-05-3007:00raymcdermottwe get a fail on the defn spec#2018-05-3007:01raymcdermotthow can I have the result as data rather than a string?#2018-05-3007:01raymcdermottis there a magic var I can set or some other compiler property?#2018-05-3012:36Alex Miller (Clojure team)In the case of a spec error, you should get an ExceptionInfo where ex-data has the explain data#2018-05-3012:37Alex Miller (Clojure team)Or you can hook s/*explain-out*
#2018-05-3012:40jumar@raymcdermott do you read and eval input from user?#2018-05-3012:41raymcdermottyes#2018-05-3012:42jumarThan this could work (although not sure if that's an optimal way to do it)
(try (eval (read-string "(defn foo 1)")) (catch Exception e (ex-data (.getCause e))))
#2018-05-3012:43raymcdermottIām reading the results of evaluation via the PREPL which gives back a string for the :val
key#2018-05-3012:44jumarI'm not familiar with prepl so don't know then#2018-05-3012:46raymcdermottIām trying to avoid having to run eval
on every input#2018-05-3012:47raymcdermottbefore I pass it to the REPL#2018-05-3012:47raymcdermottbut yes, thatās a legitimate option @jumar#2018-05-3012:50raymcdermott@alexmiller Iām using the PREPL from 1.10 and it would be nice (IMHO) if there was a property for spec fails#2018-05-3013:41Alex Miller (Clojure team)Iām a little confused by what youāre saying. In the case of an exception, I would expect prepl to be converting the Throwable to data with Throwable->map
which captures ex-data in a :data
attribute.#2018-05-3013:55Alex Miller (Clojure team)trying some things in the repl, I understand what youāre saying I think, although I donāt understand why#2018-05-3013:57Alex Miller (Clojure team)oh, Iām testing with io-prepl and itās happening due to the default valf of pr-str - is that what youāre using @raymcdermott?#2018-05-3013:59Alex Miller (Clojure team)if you use a valf of identity, then youāll get back throwables as data#2018-05-3014:07Alex Miller (Clojure team)Clojure 1.10.0-master-SNAPSHOT
user=> (require '[clojure.core.server :as ccs])
nil
user=> (ccs/io-prepl :valf identity)
(defn foo 1)
{:tag :ret,
:val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/arg-list ...",
:via [{:type clojure.lang.Compiler$CompilerException,
:message "clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform...",
:at [clojure.lang.Compiler checkSpecs "Compiler.java" 6891]}
{:type clojure.lang.ExceptionInfo,
:message "Call to clojure.core/defn did not conform ...",
#_here :data #:clojure.spec.alpha{:problems (
{:path [:args :bs :arity-1 :args],
:pred clojure.core/vector?,
:val 1,
:via ...
`#2018-05-3014:19raymcdermottthanks @alexmiller Iāll give it a shot later#2018-05-3021:01raymcdermott@alexmiller I could be doing it wrong but Iām using io-prepl as part of a socket-server like thisā¦#2018-05-3021:03raymcdermottI then have a web server to intermediate#2018-05-3021:04raymcdermottAnd this takes request from a REPL client in the browser#2018-05-3021:04raymcdermottwhich send over strings that are then read and sent to the remote prepl
#2018-05-3021:06raymcdermottRead failclojure.lang.LispReader$ReaderException: java.lang.RuntimeException: No reader function for tag object
1#2018-05-3021:08raymcdermottis this an abuse? Is there another option to read
that is more compatible?#2018-05-3021:13raymcdermottIāll keep fiddling but ideas welcome š#2018-05-3021:18raymcdermottRead fail: clojure.lang.EdnReader$ReaderException: java.lang.RuntimeException: No reader function for tag object#2018-05-3021:25Alex Miller (Clojure team)I havenāt put all these pieces together recently and donāt have time to look at it today. you might look at wrapping clj-server/remote-prepl for the client side?#2018-05-3021:26Alex Miller (Clojure team)and since stuff is mostly char stream based, you might just want the defaults in io-prepl but to read on the remote-prepl valf side, not sure#2018-05-3021:46raymcdermottthanks @alexmiller#2018-05-3111:21jumar@raymcdermott I've tried a limited version of your code and this seems to work (when tested with nc
on command line):
(defn configured-prepl
[]
(clj-server/io-prepl :valf identity))
(defn shared-prepl
[opts]
(let [socket-opts {:port 5555
:server-daemon false ; Keep the app running
:accept `configured-prepl}]
(let [server (clj-server/start-server (merge socket-opts opts))]
(println "listening on port" (.getLocalPort ^ServerSocket server)))))
Run server:
(clj-server/remote-prepl "localhost" 5555)
Later on command line:
nc localhost 5555
(+ 1 1)
{:tag :ret, :val 2, :ns "user", :ms 0, :form "(+ 1 1)"}
(defn foo 1)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException, ...
For client written in Clojure I guess you should try remote-prepl
as Alex suggested, but your problem seems to be related to the fact that read
is not able to recognize #object
in the response returned by remote repl.
The error looks is caused by sth. like this:
(pr-str (Object.))
#object[java.lang.Object 0x66a64b49 "
The #object
in response comes from spec failure (partial output, see the last line)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1
fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1]
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies]
predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body
(cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException,
:message "clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec:\nIn: [1]
val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1]
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list
:body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n #:clojure.spec.alpha{:problems
({:path [:args :bs :arity-1 :args], :pred clojure.core/vector?, :val 1, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body
:clojure.core.specs.alpha/arg-list :clojure.core.specs.alpha/arg-list], :in [1]}
{:path [:args :bs :arity-n :bodies], :pred (clojure.spec.alpha/cat :args :clojure.core.specs.alpha/arg-list
:body (clojure.spec.alpha/alt :prepost+body (clojure.spec.alpha/cat :prepost clojure.core/map?
:body (clojure.spec.alpha/+ clojure.core/any?)) :body (clojure.spec.alpha/* clojure.core/any?))), :val 1,
:via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/args+body], :in [1]}),
:spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x233319e7 \"
I'm not sure if this can be resolved somehow...#2018-05-3111:21jumar@raymcdermott I've tried a limited version of your code and this seems to work (when tested with nc
on command line):
(defn configured-prepl
[]
(clj-server/io-prepl :valf identity))
(defn shared-prepl
[opts]
(let [socket-opts {:port 5555
:server-daemon false ; Keep the app running
:accept `configured-prepl}]
(let [server (clj-server/start-server (merge socket-opts opts))]
(println "listening on port" (.getLocalPort ^ServerSocket server)))))
Run server:
(clj-server/remote-prepl "localhost" 5555)
Later on command line:
nc localhost 5555
(+ 1 1)
{:tag :ret, :val 2, :ns "user", :ms 0, :form "(+ 1 1)"}
(defn foo 1)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException, ...
For client written in Clojure I guess you should try remote-prepl
as Alex suggested, but your problem seems to be related to the fact that read
is not able to recognize #object
in the response returned by remote repl.
The error looks is caused by sth. like this:
(pr-str (Object.))
#object[java.lang.Object 0x66a64b49 "
The #object
in response comes from spec failure (partial output, see the last line)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1
fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1]
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies]
predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body
(cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException,
:message "clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec:\nIn: [1]
val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1]
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list
:body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n #:clojure.spec.alpha{:problems
({:path [:args :bs :arity-1 :args], :pred clojure.core/vector?, :val 1, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body
:clojure.core.specs.alpha/arg-list :clojure.core.specs.alpha/arg-list], :in [1]}
{:path [:args :bs :arity-n :bodies], :pred (clojure.spec.alpha/cat :args :clojure.core.specs.alpha/arg-list
:body (clojure.spec.alpha/alt :prepost+body (clojure.spec.alpha/cat :prepost clojure.core/map?
:body (clojure.spec.alpha/+ clojure.core/any?)) :body (clojure.spec.alpha/* clojure.core/any?))), :val 1,
:via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/args+body], :in [1]}),
:spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x233319e7 \"
I'm not sure if this can be resolved somehow...#2018-05-3111:23jumarThat last bit I tested in separate repl:
(require '[clojure.core.server :as ccs])
user=> (ccs/remote-prepl "localhost" 5555 *in* #(prn %))
(+ 1 1)
{:tag :ret, :val 2, :ns "user", :ms 0, :form "(+ 1 1)"}
(defn foo 1)
Exception in thread "clojure.core.server/remote-prepl" clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: No reader function for tag object
at clojure.lang.LispReader.read(LispReader.java:304)
...
#2018-05-3111:23raymcdermottyeah, he mentioned the valf option#2018-05-3111:24raymcdermottthanks for testing it out @jumar#2018-05-3111:24raymcdermottin theory with valf set to identity we should get a :data
property and we can then read that directly#2018-05-3111:25raymcdermottso Iām going to play with a remote-prepl as you suggest#2018-05-3111:26raymcdermotttbh Iām struggling on what to give it as a Reader and an out-fn ā¦. so Iām in š¤ mode at the moment#2018-05-3111:26raymcdermottif you have any ideas ā¦ I am in the market š#2018-05-3111:51jumar@raymcdermott Couldn't find anything better than leveraging *default-data-reader-fn*
somehow:
user=> (require '[clojure.core.server :as ccs])
nil
user=> (alter-var-root #'*default-data-reader-fn* (fn [_] (fn [tag value] value)))
#object[user$eval3$fn__138$fn__139 0xfd8294b "
#2018-05-3112:35raymcdermottI didnāt know about that var, so thanks ā¦.#2018-05-3112:35raymcdermottI really must look at all of those ear-muff things one day if Iām going to live down here š#2018-05-3113:00Alex Miller (Clojure team)This is interesting and probably ultimately needs a change in spec reporting. Rather than send the spec instance, it should be sending the spec name or form where you are seeing the object. For now, I think youāll need to intercept either on the server or client side and either remove or transform it yourself#2018-05-3122:29raymcdermottdo you consider this as a spec issue rather than a PREPL issue? Or a bit of both?#2018-05-3122:51Alex Miller (Clojure team)spec for the reporting#2018-05-3122:51Alex Miller (Clojure team)Maybe prepl although it gives you the hook to address it#2018-05-3113:01Alex Miller (Clojure team)I think hooking the default data reader to fall back to using tagged-literal is a pretty good dodge that covers many potential issues like this#2018-05-3113:31Andreas LiljeqvistHow would I alias a namespace that isn't required? Circular dependency is a problem if I require it#2018-05-3113:33Andreas LiljeqvistI want to keep a separate namespace/file for my spec but use another namespace#2018-05-3113:34Andreas LiljeqvistLike my-namespace
with functions and my-namespace.spec
with the spec#2018-05-3113:35Andreas LiljeqvistBut using :t/id
instead of having to write :my-namespace/id
#2018-05-3113:42guySo do you mean like.
Inside of my-namespace
you have some spec like
(s/def :t/id)
instead of (s/def ::id)
or (s/def :my-namespace/id)
#2018-05-3113:42guyAs far as i understood you have to require the namespace the spec is in, to use it in a different namespace. But i could be wrong#2018-05-3113:44guySo you might want to just move your spec somewhere else to get around the circular dependencies?
I ended up just giving certain specs fake namespaces to and not using ::id
. Instead something like :user/id
#2018-05-3113:45Andreas Liljeqvistin my-namespace.spec
I have a (s/def ::id nat-int?)
. Resulting in :my-namespace.spec/id
#2018-05-3113:46Andreas LiljeqvistI would like the spec to remain in my-namespace.spec
but be registered to :my-namespace/id
#2018-05-3113:46guytry this instead (s/def :my-namespace/id nat-int?)
#2018-05-3113:47Andreas LiljeqvistThat will work, but it feels wrong when I have a lot of defs#2018-05-3113:47guyYou will still need to require my-namespace.spec
in a different ns if you want to use the spec contained#2018-05-3113:47Andreas Liljeqvistyeah#2018-05-3113:47guybut then you can refer to it like :my-namespace/id
instead#2018-05-3113:48guyI think my counter argument to "but it feels wrong when I have a lot of defs" would be its a bit redundant to have .spec in the spec name#2018-05-3113:49guy:my-namespace.spec/id
vs :my-namespace/id
#2018-05-3113:49guylike you know they are both specs#2018-05-3113:49guyso is .spec really needed?#2018-05-3113:49guy(this is just my opinion though, so someone else feel free to correct me etc)#2018-05-3113:49Andreas LiljeqvistOnly reason for the .spec
is that I want to have a separate namespace for the specs vs the functions#2018-05-3113:50guyYeah i had user and user.spec too, because it felt nicer to separate them#2018-05-3113:50guyyeah exactly the same thought process#2018-05-3113:50guyBut i ended up moving away from ::
notation to something/id
instead#2018-05-3113:51Andreas LiljeqvistA bit worried about keyword collisions when doing it that way, but everything you say makes sense#2018-05-3113:52guyI think thats more a case of you having a consistent naming convention really#2018-05-3113:52guy>keyword collisions#2018-05-3113:52guyi mean#2018-05-3113:52Andreas LiljeqvistI would have prefered to do something like (alias 't 'my-namespace
)#2018-05-3113:53guyThe troubles i was having were similarly named fields#2018-05-3113:53guyuser/name vs something else /name#2018-05-3113:53guyso if u had a (s/def ::name)
that was a string and one that was something else#2018-05-3113:53guyit would cause headaches#2018-05-3113:53guyif that makes sense?#2018-05-3113:54Andreas LiljeqvistYeah, but I might be using a third party library that also implement :user/name
or something#2018-05-3113:54Andreas LiljeqvistBut that is a problem for future me š Thanks a lot for the help#2018-05-3113:54guysure but then you can prefix to to Andreas.user/name or Projectname.user/name etc š#2018-05-3113:54guynps! Good luck!#2018-05-3113:54guyHopefully i was helpful haha#2018-05-3114:00OlicalAfternoon. I was wondering if anyone else uses sets with their (s/keys...)
calls?#2018-05-3114:01OlicalIn the docs it mentions vectors, but wouldn't a set of keys make more sense, like: (s/keys :req-un #{::a ::b})
?#2018-05-3114:01Alex Miller (Clojure team)conceptually, yes#2018-05-3114:01OlicalI ask because I've seen a weird issue where UUIDs are not being coerced by spec-tools if I use a set š
#2018-05-3114:05OlicalYou can reproduce it with (s/def ::a (st/spec uuid?))
(s/def ::b (s/keys :req-un #{::a}))
(st/decode ::b {:a "4c6e852b-e8a3-4686-8916-4e345be53731"} st/json-transformer)
;; => {:a "4c6e852b-e8a3-4686-8916-4e345be53731"}
where st
is spec-tools.core
#2018-05-3114:06OlicalIf you replace the set with a vector it works fine and coerces the UUID string into an actual UUID. I was thinking "maybe this is an issue with spec-tools", but if you're supposed to use a vector anyway this is technically okay, just a confusing gotcha.#2018-05-3114:23ikitommiyes, the s/keys
is parsed here: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/parse.cljc#L112-L117. Hasn't been tested with sets. PR welcome ;)#2018-05-3114:27OlicalSo @U050V1N74 and I have spotted it š#2018-05-3114:27OlicalIt's because clojure.core/flatten on a set empties the set!#2018-05-3114:28OlicalTIL for the both of us.#2018-05-3114:28Olical(flatten #{1 2 3})
gives you ()
#2018-05-3114:29Olical"because sets aren't sequential, they're only seqable"#2018-05-3114:30OlicalAny idea why the call to flatten is required here though? https://github.com/metosin/spec-tools/blob/93ca24167131a2e312fed1da79beb25632090fda/src/spec_tools/impl.cljc#L53#2018-05-3114:30Olical(probably this one actually https://github.com/metosin/spec-tools/blob/93ca24167131a2e312fed1da79beb25632090fda/src/spec_tools/impl.cljc#L64)#2018-05-3114:59ikitommiI think it's for supporting the nested`or`s? See the tests
https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/parse_test.cljc#2018-05-3114:59ikitommi.. and and
s#2018-05-3115:02jarohenTIL or
s and and
s in s/keys
are a thing, too#2018-05-3114:13Andreas LiljeqvistSeems like https://dev.clojure.org/jira/browse/CLJ-2123 is related to my question about aliases#2018-06-0113:15bostonaholicIs this the proper way to spec a multiple arity function? with :args (s/or ...)
(defn foo
([a] ...)
([a b]) ...)
(s/fdef foo
:args (s/or :arity-one (s/cat :a int?)
:arity-two (s/cat :a int? :b int?))
:ret int?)
#2018-06-0113:25Alex Miller (Clojure team)Iād use s/alt there instead of s/or as itās a regex op, but thatās one option#2018-06-0113:25Alex Miller (Clojure team)Another is to push the optionality into the cat with s/?#2018-06-0113:26bostonaholics/alt
is better, thanks#2018-06-0113:29bostonaholicwhen I run my suite with lein test
and I get the generator seed āUsing generator seed: -1186346283ā, how do I pass that seed into lein test
again?#2018-06-0113:39Alex Miller (Clojure team)I donāt know of any way to do that from that level#2018-06-0113:39Alex Miller (Clojure team)stest/check takes an option map where you can pass in a seed#2018-06-0113:40Alex Miller (Clojure team)you might be able to do it through some test.check dynvar or something, not sure#2018-06-0116:16kapilIs there an idiomatic way to associate a doc-string with a spec?#2018-06-0116:21kapilI am trying to generate documentation from spec for non-clojure developers to read.
Something like this
(s/def ::id string? "Id of resource X. Ex. resource_1")
(s/def ::num integer? "Number of results for pagination")
(s/def ::result (s/keys :req-un
[::id ::num]))
(gen-doc ::result) =>
{:id "resource_1" ;; "Id of resource X. Ex. resource_1"
:num 10 ;;"Number of results for pagination"
}
#2018-06-0116:38Alex Miller (Clojure team)no, not yet but we intend to add that#2018-06-0118:37kapil@U064X3EF3 Is there any workaround I can do for now?
I tried using with-meta but it doesn't with no success since keywords are not Objects.
I could however maintain a registry of documentation.#2018-06-0118:43Alex Miller (Clojure team)there are some external libs that are providing this, but itās hard to recommend those as what they are doing is likely to break#2018-06-0118:43Alex Miller (Clojure team)you could build an external map of spec keyword to doc string#2018-06-0118:45kapil(defonce doc-registry (atom {}))
(defmacro def-w-doc
[k spec-form doc]
`(s/def ~'k ~spec-form)
(swap! doc-registry assoc k doc))
(def-w-doc ::id string? "Id of resource X. Ex. resource_1")
#2018-06-0118:47kapilThis is what I am using now. This is least intrusive change I could think. Later on when clojure.spec adds support for documenting specs I can just modify def-w-doc
.#2018-06-0118:59Alex Miller (Clojure team)sounds good#2018-06-0116:16noisesmithfor the seed, one workflow would be to keep your randomized check, and also do another with a specific seed that creates a known rare corner case#2018-06-0116:18bostonaholicSEED=<seed> lein test
is how to do it#2018-06-0116:18noisesmithoh that works? interesting#2018-06-0116:19noisesmithI like to have regression tests for things that have been known to fail in the past though#2018-06-0116:19noisesmith(I guess it's better to do that by hard coding the data that had been generated in an example based test)#2018-06-0116:25bostonaholicyup, setting the SEED
as an env var works#2018-06-0213:47gfredericksFWIW, I have never heard of this and have no idea what's going on#2018-06-0213:47gfredericksmy suspicion is it's not a test.check seed, it's something else#2018-06-0118:21rapskalianI'm curious what others' tastes are regarding namespaced keys in spec definitions? As an example, let's say we're modeling GPS coordinates. Personally, I've been doing something like this:
(s/def :coordinate/latitude double?)
(s/def :coordinate/longitude double?)
(s/def ::coordinate (s/keys :req [:coordinate/latitude
:coordinate/longitude]))
In which lat and long
are both "nested" under the :coordinate
namespace, and then the coordinate map itself is defined inside of some kind of :my-project.specs
top-level namespace (using the ::coordinate
sugar).
Does this make sense, or are there other conventions that have been established that would be clearer?#2018-06-0118:42seancorfield@cjsauer That's pretty close to my approach. Much depends on how unique you need the names to be and how your code might be used.#2018-06-0118:43seancorfieldIn a library, you'd really want all the names to be globally unique. In an application, you can use less unique names.#2018-06-0118:44Alex Miller (Clojure team)they should be sufficiently unique :)#2018-06-0120:58rapskalian@seancorfield good to hear. This is for an application, so I'll trade some uniqueness for readability š#2018-06-0219:19cvicIf I add
[tentacles.core :as c]
lein repl throws
Exception in thread "main" clojure.lang.ExceptionInfo: Call to clojure.core/defn- did not conform to spec:
In: [0] val: clj-tuple/conj-tuple fails spec: :clojure.core.specs.alpha/defn-args at: [:args :name] predicate: simple-symbol?
#:clojure.spec.alpha{:problems [{:path [:args :name], :pred clojure.core/simple-symbol?, :val clj-tuple/conj-tuple, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/defn-args], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x162e29a1 "
No idea why...#2018-06-0219:29cvicOr... some dependency that has a malformed defn- form.#2018-06-0219:29cvicTime to do some digging#2018-06-0219:35cvicAh, nvm. I'll just use
[irresponsible/tentacles "0.6.2"]
#2018-06-0222:22Alex Miller (Clojure team)The line in error is clj_tuple.clj line 556#2018-06-0222:22Alex Miller (Clojure team)Wherever that is#2018-06-0318:16cvicSometimes upgrading a lib is the shortest way of solving it#2018-06-0318:20cvichttps://github.com/ztellman/clj-tuple/blob/master/src/clj_tuple.clj
Yep. Also old.#2018-06-0504:33Vincent CantinHi. I plan to try to use spec for auto-completion. Has it been done before?#2018-06-0504:41Vincent CantinAre the specs easy to parse /navigate programmatically?#2018-06-0506:48ikitommi@U8MJBRSR5 lot of parsers in 3rd party libs, using s/form
. As a sample, one parser and a spec visitor in spec-tools (https://github.com/metosin/spec-tools).#2018-06-0506:48ikitommiAlso a issue to have a spec walker in the core: https://dev.clojure.org/jira/browse/CLJ-2251#2018-06-0506:49Vincent Cantinthank you#2018-06-0505:20Vincent Cantinā¦ I think that s/explain-data
does a lot already.#2018-06-0510:16Vincent CantinIs there some kind of s/seq-of
operator, or a way to build it?
The idea is to act as a bridge to spec the chars in a string from the string itself: (s/conform (s/seq-of char?) "abc")
#2018-06-0510:35hkjelsI donāt have a running repl atm, but I think you can use (coll-of char? :kind string?)
#2018-06-0510:41Vincent CantinIt works, thanks. Need to add :into []
to keep the chars order at conform time.#2018-06-0511:02Vincent CantinI still have trouble afterward ā¦ I would like to run some regex spec (e.g. s/cat
) on the content of a string but without having to apply s/conform
on the seq
of that string. I donāt know if it is possible.#2018-06-0514:22bbrinckFWIW, the general advice Iāve seen is that spec is not a great fit for parsing strings. Certainly people have done it by turning the string into a sequence with seq
, but often people run into further issues. More detail in comments here: https://www.reddit.com/r/Clojure/comments/7vwpu4/parsing_with_clojurespec/#2018-06-0514:25bbrinckYMMV of course#2018-06-0602:43Vincent CantinTL;DR : https://clojuredocs.org/clojure.spec.alpha/conformer ā¦with a nice example under the documentation.#2018-06-0602:47bbrinckA good discussion of pitfalls with conformers and how to work around them: https://groups.google.com/d/msg/clojure/Tdb3ksDeVnU/LAdniiZNAgAJ#2018-06-0602:48bbrinckāwe recommend that you not use conformers for coercion. Conformers were added primarily as a tool for building custom composite spec typesā#2018-06-0603:00bbrinckI know people have had success with using spec for parsing strings, just wanted to make you aware of possible issues down the road š#2018-06-0603:00bbrinckif you run into them, it may be worth looking at something like https://github.com/cgrand/regex#2018-06-0603:04Vincent CantinAh, Christophe again :-)#2018-06-0600:08noisesmithis there a way to get the individual spec keywords mapped to by an s/or call?#2018-06-0600:11noisesmithlooks like describe gets me most of the way if there's nothing better#2018-06-0611:37Alex Miller (Clojure team)s/form will be more complete than s/describe#2018-06-0621:24arohnerIs there a way to get conformed output to be different for non-regex specs? I have a bunch of strings I need to parse, and Iād like to validate a parsed string against the spec, and returned the parsed values, if they conform#2018-06-0621:44Alex Miller (Clojure team)in short, no#2018-06-0621:46Alex Miller (Clojure team)you would need a custom spec impl for that currently and I wouldnāt recommend that as the guts are likely to change#2018-06-0621:46Alex Miller (Clojure team)in the future, maybe#2018-06-0622:54dpsutton(s/valid? :fhir-type/PaymentReconciliation
examples/pay-rec)
false
com.breezeehr.specs.fhir.PaymentReconciliation>
(s/explain :fhir-type/PaymentReconciliation
examples/pay-rec)
Success!
nil
anyone familiar with something like this?#2018-06-0623:48seancorfield@dpsutton I have seen some occasional bugs reported against spec that seem to have that behavior -- I guess you'll need to dig into that :fhir-type/PaymentReconciliation
spec and see what it's doing?#2018-06-0623:49dpsuttonIt kinda sorted itself out but no idea what the transient state was. Just wondering if it was known bug or if we were way off the beaten trail#2018-06-0623:52seancorfieldAh, a Heisenspec? š#2018-06-0623:52dpsuttonI'm guessing it's more simple and just something silly we are doing. But you never know#2018-06-0700:25Alex Miller (Clojure team)Generally anything like this should be considered a bug (if itās reproducible). One thing that could possibly lead you down this path at the repl is changing a spec but not redefining a spec that used it. Specs are compiled at definition time so you can get a mixture of things that way.#2018-06-0702:08dpsuttonOh. I made a poor man's version of spec using defs and thought you got around that with the central database of specs and always getting a "fresh" copy. I'll make sure to have up to date definitions. Thanks#2018-06-0702:49Vincent CantinWhat is the most idiomatic way to apply s/cat
, knowing that it is a macro?#2018-06-0712:33Alex Miller (Clojure team)A future way to do this will be to write a spec for s/cat, write conform data, and s/unform back to a spec.#2018-06-0712:34Alex Miller (Clojure team)See CLJ-2112 for some early work on that#2018-06-0702:50noisesmithto apply a macro, you need another macro (and to do it at runtime to something that is not a literal in your source file or created when loading the file you need to be compiling new code, eg. eval)#2018-06-0703:06Vincent CantinI will use a macro then.
Do you know why it was made as a macro? (most importantly, why only as a macro)#2018-06-0703:07Vincent CantinThat would have been good to have the macro and the functions and let the users choose between them.#2018-06-1419:56johanatanCompletely agree. The prevalence of macros in spec is my number one complaint with it.#2018-06-0703:07Vincent Cantins/cat
and sm/cat
#2018-06-0703:17Vincent Cantināmacro contagionā ā¦ it reminds me of the āconst hellā in c++#2018-06-0703:42seancorfield@vincent.cantin spec was designed for humans to write, so macros are fine. There is work in the pipeline to open up programmatic access to spec, which will mean functions.#2018-06-1419:54johanatanMacros are not fine. They prevent higher-order code and partial application (to name a couple of issues). And of course they result in 'macro proliferation' as well.#2018-06-0703:43seancorfieldSo, for now, you might be going through a lot of (macro) pain for nothing, depending on how quickly you need the thing you're trying to build...#2018-06-0703:45Vincent CantinThe task I gave myself is probably making me a complainer by design: I am trying to do the most I can do with Spec.#2018-06-0703:46Vincent CantinYou are right, it is not related to the normal use case.#2018-06-0703:46seancorfieldIncluding lots of things it wasn't designed for, I'm sure š#2018-06-0703:47seancorfieldIt's interesting... over time, the Clojure/core folks have produced a number of things that in the rush of new excitement around a feature, lots of people almost immediately try to do stuff with each new feature that's outside the design goals š#2018-06-0705:40Vincent Cantin@U04V70XH6 I was doing that: https://github.com/green-coder/TIS-100-Spec
My problem with the cat being a macro was for the implementation of my token spec.#2018-06-0705:42Vincent CantinI wanted to see if it was possible to do it that way. Well ā¦ it is possible, but not convenient.#2018-06-0705:44seancorfieldAnd how is performance? š#2018-06-0705:44Vincent Cantineh eh .. I donāt know, it does not matter here.#2018-06-0705:46Vincent CantinI wished that cat
could accept nil
instead of a token for specs which we do not need to know about when we use conform
.#2018-06-0705:46Vincent Cantinin regex terms, it would mean a ānon-capturingā expression.#2018-06-0705:52seancorfieldI'm wondering how many times folks have to say "spec is not a parser"... ? :rolling_on_the_floor_laughing:#2018-06-0705:53Vincent Cantinš that means that the community would like to use it in that way too.#2018-06-0705:54seancorfieldIf you get the grammar right and each spec generates then you have the possibility of generating random but syntactically valid TIS-100 programs... that you could then run and see what they do! š#2018-06-0705:54Vincent CantinYou start to see some of my goals#2018-06-0712:05dottedmagHowever spec is definitely sold as a parser, maybe unintentionally. "Check your inputs and, oh here is the parse tree".#2018-06-0712:05dottedmagIt's not a best string parser out there, that's for sure.#2018-06-0703:48seancorfieldI think it says a lot about how the features have been such great building blocks over the years, as well as how laser-focused many of the features have been.#2018-06-0703:49seancorfieldI don't know if you've done much with clj
and deps.edn
yet? That seems to be attracting the same "let's see what crazy things we can do with this" experimentation.#2018-06-1419:57johanatanI don't consider wanting to apply or partially apply a macro to be a "crazy experimentation". It's very much just a thing that advanced [meta-]programmers want to do to keep their code concise.#2018-06-0716:58favilaI can't find a ticket for this, but I can't believe it wasn't noticed before#2018-06-0716:59favilas/every or s/coll-of will interpret :kind
as a predicate only (not spec) during s/valid?
s/conform
etc#2018-06-0716:59favilabut during explain, it does interpret as a spec#2018-06-0717:00favilaI think it is because the macros generate a cpred unconditionally interpreting kind as a pred#2018-06-0717:00favilaexample of how this can go wrong:#2018-06-0717:02favila(s/def ::vector vector?)
=> :user/vector
(s/valid? (s/coll-of any? :kind ::vector) [])
=> false
(s/explain (s/coll-of any? :kind ::vector) [])
Success!
=> nil
#2018-06-0717:03favilaHere, this is even clearer:#2018-06-0717:03favila(s/explain (s/coll-of any? :kind (s/spec vector?)) [])
Success!
=> nil
(s/valid? (s/coll-of any? :kind (s/spec vector?)) [])
ClassCastException clojure.spec.alpha$spec_impl$reify__1987 cannot be cast to clojure.lang.IFn user/eval3301/fn--3303 (form-init7886713966146839296.clj:1)
#2018-06-0717:03dpsuttonin checking against the spec, it will call (::vector [])
which returns nil and fails the spec. In explaining, it is aware that [] satisfies the ::vector
spec#2018-06-0717:04favila(macroexpand '(s/coll-of any? :kind (s/spec vector?)))
=>
(clojure.spec.alpha/every-impl
(quote any?)
any?
{:clojure.spec.alpha/describe (quote
(clojure.spec.alpha/coll-of
clojure.core/any?
:kind
(clojure.spec.alpha/spec clojure.core/vector?))),
:clojure.spec.alpha/conform-all true,
:clojure.spec.alpha/cpred (fn*
[G__3317]
(clojure.core/and ((s/spec vector?) G__3317))),
:kind (s/spec vector?),
:clojure.spec.alpha/kind-form (quote
(clojure.spec.alpha/spec clojure.core/vector?))}
nil)
#2018-06-0717:04favilaNote that :cpred is generated but nonsense#2018-06-0717:05faviladocs say#2018-06-0717:05favila> :kind - a pred/spec that the collection type must satisfy, e.g. vector?
(default nil) Note that if :kind is specified and :into is
not, this pred must generate in order for every to generate.#2018-06-0717:06favilanote "pred/spec"#2018-06-0717:12favilaok, found the ticket, nm#2018-06-0717:12favilahttps://dev.clojure.org/jira/browse/CLJ-2111#2018-06-0717:51bmaddyI'm trying to write a spec for a function that takes in another function. I have the spec for the function passed in, but am getting "Couldn't satisfy such-that predicate after 100 tries."
. Is that because I need to write a generator for functions that are arguments?#2018-06-0718:15noisesmithsuch-that acts as a filter, it's saying it wasn't finding anything that the filter accepted#2018-06-0718:27Alex Miller (Clojure team)how is it specāed?#2018-06-0720:03bmaddy(sorry, was at lunch)
Here's how I have it spec'd:
(s/def ::ruleset-id keyword?)
(s/def ::rule-name keyword?)
(s/fdef ::fetching-fn
:args ::ruleset-id
:ret (s/coll-of ::rule-name))
(s/fdef rules-for-rulesets
:args (s/cat :fetching-fn ::fetching-fn
:ruleset-ids (s/coll-of ::ruleset-id))
:ret (s/map-of (s/and set? (s/coll-of ::ruleset-id))
(s/and set? (s/coll-of ::rule-name)))
;; Ensure each rule only appears in the values only once so we
;; don't run them multiple times
:fn #(->> % :ret vals (reduce concat) frequencies vals (every? (partial = 1))))
#2018-06-0720:33taylorit might be the s/and
s where the first predicate is set?
#2018-06-0720:34taylorcould you try changing that (s/and set? (s/coll-of ::the-spec))
to (s/coll-of ::the-spec :kind set?)
@bmaddy#2018-06-0720:35taylor(because s/and
makes a generator from whatever the first predicate is, and set?
is unlikely to produce values that will satisfy ::ruleset-id
)#2018-06-0720:39tayloroh, another potential issue: ::fetching-fn
's :args
spec should probably be (s/cat :whatever ::ruleset-id)
instead of just ::ruleset-id
?#2018-06-0720:39bmaddytrying it...#2018-06-0720:43bmaddyOh geez, I think it's the :args (s/cat ...
thing. picard-facepalm#2018-06-0720:43tayloryeah I just realized the s/and
issue probably has nothing to do with the such-that
issue#2018-06-0720:44bmaddy((gen/generate (s/gen ::fetching-fn)) :foo)
[:K5.SY.if_!_._F6U/_
:p1Je.*-/*
:LbrF_/ZP]
That's so cool. š#2018-06-0720:45tayloryeah FYI the fn generators (I think) just generate functions that'll return values generated from their :ret
spec#2018-06-0720:46bmaddyYeah, that's what I would expect. It's really cool that spec does that though. š#2018-06-0720:49bmaddyThanks for the help @taylor (and others who looked at it!)#2018-06-0721:49hlshipI'm having a bit of trouble getting just the right bit of output for one of my specs. This is for expound 0.7.0.
Context: https://github.com/walmartlabs/lacinia/blob/a09684af4f2cabf23eb3d315bba0adad66787b57/src/com/walmartlabs/lacinia/schema.clj#L275
https://github.com/walmartlabs/lacinia/blob/a09684af4f2cabf23eb3d315bba0adad66787b57/src/com/walmartlabs/lacinia/expound.clj#2018-06-0721:50hlshipI've made slight changes:
(s/def ::type (s/or :base-type ::type-name
:wrapped-type ::wrapped-type))
(s/def ::wrapped-type (s/cat :modifier ::wrapped-type-modifier
:type ::type))
(s/def ::wrapped-type-modifier #{'list 'non-null})
and
(defmsg ::schema/wrapped-type-modifier "a wrapped type: '(list type) or '(non-null type)")
#2018-06-0721:50hlshipBut I get:
(binding [s/*explain-out* expound/printer]
(s/explain ::schema/field {:type '(something String)}))
-- Spec failed --------------------
{:type (something ...)}
^^^^^^^^^
should be one of: (quote list), (quote non-null)
-- Relevant specs -------
:com.walmartlabs.lacinia.schema/wrapped-type-modifier:
#{'non-null 'list}
:com.walmartlabs.lacinia.schema/wrapped-type:
(clojure.spec.alpha/cat
:modifier
:com.walmartlabs.lacinia.schema/wrapped-type-modifier
:type
:com.walmartlabs.lacinia.schema/type)
:com.walmartlabs.lacinia.schema/type:
(clojure.spec.alpha/or
:base-type
:com.walmartlabs.lacinia.schema/type-name
:wrapped-type
:com.walmartlabs.lacinia.schema/wrapped-type)
:com.walmartlabs.lacinia.schema/field:
(clojure.spec.alpha/keys
:req-un
[:com.walmartlabs.lacinia.schema/type]
:opt-un
[:com.walmartlabs.lacinia.schema/description
:com.walmartlabs.lacinia.schema/resolve
:com.walmartlabs.lacinia.schema/args
:com.walmartlabs.lacinia.schema/deprecated])
-------------------------
Detected 1 error
=> nil
#2018-06-0721:51hlshipI'd expect it to say:
should be a wrapped type: '(list type) or '(non-null type)
#2018-06-0721:58bbrinck@hlship Ah, yes, Iāve run into this as well. The problem is that defmsg
is narrowly applied to predicates, not any type of spec.#2018-06-0721:59hlshipAssociates the spec named `k` with `error-message`.
doesn't make that clear. Hm.#2018-06-0721:59bbrinckAgreed, and I donāt think itās a good design#2018-06-0721:59bbrinckItās a reflection of the original problem defmsg
solved: trying to provide more āhuman-friendlyā messages instead of string?
int?
#2018-06-0722:00bbrinckbut thatās an arbitrary restriction, and I think it should be generalized to any spec, so you can add messages across the board#2018-06-0722:00hlshipI'm trying to think of a work-around.#2018-06-0722:01bbrinckI havenāt tried it, but what happens if you replace #{'list 'non-null}
with (fn [x] (contains? #{'list 'non-null} x)
?#2018-06-0722:02bbrinck(I realize itās a hack š )#2018-06-0722:02hlshipBingo#2018-06-0722:03bbrinckIn any case, https://github.com/bhb/expound/issues/101#2018-06-0722:03hlshipPerhaps you could tweak the logic to treat a set as a predicate#2018-06-0722:03bbrinckEven more generally, I think I should make sure to respect a registered message regardless of the spec type#2018-06-0722:06bbrinckfrankly, I wasnāt sure anyone was really using Expoundās capability to register messages, so I wasnāt sure if it was really important. Glad to know you are using it š#2018-06-0722:07bbrinckIāve recently gone down a rabbit hole of trying to generate specs to test Expound but itās taken up a lot more time than expected. Iām going to pause that effort and switch over to bug fixing for a bit#2018-06-0722:11hlshipI'm very glad you implemented those messages, I think it can make a major difference. In general, I've gotten the rest of the team addicted to Expound ... we use a fair amount of clojure.spec, but previously, spec errors were treated as a boolean ā they were so hard to parse, we just looked at code changes to figure out how to make them go away. Now we have a proper guide to exactly what's wrong.#2018-06-0722:42bbrinckVery glad to hear it!#2018-06-0722:42bbrinck@hlship Iād be interested in your feedback on another potential feature: message-fns https://github.com/bhb/expound/pull/96#2018-06-0722:44bbrinckIām on the fence as to whether this would be useful enough to warrant inclusion#2018-06-0722:45hlshipWell, as a library developer, I want expound to be an optional dependency, so I don't think I could use this.#2018-06-0722:48bbrinckOh sorry, I was unclear in my description on this issue: it works like messages today in the sense that there is no effect if someone isnāt using expound.#2018-06-0722:48bbrinckThe only difference is that instead of registering a static string, you can register a function which will be called to generate the string#2018-06-0722:49bbrinckThis would let you, say, augment the existing expound error message, or use the problem to craft a custom message. But perhaps in your use case, the static strings are sufficient.#2018-06-0817:44martinklepschIs there a way to find out from what namespace/file a spec has been defined?#2018-06-0819:54seancorfieldI'd be very interested in hearing an answer to this too!#2018-06-0817:58bmaddyHere's a spec puzzle for someone. I'm trying to get a spec that defines something like this:
([-7806 "rBQUP"] :_T-7/G0X)
I thought I would need to nest calls to s/cat
to do it, but those seem to flatten or something:
> (gen/generate (s/gen (s/cat :int int? :str string?)))
(-61568 "66420tRep6jHRW6q07x647hV6qE6q")
> (gen/generate (s/gen (s/cat :pair (s/cat :int int? :str string?) :keyword keyword?)))
(-7806 "rBQUP" :_T-7/G0X)
How do I get a collection of two items, where the first is a collection of one int and one string, and the second is a keyword?#2018-06-0818:19taylorlike hiredman said below, this should work: (gen/generate (s/gen (s/cat :pair (s/spec (s/cat :int int? :str string?)) :keyword keyword?)))
just to prevent the "flattening" of the nested regex specs#2018-06-1208:14djtangolate to the party, s/tuple
isn't a regex spec so you can nest it as you like:
(s/valid? (s/tuple (s/tuple int? int?) keyword?) [[1 2] :a])
#2018-06-0818:09hiredmanwrap (s/spec ...) around the inner cat I think#2018-06-0818:14hiredmannested regex specs sort of collapse into a single regex spec (nested cats become subsequences within the outer sequence) so you have to insert something to tell spec where to stop collapsing them, and I think it is wrapping with s/spec, but it has been a while#2018-06-0819:37bmaddyThat worked! Thanks @hiredman and @taylor! š#2018-06-1207:36kurt-o-sysI'm using cljs-spec but I have issues with #object
s...:
Call to #'... did not conform to spec:
In: [0 :data :from] val: #object[b 2018-06-11T00:00:00.000+02:00] fails spec: :week.data/from at: [:args :arg-0 :data :from] predicate: vector?
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha40486]
:cljs.spec.alpha/value ({:data {:from #object[b 2018-06-11T00:00:00.000+02:00]}})
:cljs.spec.alpha/args ({:data {:from #object[b 2018-06-11T00:00:00.000+02:00]}})
:cljs.spec.alpha/failure :instrument
so, what I try to spec is #object[b 2018-06-11T00:00:00.000+02:00]
.#2018-06-1207:37kurt-o-syshow to spec that from
-field?#2018-06-1208:18djtangohow robust do you want the spec? Does that date object contain a "type" field? If you want to check if the from
field only is valid, you could pass it back into the date library to parse the string to check it's a valid time stamp?#2018-06-1208:24kurt-o-systhere is no type-field (apparently) - I'm using a js-library for datetime handling (need timezones, and there are not many options).
so, checking if the from field is a valid datetime would do...#2018-06-1208:24djtangowhat's consuming the date object further downstream?#2018-06-1208:25djtangoIf it's internal to your project, Is it good enough to spec it to be valid only for the code you know is going to use it?#2018-06-1208:26kurt-o-sysother functions that do operations on it, like .plus
etc...#2018-06-1208:26kurt-o-sysyeah, that's good enough...#2018-06-1208:27kurt-o-sysfor dev, I just want to be sure I get an kind of datetime object, on which I can perform .plus
and .minus
operations (or something like that).#2018-06-1208:27kurt-o-sysNow, I'm using any?
, but that too generic š.#2018-06-1208:29djtangoah, fair. You could store your own representation of the date object as a map, then use the library only for doing manipulations (then translate it back to your date representation)#2018-06-1208:31kurt-o-sysright... could do. that will be quite a lot of translations - not sure about performance impact here. But I may just try to do a .plus
-operation on it and check if it works, or something similar. That may do as well... will try. Thx!#2018-06-1208:35djtangogood luck#2018-06-1208:57kurt-o-sysworks fine, like it:
(s/def :luxon/plus #(try (.plus % #js {:days 1}) true (catch :default e false)))
(s/def :luxon/setZone #(try (.setZone % "Europe/Hamburg") true (catch :default e false)))
(s/def :luxon/startOf #(try (.startOf % "day") true (catch :default e false)))
#2018-06-1208:58kurt-o-sys(just testing if one can perform some operations, which should give a good enough indication that it's working)#2018-06-1208:24roklenarcicis there a way to feed existing explain
data to expound
?#2018-06-1211:57bbrinckYou can call the expound printer directly:
(let [d (s/explain-data (s/coll-of int?) [1 2 ""])]
(expound/printer d))
or
(let [d (s/explain-data (s/coll-of int?) [1 2 ""])
printer (expound/custom-printer {:show-valid-values? true})]
(printer d))
#2018-06-1307:23roklenarcicthx#2018-06-1213:08mpenetcross posting here since it got no answer in #clojure#2018-06-1213:16Alex Miller (Clojure team)If you use s/def with a resolvable symbol, the symbol itself is the spec registry key (this is his function specs are stored). So, not a bug, and matches the docs afaict#2018-06-1213:21mpenetnot sure I follow, do you have an example?#2018-06-1213:30Alex Miller (Clojure team)maybe I misunderstand what you are asking about, itās not clear to me what you think the bug is#2018-06-1213:31Alex Miller (Clojure team)the purpose of resolvable symbol k
in that docstring is to support registering function specs named by their resolvable symbol. the example given is doing something other than that, which is not what it is for#2018-06-1213:33mpenetok:#2018-06-1213:33mpenet(def x string?)
(s/valid? x "sdf")#2018-06-1213:33mpenetsomething like this then#2018-06-1213:33mpenetyeah that makes more sense#2018-06-1213:33Alex Miller (Clojure team)no, not that#2018-06-1213:34mpenetI still don't get it then š#2018-06-1213:34mpenetsince we're talking about the first arg to s/def it's a bit odd#2018-06-1213:34Alex Miller (Clojure team)x here should be a symbol referring to a var that is a function, to declare a function spec for x#2018-06-1213:36Alex Miller (Clojure team)(s/fdef clojure.core/symbol? :args (s/cat :x any?) :ret any?)
=>
(s/def clojure.core/symbol? (s/fspec :args (s/cat :x any?) :ret any?))
#2018-06-1213:37Alex Miller (Clojure team)the first is a shorthand for the second#2018-06-1213:37mpenetoh ok#2018-06-1213:37Alex Miller (Clojure team)like, literally s/fdef is a macro that does that#2018-06-1213:38Alex Miller (Clojure team)the registry has keys that are either keywords (pointing to data specs) or symbols (for function specs on the function on that symbol)#2018-06-1216:19kvltI have written a couple of fdef
specs for my functions. What is the best way to test that in a test file? I'm looking at defspec
and prop/for-all
but the path forward doesn't seem obvious#2018-06-1216:21taylorYou can instrument
the function to assert arguments conform to the :args
spec on each call. You can also check
functions to assert that it's returning valid values for randomly generated inputs.#2018-06-1216:22taylorthere's also exercise-fn
#2018-06-1216:29kvlt@U3DAE8HMG: Thanks for the response.
Are you suggesting something along the lines of:
(deftest some-test
(every? true? (map #(get-in % [:clojure.spec.test.check/ret :result])
(stest/check `sut/my-test {::stc/opts {:num-tests 1}}))))
#2018-06-1216:32taylorthat should work, although you probably want more than 1 test, and I believe there's a helper function to make the stest/check
result easier to examine#2018-06-1216:36kvltThe problem with this is that it won't give me any information about why it failed#2018-06-1218:10bbrinck@U08UTJ5PB maybe something like the following (untested)?:
(deftest some-test
(let [results (stest/check `sut/my-test {::stc/opts {:num-tests 1}})]
(is (every? true? (map #(get-in % [:clojure.spec.test.check/ret :result] results)
(stest/summarize-results results))))
#2018-06-1218:10bbrinckor something like: https://gist.github.com/Risto-Stevcev/dc628109abd840c7553de1c5d7d55608#2018-06-1218:11bbrinckhttps://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29#2018-06-1218:13bbrinckFWIW, expound can also format the check
results. https://github.com/bhb/expound#printing-results-for-check . You could call explain-results-str
for the 2nd arg to is
#2018-06-1414:40rapskalian@U08UTJ5PB here is how I've done some generative testing in the past. Works very well:
https://github.com/seesawlabs/digraph/blob/master/test/digraph/core_test.clj#2018-06-1216:20guymaybe https://clojure.org/guides/spec#_instrumentation_and_testing ?#2018-06-1216:32kvltI've been looking at that, it's still not clear to me how I run these in an automated test environment#2018-06-1217:20bortexzI have a message coming from an API in a format like '(a b c d e f)
, and Iād like to propagate that message through a channel to my program with a proper format in a map, with keys for each of the properties that come in the list.
I am using spec to conform the message, so I have something like (s/def msg (s/cat :a number? :b number?ā¦))
and then when I call conform I do have a map like {:a a, :b b, :c c, ā¦}
, which is the format I want already (a map). Okay now I have a format I like to pass it on to the rest of the program, except Iād prefer to have namespaced keys, but s/conform
does not seem to put namespaced keys even if they are specified as keys or values in the s/cat
. Is there any other way to make this happen? Is this a valid approach to ātransformā messages from the outside world to the inside language? It is really helpful as it is, but I am not sure it is the preferred way to do it. What other way then? How do you usually do it?
Thanks!#2018-06-1218:02taylors/conform
+ s/cat
does seem to preserve qualified keyword tags for me:
(s/conform (s/cat :foo/bar int? :what/ever string?) [1 "!"])
=> {:foo/bar 1, :what/ever "!"}
#2018-06-1218:17bortexzOh, youāre right, it does for me too. I got confused because proto-repl prints it out like #:bitfinex-clj.spec.data{:ts 1, :open 2, :close 3, :high 4, :low 5, :volume 6}
, but I have checked and they are namespaced.#2018-06-1218:27taylorya that's just the shorthand for when all the keys in the map share the same namespace#2018-06-1218:34bortexzAnother question: imagine I have all the namespaced keys already as predicates previously defined, so I can reuse them. Anyway to make this to not repeat each key? (s/def a (s/cat ::a ::a ::b ::b ā¦))
#2018-06-1218:35taylorthe only solution that comes to mind is writing a macro that outputs a s/cat
like you want it, but before I did that I'd probably reconsider doing the transformations outside spec, maybe before/after conforming#2018-06-1218:37taylorthis seems like it might lend itself more naturally to a s/keys
spec, and you could just handle the transformation before you conform (using zipmap
or something)#2018-06-1218:40bortexzhmā¦So the thing is the messages can come in different shapes and I use spec
and s/or
to tell me which type of message it is. But I probably can, instead of conform
, to just check which message comes in without keeping the transformation, and then do the transformation outside. On the other hand, it seems so much related that it feels everything should be together (the same keys used by s/cat
and zipmap
, etcā¦), also because conform already does the transformation I want#2018-06-1218:43bortexzAnyway, Iāll have a rethink about the current structure I am using. Thanks @U3DAE8HMG š#2018-06-1218:48taylornp, here's a macro that I think does what you want:
(defmacro kat [& ks]
(let [ks' (mapcat #(repeat 2 %) ks)]
`(s/cat
#2018-06-1312:06ackerleytngis there a way to write and use stest/instrument
or stest/check
on a function in a record?#2018-06-1312:07mpenetnope, if you have to do that you need to wrap it#2018-06-1312:07mpenet(which is not free)#2018-06-1312:08ackerleytngwhat do you mean by wrap it with another function?#2018-06-1312:09mpenet(defprotocol IFoo
(-foo [x]))
(def foo -foo)
(s/fdef foo :args (s/cat))
#2018-06-1312:10mpenetI don't think you can spec and instrument a protocol fun otherwise#2018-06-1312:40ackerleytngthanks!#2018-06-1411:40borkdudewhatās the way out here, alter-var-root? https://github.com/bhb/expound/issues/19#2018-06-1412:01Alex Miller (Clojure team)set! will only work on a dynamic var if you are in the dynamic scope of a binding call#2018-06-1412:01Alex Miller (Clojure team)Repls like clojure.main establish this#2018-06-1412:02Alex Miller (Clojure team)So you either need to use binding or modify the root value with alter-var-root#2018-06-1412:09borkdudeclear. Is there also something like *print-length*
but for strings?#2018-06-1412:11Alex Miller (Clojure team)No#2018-06-1412:14Alex Miller (Clojure team)You could override the print-method for String, although that might have some adverse consequences #2018-06-1412:15borkdudeprobably not a good idea š the reason I ask is that a spec error (with expound) flooded my emacs buffer so bad, I had to forcequit itā¦#2018-06-1413:04bbrinck@borkdude although it elides potentially useful information, you can configure expound to not print out āfailing specā info, which would likely shorten the output considerably.#2018-06-1413:45alex-dixonIs there an idiomatic way to check if a spec exists?#2018-06-1413:46alex-dixoni.e. given a namespaced keyword how to check if a spec is registered for it#2018-06-1413:48taylor#2018-06-1413:51alex-dixonThanks @taylor#2018-06-1413:53alex-dixonAlso spec is on cljdoc https://cljdoc.xyz/d/org.clojure/spec.alpha/0.1.143#2018-06-1413:57alex-dixon@taylor Iām getting a compiler exception for an unknown spec instead of a keyword#2018-06-1413:57alex-dixon@taylor Iām getting a compiler exception for an unknown spec instead of a keyword#2018-06-1413:57taylorcan you paste the exception here?#2018-06-1413:58tayloroh sorry, I think I tested this with a predicate function and not a keyword#2018-06-1413:59alex-dixonnp#2018-06-1413:59taylors/form
does throw an exception if you give it a ::keyword
that has no registered spec#2018-06-1414:00alex-dixonOk. Maybe Iāll just implement with try catch#2018-06-1414:04tayloroh there's s/get-spec
#2018-06-1414:09taylor(def registered? (comp some? s/get-spec))
#2018-06-1414:11alex-dixonOh dāoh#2018-06-1414:11alex-dixonPerfect. Thanks lol#2018-06-1414:11taylorI didn't know this existed either :man-shrugging: never had to look up a spec like that#2018-06-1414:17alex-dixon(defn registered-spec? [x]
(and (qualified-keyword? x)
(some? (s/get-spec x))))
Yeah. Working on something where a macro may receive a namespaced keyword with an associated specā¦so hopefully this all works out at compile time. Unfortunately have to do actual work now. Thanks a lot for your helpā¦was going to hack something in but I feel way better with this#2018-06-1420:03noisesmithgiven that macro expansion is a simplification step before making bytecode, you either end up with a much more complex compiler or taking the huge performance hit of having an interpreted language rather than a compiled one#2018-06-1420:04noisesmith(in order to have first class macros)#2018-06-1420:04noisesmithhttps://en.wikipedia.org/wiki/Fexpr#2018-06-1420:04johanatan@noisesmith i'm not arguing for first-class macros (although that would be nice). i'm arguing for not using second-class macros in libraries#2018-06-1420:04johanatanand forcing their proliferation onto users.#2018-06-1420:05noisesmithoh, I misinterpreted you then#2018-06-1420:06johanatan[sorry, yea, it wasn't clear from without the context]#2018-06-1420:27rapskalianCan I generate specs en masse? Something like:
(doseq [k my-keys]
(s/def k (s/double-in :infinite? false :NaN? false :min 0)))
#2018-06-1420:39taylorit's registering the specs to the literal symbol k
#2018-06-1420:40rapskalianThis was my gut feeling of what was going on. Any way to "force" it to evaluate the keyword? At this point I can see the value of writing them out by hand, but still curious...#2018-06-1420:40rapskalianSmells like a macro...#2018-06-1420:41donaldballI have done:
(doseq [[spec-kw field-spec desc] fields]
(eval `(s/def ~spec-kw ~(build-internal-spec field-spec))))
#2018-06-1420:41donaldballI donāt even feel bad about it.#2018-06-1420:49tayloryeah you could use a macro to spit out a bunch of s/def
's for your keywords:
(defmacro alias-kws [ks]
(let [defs (map #(list 's/def % ::my-spec) ks)]
`(do
#2018-06-1421:17rapskalian@U04V4HWQ4 interesting...using eval
does do exactly what I'm after. It's a bit of a hack, but it beats having to repeat these massive lists of keywords over and over again...#2018-06-1421:20donaldballIt would be nicer if spec had an explicit affordance for this use case, but I donāt really think itās abusive to take advantage of the fact that clojure is a lisp š . I wouldnāt reach for this often, mind, but in my case I had a gigantor table of fields for a structured file and this lets me leverage an edn representation for spec validation as well as actually e.g. generating serializers#2018-06-1421:23rapskalianVery cool, I agree that there are definitely valid use cases. Not to mention that the prospect of generating specifications from concretions is extremely interesting in its own right š¤
Appreciate the help (@taylor as well, thank you)#2018-06-1420:27rapskalianFor some reason I'm getting "unable to resolve spec"...#2018-06-1420:28johanatan@cjsauer most (all?) attempts to meta-program spec will fail-- it's apparently intended to be hand-written in the first-order only.#2018-06-1420:30rapskalianOoph...ok, thanks @johanatan#2018-06-1420:39rapskaliangenerates all specs directly in emacs buffer and calls it a day#2018-06-1420:39rapskalianI ā¤ļø Clojure#2018-06-1420:46seancorfield@johanatan as noted, spec is currently built for human written code/input. There are plans for a more programmatic API which will open up meta-programming. I think building the latter first would have led to a lot more breaking changes (since folks would have built code against the API). Make sense?#2018-06-1506:05ackerleytngwhat's the difference between gen/hash-map
and just using gen/fmap (fn [a] {:a a}) ...
?#2018-06-1514:33bbrinck@ackerleytng hash-map
can take multiple key-value pairs#2018-06-1514:34bbrinck(whereas in your example, the function you pass to fmap only takes a single value)#2018-06-1514:35bbrinckunder the hood, hash-map
does use fmap
with zipmap
to combine these into a map, which is just a more general version of your code above: https://github.com/clojure/test.check/blob/729de024f245c07011a2cd2fcaad04bcd90a223d/src/main/clojure/clojure/test/check/generators.cljc#L645-L646#2018-06-1516:23ackerleytngi see, thanks!#2018-06-1518:53kennyIs there a way to only check the keys explicitly written in a Spec? i.e. No implicit validation of keys. I have a case where I need to validate partial pieces of a larger map.#2018-06-1518:58noisesmith@kenny when would spec implicitly validate your key?#2018-06-1518:59kennyAny map implicitly validates all keys against existing specs.#2018-06-1518:59noisesmithselect-keys before validating?#2018-06-1518:59kennyCould totally do that but then I need to write the set of keys twice.#2018-06-1617:23alex-dixonIf I have a spec that calls conform on data that may have been specced by other users, should I expect to receive their conformed results in my conformed results?#2018-06-1722:58taylorif their specs are loaded into the spec registry in your process, I think so#2018-06-1708:46misha@kenny you can get it fairly easily from s/form
#2018-06-1714:45kenny@misha Thatās what I ended up doing š https://github.com/ComputeSoftware/spec-keys#2018-06-1800:53metametadataHi. I've noticed that CLJS impl of explain
prints more stuff than Clojure. Is it on purpose?
CLJS:
=> (s/explain ::x 23)
val: 23 fails spec: :cljs.user/x predicate: string?
:cljs.spec.alpha/spec :cljs.user/x
:cljs.spec.alpha/value 23
Clojure:
=> (s/explain ::x 23)
val: 23 fails spec: :cljs.user/x predicate: string?
#2018-06-1811:26zcljIf I would like instrument
to check my specs, what would be the preferred way of spec:ing a multimethod? Can I use a multispec with the same dispatch fn or do I need an indirection fn with a spec in each defmethod
?#2018-06-1811:48Alex Miller (Clojure team)You can make the defmulti dispatch function explicit and spec that#2018-06-1817:58zcljIn my case each defmethod
gets called with a map and the map should have different contents for each method. As I read your answer it would be the same spec for all methods?#2018-06-1812:10misha@kenny I wonder if qualified symbols can be replaced like this:
(case form-sym
(clojure.spec.alpha/keys
cljs.spec.alpha/keys) ...
;=>
(case form-sym
`s/keys ...
#2018-06-1812:10mishagiven you already required [clojure.spec.alpha :as s]
#2018-06-1812:11mishathen, getting rid of alpha
would be just 1 line :require
update#2018-06-1814:39valeraukoI recall that spec was inspired in part by RDF. Is there any recommended way for describing XML structures?#2018-06-1814:51Alex Miller (Clojure team)xml represented by what data structure?#2018-06-1814:53valeraukoI'm using clojure.data.xml, so whatever that uses internally.#2018-06-1814:53valeraukoIts sexp-as-element
is the closest I've seen it get to simple data structures, but even that seems to be pretty complex to spec#2018-06-1814:54Alex Miller (Clojure team)seems pretty straightforward?#2018-06-1814:55valeraukoI'm afraid I don't follow#2018-06-1814:55Alex Miller (Clojure team)there are at least two formats there - the input and output of sexp-as-element. they both seem pretty straightforward to spec if you want to#2018-06-1814:56valeraukothe output's clojure.data.xml.Element
s, right?#2018-06-1814:56Alex Miller (Clojure team)Element is a defrecord, so you can just spec it as a map with 3 known keys#2018-06-1814:57Alex Miller (Clojure team)the sexp form is just structured vectors and can be specāed with s/cat etc#2018-06-1814:59Alex Miller (Clojure team)oh, there actually are specs already in data.xml#2018-06-1814:59valeraukoreally?#2018-06-1814:59Alex Miller (Clojure team)https://github.com/clojure/data.xml/blob/master/src/main/resources/clojure/data/xml/spec.cljc#2018-06-1814:59Alex Miller (Clojure team)thatās for the element forms#2018-06-1815:02valeraukoit's not in the stable release yet though, is it?#2018-06-1815:03valeraukoi see you bumped its clojure dependency up to 1.9 when specs were introduced#2018-06-1815:04valeraukoas you pointed out i can just spec them as maps with :tag
:attrs
and :content
#2018-06-1818:02lwhortonif I want to declare a with-gen ::spec generator-fn
, generator fn needs to be a fn that returns a generator. how does one create a generator thatās simply a function, without using fmap or any of the other test.check.generators?#2018-06-1818:02lwhortonfor example I just want a āgeneratorā that invokes (random-uuid)
where the fn itself is already random and doesnāt need any randomness from the generator scaffolding#2018-06-1818:05lwhortoni had some garbage like
(s/exercise ::p/id 10 {::p/id (fn [] (gen/fmap #(random-uuid) (s/gen (s/int-in 1 10))))})
but this is abusing fmap to just give me access to a fn I can call that will return a value thatās wrapped in the proper generator#2018-06-1818:08guymaybe (gen/return .. ) ?#2018-06-1818:08guy(gen/return (random-uuid))
maybe?#2018-06-1818:09guyhttps://clojure.github.io/test.check/generator-examples.html
(def int-or-nil (gen/one-of [gen/int (gen/return nil)]))
(gen/sample int-or-nil)
;; => (nil 0 -2 nil nil 3 nil nil 4 2)
#2018-06-1818:09guythats using gen/return to always return nil#2018-06-1818:09guyI'm not entirely sure its what you are looking for though š#2018-06-1818:10lwhortonthat seems like the same idea as abusing fmap and an existing gen (int-in), but with different fns. it just feels like thereās a built-in fn like gen/return
for this use case, right?#2018-06-1821:13Alex Miller (Clojure team)using external sources of randomness prevents shrinking and repeatability so is discouraged#2018-06-1822:45gfredericksif you want to do the discouraged thing, fmap
will work and return
won't.#2018-06-1909:41mpenetyou can mix both to limit the awful (g/fmap rand-uuid (g/return nil)) I think#2018-06-1909:41mpenetbut it's still bad#2018-06-1914:36lwhortoni see. so if you arenāt using the generatorās api test.check canāt work its magic?#2018-06-1917:08favilatest check needs to control all sources of entropy#2018-06-1917:08favila(if you want the other features of generators eg strinking and repetition to work)#2018-06-1917:08Alex Miller (Clojure team)@lwhorton no, test.check uses the same api. the issue here is using external sources of randomness#2018-06-1917:10favilagen/uuid already exists, is that not ok for you?#2018-06-1920:45lwhortonoh, i didnāt notice that. i will check it out. my real gotchya was that i have an instance?
check against cljs.core/UUID
and wasnāt sure how to make that happen randomly.#2018-06-1922:09gfredericksThe builtin uuid generator should only generate instances of that type#2018-06-2005:41Oliver GeorgeI'm wondering how to avoid a REPL gotcha related to instrumentation. Since stest/instrument
wraps existing functions you effectively lose instrumentation by redefining a function with defn
at the REPL.#2018-06-2005:42Oliver GeorgeRe-running stest/instrument
fixes this but would be annoying to do manually every time.#2018-06-2005:43Oliver GeorgeHow do you avoid losing instrumentation at the REPL?#2018-06-2007:27sundarj@olivergeorge define a defn+instrument
macro?#2018-06-2008:11Oliver GeorgeAn instrumenting version of defn would be perfect but ideally it'd be seamless. #2018-06-2012:15gfrederickswhat tooling are you using? I'm often editing the files and running the tools.namespace refresh
function, which means I can either have the call to instrument
at the bottom of a given namespace, or else use the callback hooks in refresh
#2018-06-2012:16gfredericksrelated, what's the purpose of instrumentation? just for running tests, or for interactive development?#2018-06-2012:21Oliver GeorgeInteractive development. Just starting on a re-natal mobile companion app to a clojurescript re-frame app. #2018-06-2012:23Oliver GeorgeFigwheel has an "always reload" option for namespaces but a simple cursive repl is the case I was considering. #2018-06-2012:24gfredericksI had the idea of monkeypatching defn
, but I dunno how to accomplish that exactly in cljs#2018-06-2012:29gfredericks(alter-var-root #'defn
(fn [orig]
(fn [& args]
(let [form (apply orig args)]
`(do ~form (st/instrument '~(ffirst (drop 2 args))))))))
something atrocious like that#2018-06-2013:23Oliver GeorgeNo immediate joy but I'll give it some more time later.#2018-06-2014:28gfredericksyou'd have to somehow get that to run in the cljs compiler process
also I realized it's possible that defn
isn't even a macro in cljs; it could be a special form#2018-06-2016:20noisesmithI assume that do was meant to be doto?#2018-06-2016:22gfredericksI don't think so#2018-06-2016:22gfredericksI mean it definitely wasn't, and I believe it's right that way#2018-06-2016:22gfredericksI think st/instrument
takes a symbol, not a var#2018-06-2016:34noisesmithoh, so the intention is to no longer return the var but instead return whatever instrument returns?#2018-06-2016:35noisesmithI am probably weird for doing this, but I have a repl workflow where I count on being able to call load-file
and then (*1)
to call the last var in the file#2018-06-2016:42noisesmithanyway all I was getting at was that the original return value of defn was being dropped#2018-06-2016:59gfredericksOh good point. Need a let #2018-06-2012:29gfredericksas long as it only runs in dev mode then it's not the worst thing in the world š#2018-06-2012:58Oliver GeorgeThanks I'll give that a try#2018-06-2013:20Oliver GeorgeI might be able to do something similar via a custom command in cursive. Looks like I can send something like this to the repl with a hotkey (untested):
~top-level-form
(instrument ~current-var)
#2018-06-2013:22Oliver GeorgeAnother approach would be using a patched version of clojurescript for development. (Must try and see how hard that is now that deps.edn does github shas)#2018-06-2104:26alex-dixonDoes anyone know if thereās a proposal to add line numbers to spec error messages?#2018-06-2112:28Alex Miller (Clojure team)There should be line numbers included now for macro function spec errors#2018-06-2112:30Alex Miller (Clojure team)I think there was a ticket filed recently, or at least it came up, for the same with non macro function spec errors#2018-06-2215:09mpenet@alexmiller It seems spec might get some love given some jira activity. I know there are already a lot of things queueing on that, but do you think there's a chance we might get anything to allow to get relations between specs (like list all aliases for a registered spec, list all specs ancestors depending if it's a s/merged of others etc etc)?#2018-06-2215:09mpenetit's quite easy to add, but just curious if this is on the radar at all#2018-06-2215:28Alex Miller (Clojure team)Deep walk is of interest but I donāt think we will look at that until after Richās next batch of impl changes#2018-06-2218:22bmaddyhas there been any indication on what the next batch of impl changes might be focused on? Even if only a general description?#2018-06-2219:41Alex Miller (Clojure team)Better for data-oriented functional construction#2018-06-2219:42Alex Miller (Clojure team)Due to other priorities for Rich, I think that will slip out a bit but weāre going to try to do some bug fixing in near term#2018-06-2316:01ackerleytngwhat's the difference between s/*
and coll?
#2018-06-2316:01ackerleytnglike if i do (s/def ks coll?)
, is that the same as (s/def ks (s/* any?))
?#2018-06-2316:22Alex Miller (Clojure team)s/* is a regex op and will compose with other regex ops to describe the internal structure of a sequential collection#2018-06-2316:22Alex Miller (Clojure team)If you donāt have any structure, Iād use coll-of#2018-06-2404:59ackerleytngthanks!#2018-06-2411:00martinklepschI would like to find out where a spec has been defined (source file, namespace etc.) ā is this possible?#2018-06-2412:44ackerleytnghmm sorry not sure about that...#2018-06-2413:04Alex Miller (Clojure team)Some specs retain meta, but generally no. There is a ticket about this#2018-06-2413:20ackerleytngI'm trying to test a function where the inputs are related#2018-06-2413:20ackerleytngit's actually a bit like update-in
#2018-06-2413:20ackerleytngwhere ks
has to be related to m
#2018-06-2413:22ackerleytngI wrote a generator to generate m
, and i want the result of that generator to be inputs to a function that returns another generator for ks
#2018-06-2413:22ackerleytngthose already mostly work, but how would i supply gen-overrides
in pairs?#2018-06-2512:56ackerleytngI got it!#2018-06-2512:56ackerleytngi'm now generating all the arguments to the function as a single list
#2018-06-2512:57ackerleytnganother question! is there any "state" that carries over between instances of generators?#2018-06-2512:58ackerleytngsay i define (def string-gen (gen/string))
#2018-06-2512:59ackerleytngif i use that in two different overrides, say {::foo (fn [] string-gen)}
and {::bar (fn [] string-gen)}
#2018-06-2513:00ackerleytngwould the two overrides generate strings independently of each other?#2018-06-2513:06guyI think the generators have seeds? which might be different each time?#2018-06-2514:26ackerleytngah yes i guess so#2018-06-2514:26ackerleytngalong those lines - will a more complex generator retain any sort of state between two invocations?#2018-06-2514:32favilaa top-level generator instance and all its "sub" generator instances share a common source of randomness; it's not shared with other instances#2018-06-2514:32guyunless you pass it a seed or something? i think#2018-06-2514:32guyI canāt quite remember#2018-06-2514:32guylet me google it š#2018-06-2514:32favilayeah but even then you're regenerating the same randomness independently#2018-06-2514:33favilait's not shared#2018-06-2514:33guyahhhh#2018-06-2514:33guyyes you are correct#2018-06-2514:33guyš#2018-06-2514:33favilaso if your overrides are part of a larger generator (e.g. you are generating examples for a top-level map and it has these two keys you override) then the generators share the same source of randomness#2018-06-2514:34favilaif these are independent uses of these generators, they do not share#2018-06-2514:40ackerleytnghmm then why do the overrides have to be wrapped in a function?#2018-06-2514:41ackerleytngit has to be {::foo/bar (fn [] (gen/return inc))}
instead of just {::foo/bar (gen/return inc)}
right?#2018-06-2514:41ackerleytngdoes wrapping it in a function kind of force a new generator to be returned or something? to avoid state carrying over?#2018-06-2514:51favila@ackerleytng wrapping in a function is to allow lazy invocation of the entire generator system; It's possible to use spec without a test.check dependency (e.g. in production) if you don't use any of the generation features.#2018-06-2515:00ackerleytngah i see, thanks!#2018-06-2514:51favilaif you didn't do this, spec would always require test.check even if you only wanted to use e.g. s/valid? s/conform s/unform etc#2018-06-2514:52favilathis is also why clojure.spec.gen.alpha exists: it wraps all the test.check functions and generators so that test.check itself is never required unless actually used#2018-06-2518:29blancedo you usually define generator together with your spec when the only use case for the generator is writing generative tests? It's nice that you can combine them with with-gen
, but on the other hand I don't know if it's right to define something in src that's only needed for testing#2018-06-2519:02noisesmith@blance consider that a generator for foo will be needed for another library that uses foo, and test files are not transitive like source files are#2018-06-2519:03noisesmithso you end up hacking and adding the generator test files to exported classpath, or rewriting the generator#2018-06-2519:03noisesmithof those options, a generator in the source file is the least hackish#2018-06-2519:04blancethanks! that makes me feel better writing generator in along side with the spec itself#2018-06-2602:16athos@blance I was working on a library for managing generators separately from spec definitions a couple of months ago. If youāre interested, take a look at it! https://github.com/athos/genman#2018-06-2602:23blancethis looks promising! i'll definitely give it a try next time I write spec#2018-06-2621:03justinleequestion about spec-tools.data-spec
. it seems like with data-spec there are now two namespaces: the one in the spec registry and the symbol i define in my own code. this makes sense with the one example in the readme because it is a map. but what about for code like (def hit-box-spec
(ds/spec ::hit-box [number?]))
It seems like both ::hit-box
and hit-box-spec
are now symbols that refer to a spec. Iām just getting a bit confused about how to use this library with non-map specs.#2018-06-2621:09noisesmith::hit-box is a keyword, specs resolve via a central registry that uses keywords. hit-box-spec is a var, containing whatever ds/spec returns (not necessarily the same value stored under that key in the spec registry, but maybe?)#2018-06-2621:31justinleeyea. thatās how i understand it. i think iām just confused why ds/spec
doesnāt rely on the fact that it creates a side effect in the spec registry. iām not sure why iād want to also have a var in my namespace that points to a value that is apparently interchangeable (at least it is when using s/valid?
#2018-06-2621:44noisesmithbut it also points to the literal data,that ds/ functions can use to create other specs, right?#2018-06-2621:44noisesmithI'd assume that's the reason#2018-06-2621:45justinleeI donāt think so. In the readme, they use a different var to point to the data used to create the spec. So they might write the above as (def hit-box-spec (ds/spec ::hit-box hit-box))
#2018-06-2621:46justinleebut it might have to do with these transformations, which Iām not using and donāt understand. that would make sense#2018-06-2622:05justinleehm what it is doing is more complex than this. my statement above that ::hit-box
and hit-box-spec
are interchangeable for valid?
is wrong#2018-06-2622:57ikitommi@lee.justin.m side-effects are bad, I think the data-specs could benefit from a local-keys
variant that doesnāt need to register the keys. data-specs are anyway not in align to the Spec filosophy of reusable keys, so why not go further down the roadā¦#2018-06-2622:58ikitommibut for the original:
(ds/spec ::hit-box [number?])
can be written:
(ds/spec
{:name ::hit-box
:spec [number?]})
ā¦ and as there are no keys, the :name can be omitted:
(ds/spec
{:spec [number?]})
#2018-06-2622:59ikitommito register that, you can say:
(s/def ::hit-box
(ds/spec
{:spec [number?]}))
#2018-06-2622:59justinlee@U055NJ5CC ah i see. i should be using the map syntax.#2018-06-2622:59justinleeas for whether side effects are bad, thatās just the way spec works, right?#2018-06-2622:59justinleeyou have to register your spec with the registry#2018-06-2622:59ikitommiright.#2018-06-2623:00justinleethe thing iām not getting right now is this: (def banana-spec
(ds/spec ::banana {:id integer?
:name string?}))
(st/registry #".*banana.*"))
=> (:seekeasy.app-state$banana/name :seekeasy.app-state$banana/id)
#2018-06-2623:00justinleei see that the two subspec are registered, but why not the main banana
spec?#2018-06-2623:00ikitommiif you try to put a map somewhere in the data-spec and donāt provide a :name
, it will fail-fast:
(ds/spec
{:spec [[[[[[[{:a int?}]]]]]]]})
; CompilerException java.lang.AssertionError: Assert failed: spec must have a qualified name
; (qualified-keyword? n), compiling:(data_spec_test.cljc:380:3)
#2018-06-2623:01ikitommigood question, not sure, maybe it should?#2018-06-2623:01justinleei just donāt even know how it works š#2018-06-2623:04justinleebasically i was expecting an unmunged :seekeasy.app-state/banana
given that i provided it that as the name of the spec. although i guess thatās consistent with the readme, now that i think about it. the only thing that got registered were the subspecs.#2018-06-2623:06ikitommiif there would be the local-keys
, there would be no registering of any specs. that in mind, having a function called spec
doing registration of the top-level would be bit odd.#2018-06-2623:07ikitommithere could be ds/def
for thatā¦#2018-06-2623:07justinleeoh i see what you mean#2018-06-2623:07ikitommiā¦ but (s/def ::name (ds/spec ..))
is the way to do it now.#2018-06-2623:07justinleebecause spec does (s/def ...)
so we would expect it to cause a side effect#2018-06-2623:08justinleebut here itās just a function so we need to def it ourselves. okay. this is coming together for me.#2018-06-2623:08justinleeI guess I thought there was a separate data structure for the specs. But they are just stored on normal vars?#2018-06-2623:09justinleethe existence of the st/registry
made me think this#2018-06-2623:09ikitommiall Specs are just values, you can store them in a Var.#2018-06-2623:09ikitommi(s/valid? string? "1")
#2018-06-2623:10ikitommi(s/valid? (ds/spec {:spec [int?]}) [1 2 3])
#2018-06-2623:11ikitommi.. unless you register them and get a name than can be used in s/keys
. I think itās the only one that requires a spec to be registered?#2018-06-2623:11justinleeahhhhh okay#2018-06-2623:11justinleedamn that is confusing š#2018-06-2623:12ikitommi(s/valid? (s/coll-of int? :into []) [1 2 3])
#2018-06-2623:13justinleebasically, the thing that confused me is that if you do (s/def ::something ...)
that will show up in the registry even it if isnāt meant to be used as a key but if you do (def something (ds/spec ...))
it doesnāt show up. but both work. i hadnāt appreciated the fact that nothing cares about the registry except for s/keys
#2018-06-2623:16justinleebtw, thanks for spec tools. i really really really prefer the self-documenting format of schema, and now i have my cake and eat it too#2018-06-2623:21ikitommithanks! itās kind of a roque lib, I hope the final version of spec will make much of it redundant.#2018-06-2623:13ackerleytngwhat types of functions do you guys normally spec? do you spec utility functions?#2018-06-2623:19justinleethe answer youāll hear most commonly is āat data boundariesā or something like that. e.g. reading data from a file or network or moving from one chunk of code to another, rather than doing it wholesale on every internal function#2018-06-2623:35noisesmithI'd put it as "system boundaries" rather than "data boundaries" but yes, exactly that#2018-06-2623:35noisesmithwhere system boundaries are has a lot to do with your design, but if your system is designed it should have some :D#2018-06-2623:36ackerleytngthanks š#2018-06-2623:36ackerleytngwon't your system boundaries keep changing?#2018-06-2623:36ackerleytngas you compose functions with functions#2018-06-2623:37noisesmithif your system boundaries are changing those weren't your system boundaries#2018-06-2623:37ackerleytnganother question! how do you define an fspec
where you don't care about the name of the argument, but you care about the type?#2018-06-2623:37ackerleytngah thanks that sounds right haha#2018-06-2623:38noisesmiththe idea is that it isn't a system if you don't define some limit or border, that's really the first step. What the boundaries are, and which things cross them, should be one of the first things defined, and often you'll want to define things so it changes relatively rarely#2018-06-2623:39noisesmith@ackerleytng for the names of things in specs, the reason to have the names is for the error messages you get without a match. Otherwise the output of a failure turns into a soup of data types that isn't very helpful.#2018-06-2623:39ackerleytngoh! so it doesn't actually match against the name of the actual parameter?#2018-06-2623:41noisesmithif it is the thing I'm thinking of it's a series of name / type for each arg right?#2018-06-2623:41noisesmiththe name is used in generating the message, it doesn't have to match the arglist or anything#2018-06-2804:44ackerleytngthanks!#2018-06-2811:02gnlIntroducing Ghostwheel ā it makes the writing, reading, refactoring, testing, instrumentation and stubbing of function specs easy; introduces a concise syntax for inline fspec definitions; helps keep track of side effects; and makes general debugging almost fun, with full evaluation tracing of function I/O, local bindings and all threading macros.
https://github.com/gnl/ghostwheel#2018-06-2813:40andre.stylianosThis looks really, really nice! I will for sure give it a try soon#2018-06-2813:43gnlGreat, report back when you do!#2018-06-2812:24ackerleytnghow do i spec a function that has a variable number of arguments?#2018-06-2812:25gnlTry (s/* ...)
https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/*#2018-06-2812:26ackerleytngah i think i see where my mistake is#2018-06-2812:27ackerleytngi'm actually trying to spec a function that may have a variable number of arguments#2018-06-2812:27ackerleytngso I should have an or#2018-06-2812:45ackerleytngwonder if anyone can help me understand this#2018-06-2812:45ackerleytngHow would i write a spec such that all these conform?#2018-06-2812:45ackerleytng(defn- zzz
[i] i)
(defn- zzz-args
[i args] i)
(defn- zzz-var-args
[i & args] i)
(s/conform ::helpers/test zzz)
(s/conform ::helpers/test zzz-args)
(s/conform ::helpers/test zzz-var-args)
#2018-06-2812:45ackerleytngI've tried
(s/def ::helpers/test
(s/fspec :args (s/or :a (s/cat :itineraries (s/coll-of string?)
:args (s/* any?))
:b (s/cat :itineraries (s/coll-of string?)
:args any?)
:c (s/cat :itineraries (s/coll-of string?)))))
#2018-06-2812:46ackerleytngbut only (s/conform ::helpers/test zzz-var-args)
conforms#2018-06-2814:33favilaThey can't all conform: these are incompatible function signatures#2018-06-2814:33favilayou can write a spec such that the args for any of those signatures passes, but you can't disambiguate their meaning#2018-06-2814:35favilanor can you work backwards: generated sample args from this spec won't work for all functions. zzz
only works with :c
and some :a
for example#2018-06-2814:35favilaif you give it something matching :b
it will fail#2018-06-2815:54justinlee@clojurians.net whew! youāve been busy! iāve just switched over to using spec-tools
for its data-spec features that allow me to specify nested data types using data. I was going to try to get orchestra
installed too so I can use their defn-spec
, but your library looks like it provides that functionality and more. I think can can mix and match spec-tools
with either of these. any pitfalls you can think of?#2018-06-2816:20gnl@lee.justin.m I honestly don't know.
I have no experience with spec-tools and it is my (very superficial) understanding that they rely on some internal spec workings/APIs which are likely to change and break as spec matures and moves out of alpha, I remember reading a comment from Alex Miller to that effect, if I'm not mistaken. That's one of the reasons it hasn't been near the top of my list of new stuff to check out. Feel free to correct me if I'm mistaken and spreading FUD here, anyone.
FWIW I can say that Ghostwheel is only using public APIs and not doing anything hacky at all. And if you do end up using it - feedback would be much appreciated! I've done a moderate amount of dogfooding, but there's always all kinds of things that tend to pop up in the wild.#2018-06-2816:21gnlOh and btw, Ghostwheel also supports orchestra for instrumentation.#2018-06-2816:22justinlee@clojurians.net thatās true about spec-tools. it says so right in the readme. (doesnāt bother me though. code can always be changed š )#2018-06-2816:23noisesmith> code can always be changed
tell that to python3#2018-06-2816:24justinleeiāve never kept up with python#2018-06-2816:25justinleeand at any rate, spec tools is a small library. if they break it, iāll just not upgrade or iāll rewrite my types. itās not mission critical#2018-06-2816:25noisesmithpython3 was introduced in 2008. most of the community still uses python2 because they use libraries that are incompatible with version 3#2018-06-2816:26justinleeoh. well thatās not exactly comparable. thatās changing out the whole language š#2018-06-2816:27noisesmithif you are relying on the internals of spec, you need to change your code with updates to spec, and if you aren't careful your users need to change code too - the point here is the way that compatibility issues can fragment a community and get things stuck, and spec marks things as internal to avoid this kind of scenario#2018-06-2816:31justinleewell spec should have given me a usable tool instead of cramming its way of doing things down my throat. by releasing this thing as an official library schema lost its mindshare so iām kind of forced to use spec and iām doing my best. iām working on a product so there are no consumers of my code.#2018-06-2816:23justinleeI basically just want to spec out the args inline using a macro. I think Iāll try yours out because maybe I can get tracing too. I never could get debux
to work with shadow#2018-06-2816:24gnlAnother thing that comes to mind regarding orchestra's defn-spec
is that if I'm not mistaken it's not quite straightforward to do multi-arity functions with varying return specs between the different arities. With Ghostwheel it's just like writing single-arity functions and it automatically does the right thing.#2018-06-2816:25gnlAnd last but not least - migration to and from ghostwheel is easy, because you are not interspersing specs between the args, etc, you are basically just adding a single vector to the standard defn
syntax#2018-06-2816:26gnlRegarding debux
- I got it to work, sorta, and was considering using that, but I much prefer the clairvoyant+re-frame-tracer output, so I took those and pimped them a bit instead.#2018-06-2816:29justinlee@clojurians.net does all of this stuff print back to the repl? it seems like shadow-cljs defaults to the browser console. iām just trying to figure out how it will work with all of this color output#2018-06-2816:29gnlIt prints to the browser console, js/console.log
basically.#2018-06-2817:00metametadataThere's also this nice little alternative to orchestra
- https://github.com/Provisdom/defn-spec. I like that it's friendly to Cursive IDE syntax highlighting.#2018-06-2817:11gnl@metametadata So is Ghostwheel! Cursive-friendly that is.#2018-06-3013:25kurt-o-sysif I have a map:
(def m #::{:item1 #::{:name "whatever"
:description "..."
:ref []}
:item2 #::{:name "another one"
:description "...?"
:ref [:item1]})
can I write a spec so that all items in ::ref
are keys of the containing map?
(s/def ::ref (coll-of ...??...))
or
(s/def ::ref (key-set ...??...))
#2018-06-3015:44ackerleytnghmm yes#2018-06-3015:44ackerleytngi think you can spec it as a predicate#2018-06-3015:44ackerleytngso#2018-06-3015:44ackerleytnglet me try it out on my side first#2018-06-3015:50ackerleytngsomething like that
(def m1 {:a {:b 1
:ref [:a]}})
(def m2 {:a {:b 1
:ref [:c]}})
(defn ref-contains-keys-of-containing-map [m]
(let [ks (keys m)]
(= (:ref (first (vals m))) ks)))
(s/def ::m (s/and map?
ref-contains-keys-of-containing-map))
(s/conform ::m m1)
(s/conform ::m m2)
#2018-06-3015:51ackerleytngm1
conforms, m2
doesn't#2018-06-3016:02kurt-o-sysright, thx... will try.#2018-06-3016:35kurt-o-sysoh right, but will see if I can make that more dynamic in some way... I'm fetching the map from outside. well... will see š#2018-06-3016:40kurt-o-sys@clojurians.net I like ghostweel - just trying it now - but it doesn't validate the return value of >defn
. I seem to remember clojure.spec
itself doesn't either, but orchestra does so... Any plans to add this to ghostweel as well (validation of return values of functions)?#2018-06-3016:43justinlee@kurt-o-sys have you configured to do so? thereās a separate config option to turn that on#2018-06-3017:25andre.stylianos;; Spec-instrument functions on namespace reload.
::instrument false
;; Spec-instrument functions on namespace reload using orchestra,
;; which spec-checks the output in addition to the input.
::outstrument false
#2018-07-0118:20kurt-o-sysI didn't... - I didn't know it was an option, but I do now š.#2018-07-0119:43andre.stylianosš
Glad to help!#2018-06-3017:25andre.stylianosfrom the README#2018-06-3018:25gnl@kurt-o-sys It's already in there! The option is ::g/outstrument
.#2018-06-3018:28gnlDidn't read the other replies before I answered. By the way there's a bug in the current stable version, where ::g/instrument
and ::g/outstrument
require ::g/check
to be enabled to have any effect. This doesn't really make any sense and is fixed in 0.2.2-SNAPSHOT, soon to be released as a stable 0.2.2 once some Figwheel issues have been cleared up.#2018-06-3018:28gnlDidn't read the other replies before I answered. By the way there's a bug in the current stable version, where ::g/instrument
and ::g/outstrument
require ::g/check
to be enabled to have any effect. This doesn't really make any sense and is fixed in 0.2.2-SNAPSHOT, soon to be released as a stable 0.2.2 once some Figwheel issues have been cleared up.#2018-07-0118:22kurt-o-sysnice, thx! I did figure it out concerning that ::g/check
issue, not about the ::g/oustrument
š. Thanks a lot. Just one more question: how would you recommend instrumenting only during dev, but not for a prod build? (Just wondering what you consider as best practice)#2018-07-0118:30kurt-o-sysalso, I must be doing something wrong:
(>defn target
"target function to be minified"
{::g/instrument true
::g/outstrument true}
[in]
[any? => double?]
"wrong")
This should fail, right? (return value should be of type double
, but it's a string). However, it doesn't fail at all.
Adding the config to the ns
doesn't help either:
(ns test-gw.core
#:ghostwheel.core{:instrument true
:outstrument true}
(:require [clojure.spec.alpha :as s]
[ghostwheel.core :as g
:refer [>defn >defn- >fdef ? => | <-]])
...)
#2018-07-0118:33gnlDon't use ::g/instrument
and ::g/outstrument
together, only one will be used, in this case ::g/instrument
. What you want here is only ::g/outstrument
.#2018-07-0118:34gnlAnd regarding not instrumenting in production ā just make sure that ghostwheel isn't enabled in your prod build config#2018-07-0118:35gnlNo way to do that in Clojure at the moment, so just don't do instrumentation there in production. š#2018-07-0118:46kurt-o-sysright, perfect :+1: :+1:#2018-06-3021:38gnlSpeaking of which ā 0.2.2 is out:
https://github.com/gnl/ghostwheel/blob/master/CHANGELOG.md#2018-07-0101:22codonnellDoes anyone know of an efficient way to generate a multi-spec value with a particular tag? (ie. without doing a such-that and hoping to get lucky)#2018-07-0106:09ackerleytng@codonnell what do you mean by a multi-spec value? have an example?#2018-07-0106:59ackerleytngIf i run a stest/check
and hit an error, how do i inspect the value that was generated? my return value is a function. I fspec
ced it, and one of the inputs to that function, generated during stest/check
is causing the spec to fail#2018-07-0112:39codonnell@ackerleytng
(s/def ::type #{::a ::b})
(s/def ::a-value pos-int?)
(s/def ::b-value keyword?)
(defmulti mymap-type ::type)
(defmethod mymap-type ::a [_]
(s/keys :req [::type ::a-value]))
(defmethod mymap-type ::b [_]
(s/keys :req [::type ::b-value]))
(s/def ::mymap (s/multi-spec mymap-type ::type))
;; How to generate a mymap value with tag ::a here without using gen/such-that
#2018-07-0113:18ackerleytngsomething like this?
(s/conform ::mymap {::type ::a
::a-value 1})
(s/conform ::mymap {::type ::b
::b-value :foo})
(map (partial s/explain-data ::mymap)
(gen/sample (tgen/let [type_ (gen/elements [::a ::b])
a-value tgen/pos-int
b-value (gen/keyword)]
(conj {::type type_}
(if (= type_ ::a)
{::a-value (inc a-value)}
{::b-value b-value})))))
#2018-07-0113:19ackerleytngtgen
is clojure.test.check.generators
#2018-07-0113:19ackerleytng[clojure.spec.gen.alpha :as gen]
#2018-07-0113:20ackerleytngpos-int
generates 0
sometimes, hence the inc
#2018-07-0113:21ackerleytngthis talk was super helpful for me https://www.youtube.com/watch?v=F4VZPxLZUdA#2018-07-0113:24gfrederickss-pos-int
doesn't generate zeros, fyi#2018-07-0113:25codonnellI'd prefer not to build up the value manually like that, since the actual spec I'm working with is much more complex than the toy example I posted above.#2018-07-0113:26codonnellI'll definitely check out that talk, thanks.#2018-07-0113:44ackerleytng@gfredericks thanks for giving that talk!! I loved the pictures too#2018-07-0113:45ackerleytng@codonnell hmm...#2018-07-0122:19Andreas Liljeqvist@codonnell (gen/sample (s/gen ::mymap {::type #(gen/return ::a)}))#2018-07-0122:20Andreas LiljeqvistIt would have been, but there is a bug with providing a generator for the dispatch key - See my bug report and patch https://dev.clojure.org/jira/browse/CLJ-2311#2018-07-0122:31codonnellthanks @andreas862, that is exactly what I was looking for. Hopefully your patch is accepted!#2018-07-0200:13ackerleytngs/gen allows you to override a specific generator?#2018-07-0200:13ackerleytngI see, thanks!#2018-07-0209:24otfrommorning#2018-07-0209:26otfromI have a map where one of the namespaced keys ::foo
for example, in the whole program can be one of 3 values #{"foo" "bar" "baz"}
, but in one part of the program can only be #{"foo" "bar"}
because I've filtered out "baz". Is there a way of expressing this in spec? I've been looking around and I haven't found anything (might be morning brane and solved by more coffee, but I'm not sure)#2018-07-0711:53rickmoynihanInterestingā¦ I think this is awkward because itās expressing something thatās brittle and contrary to the growth ideals of spec?
i.e. saying āthis set must have only these keysā is analogous to saying āthis map must have these keys and no othersā.
The solution with maps in this situation is often to use select-keys
in your function, so the function is robust to inputs with arbitrary other keys as it will ignore them.
Could the solution in your situation be to intersect the set-argument with #{"foo" "bar"}
so anything else is ignored?#2018-07-0815:59otfromhmm.. I hadn't thought about the growth ideals properly when thinking about this. It makes me think that the envelope should be very permissive and that I should create new maps for the payload with new specs going through the rest of the system. I think that is why I keep asking these questions as spec affords some forms and punishes others (for good reasons such as growth), but it can be difficult to see it all the time and design accordingly.#2018-07-0907:58rickmoynihanYes, if you build specs where you find yourself trying to spec āand nothing elseā youāll struggle.
Obviously I donāt really know what youāre doing, but your new ideas seem to make perfect sense from a general architectural perspective i.e. the transport layer of an architecture typically shouldnāt know about specifics of the application/content layer.#2018-07-0908:58otfromI often find if I'm struggling against one of the affordances of something in clojure it is because I'm not thinking about it very well. I think this might be one of those cases. Thx!#2018-07-0914:17carocad@U0525KG62 I have had that case as well though not for part of the program but rather for part of an api response. My approach was to split the set into the two possible solutions and then use s/or
and s/merge
to express the parts that are common and differentiate the other ones#2018-07-0914:18carocadthat might sound too abstract so here is an example: https://github.com/hiposfer/kamal/blob/master/src/hiposfer/kamal/specs/directions.clj#L55#2018-07-0210:07andre.stylianosDepends on what you want out of it I think, and what those things mean in your code.
You can make ::filtered-foo
which is #{"foo" "bar"}
and ::foo
is now s/or
of ::filtered-foo
or #{"baz"}
.#2018-07-0210:09andre.stylianosJust avoid having ::foo
itself mean two different things depending on context, as that breaks specs being globally consistent#2018-07-0213:03otfromhmm... makes me want to use unqualified keys for things like that then#2018-07-0213:16otfromI basically have a lot of "type fields" and often filter down to particular types in certain parts of the system and would like to constrain things in those sections (esp things like generative testing)#2018-07-0213:16otfromand different affordances drive different design styles in maps#2018-07-0213:38andre.stylianosYou could also leave ::foo
as being #{"foo" "bar" "baz"}
and in specific places just say that ::foo.subset
is (s/and #(not= % "baz") ::foo)
#2018-07-0214:21otfromyeah, it just means that the key has to change going through so any code that does work on the supertype wouldn't be pointing at the right key.#2018-07-0214:22otfromit just makes an envelope and payload style tricky to do (as payload might have loads of types)#2018-07-0214:37otfromso this is tricky to model in spec then
{::id <some uuid>
::timestamp <some timestamp>
::payload <a number of different things that are maps that vary depending on domain>}
#2018-07-0711:55rickmoynihanisnāt this the use case for a multi-spec
on ::payload
?#2018-07-0214:39otfromif you want to be able to constrain ::payload
later as you are only dealing with one of the domains.#2018-07-0214:40otfromI suppose you could leave ::payload as just a map#2018-07-0214:40otfromand deal with the payload maps separately#2018-07-0214:45andre.stylianosWell, you can either have ::domain-1/payload
, ::domain-2/payload
and so on, or you could spec ::payload
with s/or
, conform it and check if the conformed value is the branch you expected#2018-07-0214:47andre.stylianosdo keep in mind that I'm no clojure.spec expert, just brainstorming a bit :man-shrugging:#2018-07-0214:58otfromor have a non-namespaced :payload
and define it as :req-un
as needed#2018-07-0214:59seancorfieldWell, you can use namespaced variants of :xxx/payload
and still define it as :req-un
-- that way you can see which version you're working with -- and still just use :payload
in the map.#2018-07-0215:03otfromI'm not sure I understand you @seancorfield. Does that give me the ability to say "this bit of code here only works with this domain of what might go in payload rather than all the domains"#2018-07-0215:14seancorfieldYou can have multiple specs with the same set of (unqualified) keys -- but each spec can use different definitions for those keys by using different qualified versions of a key.#2018-07-0215:15seancorfield(s/keys :req-un [:foo/bar])
and (s/keys :req-un [:quux/bar])
-- the actual map has :bar
in both cases but the specs are distinct.#2018-07-0215:25otfromcool. That was what I thought. That you could basically redefine what the *un*qualified keys meant where you wanted, but that you couldn't do that with qualified keys in a map. I didn't know about the :req-un
sugar you had their tho (or had forgotten)#2018-07-0215:32guyI mean you still have to s/def the keys right#2018-07-0215:33guy(s/def :quux/bar string?)
and (s/def :foo/bar pos-int?)
for example#2018-07-0215:33guyu canāt just use :req-un
to redefine them#2018-07-0215:33guyiām pretty sure#2018-07-0215:33otfromyeah, but I can do that per domain :domain1/bar
:domain2/bar
#2018-07-0215:33otfrometc#2018-07-0215:33guyye š#2018-07-0215:34guy>That you could basically redefine what the *un*qualified keys meant where you wanted,
Just wasnāt sure what u meant here#2018-07-0215:34otfromwhich is exactly what I want to do here#2018-07-0215:34guycool cool#2018-07-0215:34guyš#2018-07-0215:34otfromfeels like a bit of a cheat, but it does work with what spec affords#2018-07-0215:34otfromand I'm all about the affordances#2018-07-0215:34guyitās in the spec guide if it makes you feel better š#2018-07-0215:34otfromis it?#2018-07-0215:34guyya i believe so#2018-07-0215:35guyhttps://clojure.org/guides/spec#_entity_maps#2018-07-0215:35guyscroll a bit down#2018-07-0215:35guyMuch existing Clojure code does not use maps with namespaced keys and so keys can also specify :req-un and :opt-un for required and optional unqualified keys. These variants specify namespaced keys used to find their specification, but the map only checks for the unqualified version of the keys.
#2018-07-0215:39guy:+1:#2018-07-0215:41otfromthx#2018-07-0318:28fominokHi there! I suppose this is a right place for questions and I have one: is it possible to have references with spec generators? Like this: I have an author spec and a book spec, book has an author_id, can I generate one author and n books with proper id?#2018-07-0318:31noisesmithI think gen/fmap could do that#2018-07-0320:00taylor@fominok you could also use test.check's let
macro like this:
(s/def ::author-id uuid?)
(s/def ::name string?)
(s/def ::author (s/keys :req-un [::author-id ::name]))
(s/def ::book (s/keys :req-un [::author-id ::name]))
(s/def ::books (s/coll-of ::book))
(gen/sample
(gen/let [author (s/gen ::author)
books (s/gen ::books)]
{:author author
:books (map #(assoc % :author-id (:author-id author))
books)}))
which is generating a map of author + books from their individual specs, and just assoc
ing the author's ID over each book#2018-07-0320:10fominokThank you, @taylor #2018-07-0415:13ackerleytnggenerative tests with specs take really long to run. what are some tips to try and reduce the testing time?#2018-07-0415:37taylorThis can depend on your specs. Recursive specs can be especially costly. Can you give an example thatās taking a while to run?#2018-07-0423:22ackerleytngit's not a recursive spec though#2018-07-0417:02seancorfield@ackerleytng I don't feel generative tests should be run automatically as part of your (fast) unit test suite. They can/should be run separately or as part of CI for example. Because generative testing can definitely take a while.#2018-07-0417:24dominicmRich has mentioned a number of times that generative tests should only run when the relevant code changes. I'm sure he'd be very happy if someone built that for him.#2018-07-0418:13Alex Miller (Clojure team)I have built the hard parts of it, just needs a lot of productization#2018-07-0421:11dominicmDo you mean to suggest that it's going to be a product that's sold or branded? Or just that it's rough?
If it's rough, is it in such a state that the community might learn much from it by seeing it and seeing where they can take it?#2018-07-0421:30Alex Miller (Clojure team)It will be a contrib library. Itās not at a state where Iām ready to publish it yet, both for design and impl#2018-07-0421:32Alex Miller (Clojure team)We may end up sharing some code with codeq 2 also, which is similarly unfinished. Just a lot of stuff to shake out and none of us have time to work on either atm#2018-07-0420:29gnl@dominicm Shameless plug ā https://github.com/gnl/ghostwheel runs gen-tests per namespace on hot-reload which is effectively just that (or close to it), with the option to make re-rendering dependent on successful test completion.#2018-07-0421:11dominicmI hadn't appreciated that's why it did those things. Very impressive! #2018-07-0421:28Alex Miller (Clojure team)Thatās great but what Iāve been working on is code analysis at the function dependency level#2018-07-0421:44gnlFunction-level granularity would be even better for this of course ā looking forward to it. Is this going to be a core thing or a third-party library?#2018-07-0421:45gnl@dominicm It's really just a fancy (run-tests)
but it gets the job done. š#2018-07-0421:47gnlThe important thing is that if you're working on layout/view stuff where quick hot-reloading is most valuable, it doesn't run any expensive event handler gen-tests, etc.#2018-07-0421:49gnl@U064X3EF3 Oh, I just saw the other thread where you answered that, nevermind.#2018-07-0423:23ackerleytng@seancorfield Thanks! I'm writing my own custom generators, could that contribute to slower testing?#2018-07-0423:24ackerleytngeven with overrides like {::foo (fn [] (gen/return <something pre-generated>}
does not improve testing speeds much. why would this be?#2018-07-0423:24ackerleytnggen/return
probably doesn't run the generator at all, right?#2018-07-0423:43gfrederickswell it is a generator, but everything it does is trivial, so it by itself should never make anything slow#2018-07-0616:50twsIād like to try and do this without custom generatorsā¦ any ideas? I want to spec out a string of digits, with each having certain restrictions. e.g. each digit is (s/int-in 2 10)
but Iād like 3 in a row catted into a string. using regexes like #(re-matches #"[2-9]{3}" %)
requires me to make a custom generator I believe. Struggling with the syntax.#2018-07-0616:51twsI can get an s/tuple
easy enough, but not sure how to jam that into a string inside the s/def
#2018-07-0617:06twsso I have (s/def ::digit (s/int-in 2 10))
(s/def ::code (s/tuple ::digit ::digit ::digit))
but instead of a tuple/vector I want them jammed into a string#2018-07-0617:08noisesmithI can't answer this question, but as an aside, spec is explicitly not for parsing#2018-07-0617:15twsItās not to parse itās to validate data. (And generate it)#2018-07-0617:19noisesmithwhy not use a regex as your spec validator, and write a custom generator?#2018-07-0617:20guyYou can give a spec a predicate and a generator#2018-07-0617:20twsI could. Just wanted to know if there was a cool way to do without #2018-07-0617:20guyso just make a custom predicate#2018-07-0617:20guylet me find an example 2 secs#2018-07-0617:20guyhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/spec#2018-07-0617:21guyso you can have say#2018-07-0617:21guy(s/def ::digit my-pred-fn my-gen)
#2018-07-0617:21guy(defn my-pred-fn [x] your logic)
#2018-07-0617:22guyif that makes sense?#2018-07-0617:22guybut you need a generator that would then fulfil that predicate#2018-07-0617:22twsYes thanks. Iām familiar with custom generators. It just seems like Iām codifying the same rule in two places (pred and gen) which I would like to avoid if I could. #2018-07-0617:23twsBut Iāll just move ahead with a custom gen. #2018-07-0617:24guy@tws oh sorry just read your first line facepalm#2018-07-0617:24guy>Iād like to try and do this without custom generators#2018-07-0617:24twsCommunication is hard!#2018-07-0617:25guyhaha nah i just need more coffee so i can read better š#2018-07-0713:43ackerleytngwhy does this
(ns foo.bar
(:require [clojure.spec.test.check]))
error out with
Could not locate clojure/spec/test/check__init.class or
clojure/spec/test/check.clj on classpath.
#2018-07-0713:45gfredericksI don't think that's a real ns#2018-07-0713:47ackerleytnghmm there's no file associated with it#2018-07-0713:47ackerleytng(ns clojure.spec.test.alpha
(:refer-clojure :exclude [test])
(:require
[clojure.pprint :as pp]
[clojure.spec.alpha :as s]
[clojure.spec.gen.alpha :as gen]
[clojure.string :as str]))
(in-ns 'clojure.spec.test.check)
(in-ns 'clojure.spec.test.alpha)
(alias 'stc 'clojure.spec.test.check)
#2018-07-0713:47ackerleytngi guess you're right, thanks!#2018-07-0713:47gfredericks:+1:#2018-07-0714:06ackerleytnghow do i use the default-reporter-fn
from test.check
(https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/clojure_test.cljc) with a specific deftest?#2018-07-0714:07gfredericksWhat's your entry point? The stest/check function?#2018-07-0714:09ackerleytngyup#2018-07-0714:09ackerleytngi have a deftest which calls stest/check#2018-07-0714:11gfredericksDoesn't it have an arg for arbitrary extra test.check args?#2018-07-0714:12gfredericksKeep in mind the reporter-fn stuff is only in 0.10.*#2018-07-0714:20ackerleytngit does? i'll look that up#2018-07-0804:52ackerleytngah actually 0.10.0-alpha3
improves testing error messages, it's exactly what i wanted!#2018-07-0811:52gfredericksI'm curious which improvement you're referring to#2018-07-1401:31ackerleytng(deftest generative-tests
(doseq [test-output (stest/check function-under-test {:gen gen-overrides
::stc/opts {:num-tests 10}})]
(testing (-> test-output :sym name)
(is (true? (-> test-output ::stc/ret :result))))))
#2018-07-1401:32ackerleytngI do this, so previously it was just something like expected: true and actual: false#2018-07-1401:32ackerleytngwhich is meaningless#2018-07-1401:32ackerleytngbut now it shows #error and dumps the error from spec
#2018-07-1412:55gfrederickshmm#2018-07-0804:52ackerleytngalso, youre right, it should be :reporter-fn
#2018-07-1113:27roklenarcicdoes calling instrument
with no arguments instrument all symbols in current namespace or all instrumentable symbols in all the loaded namespaces?#2018-07-1113:28taylorall instrumentable vars in all loaded namespaces AFAIK#2018-07-1113:31Alex Miller (Clojure team)yes#2018-07-1210:09Matt ButlerIs there any way to use predicates inside the literal comparator to achieve something like this in spec.
(s/valid? #{["foo" "bar" string?]} ["foo" "bar" "baz"])
#2018-07-1212:10tayloryou could spec that with s/cat
#2018-07-1212:44Alex Miller (Clojure team)Or s/tuple#2018-07-1214:10Matt Butlerso (s/tuple #{"foo"} #{"bar"} string?)
? but if I wanted to combine that with other fixed values where I know all 3 elements, I'd have to use an s/or like so?
(s/or :set #{["a" "b" "c"]} :tuple (s/tuple #{"foo"} #{"bar"} string?))
#2018-07-1214:10Matt ButlerThere is no way to next dynamic values inside the #{}
#2018-07-1214:13Matt ButlerThanks btw š jut wanted to check my understanding.#2018-07-1214:17taylorif I understand the use case, s/or
seems reasonable to me#2018-07-1214:19Matt ButlerYeah I have a set of triple store values, most are known but for some subset the final key is freetext.#2018-07-1215:58akielIs there a way to override key-specs in test like it is possible for fn-specs with instrument
?#2018-07-1216:00ghadisee the second argument to instrument, specifically :overrides :stubs :gen#2018-07-1216:03akielI donāt see :overrides in the doc string of instrument
. I also donāt see a possibility to override key-specs.#2018-07-1216:04akielWith key-spec I mean (s/def ::a int?)
. I like to override ::a
.#2018-07-1216:04ghadisorry, I goofed, that's not an actual config key#2018-07-1216:05ghadihttps://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument
You cannot override key specs. You can only override which specs are bound to particular functions, and which generators are used for particular specs#2018-07-1216:05ghadiOverriding the key spec doesn't really make sense conceptually -- you'd be changing the contract but preserving the same global name#2018-07-1216:10akielI have a function which gets a map of values and I donāt like to generate the real values in test. So I would like to override the specs for the keys in the map. But if I think more about it, why should a supply a map to the function at all? So I can change the fn-spec to take a custom test value instead of a map which custom test values in it. That should work.
Oh I also stub the function.#2018-07-1314:48roklenarcicHow can one spec maps with string keys?#2018-07-1314:51mpenet(s/map-of string? ....) if you want to use s/keys you will need to convert them first#2018-07-1421:26roklenarcicHm I thought this would work:
(alter-var-root #'s/*explain-out* (constantly exp/printer))
#2018-07-1609:31bbrinckIs this at the REPL? If so, youāll want to use āset!ā not ālater-car-rootā#2018-07-1609:33bbrinckSorry, autocorrect on my phone makes talking about code difficult š#2018-07-1609:34bbrinckHereās a gist that shows why alter-var-root doesnāt work as you might expect in this context https://gist.github.com/bhb/d82fcf0f80f555c28afa8db320be16c8#2018-07-1609:36bbrinckAnd hereās a different gist with an example of how Iād recommend setting up expound for instrumentation (if thatās what you are looking to do) https://gist.github.com/bhb/649c46ac6dfa290fa6a62bb96fb66f62#2018-07-1609:38bbrinck@U66G3SGP5 Let me know if you encounter any problems with the above code#2018-07-1611:12roklenarcicI tried set!
before, but it's been my understanding that it only sets var value for current thread#2018-07-1611:18bbrinckHm, Iām not sure what happens in threads started within the REPL - I can check later. You might have to call āset!ā in each thread. But in any case, alter-var-root! wonāt work in the REPL context since the var is bound by Clojure before your code is run, so I think āset!ā is the only solution. #2018-07-1706:59athosA quick workaround for this is to try (alter-var-root #'s/*explain-out* (constantly exp/printer))
in your user.clj
. Clojure runtime should automatically load user.clj
(if it exists) before launching the REPL.#2018-07-1713:17bbrinck@U66G3SGP5 Are the threads being created within the REPL context? If so, it looks like set!
will apply. See this gist. https://gist.github.com/bhb/910f718e2da57793bc0f5817f006f28a#2018-07-1713:18bbrinckIf threads are being created outside the REPL, then yes, I would try what @U0508956F suggested above.#2018-07-1713:19bbrinck@U7PBP4UVA Did you end up getting expound to work with multiple threads?#2018-07-1421:27roklenarcicto make all my stuff print spec errors with expound#2018-07-1711:47marktThink I asked in the wrong channel... too many channels https://clojurians.slack.com/archives/CB19ETU0N/p1531736138000105#2018-07-1711:53mpenetnot really#2018-07-1711:54mpenetmaybe some external lib does, but not spec itself afaik#2018-07-1712:14ikitommi@markt you were using schema-tools.core/select-schema
? By design, Spec doesn't support closed keys
. 3rd party?, there is at least spec-tools.core/select-spec
for this#2018-07-1712:16marktAh thought that would be the case, it's a bit of a odd case we have.
Cool thanks for the heads up, i'll have a look at spec-tools š#2018-07-1712:37ikitommiI think itās the odd case not to strip out extra data at the borders.#2018-07-1712:50marktInteresting, so you would prefer to have a function only return what the spec dictates?#2018-07-1714:14marktJust noticed your the author of the library!
Thanks for writing the library š#2018-07-1715:26ikitomminot at function level, but at system boundaries: Reading JSON sent by a (potentially rogue) third party, or when writing to a (document) database. Not removing extra keys is a security risk, stripping keys manually is both error prone & extra work.#2018-07-1812:25marktabsolutely agree, and thanks for all the help#2018-07-1818:31gklijsIt's not that hard using specs to implement losing the rest. I rather have a bit of code than a dependency. That's the road I took, also added the option to have default values for keys, to keep specs backwards compatible.#2018-07-1912:35ackerleytngMay I trouble someone to explain conformer to me? Don't really understand the docs#2018-07-1913:10claudiuDid you check out https://lambdaisland.com/episodes/clojure-spec ?#2018-07-2413:12ackerleytngthanks! I meant conformeR not just conform#2018-07-2413:34claudiuahh š Also trying to learn this stuff. Stuart used it in the fizzbuzz example, maybe that example helps https://gist.github.com/stuarthalloway/01a2b7233b1285a8b43dfc206ba0036e#2018-07-2016:38ghadihttps://twitter.com/smashthepast/status/1020347051589144578#2018-07-2101:24jkrasnayHi folks, I have a Re-frame app that is nicely checking my function invocations with spec but itās not showing file names / line numbers.#2018-07-2101:25jkrasnayCall to #' did not conform to spec:āµ
<filename missing>:<line number missing>āµ
āµ
-- Spec failed --------------------āµ
āµ
Function argumentsāµ
āµ
(... :foo ... ...)āµ
^^^^āµ
āµ
should satisfyāµ
āµ
string?āµ
āµ
#2018-07-2101:26jkrasnayIs this normal, or should I be able to get a file name and line number here?#2018-07-2102:31seancorfield@jkrasnay Are you using Expound there? That doesn't look like a raw clojure.spec
error... Just wondering if it's a bug in the reporter itself...?#2018-07-2102:56jkrasnayYep, itās expound. Iāll try taking that out.#2018-07-2103:32jkrasnayLooks like Expound has that code there waiting for the day when CLJS will send caller info. For now itās just teasing me.#2018-07-2104:18seancorfield@jkrasnay Good to know. I don't use cljs myself but I know there are all sorts of weird edge cases/differences from clj...#2018-07-2204:32alex-dixonIs it possible to define a spec with a custom generator but have it separate from the spec itself? Something like
(s/def ::my-spec (s/or :keyword keyword? :symbol symbol?))
(s/with-gen ::my-spec (gen/symbol))
Seems like it could be looking at the defn for s/with-gen
but not working at the REPL atmā¦#2018-07-2204:41athosYou can replace spec generator impls with s/gen
's overrides
argument or via :gen
option of instrument
and check
.#2018-07-2204:44athosA library of mine called Genman also does that well https://github.com/athos/genman
If you're instested, give it a shot š#2018-07-2205:19alex-dixonWorks!
(gen/generate
(s/gen ::my-spec
{::my-spec gen/symbol}))
#2018-07-2205:19alex-dixon@athos Youāre awesome. Thank you š#2018-07-2311:58roklenarcicI have a problem with clojure.java.jdbc.spec
, namely when calling query
fn which is instrumented with this spec:
(s/def ::result-set-fn (s/fspec :args (s/cat :rs (s/coll-of any?))
:ret any?))
It calls my result set fn with a bunch of garbage#2018-07-2311:58roklenarcicSo when callbacks have a fspec defined, they get called with random data if they are instrumented?#2018-07-2312:03gfredericksyep#2018-07-2312:03gfredericksthat's how the function is validated#2018-07-2312:03gfredericksit is a common source of surprise#2018-07-2312:29roklenarcicI guess i need to instrument just a specific ns#2018-07-2312:34mpenetthere are options also on instrument you can use to bypass this#2018-07-2312:34mpenetwell, to provide stubs at least#2018-07-2316:30fedregIs there a way (or another function) to have spec/explain
return just the spec errors and not the original data as well?#2018-07-2316:37samueldev@fedreg why not just pull the errors out of the result of spec/explain
? :shrug:#2018-07-2316:37samueldevor create your own fn explain-errors
that does that and use it instead :shrug:#2018-07-2316:39fedreg@samueldev I was about to do just that but then decided to check if I had missed something that already existed. Guess thatās a noā¦ thx!#2018-07-2317:02seancorfield@roklenarcic Yeah, I may revisit some of those fspec
s in clojure.java.jdbc.spec
as I get more feedback from people. As @gfredericks says, the behavior you're seeing is what clojure.spec
instrumentation is intended to do, but it can be surprising at first. And of course your specific :result-set-fn
may well not accept any old collection -- but given that :row-fn
can transform the result set data into, essentially, an arbitrary collection of any type of element, the :result-set-fn
could potentially be called on any?
types... We don't have parameterized types so we can't constrain the row function to map? -> T
and then the result set function to coll-of T -> U
...#2018-07-2319:38manutter51This surprised me 1. In a CLJS namespace, define a function with a name that matches a valid HTML tag (e.g. "table" or "br" or whatever)
2. Create an (s/fdef ...) spec for it.
Result: every function in that namespace now evaluates to nil.
#2018-07-2319:38manutter51Just a local fluke in my current project, or something that other people can reproduce?#2018-07-2319:39manutter51If the latter, known issue, or new?#2018-07-2407:49claudiuThere seem to be a lot of nice tools/libs like expound, ghostwheel, orchestra, phrase etc... Is there an official/unoficial list with most of these like tools.deps wiki š ?#2018-07-2408:27cvichttps://www.clojure-toolbox.com was a good list, but it's not updated anymore.
Same goes with
https://github.com/mbuczko/awesome-clojure
https://github.com/razum2um/awesome-clojure#2018-07-2408:28cvicMaybe
https://crossclj.info#2018-07-2409:27claudiuCool, but still just partials in multiple sources š Was banging my head with tools.deps stuff and noticed that there was a really nice page. seems to be pretty complete. Was hoping for something similar for spec. https://github.com/clojure/tools.deps.alpha/wiki/Tools#2018-07-2409:28cvicYeah. In an ideal world you would find all of these on https://clojure.org
Work in progress (I hope).#2018-07-2415:02avi@U064X3EF3 I tried to add a Tools page to the wiki in this repo:
https://github.com/clojure/spec.alpha
but canāt navigate to the wikiā¦ I just get redirected back to the home page of the repo. Never seen that before. Any idea whether itās a repo setting problem or a bug in GitHub maybe?#2018-07-2415:25eggsyntaxhttps://cljdoc.xyz/
is trying to become the new center for documentation of clojure/script libraries. But doesn't especially help with finding good libs.
I know https://www.clojure-toolbox.com/ was updated with at least one lib recently, so it's not abandoned.#2018-07-2419:25Alex Miller (Clojure team)Itās a repo setting. Happy to change it, but Stu and I are both on vacation atm. Ping me again next week or send me an email or Iāll forget.#2018-07-3120:43avi@U064X3EF3 ping! āļø#2018-08-0217:56Alex Miller (Clojure team)hey, fyi email is always better than dms as it can stick around in my inbox rather than disappear#2018-08-0217:57Alex Miller (Clojure team)I donāt have admin access to that repo#2018-08-0217:57Alex Miller (Clojure team)Stu does, but he is still out#2018-08-0217:57aviah ok cool#2018-08-0217:57aviwill do ā thanks!#2018-07-2416:42martinklepschIs there a reason spec doesnāt have itās own project in JIRA?#2018-07-2416:42martinklepschI assume itās mostly historical?#2018-07-2419:24Alex Miller (Clojure team)Because spec and the core specs are tied into Clojure, we manage them in the context of Clojure even though itās a different repo. That was an intentional choice. It may not always remain so.#2018-07-2419:34claudiuIn the next release will spec graduate from alpha
? #2018-07-2419:36Alex Miller (Clojure team)I hope so, no promises#2018-07-2512:47jumarLet's say I have a strange function that has two arities:
1-arity computes some results and returns them
2-arity uses 1-arity to compute the same results and save the output to file.
How can I write :ret
spec for this?
I tried :fn
but struggling with reusing an existing spec
:fn (s/or :results (s/and #(= 1 (-> % :args count))
::result-spec)
:ouput-in-file (s/and #(= 2 (-> % :args count))
#(-> % :ret empty?))))
Here, the ::result-spec
isn't properly used - I need to apply it only to the :ret
key value, not the whole map passed to :fn
.#2018-07-2512:55gfredericksif it were me I'd have two different functions#2018-07-2512:57jumarThat was my idea too, but let's say it's a "convention" to do it this way. Can I still somehow reuse the ::result-spec
or is there a better way to write :ret
spec for this function?#2018-07-2520:34hiredman(s/def ::ret ::result-spec)
(s/and ... (s/keys :req-un [::ret]))
#2018-07-2708:12jumarNice, thanks! I guess the only problem would be if I had more than one such function in the same ns (conflicting ::ret
specs) but that's not an issue in my case.#2018-07-2618:40EthanHello, I'm working on creating specs and I would like to use ::bindings from the core.specs.alpha in the alpha.clj file in my code. I was wondering how I would go about doing that without all copying the code to my file.#2018-07-2618:54Alex Miller (Clojure team)Just refer to the full spec keyword#2018-07-2618:54Alex Miller (Clojure team):clojure.core.specs.alpha/bindings#2018-07-2619:00Ethanok, thank you#2018-07-2713:21burbmaCan anyone provide insight as to whatās going on here?#2018-07-2713:42jumar(type (resolve 'string?))
;;=> clojure.lang.Var
user> (type string?)
;;=> clojure.core$string_QMARK___5132
#2018-07-2713:44guyIs resolve a spec function?#2018-07-2713:44guyš¤#2018-07-2713:44guyIāve not seen it before#2018-07-2713:47burbmaresolve isnāt a spec function itās in core. Suppose I started with "string?"
, how can I get to the clojure.core$string_QMARK_...
? (resolve (symbol "string?))
wonāt do it as weāve seen.#2018-07-2713:48guyreally? ill check it out thanks!#2018-07-2713:49guyohh right got ya#2018-07-2713:49guyneat#2018-07-2713:49guyTIL;!#2018-07-2713:50guyif u want to see what it does, you can generally just try s/exercise
and see what it produces#2018-07-2713:51burbmas/exercise
gives the same error for the (resolve (symbol "string?"))
version of the spec.#2018-07-2713:56guyYeah i mean just for (s/exercise string?)
#2018-07-2713:56guysorry#2018-07-2713:57burbmaAh, no worries. Thanks for the pointer.#2018-07-2714:01burbmaUsing (var-get (resolve (symbol "string?")))
does the trick.#2018-07-2716:39noisesmithor deref, or the shorthand @
#2018-07-2715:49dspiteselfAny idea when we will get this fix https://dev.clojure.org/jira/browse/CLJ-2003 in? We are still running a patched version.#2018-07-2715:57Alex Miller (Clojure team)In queue to get screened#2018-07-2716:46dspiteselfThanks#2018-07-2718:03aviLast week at clojure/nyc I shared my experience, as a spec n00b, using spec to specify other peopleās data structures ā¦ hereās the video if anyoneās interested: https://www.youtube.com/watch?v=eqfSifXaXnw#2018-07-2718:04aviAnd for those in the Boston area Iāll be sharing this again at the Boston Clojure Group on 9 August: https://www.meetup.com/Boston-Clojure-Group/events/tjztcpyxlbmb/#2018-07-2718:11fabraoHello all, how
(defn ^:private menor? [n t]
(<= (count n) t))
(s/def ::limite-tamanho menor?)
(s/valid? ::limite-tamanho <what is in here?>)
#2018-07-2718:11fabrao?#2018-07-2718:13noisesmithI don't understand how your spec predicate could be a function of two args#2018-07-2718:13noisesmithperhaps (partial menor? some-n)
?#2018-07-2718:14fabraowell, itĀ“s for dynamic size#2018-07-2718:14noisesmithright, but how can you use that directly as a spec predicate? where would the other arg come from?#2018-07-2718:28fabrao@noisesmith IĀ“ll check many specs from vector, like fields in form -> [[:code "Error code" ::limite-tamanho 6] [:name "Error name" ::limite-tamanho 15]]
#2018-07-2718:32noisesmithmaybe I'm missing some spec feature that would make this work, but I still don't understand how you'd make spec supply two args to your spec#2018-07-2718:32noisesmithand as far as I know you can't use a neighbor or parent data in the check for a key#2018-07-2718:33fabraoWell, IĀ“m thinking use like this so, [:codigo "CĆ³digo do vendedor" #(<= (count %) 6)]
#2018-07-2718:34noisesmithwhich is equivalent to what I proposed with partial above, right?#2018-07-2718:34fabraoyes, thanks a lot#2018-07-2803:00burbmaHow can I get the forms to evaluate before going into the macro?#2018-07-2806:41valeraukonot sure if this is what you're looking for, but i have something like this
(def context-kw (keyword "ap.object" "@context"))
(eval `(s/def ~context-kw map?))
#2018-07-3017:33djtangoSo I am pretty sure this is absolutely not how spec was intended to be used, but has anyone gotten mileage from leaving stest/instrument
turned on in production to use spec for runtime contract assertion? Even more so with jeaye/orchestra#2018-07-3017:33djtangoOne reason I can think of why this may not be such a great idea is that stest/instrument
validates fspec
using generated inputs#2018-07-3017:35djtangoare there any other reasons not to leave stest/instrument
turned on?#2018-07-3017:48seancorfield@djtango The generative validation of fspec
is one reason. Performance is another. You'll also get bare AssertionError exceptions instead of anything meaningful you might normally catch and handle (assuming you try to (catch Exception e ...)
)#2018-07-3017:59noisesmithyeah, my understanding is that AssertionError is for things you expect to throw during development - the vm even has a flag to ignore them#2018-07-3018:02djtangoI suppose, to give more context on my usecase, it is useful (for me) to declare some invariants about a function (e.g. a relationship between it's args and return value) but in a sufficiently large project, it's possible for inputs in runtime to fail that invariant and it's nicer when that failure comes at the point of the invariant failing rather than some arbitrary of steps later like a null pointer error#2018-07-3018:04djtangospec happens to be a great way for declaring properties about lots of things, and while generative testing is meant to give better guarantees about over your testing space, the associated complexity with them is a harder sell for the team#2018-07-3018:04djtangoIf that makes sense?#2018-07-3018:05djtangoComing from Racket, I'm kind of used to contracts always being on#2018-07-3018:24noisesmiththe pragmatic thing might just be overriding fspecs for the instrumenting#2018-07-3021:57nopromptitād be great if there were a spec-impl that āmergedā two s/or
s or s/alts
without having to reify
the protocols (which is a dangerous game as i understand it).#2018-07-3022:01nopromptfor example, if you want to express that clojure.core/unquote-splicing
forms are not permitted at the top level youād have a union for your legal top level forms and then another union for your subforms in, say, a list that includes unquote-splicing forms in addition to what is legal at the top level.#2018-07-3022:02nopromptto fully leverage all of spec, including explanations, it seems like i have to spell this out entirely in both cases if i want to avoid an additional layer of rettags.#2018-07-3022:05seancorfield@noprompt Or wrap s/or
in s/nonconforming
I guess...?#2018-07-3022:07noprompt@seancorfield thatās not quite the ticket. lemme demonstrate.#2018-07-3022:08noprompt(s/def :clj.form/top-level
(s/or :list :clj.form/list
:number number?))
(s/def :clj.form/list
(s/coll-of
(s/or :a :clj.form/top-level
:b (s/or :unquote-splicing unquote-splicing-form?))
:kind list?))
(s/conform :clj.form/top-level '(1
#2018-07-3022:09nopromptwhat i do not want is the :a
and :b
tags. i still want to retain the :number
and :unquote-splicing
tags.#2018-07-3022:09nopromptby using s/nonconforming
i lose the tags. š#2018-07-3022:10nopromptessentially, i want to extend the :clj.form/top-level
union without adding an additional layer of rettags.#2018-07-3022:10nopromptof course, i can do this by using s/with-gen
, s/conformer
, etc. but i lose out on s/explain-data
.#2018-07-3022:12nopromptif there was an s/explainer
thisād be a done deal. š#2018-07-3022:13nopromptin short, i think what i want is an or-spec-impl
or an alt-spec-impl
that does not require tagging the options.#2018-07-3022:14nopromptwhen it conforms returns whatever the conformers itās compose of returns.#2018-07-3022:15noprompti think there was previous discussion along same lines but for s/cat
with an s/concat
conformer..#2018-07-3022:29seancorfieldAlex has talked several times about the possibility of adding a specific non-conforming or
variant so you'd have that sort of level of control...#2018-07-3022:32arohnerI have a big complicated s/keys, and Iām getting "Couldn't satisfy such-that predicate after 100 tries."
. How do I discover which predicate is failing?#2018-07-3022:58gfredericksif you're using a new enough test.check, the ex-data should have the generator and predicate attached, at least#2018-07-3118:15lilactownhow do I spec a function that takes only one argument?#2018-07-3118:27seancorfield(s/fdef my-func :args (s/cat :arg ::spec))
(to be more specific)#2018-07-3118:29seancorfield@lilactown Here's an example from our codebase at work (s/fdef me
:args (s/cat :access-token string?)
:ret (s/nilable (s/keys :req-un [::id ::name])))
#2018-07-3120:48lilactownhow are people enabling instrumentation in development?#2018-07-3120:49lilactownI'm using CLJS. I'm thinking of doing something like:
(when js/goog.DEBUG
(stest/instrument))
but I'm worried that clojure.spec.test.alpha
is going to end up in my release bundle#2018-07-3121:04justinleelet me know if you figure out the right solution š#2018-07-3121:55seancorfield(we use clojure.spec
in production code for validation and conformance so I'm puzzled as to why you'd want to avoid it? doesn't cljs use tree-shaking anyway to remove unused code?)#2018-07-3121:57lilactownvalidating and conforming can be done without including clojure.spec.test#2018-07-3121:57lilactownright?#2018-07-3121:57lilactownand I'm not sure if tree-shaking is smart enough to completely eliminate the lib if it's not used in a release build. will have to test#2018-07-3121:57lilactownI have a second stupid question: is there an easy way to alias a spec?#2018-07-3121:58lilactowne.g.
(s/def ::my-spec <point to ::other-spec>)
#2018-07-3121:59hiredmanspec validates functions by generating example arguements and calling the function on the examples, which is not something you'll want to do in production#2018-07-3121:59hiredmanif you are strictly validating datastructures that should be fine#2018-07-3122:00lilactownright. I know I definitely don't want to instrument my fdefs in prod š#2018-07-3122:09aviš¤ I donāt think instrumentation involves generators ā¦ not saying instrument
is appropriate for runtime, just might be helpful to keep things clearā¦#2018-07-3122:17noisesmithdoesn't it, for the case where the arg is specced to be a function, it would be checked by exercising it#2018-07-3122:18noisesmith(let me know if I misunderstand something here...)#2018-08-0113:51avi:man-shrugging: I dunno, I think I spoke beyond my level of understanding ā sorry#2018-08-0113:52aviyour example is fascinating!#2018-08-0113:53avithanks!#2018-07-3122:31noisesmithuser=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (s/fdef bar :args (s/cat :f (s/fspec :args (s/cat :x int?) :ret int?)) :ret int?)
user/bar
user=> (defn bar [f] (f 0))
#'user/bar
user=> (stest/instrument `bar)
[user/bar]
user=> (bar (fn [x] (println "called with arg" x) x))
called with arg 0
called with arg 0
called with arg 0
called with arg 0
called with arg 1
called with arg 2
called with arg -1
called with arg 0
called with arg -2
called with arg -24
called with arg 3
called with arg -2
called with arg 9
called with arg -1
called with arg -757
called with arg -1
called with arg 3
called with arg 1963
called with arg 1165
called with arg -359
called with arg -3956
called with arg 0
0
#2018-07-3122:31noisesmithexample of the generation behavior with instrument#2018-08-0104:48tianshuHi, how can I write a spec for arguments like [name & opts]
, opts is a hash-map here, I already have a ::opts
spec for a hash-map, what a spec looks like for this function?#2018-08-0104:56tianshuIt's likely I can use (s/cat :name ::name :opts (s/keys* :req-un [...]))
#2018-08-0104:58tianshubut If I do (s/def ::opts (s/keys* ...))
, I can't use ::opts
here. It requires me to provide a vector#2018-08-0105:00tianshuoh, my mistake, if I use s/keys*
it's ok, but can't use (s/and (s/keys* ...) (...))
.#2018-08-0207:18athos(s/& (s/keys* ...) (...))
doesnāt work?#2018-08-0117:08arohnerIf I have a multi-spec, whatās the best way to get an instance of a single dispatch value? Occasionally (-> (s/spec ::foo) (s/gen) (gen/such-that [f] (= :bar (:type f))
will ācouldnāt satisfy such-thatā#2018-08-0117:09arohnerbut if I call the multimethod directly, it can return nonsense values, because (defmethod :foo [_] (s/keys :req-un [::type]))
, the :type
doesnāt necessarily match :foo
#2018-08-0204:28andy.fingerhut@noisesmith Do you know why instrument of function foo causes a function arg of foo to be called additional times with arguments that the intrumented call is not itself calling? I can understand doing that if you were doing generative testing on foo, but don't see the rationale for why one would want to do that for instrument.#2018-08-0204:35noisesmithMy understanding is that's the only way spec has to validate a function argument#2018-08-0209:38mpenetIt could validate it at invoke time, just once. #2018-08-0209:39mpenetAt least in theory. A few people suggested that on jira among other places. I can imagine that being an option#2018-08-0204:49andy.fingerhutMakes sense. 20 times may be more than needed for that purpose, but probably tunable.#2018-08-0316:34kvltSo question:
I have the following...
(s/def ::name string?)
(s/def ::age (s/and integer? pos? #(< % 100)))
(s/def ::occupation #{:engineer :mechanic :manager :clerk})
(s/def ::person (s/keys :req [::name ::age ::occupation]))
(s/fdef do-it
:args (s/cat :people (s/coll-of ::person))
:ret (s/coll-of ::age))
(defn do-it
[people]
(map ::age people))
Running this does not work:
(stest/check `do-it)
=>ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4739)
However, I can exercise the fn:
(s/exercise-fn `do-it)
([([#:gentest.core{:name "", :age 1, :occupation :manager}]) (1)]
[([#:gentest.core{:name "A", :age 15, :occupation :engineer}]) (15)]
[([#:gentest.core{:name "", :age 3, :occupation :manager}]) (3)]
[([#:gentest.core{:name "nA0", :age 3, :occupation :clerk}]) (3)]
[([#:gentest.core{:name "1ZJO", :age 3, :occupation :engineer}]) (3)]
[([#:gentest.core{:name "80v", :age 20, :occupation :mechanic}]) (20)]
[([#:gentest.core{:name "M", :age 98, :occupation :engineer}]) (98)]
[([#:gentest.core{:name "", :age 2, :occupation :engineer}]) (2)]
[([#:gentest.core{:name "drn9G", :age 59, :occupation :mechanic}]) (59)]
[([#:gentest.core{:name "2p", :age 72, :occupation :clerk}]) (72)])
Why is this?#2018-08-0316:49favila(s/and integer? pos? #(< % 100)))
#2018-08-0316:50favilathis value space is too large#2018-08-0316:50favilait can't create an efficient generator#2018-08-0319:24seancorfield@petr (s/def ::age (s/int-in 1 100))
should solve that.#2018-08-0319:31favilaExceptionInfo Couldn't satisfy such-that predicate after 100 tries.
almost always means there's an s/and with values generated from earlier predicates could not satisfy later predicates#2018-08-0319:31favilavery often you need a custom generator#2018-08-0319:32favilabtw it would be really nice to somehow have a generator associated with a predicate#2018-08-0319:33favilais there some trick to this? I end up manually using s/with-gen everywhere I use a predicate (and having to remember to do that)#2018-08-0405:03kvltHey guys, sorry for the slow reply. I had ended up with this:
(s/def ::age (s/with-gen (s/and integer? pos? #(< % 100)) (clojure.spec.gen.alpha/large-integer* {:min 0 :max 100})))
#2018-08-0405:40seancorfield@petr Did you see my suggestion of using s/int-in
?#2018-08-0405:42seancorfielduser=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::age (s/int-in 1 100))
:user/age
user=> (s/exercise ::age)
([1 1] [1 1] [2 2] [2 2] [1 1] [6 6] [5 5] [1 1] [16 16] [7 7])
user=> (s/exercise ::age)
([1 1] [2 2] [1 1] [3 3] [8 8] [2 2] [2 2] [9 9] [2 2] [69 69])
user=>
#2018-08-0514:23kvltI did thanks!#2018-08-0406:45mishawhat is the practical purpose of s/keyās :opt
and :opt-un
?
- documentation(?)
- include :opt keys during s/exercise
- anything else?
does s/conform conforms all known keys in a map, or just listed in :req/:opt/-un?#2018-08-0407:23valeraukoin my understanding it's that the presence of the key is optional, but if it's present, its shape has to fulfill its spec#2018-08-0407:48guy^#2018-08-0407:49guyThats what ive used it for before#2018-08-0408:20misha@vale s/keys checks against all the known key-specs, you dont have to list them in the s/keys. empty s/keys will test all the registered keys#2018-08-0408:21valeraukoare you sure about that?#2018-08-0408:21mishayes#2018-08-0408:22mishathis is one of the goals or benefits of having global specs registry and namespaced keywords#2018-08-0408:24misha(s/def :foo/bar int?)
=> :foo/bar
(s/valid? (s/keys) {:foo/bar 1})
=> true
(s/valid? (s/keys) {:foo/bar "1"})
=> false
#2018-08-0408:26valeraukoi see... well it's still meaningful for the un
ones where you have to be explicit#2018-08-0408:26valeraukootherwise the opt
is probably just for what you said#2018-08-0408:27mishatrue#2018-08-0408:27mishaanything else?#2018-08-0408:37guy>you dont have to list them in the s/keys.
I thought part of the reason u list them in s/keys is to show easily what the shape of the data is?#2018-08-0408:37guyusing (s/keys) like you did seems to be interesting, iāve never seen that before#2018-08-0408:39misha@guy āshowā is documentation + exercise I mentioned above#2018-08-0408:39guyoh right sorry#2018-08-0408:39guyš#2018-08-0408:39guyyeah i think uve covered it imo#2018-08-0408:39mishaIād not use empty s/keys, it is just an illustration#2018-08-0413:23ackerleytngdoes anyone have a good way of writing specs for ring middleware?#2018-08-0413:24ackerleytngfor every middleware function along the chain, the inputs and outputs change#2018-08-0413:24ackerleytngseems tedious to spec all the middleware functions#2018-08-0413:36gfredericksyou shouldn't have to fully describe the req/resp at each point though; you could describe just what's pertinent to that piece#2018-08-0413:39ackerleytngdo you mean like... if this middleware augments the request with a new key value pair#2018-08-0413:39ackerleytngthen perhaps the :args
request should be just map?
#2018-08-0413:40ackerleytngand the :ret
is like a function with :args
including just the new key-value pair?#2018-08-0413:40ackerleytngsort of like only writing spec for the change in this particular middleware?#2018-08-0414:00gfredericksyeah#2018-08-0414:01gfredericksyou could also write a :fn
saying everything else gets passed through#2018-08-0423:01ackerleytngthanks!#2018-08-0500:23ackerleytngWhat's an example of a well or appropriately specced codebase?#2018-08-0500:26taylorI donāt think thereās only one correct approach but hereās a popular lib with specs https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2018-08-0500:58seancorfieldNote: that deliberately puts all the specs in a separate namespace so that they are optional and clojure.java.jdbc
can still be used with earlier versions of Clojure.#2018-08-0500:59seancorfieldAlso, if you instrument that library, there's a noticeable overhead in performance (running the test suite takes much, much longer) and that's partly due to the complexity of some of the specs. I have a ticket open to look into optimizing them.#2018-08-0501:03seancorfieldAnother point to consider: those specs focus on functions. Spec is amazing with data structures -- and that's mostly how we use it at work: to specify data structures (including API parameters) and to validate values against those specs (with some coercions). We also use specs for tests, but that's less of our focus.#2018-08-0511:53hmaurerSo you use specs to validate inputs from APIs?#2018-08-0600:20seancorfieldYes, we have specs for all our (public/external) APIs and then s/conform
the data on entry to the API. If that is s/invalid?
then we use s/explain-data
to construct appropriate error codes and messages for the client/caller, otherwise we go forward with the conformed data.#2018-08-0600:23seancorfieldOur specs include some coercion, since API input data can be either strings or "values" -- so we conform strings to values (and pass values through). So true
is acceptable where we expect a Boolean, but so is "true"
and that is conformed to true
. Same with long, double, and date values. We have custom generators that produce strings (mostly they just use the "expected" value spec as a generator and then call fmap str
on the result).#2018-08-0610:48hmaurer@U04V70XH6 thanks for the detailed reply! considering that specs are arbitrary predicates, do you find it easy to construct human-readable error messages?#2018-08-0610:48hmaurer(I am aware of expound)#2018-08-0616:04seancorfieldIt took a while to develop heuristics for walking the explain data but, given our domain and the specific set of specs and predicates in use, it isn't too bad.#2018-08-0501:54ackerleytngi think i've a vague idea of what you mean by focus on functions, could you elaborate on what you mean by focus on functions vs focus on data structures?#2018-08-0501:54ackerleytngalso, whats the difference between s/nilable
and s/?
?#2018-08-0504:26seancorfield@ackerleytng Sorry, I was off having dinner... s/nilable
wraps a spec or predicate and produces a new spec that allows a value to be nil
or conform to the wrapped spec.#2018-08-0504:27seancorfields/?
is a regex spec (in an s/cat
sequence) for something that is optional.#2018-08-0504:27seancorfieldSo for a data structure (or argument list) that may have two or three elements, the third one is optional and would be specified with s/?
.#2018-08-0504:28seancorfieldfor an element that could be nil
or a string, you would use (s/nilable string?)
-- it's a value either way, it's just allowed to be nil
or else conform to string?
.#2018-08-0508:06ackerleytngThanks! So it's like... Elements can be nothing, nil or something. s/nilable
allows the element to be either nil or something, but s/?
allows the element to not be there at all.#2018-08-0504:28seancorfieldMake sense?#2018-08-0504:30seancorfieldAs for focus on function vs data, it's about whether you're spec'ing function arguments (and return values and the relationship between them), vs whether you're using s/def
to spec entities that are part of a data structure (and the data structures themselves). Not sure how to explain it beyond that. Does that help @ackerleytng?#2018-08-0508:02ackerleytngBut without spec'ing functions, there's no way to trigger checks unless you manually call functions and use s/conform
on the outputs right? So we're better off spec'ing the functions then running check in a deftest
#2018-08-0508:04ackerleytngAs in, I thought whether or not you spec functions, you would have to spec the data going in and out of functions#2018-08-0508:04ackerleytngSince you kind of want to exercise the specs in some way, it makes sense to just spec the functions to run tests, does that make sense?#2018-08-0600:24seancorfieldWe call s/conform
explicitly because we're using spec in production code. And we conform just the data (arguments) we want to, whereas instrument
checks all of them -- so you have to write specs for all the arguments.#2018-08-0600:26seancorfieldSpec'ing functions and spec'ing data are very different approaches. Typically the former is for dev/test since the overhead of instrument
can be high -- and it will use generative testing on fspec
function arguments. The latter, however, is a great way to do data validation in production code.#2018-08-0823:47ackerleytngI see, thanks! So if i'm using spec in production, then s/conform
is the way to go, and if i'm using spec for testing, then fspec
is a good idea#2018-08-0520:20ikitommicould the Spec
s have a concise toString representation? example with expound:
-- Spec failed --------------------
{:coercion ...,
:middleware ...,
:summary ...,
:swagger ...,
:parameters {:body ..., :header #object[clojure.spec.alpha$map_spec_impl$reify__1931 0x5ac022 "
#2018-08-0520:24ikitommiā¦ or print-method
implemented.#2018-08-0613:31ben.mumfordhi spec, i have two types i want to spec that have some fields in common. is it possible to derive specs from a common base spec (like a superclass in java)?#2018-08-0613:31ben.mumfordin schema i could merge the base schema with new stuff.#2018-08-0613:46ben.mumfordnvm sorted it using a combination of s/and s/keys#2018-08-0614:32favilas/merge is a thing also#2018-08-0616:44Alex Miller (Clojure team)s/merge is better for this#2018-08-0620:08hmaurer@U064X3EF3 @U09R86PA4 I was wondering a couple of months back if there would be a way to do this using derived keywords. i.e. with two specs :foo/a
and :foo/b
and the relation :foo/b derives :foo/a
, enforce that :foo/b
has to include :foo/a
ās spec (or pass both, equivalently)#2018-08-0620:17favila(s/def :foo/b (s/and :foo/a EXTRA-STUFF))
#2018-08-0620:18favilaif :foo/b and :foo/a are both map specs, s/merge is better because it can intelligently combine the key's generators#2018-08-0620:19favilas/and of two map specs likely cannot create a usable generator#2018-08-0711:43Alex Miller (Clojure team)There is no support for keyword hierarchies in spec, sorry#2018-08-0712:14hmaurer@U064X3EF3 out of curiosity, is that a design choice? or just a ānot for nowā kind of thing?#2018-08-0712:34Alex Miller (Clojure team)Weāve never talked about it#2018-08-0712:35Alex Miller (Clojure team)So, neither :)#2018-08-0712:01scaturrIs there a conventional way to describe a spec as an argument? Like if I have a function that takes a spec? I see s/spec?
- but that doesnāt seem to work with something like (s/spec? ::my-spec)
#2018-08-0712:36Alex Miller (Clojure team)There is no single predicate for it right now#2018-08-0718:57hmaurero/. Is there a reason why s/def
does not accept an optional docstring?#2018-08-0719:00favilaProbably an accident of implementation; A spec is put in a global registry atom, which contains a hashmap from key/symbol to spec objects. There's no clear place to put metadata like there is with vars.#2018-08-0719:01hmaurer@U09R86PA4 isnāt it possible to attach metadata to the keyword representing the spec?#2018-08-0719:01favilayou cannot attach metadata to keywords#2018-08-0719:02hmaurer@U09R86PA4 ah š#2018-08-0719:02favila(with-meta :mykw {:abc 123})
ClassCastException clojure.lang.Keyword cannot be cast to clojure.lang.IObj clojure.core/with-meta--5142 (core.clj:217)
#2018-08-0719:02hmaureritās really unfortunate; being able to attach doc to specs in this way would be really neat#2018-08-0719:02hmaurer(yep I just tried that :()#2018-08-0719:02favilathey could do something, but it wouldn't be the normal var thing#2018-08-0719:02favilathat's all#2018-08-0719:02hmaurerespecially since the doc
function is already able to pull specs from the registry and show the predicate#2018-08-0719:03hmaurerit would be super nice if it could also show an extra description#2018-08-0719:03hmaureruseful in cases where the predicate doesnāt speak for itself#2018-08-0719:04hmaurerI guess the implementation doesnāt really matter; it could be an extra global hashmap from keywords to strings#2018-08-0718:57hmaurer@alexmiller I found this but it hasnāt moved in a while: https://dev.clojure.org/jira/browse/CLJ-1965#2018-08-0719:05Alex Miller (Clojure team)Itās the most voted issue in jira and we are def on board with fixing it#2018-08-0719:06Alex Miller (Clojure team)But there are impl factors meaning it will be a while before itās implemented#2018-08-0914:46trissIs there an easy way to flush the clojure spec library? Iām refactoring and am never sure Iāve got ll my specs right until I restart clojure.#2018-08-0914:50favilareset! the registry to empty then re-evaluate all namespaces with specs in them?#2018-08-0914:52trissah ok. Just reset an atomā¦ but where is the registry atom?#2018-08-0915:35favila(reset! @#'clojure.spec.alpha/registry-ref {})
?#2018-08-0915:36favilait's private so you do need to do some poking#2018-08-1000:07ackerleytngWhat's a good way to kind of namespace specs internal to a file? So the issue is that s/keys
needs to have a spec named, say foo
#2018-08-1000:09ackerleytngSo I would do ::foo
. However, my function returns a modified map, hence the :ret
also has to be something that has s/keys
with ::foo
#2018-08-1000:09ackerleytngAnd many functions in that file return modified maps#2018-08-1000:10ackerleytngSo I kind of want to namespace the spec for this function#2018-08-1000:11ackerleytngI'm now doing :function-name-pre/foo
#2018-08-1000:11ackerleytngAnyone has a better solution?#2018-08-1000:12taylordo you need qualified keys in the map? or are you going to have multiple */foo
keys?#2018-08-1000:12taylorif not, why not use unqualified keys?#2018-08-1000:13tayloror maybe I misunderstand the question#2018-08-1000:15ackerleytngThe s/keys
already spec it as unqualified#2018-08-1000:15ackerleytngBecause of that#2018-08-1000:15ackerleytngThe actual key has to have a name that matches the key in the map that is specced#2018-08-1000:17taylorsorry, not sure I understand what the question is then#2018-08-1000:25ackerleytngThanks! Haha#2018-08-1000:26ackerleytngMaybe in short it's "how do you group specs for a function?"#2018-08-1000:37seancorfield@ackerleytng I'm not sure I understand the problem you think you have... why does :ret
come into this and cause you a problem?#2018-08-1000:37seancorfieldPerhaps if you shared a concrete example?#2018-08-1000:38ackerleytngSure hmm I'll work one out and post it later#2018-08-1017:01bjais ::s/invalid
part of the public api?#2018-08-1017:02seancorfieldI don't believe so but there's a function s/invalid?
you can use to test for that value.#2018-08-1017:03seancorfield(that said, I've written conforming predicates that return either an updated value or ::s/invalid
a few times so...)#2018-08-1017:18dadairI have a spec that checks that one of the keys is a core.async channel; Iād like to write a custom generator for that specific key spec, but I canāt seem to determine how to write a generator that returns a channel; I feel like Iām missing something obvious here#2018-08-1018:12taylorcan you use https://clojure.github.io/test.check/clojure.test.check.generators.html#var-return#2018-08-1018:18dadairHmm that would probably work! Thanks!#2018-08-1018:10bjathanks @seancorfield#2018-08-1020:17Alex Miller (Clojure team)yes, ::s/invalid is part of the public api @bja @seancorfield#2018-08-1021:04seancorfieldThanks for the clarification @alexmiller!#2018-08-1023:47bbrinckIāve created a new channel for expound-related questions - #expound#2018-08-1103:41myguidingstarthis spec used to work in 0.1.x (s/explain (s/and map? (s/+ (s/or :pair (s/cat :key keyword? :value boolean?)))) {:foo true})
but with 0.2.168 it fails predicate: (or (nil? %) (sequential? %))
#2018-08-1103:44myguidingstarI use that spec to conform some hash-map's key/value#2018-08-1103:47myguidingstarbut it looks like new clojure.spec use sequential? for s/+ therefore doesn't allow hash-maps#2018-08-1103:48myguidingstarwhat should I do?#2018-08-1103:52myguidingstarthat spec is actually used to describe a flexible dsl composed of vectors and hash-maps (the dsl itself is used as part of om.next queries)#2018-08-1103:53myguidingstarI can't think of any viable workaround#2018-08-1103:55myguidingstaralso, is there any reason clojure.spec use sequential?
there instead of coll?
which is almost identical except that it returns true for sets and maps?#2018-08-1104:54seancorfield@myguidingstar Why not just use (s/map-of keyword? boolean?)
#2018-08-1104:58seancorfield(it took me a while to read your spec -- what does s/or
even mean with just one named branch?)#2018-08-1105:05lilactownis it possible to have a recursive spec?#2018-08-1116:07taylorI wrote a post that covers this https://blog.taylorwood.io/2017/10/04/clojure-spec-boolean.html#2018-08-1105:28seancorfieldYes.#2018-08-1108:59myguidingstar@seancorfield yup, that s/or
is for named branch#2018-08-1108:59myguidingstarI need to tag the key and value, hence the use of s/cat
#2018-08-1109:00myguidingstarso s/map-of
doesn't work for me#2018-08-1109:10myguidingstarthat spec's purpose is to used with s/conform
, kinda like a dsl parser, not just to check validation#2018-08-1118:31seancorfield@myguidingstar I wonder if (s/and map? seq (s/+ ...))
would make it so what you need?#2018-08-1122:28beta1036I have specd a domain entity in my application and have to read instances encoded as JSON where the keys are without namespaces. What's the recommended way to validate and convert the JSON encoded instances to the internal representation?#2018-08-1313:11rapskalian@U4844LY6A I wrote a little library that uses spec to automatically qualify/unqualify domain entities. Given that JSON uses string keys, youāll have to walk/keywordize-keys
first on your map as this library currently only works with keyword keys.
https://github.com/cjsauer/disqualified/blob/master/README.md#2018-08-1317:01beta1036You use qualify-map
first and then conform the result to validate the input, right?#2018-08-1317:46rapskalianCorrect. qualify-map
only translates unqualified keys into qualified keys. It doesn't do any conforming of its own. The source code is less than 50 lines in total if you'd like some inspiration for your own implementation:
https://github.com/cjsauer/disqualified/blob/master/src/cjsauer/disqualified.cljc#L32-L47#2018-08-1320:46beta1036I've had a look. I'm afraid in order to extend it to my use case (embedded maps, collections, conjunctions, etc.) I'd have to re-implement a lot of spec. handling or
this way seems to be particularly tricky.#2018-08-1203:53ackerleytng@seancorfield I have an example here: https://pastebin.com/Kuat3MPV . That's what i'm doing now, but is there a better way of "namespacing" specs?#2018-08-1204:40seancorfield@ackerleytng OK... and what's the question again? That seems OK to me on the face of it...#2018-08-1204:45ackerleytngah ok. I just thought there might be a better way of namespacing all of that#2018-08-1204:45ackerleytngthanks then!#2018-08-1215:36lilactownso I have this spec:
(s/def ::projection
(s/* (s/alt :key keyword?
:sym symbol?
:projection ::projection)))
and when I try and run valid?
, I get StackOverflowError clojure.spec.alpha/deriv (alpha.clj:1526)
#2018-08-1215:36lilactown(s/valid? ::projection [:a :b])
#2018-08-1215:44taylorIām not sure if youāre intending to validate nested collections, but you can do this:
(s/def ::projection
(s/* (s/alt :key keyword?
:sym symbol?
:projection (s/spec ::projection))))
#2018-08-1216:53taylorI should mention this only compiles when ::projection
has already been defined#2018-08-1215:44taylor(s/valid? ::projection [:a :b 'c])
=> true
(s/valid? ::projection [:a :b 'c "x"])
=> false
#2018-08-1215:47lilactownš okay awesome. yes, I am intending to validated nested collections#2018-08-1215:48taylor(s/valid? ::projection [:a :b [:c ['d]]]) => true
#2018-08-1215:49lilactown:+1:#2018-08-1215:48lilactownwhatās the difference between writing :projection ::projection
and :projection (s/spec ::projection)
?#2018-08-1215:50taylorby default all the regex-type specs āflattenā themselves when nested#2018-08-1215:52taylorwrapping a regex-type spec in s/spec
prevents that flattening, so you can use them to describe nested sequences of things#2018-08-1216:08lilactownI donāt understand what you mean by flattening#2018-08-1216:09lilactownor why something āflatteningā would cause a stackoverflow#2018-08-1217:00taylorthe stack overflow is just because of how spec is implemented, if you have a certain form of nested/recursive regex specs it recurses infinitely trying to evaluate it#2018-08-1216:48taylor@lilactown FWIW you can also define your spec using s/or
instead of s/alt
, and then you donāt need the s/spec
call to prevent the regex flattening:
(s/def ::projection
(s/* (s/or :key keyword? :sym symbol? :projection ::projection)))
#2018-08-1216:49taylorby āflatteningā I mean that when you nest the regex-type specs, by default they merge/compose/combine/maybe-a-better-word to describe a single sequence#2018-08-1216:51taylor(s/conform
(s/+ (s/alt :n number? :s string?))
[1 "2" 3])
=> [[:n 1] [:s "2"] [:n 3]]
#2018-08-1216:55taylor(s/conform
(s/+ (s/alt :n (s/+ number?) :s (s/+ string?)))
[1 "2" 3])
=> [[:n [1]] [:s ["2"]] [:n [3]]]
#2018-08-1216:57taylorbut what if you wanted to use regex specs to describe nested sequences? the āflatteningā behavior wouldnāt allow it, so you could use s/spec
to avoid it:
(s/conform
(s/+ (s/alt :n (s/spec (s/+ number?)) :s (s/spec (s/+ string?))))
[[1 2 3] ["1 2 3"]])
=> [[:n [1 2 3]] [:s ["1 2 3"]]]
#2018-08-1301:48lilactownI think I understand. thank you for the thorough explanation!#2018-08-1315:12trissso Iāve got a few namespaces - each one responsible for managing the contents of particular type of map. Is there a convention in clojure for what to call your āconstructerā method? Do you name it after the type of the type of object youāre going to create (already in the namespace?) or use a word like create
or construct
?#2018-08-1315:38guy(def job {:name "some job"})
like that you mean?#2018-08-1315:38guy(defn job [some-args] ...)
#2018-08-1315:42lilactown@triss I think this is a better discussion for #clojure , but I try and elide create
or make
whenever possible#2018-08-1315:44trissah yes youāre probably right re: #clojureā¦#2018-08-1315:45trissthe annoying thing is most of my constructers just do (s/conform ::definition args)
#2018-08-1315:45trissbut thatās way too much to typeā¦#2018-08-1315:46lilactownI see what youāre saying though. If i had a particular namespace, like my-app.job
and in there were all functions that work on the job
entity type, then I would probably just have a create
fn#2018-08-1315:46trissor is itā¦ maybe I should just call (s/conform directly) in other namespaces?#2018-08-1315:46lilactownthat might be my OCaml bleeding through tho#2018-08-1315:47lilactownitās probably better to hide the fact that all youāre doing is s/conform
in case you ever want to do more?#2018-08-1315:48trissthat was my thinking I guess#2018-08-1315:48trissbut it is very rare I do more these daysā¦#2018-08-1319:28mikerodIs it necessary to forward-declare specs that are referenced from other specs in a case such as mutually recursive specs?
Contrived example:
(s/def ::a
(s/map-of keyword? (s/or :b ::b :str string?))
(s/def ::b
(s/map-of keyword? ::a))
#2018-08-1319:29taylornot in my experience, as long as they're both registered by the time you use them#2018-08-1319:29mikerodI believe, at least in some circumstances, that the forward-use of a spec isnāt resolved until later anyways, so the forward declaration isnāt needed (maybe Iām wrong)#2018-08-1319:29mikerod@taylor that is what I thought I was seeing as well#2018-08-1319:29mikerodand looking at how the references are used in a various implās I checked in clojure.spec.alpha
it seemed like it was ok#2018-08-1319:30mikerodbut I havenāt seen any direct discussions on it really. Iāve seen at least one posts in the wild that did something like
;; Forward-declaration, replaced later
(s/def ::b any?)
(s/def ::a
(s/map-of keyword? (s/or :b ::b :str string?))
(s/def ::b
(s/map-of keyword? ::a))
but that doesnāt seem necessary#2018-08-1319:31tayloragree, seems unnecessary#2018-08-1319:32taylorthe keyword names are like "pointers" to the actual specs that get stored in the spec registry, so as long as the pointer points to a registered spec by the time it's used, I'd assume all's well#2018-08-1319:52mikerodyep#2018-08-1416:45trissis it just me or does spec cause me to end up with circular dependencies in namespaces? whatās the workaround?#2018-08-1417:35guyinstead of using ::my-spec
i create fake nsās
(s/def :guy-project/user (s/keys :req-un [:guy-project/name]))
#2018-08-1417:35guyThen u can store those spec wherever u want#2018-08-1417:36guyspec/user.clj
#2018-08-1417:36guyand import it where u need it#2018-08-1417:36guyif that makes sense? š#2018-08-1417:43manutter51Thatās how I do it too. I think it makes for more fine-grained namespacing.#2018-08-1418:50EthanI am working on a project to rewrite error messages and I was wondering if anyone knows any libraries that are well specced so I can test my spec error rewriting on more than just my specced functions and the specs in clojure.spec.alpha#2018-08-1515:25bbrinckThe libraries I test against in expound are: ring/ring-spec
and org.onyxplatform/onyx-spec
. I also have a partial spec for specs that can generate some specs ā¦ but itās not quite reliable enough for me to depend on it. https://github.com/bhb/expound/blob/master/test/expound/alpha_test.cljc#L2877-L3124#2018-08-1515:26bbrinckFeel free to use that spec, or frankly you can use any tests in the test suite and update it for your library š#2018-08-1509:55djtangoI think clojure.java.jdbc
has good spec coverage#2018-08-1509:55djtangohttps://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2018-08-1516:51Ethanthank you#2018-08-1520:06ericnormandHello!#2018-08-1520:06ericnormandCan someone tell me what :opt
is for? How is validation affected by its presence?#2018-08-1520:07ericnormandI'm talking about :opt
as an option to keys
#2018-08-1520:10guyAs far as iāve understood it,
if you have say (s/keys :opt [::name])
If the key ::name
is present it will check it against the spec.#2018-08-1520:11guySo iāve used it in the past to say, If this thing is present then it has to look like this spec. Otherwise carry on.#2018-08-1520:12guySo if i just make up an example with some pseduo code#2018-08-1520:14guy(s/def ::name string?)
(s/def ::type string?)
(s/def ::person (s/keys :opt [::name] :req [::type]))
(s/valid? ::person {::type "human})
=> probably true
(s/valid? ::person {::name 124 ::type "human"})
=> false because name is not a string
#2018-08-1520:14guyIāve just written these by hand so donāt trust the code 1 for 1#2018-08-1520:16guyFor more reading i would recommend https://clojure.org/guides/spec#_entity_maps#2018-08-1520:16guyThis registers a ::person spec with the required keys ::first-name, ::last-name, and ::email, with optional key ::phone. The map spec never specifies the value spec for the attributes, only what attributes are required or optional.
When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. We'll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.
A rather large snippet#2018-08-1520:16guyHope that helps!#2018-08-1522:02ericnormandHey @guy! Thanks!#2018-08-1522:02ericnormandHowever, if I do (s/keys)
, it still checks that ::name
is valid.#2018-08-1522:03ericnormand(s/def ::name string?)
(s/def ::person (s/keys))
(s/valid? ::person {}) ;=> true
(s/valid? ::person {::name "Eric"}) ;=> true
(s/valid? ::person {::name 15}) ;=> false
#2018-08-1522:04guyYeah but what does ::person check for#2018-08-1522:04guy:man-shrugging:#2018-08-1522:04guypart of why i use opt is documentation#2018-08-1522:04guyyour point is valid though and its been asked before#2018-08-1522:04ericnormandyes, documentation is good#2018-08-1522:04guylet me find it#2018-08-1522:04ericnormandthanks#2018-08-1522:05guyah bummer the historys gone#2018-08-1522:05guyAnother person asked something similar but a different way#2018-08-1522:05guysaying why use :opt at all when (s/keys)
does the job#2018-08-1522:06guyI think for me its documentation + being explicit about what you want to check for#2018-08-1522:07ericnormandokay, thanks#2018-08-1522:07ericnormandthe spec guide on http://clojure.org hints that it'll explain later, but it never does#2018-08-1522:07ericnormandI thought it might be more than just docs#2018-08-1522:07mpenetAlso gen#2018-08-1522:07ericnormandoh!#2018-08-1522:08ericnormandgood call#2018-08-1522:08mpenetKind of important ;)#2018-08-1522:08guyI think when it comes to speccing out ur fn as well it might be awkward depending on how u use (s/keys)
#2018-08-1522:09guyIām unsure if you would really generate a spec like (s/def ::person (s/keys))
#2018-08-1522:09ericnormandfor sure#2018-08-1522:09guyyou might just use (s/keys)
as an anonymous spec inside a (s/fdef ..)
#2018-08-1522:10guyiāve probably got the terminology wrong#2018-08-1522:10guybut that would also lead to harder reading when it comes to spec messages when they have failed#2018-08-1522:10guybut yeah#2018-08-1522:11guyi would favour :opt over (s/keys)
#2018-08-1522:11guyjust from a readability factor#2018-08-1522:11guybut im just one guy š#2018-08-1522:34lwhortontrollface#2018-08-1522:47gfredericksis there also a difference when the spec is not defined?#2018-08-1522:48gfredericksi.e., putting things in :opt
forces you to define the specs you listed (i.e., prevents you from accidentally forgetting them)?#2018-08-1523:47ericnormand@gfredericks a little repling shows that opts does not check if the spec exists#2018-08-1615:15guyCan you give an example?#2018-08-1615:15guyLike do you mean just doing something like
(s/keys :opt [::fake-spec])
?#2018-08-1615:24dadairThat fact allows me to avoid circular spec dependencies; the s/keys doesnāt validate that the spec exists#2018-08-1618:59ericnormand@guy that's what I mean#2018-08-1600:37gfredericksoh weird#2018-08-1600:37gfredericksdoes :req
?#2018-08-1602:20ericnormanddon't think so#2018-08-1602:20ericnormandthere's a pretty strong separation between checking keys and checking their values#2018-08-1602:20ericnormandall values are checked, even if they're not required#2018-08-1602:21ericnormandand all keys are allowed, even if they're not defined#2018-08-1612:03gfredericksyeah I guess that's consistent at least#2018-08-1618:59ericnormandyeah#2018-08-1618:59ericnormandI think that's why it's called "keys". It's just about the keys, and whether they exist.#2018-08-1619:34joost-diepenmaatthough you will get errors if you try to generate values for keys specs with unspecād keys#2018-08-1620:13guy^ yeah thats why i think iāve never found it to be a problem#2018-08-1620:14guyWhenever iāve tried to exercise it for an fdef or whatever#2018-08-1620:14guyit would throw that 100 tries error etc#2018-08-1620:14guyiirc#2018-08-1622:04richiardiandreacan I stest/instrument
a function that has been replaced with with-redefs
? would the spec still work on it?#2018-08-1701:50Alex Miller (Clojure team)I think both are working by swapping vars, so probably, but it might get weird. try it and see.#2018-08-1702:03richiardiandreaSeems like it is when the with-redefs
happens before the instrument. But I've got mixed feelings about this idea š #2018-08-1702:04richiardiandreaProbably better not to do that#2018-08-1702:04Alex Miller (Clojure team):)#2018-08-1710:08Oliver GeorgeIs this a common pattern: isolate a fn by stubbing everything else and running stest/check on it.
(defn check-all []
(let [syms (stest/instrumentable-syms)]
(doseq [sym syms]
(let [stub-syms (disj syms sym)]
(stest/instrument stub-syms {:stub stub-syms})
(stest/summarize-results (stest/check sym))
(stest/unstrument)))))
#2018-08-1712:10Alex Miller (Clojure team)First time Iāve seen it, but interesting#2018-08-1801:28Oliver GeorgeI was thinking it would give you confidence that the inputs and outputs through the code match up. So essentially a cheap type checking substitute.
The downside of that approach is that for full code coverage the generated data needs to trigger all branches of conditionalsā¦ I think thatās probably not something you can rely on.#2018-08-1717:21colinkahnLooks like it isn't an option, but curious if there's a way to specify an fspec for check
like you can for excercise-fn
#2018-08-1717:25colinkahnMy goal is to check a fn with an fspec whose :fn
is specific to a more narrow generator.#2018-08-1718:00Alex Miller (Clojure team)you can do that with instrument
first#2018-08-1718:01Alex Miller (Clojure team)with the :replace
functionality#2018-08-1718:01Alex Miller (Clojure team)if I understand your goals that is#2018-08-1718:19colinkahnLooking at the docs for instrument
I think :spec
should do what I want, but trying the following still uses the registered fdef
for files-reducer
instead of the override fspec
#2018-08-1718:19colinkahn(st/instrument `files-reducer {:spec {`files-reducer (s/fspec :args (s/cat) :ret any? :fn (fn [_] false))}})
(expound/explain-results (st/check `files-reducer))
#2018-08-1904:32pablorenoob question here, how do I make a spec for a fixed length collection of another spec?#2018-08-1904:34pabloreIs this a good solution?
(s/def ::my-10-specs (s/and #(= 10 (count %)) (s/+ ::my-spec)))
#2018-08-1904:47pablorewell, using (s/def ::my-10-specs (s/& (s/+ ::my-spec) #(= 10 (count %)))
fails after 100 tries#2018-08-1904:47pablore(when generating)#2018-08-1906:54fedregHi all, Is there a way to see the data that is returned from a function that I call stest/check
on? For example, I can see all the data generated for that functionās inputs by passing in a reporter-fn
to the opts
but I canāt seem to find a way to access that data for the returns. Iād love to pass a function in to format the output data as needed. thx for any pointers!#2018-08-1911:32gfredericksis there a coll-of
?#2018-08-1913:39pabloreyes, but it doesnāt work well with s/cat
.#2018-08-1913:44gfredericksoh so you want something that can go in a regex in particular?
like (){12}
from string regexes?
I can imagine a macro that expands to an s/cat
with 12 instances of the spec, but that'd be unfortunate. seems like something that could be done better built-in
it does seem uncommon though; where did this come up for you?#2018-08-1911:32gfrederickssorry, that was for @pablore#2018-08-1911:34gfredericks@fedreg there might not be a supported way, but if you make a TCHECK jira ticket I'll make sure it's at least available to the reporter-fn
#2018-09-0416:43djtangocurrently we are seeing this exception:
Caused by: clojure.lang.ExceptionInfo: Invalid keyword: ::clj-time.s/local-date. {:type :reader-exception, :ex-kind :reader-error, :file "/x/spec.cljc", :line 16, :col 54}
#2018-09-0416:47Alex Miller (Clojure team)Autoresolved keywords use the current env at read-time to resolve the ns#2018-09-0416:48Alex Miller (Clojure team)And this is reading, which happens regardless of the chosen branch#2018-09-0416:48Alex Miller (Clojure team)A workaround is to use the full keyword with namespace#2018-09-0416:49mishaharold#2018-09-0416:56djtangoAh so the reader for your cljs doesn't ignore the clj branches?#2018-09-0421:35Alex Miller (Clojure team)Itās read before itās ignored #2018-09-0416:57djtangoand so it gets to the auto-resolved keywords and if the clj-only ns isn't in the env at that point you get an exception#2018-09-0416:57djtangoHave I understood that right?#2018-09-0417:27seancorfieldYes, the expressions are read and auto-resolving keywords is part of that, prior to the conditional being applied. Then the result of reading is compiled.#2018-09-0417:28seancorfieldSo reader conditionals can only select between syntactically valid expressions that can be read in -- :some-ns/some-thing
is read "as-is" but ::some-alias/some-thing
has some-alias
expanded as part of the reading process.#2018-09-0417:29seancorfield(I hope I explained that accurately!)#2018-09-0509:33djtango^ makes sense. Thanks for the explanations @seancorfield @alexmiller#2018-09-0512:45ikitommicross-posting here too: a sample app with clojure.spec & ring, with conforming-based coercion: https://github.com/metosin/reitit/blob/master/examples/ring-spec-swagger/src/example/server.clj#2018-09-0512:47ikitommiin the example, the ::x
and ::y
specs are coerced either from strings (the get-endpoint with query-parameters) or from json (the post-endpoint if client sends json), or just validated (the post-endpoint if client sends edn or transit).#2018-09-0513:17ChrisIs s/fspec the proper way to define a spec for a higher order function? When I declare something with s/fspec and then try to generate a value, it appears to be called many times (perhaps by conform?) is there a way to declare a higher order function without it being invoked with conform?
Example:
`(s/def ::post-download-fn (s/fspec :args (s/cat :size nat-int?) :ret any? :gen
(fn [] (gen/return (fn post-dl-fn [size] (log/info (str "Post Download fn called! n: " size)))))))`
(gen/generate (s/gen ::post-download-fn))
Gives a bunch of these before returning the single value generated:
Post Download fn called! n: 0
Post Download fn called! n: 1
Post Download fn called! n: 1
....
Am I missing something obvious?#2018-09-0514:07ChrisI believe it to be instrumentation validating conformance - but it's odd that we can't turn it off in the case of a side effecting higher order function.#2018-09-0514:29taylor@chris547 there might be a better way, but there's a *fspec-iterations*
dynamic binding you can use to sidestep that behavior:
(defn foo [f] (f 1))
(s/fdef foo :args (s/cat :f (s/fspec :args (s/tuple int?))))
(st/instrument `foo)
(binding [clojure.spec.alpha/*fspec-iterations* 0]
(foo #(doto % prn inc)))
#2018-09-0514:30ChrisOh great, I wasn't aware of that binding - thanks @taylor!#2018-09-0514:50taylornp, I'm sure there's better advice/guidance on how to handle this for higher-order functions that take side-effecting functions, and I'm sure it's something that must be considered as clojure.core gets specs. One obvious workaround is to sacrifice specificity by just using e.g. ifn?
as a predicate#2018-09-0519:36bastiHi, is it possible to nest specs, rather than creating for each tiny bit a new s/def
?
So an inline version of e.g.
(s/def ::key1 string?)
(s/def ::my-map (s-keys [::key1]))
?#2018-09-0519:37taylorfor s/keys
specs, you need registered specs for the keywords. For other non-`s/keys` types of specs, you can nest spec definitions#2018-09-0519:38bastik thanks for the info - appreciated! how would that look like for that the spec above?#2018-09-0519:38taylor(s/def ::my-map (s/keys :req [::key1]))
#2018-09-0519:39tayloror :req-un
for unqualified keywords#2018-09-0519:41bastiOh I rather meant to define the spec for ::key1 in the ::my-map definition. I donāt want to reuse ::key1#2018-09-0519:42taylorI don't think that's possible#2018-09-0519:42taylornot with an s/keys
spec, at least#2018-09-0519:36smothersDo you know if there is a way for a recursive spec to nest uniquely? So the same tag canāt repeat across a branch
e.g. NO ā
[:paragraph
[:paragraph "foo"]]
or ā
[:link
[:link "foo"]]
or ā
[:italic
[:bold
[:italic "foo"]]]
but repeating siblings are fine ā
[:paragraph
[:bold "foo"]
[:bold" "bar"]]
Hereās the spec Iām currently using
(s/def ::attrs (s/map-of #{:url} string? :gen-max 2))
(s/def ::hiccup (s/or :string string?
:element (s/cat :tag #{:paragraph :link :bold}
:attrs (s/? ::attrs)
:content (s/* ::hiccup))))
#2018-09-0519:38Alex Miller (Clojure team)Generally, I would say no#2018-09-0519:39Alex Miller (Clojure team)You can of course write any arbitrary predicate you want and include it#2018-09-0519:41smothersThanks. Iāll give that a try#2018-09-0701:43bbrinckIn clojure.spec.alpha ā0.2.176ā, is there a new way to install a custom printer for macro-expansion errors?#2018-09-0701:44bbrinckIt seems to work differently than in ā0.2.168ā, unless Iām missing something#2018-09-0701:44bbrinckhttps://gist.github.com/bhb/1948bfca870fbc8758b515a4d0acc5ff#2018-09-0701:47bbrinckThere may be some subtlety of https://dev.clojure.org/jira/browse/CLJ-2373 that Iām not understanding#2018-09-0701:54seancorfieldIt's this patch https://dev.clojure.org/jira/secure/attachment/18391/clj-2373-spec-alpha-2.patch that removes the call to s/explain-out
as part of simplifying the error messages.#2018-09-0701:55seancorfieldAnd specifically this part of the proposal:
* stop printing data into message strings
ExceptionInfo.toString should list keys, not entire map contents
spec.alpha/macroexpand-check should stop including explain-out in message#2018-09-0701:59seancorfield@bbrinck Interesting. I can reproduce the behavior in a Boot repl but if I use clj
I get different behavior: (! 504)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-alpha7"}}}'
src/user.clj was loaded
Clojure 1.10.0-alpha7
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (set! s/*explain-out* (fn [ed] (println "failed")))
#object[user$eval5$fn__141 0x5b07730f "
#2018-09-0702:01bbrinck@seancorfield Good to know - I am still using Clojure 1.9 in my repro. Maybe itās an issue with using new clojure.spec without new clojure alpha?#2018-09-0702:01seancorfieldCompared to alpha 6 (! 505)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-alpha6"}}}'
src/user.clj was loaded
Clojure 1.10.0-alpha6
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (set! s/*explain-out* (fn [ed] (println "failed")))
#object[user$eval5$fn__140 0x502f1f4c "
#2018-09-0702:02bbrinckThe announcement for spec.alpha 0.2.176 and core.specs.alpha 0.2.44 says āthese libraries can be used now with Clojure 1.9.0ā, so I expected that to work#2018-09-0702:03seancorfieldConfirmed, yes, the printing has changed: (! 506)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.9.0"} org.clojure/spec.alpha {:mvn/version "0.2.176"}}}'
src/user.clj was loaded
Clojure 1.9.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (set! s/*explain-out* (fn [ed] (println "failed")))
#object[user$eval15$fn__150 0x590c73d3 "
#2018-09-0702:06bbrinck@seancorfield Thanks! Since it works in 1.10.0-alpha7, I may just add a note to Expound to suggest that people donāt upgrade to newest spec.alpha unless they also upgrade clojure. Or there may be a workaround#2018-09-0702:10Alex Miller (Clojure team)I am too tired to grok the issue - can you recap?#2018-09-0702:45seancorfield@alexmiller The change in 2373 that removes the explain-out
call from several spec failures (and instead adds that call to the default REPL?) means that The details of the spec failure aren't run through a custom printer now in Clojure 1.9 or Boot REPL (and probably not in Leiningen's REPL -- I think both lein
and boot
use REPL-y?).#2018-09-0702:46seancorfield(it's the right choice but it means that tools that print explain-data
in fancy ways now need to hook into the REPL instead of expecting to be invoking in the spec failures themselves)#2018-09-0703:12Alex Miller (Clojure team)Itās not that they arenāt run through a custom printer though, itās that theyāre not run through any printer, right?#2018-09-0703:13Alex Miller (Clojure team)Previously they were printed into the exception message and now theyāre not#2018-09-0703:15Alex Miller (Clojure team)The hooking is still exactly the same, its just that repls need to be better about how they print. The 1.10 clojure.main has a new function exposed to help with this#2018-09-0705:13seancorfield> Itās not that they arenāt run through a custom printer though, itās that theyāre not run through any printer, right?
Yes, right. I could have worded that better š#2018-09-0705:15seancorfield> The hooking is still exactly the same, its just that repls need to be better about how they print. The 1.10 clojure.main has a new function exposed to help with this
Boot and Leiningen have some work to do. Or REPL-y. š#2018-09-0714:08bbrinck@alexmiller I suspect Iāll get questions about this for Expound, so Iād like to put a notice for anyone who upgrades clojure.spec.alpha to ā0.2.176ā. What would you suggest? Should I recommend not upgrading to 0.2.176 unless they also upgrade to clojure 1.10.0-alpha7? Or until they upgrade to a newer version of Lein or Boot that changes the REPL printing?#2018-09-0714:10bbrinck> The 1.10 clojure.main has a new function exposed to help with this
which function is this? Iām toying around with a custom REPL built on top of rebel-readline#2018-09-0714:20Alex Miller (Clojure team)clojure.main/ex-str
#2018-09-0714:21Alex Miller (Clojure team)thatās the function that builds the string to print in the repl#2018-09-0714:22Alex Miller (Clojure team)for the note, I think the key is to just be clear about the combination of circumstances youāre warning about#2018-09-0714:27Alex Miller (Clojure team)as those circumstances will change with subsequent releases of clojure, spec.alpha, and the tools involved#2018-09-0716:18bbrinckthanks, iāll add a warning to the readme#2018-09-0716:53JHHow would you spec this map?#2018-09-0717:13dadairWhat do you want the spec to express? Have you read the spec guide? https://clojure.org/guides/spec#2018-09-0717:21dadairāhow would you spec this mapā is very subjective; it depends on what you need. For example, a perfectly valid spec is (s/def ::map-thing map?)
, if that is a robust enough spec for your particular problem.#2018-09-0719:20JHThat makes sense, I just read through the guide and was able to figure it out. I wanted to spec that every key in the map was of certain type. Moreover that the :on-click was a function.#2018-09-0719:21JHThanks @dadair#2018-09-0802:32hoynkCan anyone point me to some GitHub code with a good use of spec?#2018-09-0802:32hoynkAlso, should I use a newer version of spec than the one that comes with 1.9.0?#2018-09-0802:34seancorfield@boccato If you're just getting started with spec, you might as well stick with 1.9 and the included version.#2018-09-0802:36hoynkThat is exactly my case, @seancorfield.#2018-09-0802:51seancorfieldThere are also some changes in the latest spec version that sort of assume some corresponding changes in 1.10 (and I think some of the REPL tools will need some changes to catch up as well).#2018-09-0901:04bastiI there a nested version of s/form
?#2018-09-0901:17bastito back up a little bit.. suppose I have a deeply nested map data structure, how can I inspect its structure without manually traversing all its composed (s/def)
s ? I could generate an example, but that doesnāt tell me that a specific leaf is specd with a string?
predicate e.g.#2018-09-0901:18bastiplus the example generation is also kinda limited because its predicated on the fact that every predicate I use has a generator#2018-09-0905:53bastiOther question, how do I spec a collection which has the sequence: [string? int? keyword?]
.. Whereby int? keyword? can repeat arbitrary. So something like [sting? int? keyword? int? keyword? int? keyword?]
would be valid?#2018-09-0905:56seancorfield@basti untested but I think (s/cat :s string? :rest (s/+ (s/cat int? keyword?)))
#2018-09-0906:00seancorfieldYeah, (s/def ::x (s/cat :s string? :rest (s/+ (s/cat :i int? :k keyword?))))
should do it @basti#2018-09-0906:00bastithatās awesome, thanks @seancorfield š#2018-09-0907:10mishawell, until skynet comes around, it's up to you to assess which spec is trivial to generate automatically, and which ones need your help, @basti#2018-09-0907:14mishawhat exactly do you mean by inspect
? there is an s/conform, which destructures and assigns labels to most of the nodes. in which usecase it is not enough for you?#2018-09-0907:43bastibasically convert
(s/def ::name string?)
(s/def ::person (s/keys :req-un [::name]))
(s/def ::registration (s/keys :req-un [::person]))
to
(mapify ::registration)
=> {:person {:name 'cljs.core/string?}}
So that I see the whole structure at one glance.
Meanwhile I wrote a small utility function that converts map specs to my desired structure - will write a blog post about that eventually š#2018-09-0907:56ikitommi@basti at least spec-tools has a spec form parser & visitor, which can walk over all(?) specs. But I guess you already rolled your own.#2018-09-0907:57ikitommicoverage on the visitor: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/visitor_all_test.cljc#2018-09-0908:03bastiawesome, yeah I hand-rolled my own visitor, but happy to switch to a library š Iāll have a look!#2018-09-1016:56snowellI have a function that takes 2 arguments, the second of which needs to conform to a spec: (s/def ::MySpec (s/cat ::foo string? ::bar keyword? ::baz vector?))
(defn myFunc [one two])
(s/fdef myFunc :args (s/cat :one ::One :two ::MySpec))
#2018-09-1016:57snowellWhen running stest/check
on this function, I get an ArityException because the (s/cat a (s/cat b c d))
is becoming 4 arguments#2018-09-1016:57snowellHow can I tell it that the second cat
needs to be as one arg?#2018-09-1017:00favilasurround with s/spec
#2018-09-1017:05snowellOK cool...that at least gets me a different failure#2018-09-1018:50borkdudeI listened to the REPL podcast today and the interviewee (Higginbotham) said there were benefits with putting specs in their own namespace. How exactly?
For fdefs I find it useful to have them close to the function as documentation?#2018-09-1018:55seancorfield@borkdude It means your code can still be used without spec since the specs -- and those namespaces -- are optional.#2018-09-1018:56seancorfieldclojure.java.jdbc
does that so it can still support back to Clojure 1.7.#2018-09-1018:56borkdudefor libraries thatās nice yes#2018-09-1018:56seancorfieldAlso, most of our specs at work are data specs, not function specs -- and we find it easier to have a namespace for data specs. One place to go read them all (for a given data concern).#2018-09-1018:59borkdudeok#2018-09-1018:59misha+1 to data specs mostly#2018-09-1018:59borkdudedo you use instrumentations with these specs at all?#2018-09-1019:00borkdudee.g. in jdbc#2018-09-1019:02seancorfieldI instrument the java.jdbc
lib for the tests within that lib -- which slows things down dramatically.#2018-09-1019:03seancorfieldAt work, we tend to instrument specific functions (that have fdef specs) only within their specific tests.#2018-09-1019:03seancorfieldWe use the data specs in production code (to validate/conform data).#2018-09-1019:03borkdudeI find it annoying to get the wrong keys. Whenever I encounter that I tend to put an fdef now and instrument it (only for dev)#2018-09-1019:03seancorfieldWe also use the data specs in tests -- for generative testing, or just for random example-based test data.#2018-09-1023:04Jon WalchIs there anyway to force a specific path with a call to s/gen
?
Hereās a trivial example:
(s/def ::value (s/or :a map? :b int?)
(s/def ::message :req-un [::value])
(gen/generate (s/gen ::message))
I want the final line to always return with the ::value
codepath of :a
for the purpose of generating data for a test. Is this possible?#2018-09-1023:05taylortake a look at the 2-arity version of s/gen
https://clojuredocs.org/clojure.spec.alpha/gen, it takes an overrides map#2018-09-1023:06tayloryou might be able to override ::value
there by providing a different generator#2018-09-1023:10Jon Walchperfect thanks#2018-09-1114:41slipsetSorry if this is a faq, but Iād like to generate specs at runtime to use for validation eg:#2018-09-1114:42slipsetSay I receive something like:#2018-09-1114:42slipset{:id 1, :model-id 4711, :foo 'bar}
#2018-09-1114:42slipsetIn order to know how to validate this thing, Iād need to look up what the model is in a database, and then construct a spec based on that model.#2018-09-1114:44ghadieval
#2018-09-1114:45lilactownunfortunately, clojure.spec is very unwieldy for this type of use case#2018-09-1114:45slipsetWhat Iād be looking for is something like:#2018-09-1114:46slipset(s/valid? (model->spec (find-model (:model-id entity))) entity)
#2018-09-1114:46ghadi@slipset (-> model-id (grab-from-database db) read-string eval)
#2018-09-1114:46ghadithat'll give you a spec, if the form is saved in the db#2018-09-1114:46slipsetAnd with my understnading of spec, this is not possible, but Iād like to flag it as a use case, just in case.#2018-09-1114:46ghadiand all of what it refers to is already loaded and available#2018-09-1114:47ghadiit's totally possible, you just have to dynamically load the spec#2018-09-1114:47lilactownI know that there seems to be some disinclination to adding more support this. itās my most-desired feature#2018-09-1114:48ghadiit's not a feature you add to spec, you can already do it#2018-09-1114:48slipset@ghadi but my specs are not āknownā or stored anywhere, and there might be a bunch of models (a bunch being in the thousands).#2018-09-1114:49slipsetWhile Iām typing this, I guess I think that if specs were data, this would be feasible, right?#2018-09-1114:49lilactownthe ability to have āanonymousā specs / specs-as-data would have a lot more utility as a general purpose way of describing data. but I think that it rubs against the desire of the core team to encourage people to use namespaced keys#2018-09-1114:50ghadiregardless of the mechanism, your specs have to be known somehow if you want to dynamically load them#2018-09-1114:51slipset@ghadi: in my db I would typically have someting like this (for a model)#2018-09-1114:52slipset{:id 4711, :foo "integer", :bar "list-of-strings", :baz "string"}
#2018-09-1114:52slipsetand, while this might be fortunate, itās how my world is.#2018-09-1114:53guyDo you have lots of models?#2018-09-1114:53slipsetyes.#2018-09-1114:53slipsetand theyāre user defined#2018-09-1114:54slipsetso my model->spec
would return, for the model above, something like:#2018-09-1114:54slipset(s/keys :req-un [::foo ::bar ::baz])
#2018-09-1114:54slipsetand there I see where the eval comes in š#2018-09-1114:54ghadiany validation system that stores stuff as data needs to convert list-of-strings
into some predicate#2018-09-1114:54slipsetThanks, @ghadi š#2018-09-1114:55slipsetYeah, I see that, list-of-strings
which is one of a few predefined types, would be mapped to (s/coll-of string?)
#2018-09-1114:56slipsetThe main problem here, I guess is ::foo
, ::bar
, and ::baz
needs to be defined.#2018-09-1114:57lilactownright. everything in clojure.spec must be named#2018-09-1114:58guyAre the types of your data always quite static?#2018-09-1114:58ghadinamed only if you refer to the spec by name#2018-09-1114:58ghadi(keys or def / fdef)#2018-09-1114:59ghadiyou can call (valid? (coll-of string?) ["a" "b"])
without naming the spec#2018-09-1115:00ghadiof course if you're checking a map, you'll want to handle that yourself#2018-09-1115:00ghadii.e. just because s/keys is available, you don't have to use it#2018-09-1115:01slipsetThe types are static#2018-09-1115:02slipsetbut the field names are not.#2018-09-1115:03lilactownš¤ maybe using map-of instead of keys?#2018-09-1115:03lilactownthereās also https://github.com/metosin/spec-tools which might be a more ergonomic solution#2018-09-1115:05ikitommiwith spec-tools, without macros, on top of the non-documented parts of clojure.spec:
(require '[spec-tools.data-spec :as ds])
(require '[clojure.spec.alpha :as s])
(let [model {:foo integer?
:bar [string?]
:baz string?}
spec (ds/spec
{:spec model
:name ::spec})]
(s/valid? spec {:foo 1
:bar ["kikka" "kukka"]
:baz "abba"}))
; true
#2018-09-1116:12borkdude@ikitommi thatās cool. kind of like Schema notation for clojure.spec?#2018-09-1116:17borkdudefair reminder, go on š#2018-09-1116:17ikitommi@borkdude yes, it is. Quite nice for ad-hoc stuff like web apps, where one would use Schema#2018-09-1116:18ikitommihttps://github.com/metosin/reitit/blob/master/examples/ring-swagger/src/example/server.clj#2018-09-1116:20ikitommiIām actually looking forward to seeing things break.#2018-09-1116:20ikitommiany plans on doing that @alexmiller?#2018-09-1116:21borkdudeeven the public parts of spec itself may break if itās for the better, itās alpha š#2018-09-1116:33Alex Miller (Clojure team)Rich has been getting back to spec and right now I think itās likely that weāll try to get 1.10 out the door, then start working on spec in earnest#2018-09-1116:33borkdudecool#2018-09-1116:38borkdudewhat are the main features of 1.10? - I saw you were working on better error messages, very nice work#2018-09-1116:43Alex Miller (Clojure team)updated JVM/JDK requirements and compatibility, prepl (kind of alpha), error messages#2018-09-1116:45seancorfieldDoes that mean we'll get some new tooling from Cognitect based around the prepl?#2018-09-1116:46Alex Miller (Clojure team)unlikely for 1.10#2018-09-1116:47borkdudeprepl.alphaā¦ what are we going to do with the prepl?#2018-09-1116:47Alex Miller (Clojure team)whatever you like :0#2018-09-1116:47Alex Miller (Clojure team)thereās a bunch of things in work. rather than wait for them all to reach completion, I think weāll just release 1.10 and keep working on them#2018-09-1116:48borkdudeanything to read on prepl?#2018-09-1116:48seancorfieldThe source code? š#2018-09-1116:48seancorfieldI had a quick play with it when it dropped... I can imagine some interesting tooling options based on it...#2018-09-1116:48Alex Miller (Clojure team)thereās a small chance it might even be removed before 1.10 releases#2018-09-1116:48borkdudeI have this vague notion itās something similar to unrepl, but thatās all I know#2018-09-1116:49Alex Miller (Clojure team)it shares many goals with unrepl#2018-09-1116:49Alex Miller (Clojure team)both are data-oriented stream repls#2018-09-1116:50seancorfieldWe're thinking a bit about building some internal tooling on top of prepl. Automating some of the stuff we currently do today (via a REPL manually).#2018-09-1116:51seancorfield(we run a Socket Server REPL inside one of our production processes and currently use unrepl to talk to it over an ssh tunnel for certain things)#2018-09-1116:51borkdudewe run nREPL in production, but I only talk to it myself š#2018-09-1116:52seancorfieldWe used to -- but I didn't like the stack of dependencies that nREPL brought in. Running just a Socket REPL is nicer. And of course we can start any process up with a Socket REPL without needing any code inside the process.#2018-09-1116:53borkdudeI did that once for a Scala project where I added clojure as a dependency š#2018-09-1116:54borkdudeit wasnāt a serious project though, but it worked š#2018-09-1116:56seancorfieldWe have a REPL running inside our legacy CFML app so I won't judge š#2018-09-1117:08slipset@ikitommi nice! Iāll have a look at the data specs.#2018-09-1119:01arohners/keys
supports and
in :req
, but not :opt
, right? Is there any other way to express ā:foo is optional, but if present, :bar must also be presentā?#2018-09-1119:04mishas/multispec, s/or#2018-09-1119:06mishabtw, the exact behavior of or
and and
inside s/keys
is still a mystery to me. is there something official to read about it, @alexmiller? or is it all unintended side effect, which should not be relied upon?#2018-09-1119:15seancorfield@arohner That's usually when I reach for s/and
wrapped around s/keys
and a predicate for the present/not-present relationships...#2018-09-1119:24arohner@seancorfield thanks. The combinations here get a little gnarly to multi-spec, so I guess Iāll punt#2018-09-1119:25arohner@misha itās documented in s/keys
docstring:
The :req key vector supports 'and' and 'or' for key groups:
#2018-09-1215:08samedhiIs it possible to write a spec that generates a sequence where each element in the sequence has some relationship to the previous element(s) in the sequence?#2018-09-1215:09samedhiAs an example, like one that multiplies the last element of a sequence by 2 or 3 (random).#2018-09-1215:09samedhiso [1 2 4 8 24 48 144]
and [1 3 9 18]
would both be sequences it might generate.#2018-09-1215:10taylorare you only interested in generation, or do you want a spec to validate inputs too?#2018-09-1215:10samedhiI guess both#2018-09-1215:10samedhiI mean I would like to use test.check
with this generated sequence, so I assume that means using both (I think).#2018-09-1215:13Alex Miller (Clojure team)the way I would handle this is to use gen/bind#2018-09-1215:13Alex Miller (Clojure team)generate a series of random multipliers (a sequence of random 2s and 3s)#2018-09-1215:14samedhiAh, I see where you are going.#2018-09-1215:14Alex Miller (Clojure team)then gen/bind to turn the sequences of multipliers into a sequence of values#2018-09-1215:14Alex Miller (Clojure team)or rather, I guess gen/fmap is sufficient here#2018-09-1215:14Alex Miller (Clojure team)itās just apply a function to that sequence#2018-09-1215:16samedhiYeah, so what I was really hoping was to generate the initial (seed) state of my program as the first element in the sequence, and generate the rest of the sequence by constantly looking at the last element in the sequence, determining what actions can be taken at that state, and then picking one of those actions and applying it to said state.#2018-09-1215:16Alex Miller (Clojure team)yeah, same idea#2018-09-1215:17Alex Miller (Clojure team)start from a gen of a tuple feeding the fmap#2018-09-1215:17Alex Miller (Clojure team)(s/gen (s/tuple int? (s/coll-of #{2 3} :max-count 10)))#2018-09-1215:18Alex Miller (Clojure team)then gen/fmap a function over that that takes the first element in the tuple and repeatedly applies the multipliers in the second element to produce a sequence#2018-09-1215:18samedhiOk, the only wrinkle I see there is that only a subset of all of the actions are valid for some of the states.#2018-09-1215:19samedhiBut that is an idea.#2018-09-1215:19samedhiMaybe, I can just make it so that if the action taken is NOT a valid action for that state, it just returns the original state (identity).#2018-09-1215:19Alex Miller (Clojure team)if you need to randomly choose an action per state, you will need to move to gen/bind#2018-09-1215:20Alex Miller (Clojure team)but I think you could make that work too#2018-09-1215:20samedhiInteresting, interesting. I mean I tried to read through the "rose tree" thing for test.check generators. And... well, it looks like it is generating some sort of tree using something like a recursive algorithm. But, yeah, got lost there. š#2018-09-1215:20Alex Miller (Clojure team)in that case, Iād probably gen the initial state and the number of actions to gen#2018-09-1215:21Alex Miller (Clojure team)yes, but you donāt need to worry about that - the key is just to capture all randomness via generators#2018-09-1215:21Alex Miller (Clojure team)test.check will do the rest#2018-09-1215:21samedhiOk, thank you for your help, I'll look at it some more this evening.#2018-09-1215:22Alex Miller (Clojure team)this is advanced generating, but itās where things get interesting#2018-09-1215:22Alex Miller (Clojure team)a lot of interesting problems in simulation testing look like this#2018-09-1215:22samedhiReally, it is just the fantasy I have of letting spec generate every possible way that my program could run and being like "yep, that one thing encompasses all the test".#2018-09-1215:23samedhiYou still get a great deal of benefit out of using the normal fdef
and test.check
per function imo, but the idea of actually generating all states of the program would be really cool.#2018-09-1215:28gfredericks@samedhi @alexmiller this is exactly what the stateful-check library tries to automate, I believe. I haven't used it#2018-09-1215:33samedhi@gfredericks Awesome š#2018-09-1215:34taylorwould there be a way to spec this kind of sequence without a custom predicate?#2018-09-1215:34roklenarcicIs there some way to specialize a key in s/keys
? Like I have a (s/keys :req [.... ::type])
where type is #{"ORG", "USER"}
, but now I'd like that same spec but where type is limited to `#{"ORG"}#2018-09-1215:35Alex Miller (Clojure team)@taylor do you mean custom generator?#2018-09-1215:36taylorno I mean if you wanted to conform/validate a sequence like that, asserting that each element is a multiple of the previous#2018-09-1215:36taylorI was trying to imagine how you might use the spec primitives to do that but I can't imagine how#2018-09-1215:37Alex Miller (Clojure team)not as a pred in the coll spec, but you could s/and with a function that verified that#2018-09-1215:37Alex Miller (Clojure team)or, maybe you could put that in :kind if you had a seq-of-mults?
pred#2018-09-1215:35Alex Miller (Clojure team)@roklenarcic no, but you can s/and it with a more restrictive predicate#2018-09-1215:36roklenarcicIt tried to s/merge
it with another keys spec which has key that is more restrictive, with different keyword ns and same name#2018-09-1215:37roklenarcicit kinda works, in that it will check both#2018-09-1215:38Alex Miller (Clojure team)there are pros/cons to those approaches wrt conform and gen#2018-09-1215:38roklenarcicah, haven't thought about that#2018-09-1215:39Alex Miller (Clojure team)I think s/and is the most direct way to say it: āI have this map AND it must meet this extra constraintā#2018-09-1215:39Alex Miller (Clojure team)s/multi-spec can also be handy for some cases like this but itās a lot of weight to add if itās only this#2018-09-1215:45roklenarcicI'll need to check out multi-spec#2018-09-1215:53roklenarcicUsing spec-tools
's data-spec solves a couple problems, but sadly data-spec has a bug where they don't name their specs#2018-09-1216:02roklenarcichttps://github.com/metosin/spec-tools/pull/125#2018-09-1219:48misha@arohner well, that's embarrassing opieop (re: or in s/keys)#2018-09-1219:58taylorwhat's opieop all about#2018-09-1220:01mishaface expression#2018-09-1220:02taylorit's so small I can't make out his expression#2018-09-1220:03taylorI only see it used on this Slack and I feel like I'm missing... something#2018-09-1220:04mishait is from twitch .tv chats originally#2018-09-1222:23lilactownclojure.spec has the ability to transform data from one form into another, right?#2018-09-1222:24lilactownare there any ground-up guides around data transformation?#2018-09-1222:25seancorfieldThe basic rule of thumb is "don't do that"#2018-09-1222:26seancorfieldSpec isn't designed as a data transformation system and when folks ask about doing it, @alexmiller usually pops up and says that's not a good idea š#2018-09-1222:28lilactownbummer#2018-09-1222:29seancorfieldThat said, I think there are some use cases where limited coercion is acceptable. For example, we use spec for API / form input validation so data is mostly strings but we want strings, longs, doubles, Booleans, and sometimes dates out of it. I think that sort of spec is "OK" if you're careful (in particular, think about generation of data using such specs, as well as applicability of the spec elsewhere in your system -- just because you accept strings at the API/form level doesn't mean you want to accept strings elsewhere in your domain model).#2018-09-1222:29seancorfieldWhat sort of data transformation were you thinking about @lilactown?#2018-09-1222:31lilactownI'm creating an API for creating content schemas in a CMS#2018-09-1222:32lilactowni have a general purpose edn format I'd like to use, and then coerce it into the JSON representation for the particular CMS we're using#2018-09-1222:32Alex Miller (Clojure team)to be specific, transforming data use conformers in spec is imo not a good idea. using spec as a target for driving transformations is potentially a good idea. I think https://github.com/wilkerlucio/spec-coerce is pretty good.#2018-09-1222:50lilactownthanks! I'll take a look#2018-09-1302:46donaldballIām working on a spec for an asynchronous message format and am curious about versioning. My instinct is to declare a versioned namespace for the message envelope - e.g. :my.message.v1/schema
, declare all my envelope fields in that same namespace, and then just make a new one if and when I realize the need to make an incompatible change to the envelope. It occurred to my colleague that multi-spec was another way to accomplish this by including the version as a value, e.g. :my.message.schema/version 1
and using multi-spec to choose the correct spec.
Are both of these viable options and is there anything to recommend one over the other?#2018-09-1303:06Alex Miller (Clojure team)Iād recommend not making incompatible changes#2018-09-1303:06Alex Miller (Clojure team)then you donāt need versions :)#2018-09-1303:07Alex Miller (Clojure team)if the attribute needs to change, give the attribute a new name#2018-09-1303:08Alex Miller (Clojure team)if youāre talking about adding attributes, thatās compatible. if you need to remove attributes from a map, then you should have a new name for the container, which I guess version is one kind of new name#2018-09-1314:29donaldballAlas, my foresight is sufficiently imperfect that I donāt believe I can develop the correct message format for all time. Incompatible changes I believe might be reasonably desired include (and are probably limited to?): adding and removed required keys.#2018-09-1314:33Alex Miller (Clojure team)well, Iāll refer you to Richās Spec-ulation keynote for the long form argument#2018-09-1314:35ghadiwhat is an example of adding keys that is a breaking change @donaldball?#2018-09-1314:35tayloradding new required keys would be breaking#2018-09-1314:36ghadiimportant word simple_smile#2018-09-1314:37taylorbut yeah I think the gist is "don't make breaking changes" which I generally take to mean "if you need to break something, make a new something, and leave the old something alone"#2018-09-1314:38taylorif you need to add a new required key to some map, it's no longer the same thing. Make a new map spec with a different name that requires the new key#2018-09-1314:40Alex Miller (Clojure team)^^#2018-09-1314:41ghadiif it's for a message on something like Kafka, the new name might be an entirely new Topic#2018-09-1314:44donaldballI agree thatās very much the route spec wants you to go down and Iām on board the train, but I am curious what makes the multi-spec idea a poor solution. It seems like it does constrain you forever to having a message be a map, and for the map to require the multi-spec keyword, but otherwise the map contents could vary as required by the versions.#2018-09-1315:17lilactownhow do I put a bound on how many times clojure.spec.test.alpha/check generates a test?#2018-09-1315:17lilactownthe documentation is very nebulous#2018-09-1315:22taylor@lilactown (st/check `foo {:clojure.spec.test.check/opts {:num-tests 10}})
#2018-09-1315:24taylor>The opts map includes the following optional keys, where stc aliases clojure.spec.test.check
>The ::stc/opts include :num-tests in addition to the keys documented by test.check.#2018-09-1315:24taylorit confused me at first too b/c test.check/quick-check takes the number of tests as its first argument#2018-09-1315:33lilactownyeah it was completely meaningless to me#2018-09-1315:34lilactownhm. even specifying {:num-tests 5}, my generative testing is taking minutes#2018-09-1315:42lilactownin fact, specifying 0 takes minutes? I'm not even sure this is working#2018-09-1315:48taylorthis works for me:
(defn foo [x] (str x))
(s/fdef foo :args (s/cat :x any?) :ret string?)
(st/check `foo {:clojure.spec.test.check/opts {:num-tests 1}})
(st/check `foo {:clojure.spec.test.check/opts {:num-tests 100}})
#2018-09-1315:49taylorand the :num-tests
is accurately reflected in the check
result#2018-09-1315:51lilactownI think some of my specs must be really expensive. is there a way for me to selectively narrow some of the child specs?#2018-09-1315:51lilactownI think some of my specs must be really expensive. is there a way for me to selectively narrow some of the child specs?#2018-09-1315:54taylorany recursive specs involved?#2018-09-1315:54lilactownnope#2018-09-1315:55taylorhmm hard to say w/o seeing all the relevant code#2018-09-1315:58Alex Miller (Clojure team)Expensive to check or to gen?#2018-09-1315:59lilactownI'm not sure š#2018-09-1316:00lilactownI apologize, it's quite long. but I'm not confident what might be what's causing the slowness#2018-09-1316:11lilactowndefinitely seems to be in the generation of :torch/fields#2018-09-1316:13lilactownI'm going through and removing everything and slowly adding back in. adding back in :torch.field/key
:torch.field/name
seems to slow it down, but not astronomically#2018-09-1316:38Alex Miller (Clojure team)I meant what operation are you invoking when itās slow?#2018-09-1316:39Alex Miller (Clojure team)Sounds like gen #2018-09-1316:40Alex Miller (Clojure team)If you have coll-of, itās good to set :gen-max to something small like 3#2018-09-1316:51lilactownthat did help! thanks!#2018-09-1317:06Alex Miller (Clojure team)The default is 20 and that can get a little out of hand#2018-09-1316:27lilactownhow do I selectively narrow the values generated by stest/check?#2018-09-1316:28lilactowne.g. if my spec has a named child-spec somewhere that is defined as keyword?
, but I don't really care about testing 1000s of keywords#2018-09-1316:29taylorI think you can pass in an override map in check
opts :gen
#2018-09-1316:29taylor>:gen map from spec names to generator overrides#2018-09-1316:30lilactownthanks!#2018-09-1415:28lilactownhow would I represent a constraint on two related fields? e.g.:
{:type :type/json
:cardinality :cardinality/many} ;; invalid! :cardinality/many can only be used with :type/string and :type/ref
#2018-09-1415:30dominicm@lilactown use a predicate on the map? #2018-09-1415:35lilactownright.... keep thinking of specs at the key level. thanks#2018-09-1417:01Spencer AppleHello, is it possible to include external data when using spec to validate data - or would a separate function be better? E.G. validating an api response, and using a key from the request (such as the length of the pagination to validate the number of values returned).#2018-09-1417:08dadairCould you define a spec that is the combination of the request + the response (i.e., the context), and then use a predicate to check the mentioned constraints?#2018-09-1417:10dadairsomething like the following?
(s/def ::request ,,)
(s/def ::response ,,)
(s/def ::context
(s/and (s/keys :req [::request ::response])
#(= (-> % ::request :pagination :length) (count (-> % ::response :values)))))
#2018-09-1417:13Alex Miller (Clojure team)I think the question is whether thatās better than just writing the equivalent Clojure code#2018-09-1417:13Alex Miller (Clojure team)(I tend to think not in this case)#2018-09-1417:19Spencer Applehmm @dadair that is a really good example, and I think it would work. Thanks for writing that out. I was curious about if it made sense to include in the spec - and I kind of think the length of a vector is part of the data shapeā¦ but perhaps it still doesnāt fit in the spec#2018-09-1417:20Spencer AppleI will eventually get to the data generation side of this spec, in which I will want to include the :pagination :length
parameter from the request to generate the correct length of ::response ::values
. At that point I might need to combine the ::request
and ::response
specs?#2018-09-1418:02Alex Miller (Clojure team)yes, if you want to generate one thing, they need to be in the same spec, although Iām not sure why you need to generate responses (presumably your code is making those)#2018-09-1418:12Spencer AppleGotcha, and thanks for you responses!
The request and response are to and from an external API. I want to generate responses to test our data munging flows. Since the data munging fns rely on the request parameters, the generated responses also have to conform (at least partially) to the requests that āgenerated themā.#2018-09-1419:46Alex Miller (Clojure team)gotcha, that makes sense#2018-09-1419:46Alex Miller (Clojure team)depending how much stuff there is, you might able to gen both independently, then just copy values that have to match from the request into the response or something like that#2018-09-1420:02Spencer Appleah interesting idea!#2018-09-1518:36richiardiandreawe have found that ::
is a bit clunky for refactoring, is other people using the long :ns.foo.bar/quuz
for specs?#2018-09-1518:39mpenetNot really. You change the name just in the ns declaration vs everywhere you use it. I guess it s quite personal #2018-09-1518:39richiardiandreawell you have to change the namespace and the alias in case it changes#2018-09-1518:40mpenetYes that's what I meant#2018-09-1518:40richiardiandreawhile with unqualified you don't have to do anything else, just copy them around#2018-09-1518:40richiardiandreaI mean fully qualified š#2018-09-1518:41richiardiandreafollowing along these lines, sometimes you need to make them unique...you cannot use :foo/bar
or they would clash#2018-09-1518:42mpenetThe alias doesn't change really, just the thing it points to. #2018-09-1518:43richiardiandreathis though means you need to give names#2018-09-1518:43richiardiandreaand naming is hard and requirements change and all that#2018-09-1518:44richiardiandreaso we found fully qualified better for us - for now... š#2018-09-1518:44mpenetSame argument that's sometimes mentioned with clj.spec.alpha, just use aliases ex s/def and the day it changes to beta or whatever you just update the alias target#2018-09-1518:45mpenetIn the end it s just a search & replace with less/more hits, I guess both ways are fine#2018-09-1518:45richiardiandreawell my argument would be that you don't even need search and replace#2018-09-1518:45richiardiandreaat the price of spec verbosity#2018-09-1518:46mpenetYou d need to update every kw vs just ns declarations so yeah, there is some s&r :p#2018-09-1518:48richiardiandreamy question was something else though.. š#2018-09-1518:49richiardiandreakind of a feature request, but at this point I don't know if it makes sense š#2018-09-1518:49richiardiandreait would be great if you could have aliases of keywords that are not tight to a namespace#2018-09-1518:50Alex Miller (Clojure team)we do expect to have something like this, but not sure yet exactly what it will look like#2018-09-1518:50Alex Miller (Clojure team)didnāt want to couple it tightly to existing namespace/alias support#2018-09-1518:50richiardiandrealike (defkw :my-long.foo.bar :mfb)
#2018-09-1518:50richiardiandreaoh cool ok#2018-09-1518:51richiardiandreais there a JIRA to upvote? I can open one#2018-09-1518:52richiardiandreaI'll check š#2018-09-1518:52mpenetWould be interesting if spec understood hierarchies, but I already mentioned that #2018-09-1518:53Alex Miller (Clojure team)there is one I think#2018-09-1518:54Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2123#2018-09-1518:54richiardiandreaoh nice#2018-09-1518:56richiardiandreawill track it and write there if I have good ideas š#2018-09-1518:56richiardiandreathank you Alex#2018-09-1520:25lilactownyeah I ended up writing the fully qualified keyword for my specs lately and really am glad I did#2018-09-1520:25lilactownit made it really easy to e.g. move all of my specs to their own ns#2018-09-1523:27dadairDoes the thing change if it moves namespaces? Is it fundamentally different? If not, I think aliasing conflates location with name with regards to specs. I tend to prefer something like :transport/event
because what it is doesnāt change based on where it is.#2018-09-1606:51ikitommiFrom reddit:
> There are a couple areas (spec programmability and some map-related stuff) where we have some work to do.#2018-09-1606:56ikitommisounds good, despite doesn't say much about what kind of changes. As there are lot of libraries trying to extend spec for different use-cases, it would be great if the need of the libs could be taken into account. Personally would like to see some public spec walking protocols.#2018-09-1607:00ikitommiand not based on a parsing spec forms, that we can do outside, more like CLJ-2251.#2018-09-1607:02ikitommia library developer feature poll/discussion?#2018-09-1607:03misha@richiardiandrea you can create ns and give it an alias, it will not be tied to an actual file, but you will be able to use that alias to shorten keywords. it is not pretty though. clojure.core/create-ns
, clojure.core/alias
. I found, that having shorter spec names is less tedious and more readable (possible in app layer, where your system is pretty much the only system which sees them). In libraries you probably would want to use longer "truly" global long keywords for under the hood things.#2018-09-1615:50richiardiandreaThis is an interesting idea, but feels like a hack :) I will give it a try anyways, thank you!#2018-09-1616:01mishathis is sort of what ns
does, but it looks alien and hacky, I agree with that.#2018-09-1818:30richiardiandreaunfortunately alias
does not seem to work in cljs#2018-09-1818:36mishai'd cross-post this to #clojurescript, seems critical(?)#2018-09-1818:37richiardiandreadone that#2018-09-1608:49Ho0manHi, I am using spec for the first time. And it takes too long to generate my specs using (test.check/generate (spec/gen ,,, ))
.
I had previously used test.check
for similar data generation and did not had the problem.
The spec if the configuration map having many nested spec/keys
and spec/every
.
Limitin spec/every
with :gen-max
did not help. What am I doing wrong ?#2018-09-1608:50Ho0manHeres part of the spec ... the final configuration contains a numbeer of :ecosystem/service-instance
s#2018-09-1612:38Alex Miller (Clojure team)If you have a lot of s/every, they will gen up to 20 elements. Setting :gen-max in those to something like 3 would probably help#2018-09-1707:38Ho0manThanks, Alex.
I did try that and it helped.
My question is whether there is a fundamental difference in spec
's data generation and that of test.check
's ?
And should I be concerned about using these specs in large generative tests as their size grows ?
Thanks again#2018-09-1712:26Alex Miller (Clojure team)spec uses test.check generators so there is literally no difference. There will always be the need to tune the gens a little bit#2018-09-1618:00dadairWhatās the preferred way of specing a spec? I have an internal lib function that takes a spec and I want the s/fdef to check that the arg is a spec. So far I have qualified-keyword? But thatās a bit of a shallow spec? Tried spec? But it was returning false on s/defād qualified keys.#2018-09-1618:02dadairThat spec arg is later used in internal machinery to validate some request routing#2018-09-1618:08seancorfield@dadair Remember that a predicate function could also be (treated as) a spec...#2018-09-1618:08dadairYeah, at this point I force using a ādefinedā spec#2018-09-1618:09seancorfieldMaybe call s/form
on it and succeed if you get truthy?#2018-09-1618:12dadairMaybe expect users to wrap in (s/spec ..)
prior to passing argument, then internally spec against s/spec?
?#2018-09-1618:13dadairthat would also get around the limitation of removing the ability to use a raw predicate#2018-09-1618:14dadairAre there any downsides to that approach? Iām not super familiar with the effects of (s/spec ..)
#2018-09-1618:18seancorfieldThere's a get-spec
function you can use with the qualified keyword (rather than s/form
). You can wrap anything in s/spec
: (s/spec 42)
#2018-09-1618:19seancorfieldSo (s/spec? (s/spec 42))
is truthy -- so it won't tell you it's really a spec that is wrapped.#2018-09-1618:19dadairYeah maybe I need to do some code-based validation rather than relying on the fdef here#2018-09-1618:20dadairItās not a critical check, Iām just trying to add some guardrails for the other devs that arenāt super familiar with this part of the system#2018-09-1618:20seancorfieldI guess I'd take a step back and ask: Am I over-constraining this function?#2018-09-1618:21seancorfieldCan it accept a predicate and use it like a spec, for example?#2018-09-1618:22dadairYeah, thatās what Iām thinking too.#2018-09-1618:49Alex Miller (Clojure team)specs can be qualified-keyword?, set?, ifn?, or spec instances (thereās a s/spec? for those)#2018-09-1620:28samedhiShould I be using (spec/fdef ...)
for multimethods (does not seem to work)? Guess I can just wrap the defmulti invocation in a function and test that...#2018-09-1620:30samedhiErroneously thought that multi-spec
was for multimethods, but it actually appears to be for dispatching data based on "kinds".#2018-09-1620:33samedhinm, silly ns error. spec/fdef
does appear to work with defmulti
#2018-09-1719:49spiedenis anyone aware of a way to strictly validate some data against a spec? e.g. fail on extraneous map keys, etc.#2018-09-1719:50taylorhere's one way https://github.com/gfredericks/schpec/blob/master/src/com/gfredericks/schpec.clj#L13#2018-09-1719:50taylorand I think spec-tools has another way, although the general advice is to not make map/keys specs "closed"#2018-09-1721:51bbrinckIf you just want to guard against typos, check out https://github.com/bhauman/spell-spec#2018-09-1721:51bbrinckbut yes, the general advice is not to make specs closed#2018-09-1722:27spiedenthanks!#2018-09-1722:27spiedenyes generally i leave them open#2018-09-1722:28spiedenin this case itās closed, though, and the user can make all kinds of interesting typlos#2018-09-1722:29bbrinckYep, I think spell-spec is a good fit then#2018-09-1806:41misha@U0HM5MJ5V you also could do (s/and (s/keys ...) (s/map-of #{<allowed-keys-only>} any?))
to see whether this behavior is actually what you need. saves you custom macro or dependency.#2018-09-1816:46spieden@U051HUZLD great, thanks! didnāt realize map-of
was strict#2018-09-1817:00mishawell, #{}
is strict, but yeah#2018-09-2019:22spiedenah of course#2018-09-1806:22esp1Is there a way to have predicates return more detailed error messages? I have some complex predicates that operate on subtrees to do things like make sure all branches of the subtree are of the same type. Instead of just having the predicate return false at the root, I'd like to be able to have the predicate provide detailed path information about where within the subtree it detected an error. It would be great if I could have the predicate behave like the built-in spec functions so it could push this detailed information into the explain-data format. Is this possible to do in any supported way without reverse engineering spec internals?#2018-09-1806:45mishadecomposing your predicates in simpler ones, and recomposing them with spec's s/and, s/or
and s/*?+
comes to mind. former will bring you closer to "can understand the exact issue from a predicate form w/o error messages", latter will give you exact paths in s/explain-data. @esp1#2018-09-1806:46mishathere is also https://github.com/bhb/expound#error-messages-for-predicates#2018-09-1807:46esp1Thanks for the reply @misha! I don't think decomposing the predicate will really work for me though; let me give some more detail about why. The structure I'm spec'ing is essentially a lisp-like expression syntax. For instance I want to validate that in the expression [ := :Foo/bar {:a 1, :b {:c "blah"} } ]
that the arguments to the :=
comparison are both of the same type. The predicate to do this looks up the type schema I have associated with the :Foo/bar
argument (which looks something like {:a :int, :b {:c :int} }
and checks if the map argument matches this structure. In this example the type schema says that the value of :c
should be an integer, but the value is actually a string, so validation should return an error. I'd like the error message to report the expected and actual types, along with the fact that the type mismatch error occurs in the second argument value at the path [:a :c]
. If I just have a boolean predicate I'd just get a spec error indicating that a type mismatch error occurred at the root level. The ability to associate error messages with expound is nice - I hadn't realized you could do that. I suppose I could use a custom printer in expound that essentially walks the argument subtree again once a spec error occurs and prints out the error detail. This feels kinda hacky tho. I was hoping I could make the predicate such that it could provide detailed error info directly into the structure returned by s/explain-data
.#2018-09-1812:47Timo Freibergcan i use a s/or
spec but have conform only return the value instead of [:key val]
?#2018-09-1813:10Timo Freibergnot sure how ugly this is, but i "solved" it by wrapping the s/or
in an s/and
and putting a (s/conformer second)
at the end#2018-09-1813:11Timo Freibergso
(s/and
(s/or :a #{"a"} :b ::b-spec)
(s/conformer second))
#2018-09-1813:12taylor(s/conform (s/nonconforming (s/or :i int? :s string?)) "foo")
=> "foo"
#2018-09-1813:13taylor@UA11M92KE s/nonconforming
is what you want, I think#2018-09-1813:20Timo Freibergah, awesome! that makes it easier#2018-09-1813:29misha@esp1 well, sounds like you already have destructured spec for :Foo/bar, which should give you detailed paths to errors. You just need to figure out how to validate against it w/o enclosing it in an opaque predicate#2018-09-1813:49misha#2018-09-1813:54mishanumber of layers and complexity of this would depend on the diversity of the expressions you want to support.#2018-09-1813:56mishaway simpler version might be just
(defn validate-expr [expr]
(s/assert (s/tuple keyword? any? any?) expr)
(let [[op spec y] expr]
(s/assert spec y)))
#2018-09-1813:57mishaI assume, this is not the case, hence extra layer of case
or multimethod
#2018-09-1818:56jrychterLooking for advice ā I've been looking to eliminate some verbosity in my specs and came up with this:
(defn sorted-set-of [spec]
(s/and (s/coll-of spec)
(s/conformer (partial apply sorted-set))))
Will this bite me in the future? (using a defn
to define specs) Will it play well with generators once I get to those?#2018-09-1819:00Alex Miller (Clojure team)You donāt need that - look into :into in coll-of#2018-09-1819:03Alex Miller (Clojure team)(s/coll-of int? :into (sorted-set))
does what you want#2018-09-1819:03Alex Miller (Clojure team)With whatever for int?#2018-09-1819:06jrychterOh, thank you, I didn't know about :into
! Seems like it's exactly what I need. Trying it out.#2018-09-1819:07Alex Miller (Clojure team)Will work with gen automatically too#2018-09-1901:25richiardiandreawhat is a good intermediate value for quick-check's :max-size
?#2018-09-1901:26richiardiandreaThe default kind of takes forever for me for a couple of generative tests#2018-09-1901:26richiardiandreaI am afraid 5, which is fast, is too small#2018-09-1901:26taylorFWIW I think the default is 200#2018-09-1901:28richiardiandreayep it is 200
#2018-09-1901:41richiardiandreawow, I had to reduce :max-size
to 10
#2018-09-1901:44richiardiandreadon't even know what or why I am doing this frankly, any feedback is very welcome š#2018-09-1901:45taylora lot depends on what youāre generating and what youāre doing with the generated data. if itās any thing non-trivial, itās not unusual for generative tests to take a little while#2018-09-1901:45Alex Miller (Clojure team)the slow part is usually generating (too large) input values#2018-09-1901:46richiardiandreaI also added a s/coll-of
:gen-max
to 5#2018-09-1901:46Alex Miller (Clojure team)the most common thing I find is large collections (default max size per coll is 20) - usually I set :gen-max 3
in all of my s/coll-ofās#2018-09-1901:46richiardiandreaš#2018-09-1901:46Alex Miller (Clojure team)recursive / nested specs can also be a problem but less common than the above#2018-09-1901:47richiardiandreaI have some kind of nasty :fn
checks:
(s/fdef generate-events
:args (s/cat :options :my-ns/options)
:fn (s/and #(count-matches-number? (-> % :args :options :number) (:ret %))
#(data-nil-or-object? (:ret %))
#(event-key-matches? (-> % :args :options) (:ret %))))
#2018-09-1901:49taylor:fn
can also just be a simple predicate function, not sure if thatād have much impact#2018-09-1901:50taylor#(and (count-matches-number? (-> % :args :options :number) (:ret %))
(data-nil-or-object? (:ret %)) ...)
#2018-09-1901:50richiardiandrearight, lemme try that#2018-09-1901:52richiardiandreathis seems to be definitely faster!#2018-09-1901:52richiardiandreathanks! didn't thing about that š#2018-09-1901:53taylorI imagine evaluating that s/and
spec in each iteration is more expensive than a single predicate#2018-09-1901:54richiardiandreaI was able to get up to :max-size 50
with that simple trick, I guess reporting will be different but that's ok for now#2018-09-1901:48richiardiandreaso I think this is where it's all clogged...#2018-09-1901:48richiardiandreain ClojureScript too so I don't know, i might be many things#2018-09-1901:48richiardiandreabut thanks for the answer#2018-09-1901:49richiardiandreaif I go :max-size 20
it is taking more than I can wait for š 10
is good#2018-09-1901:55richiardiandreathe trick in the tread above by @taylor makes things acceptable again:
:fn #(and (count-matches-number? (-> % :args :options :number) (:ret %))
(data-nil-or-object? (:ret %))
(event-key-matches? (-> % :args :options) (:ret %))))
#2018-09-1901:56richiardiandreainstead of s/and
#2018-09-1905:36mishadid you try to rearrange s/and
clauses?#2018-09-1905:39mishait seems to me, at this point you could benefit from custom generator(s)#2018-09-1907:11tianshu(s/valid? any? ::s/invalid)
clojure.spec.alpha/invalid will always fail on spec vaild? so I can't write code like this:
(if-let [x x]
x
::s/invalid)
because if-let
use spec to validate the arguments.#2018-09-1907:14mpenetyou can cheat your way around this:#2018-09-1907:14mpenet(def si :clojure.spec.alpha/invalid)
(if-let [x si]
:foo
:bar)
#2018-09-1907:15mpenetit works also for your example#2018-09-1907:15mpenet(if-let [x true]
si
:bar)#2018-09-1907:15mpenetbut yeah that's unfortunate#2018-09-1907:16mpenetthere are a few jira issues about it already : https://dev.clojure.org/jira/browse/CLJ-1966#2018-09-1907:19tianshuyeah, if this is the only problem I think it's okay#2018-09-1907:22mpenetI only encountered this when writing custom conformers, which is something quite rare#2018-09-1907:23tianshuYes, I'm writing custom conformers...#2018-09-1921:22roklenarcichm I've got a head scratcher. I have two types of keys in a hashmap (keywords and strings). If key is string then spec A
needs to be applied to value, if not then spec B
is applied to value#2018-09-1921:23roklenarcicwhat's the best way to achieve this#2018-09-1921:23roklenarciccurrently I use seq
to change map to sequence of pairs#2018-09-1921:23roklenarcicthen it's trivial to spec#2018-09-1921:44richiardiandrea@roklenarcic I do that with exclusive maps and I use s/or
#2018-09-1921:45richiardiandreayou could also use multi specs#2018-09-1921:45richiardiandrea(if I understand the problem correctly)#2018-09-1922:16Alex Miller (Clojure team)btw, rather than seq to pairs, you can just spec as s/every of s/tuple on the map#2018-09-1922:17Alex Miller (Clojure team)thatās a perfectly valid way. alternately, I think given just two choices, Iād lean on s/or over s/multi-spec for this#2018-09-1922:18roklenarcicthx#2018-09-2007:07jaihindhreddyIs there a catenation which doesn't care about order?
I have specs ::a
, ::b
, ::c
and ::d
.
I want a catenation in which ::a
and ::b
are required, ::c
and ::d
are optional and order doesn't matter. Is this possible with spec?#2018-09-2012:01Alex Miller (Clojure team)Sounds like s/keys*#2018-09-2012:02Alex Miller (Clojure team)(s/keys* :req [::a ::b] :opt [::c ::d])
#2018-09-2012:03Alex Miller (Clojure team)Does what you said#2018-09-2012:12djtango^ does keys work on non-maps?#2018-09-2012:54Alex Miller (Clojure team)s/keys* works on sequential collections#2018-09-2012:58Alex Miller (Clojure team)user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::a int?)
:user/a
user=> (s/def ::b string?)
:user/b
user=> (s/def ::c keyword?)
:user/c
user=> (s/def ::d symbol?)
:user/d
user=> (s/def ::s (s/keys* :req [::a ::b] :opt [::c ::d]))
:user/s
user=> (s/conform ::s [::b "abc" ::c :wow ::a 100])
#:user{:b "abc", :c :wow, :a 100}
user=> (pprint (take 5 (s/exercise ::s)))
([(:user/a 0 :user/b "" :user/d q :user/c :T)
#:user{:a 0, :b "", :d q, :c :T}]
[(:user/a -1 :user/b "" :user/d !D/- :user/c :oh)
#:user{:a -1, :b "", :d !D/-, :c :oh}]
[(:user/a 1 :user/b "kK") #:user{:a 1, :b "kK"}]
[(:user/a -4 :user/b "3" :user/d Jw/R5 :user/c :?jH)
#:user{:a -4, :b "3", :d Jw/R5, :c :?jH}]
[(:user/a 0 :user/b "Y6" :user/d NS.+_.H/-)
#:user{:a 0, :b "Y6", :d NS.+_.H/-}])
#2018-09-2017:14fbelinethere is a way to do exactly what keys*
does but using quoted symbols instead of keywords?
ex:
(s/conform ::foo [:source1 '-> :target1 'guard #() 'action #()])
Where the 'guard and the 'action maybe defined or not but the :source1 '-> :target1
should be required.#2018-09-2017:16Alex Miller (Clojure team)no#2018-09-2017:17Alex Miller (Clojure team)keys* is capturing the common Clojure idiom of kwargs passed to a function (but those are always keywords)#2018-09-2017:17Alex Miller (Clojure team)I would probably use regex for that#2018-09-2017:19Alex Miller (Clojure team)(s/cat :s keyword? :_ #{'->} :t keyword? (s/* (s/alt :guard-pair (s/cat :_ #{'guard} :g any?) :action-pair (s/cat :_ #{'action} :a any?))))
#2018-09-2017:21Alex Miller (Clojure team)or could be simpler if you want to spec less of the options (s/cat :s keyword? :_ #{'->} :t keyword? (s/* (s/cat :op symbol? :val any?)))
#2018-09-2110:44jaihindhreddy@U064X3EF3 thank you sir! picard-facepalm I should have perused the clojure.spec.alpha
docs.#2018-09-2414:05djtangooh wow didn't know that - but I suppose it makes sense for s/keys
to work like that#2018-09-2007:13andy.fingerhutYou could probably do the s/or of all the possible orders, but that would be pretty tedious and error prone.#2018-09-2007:14andy.fingerhutWhy do you want a catenation if the order doesn't matter? Are these args to a function?#2018-09-2007:19jaihindhreddyTrying to implement serialization and deserialization of a protocol using conform
and unform
.#2018-09-2007:19jaihindhreddyTo see just how much of it can be pushed into spec? This is the last step.#2018-09-2007:22jaihindhreddyActual use case contains ~20 specs, so s/or
ing 20 factorial orders is out of the picture.#2018-09-2016:31lilactownI think I keep asking roughly the same question in different contexts but hoping someone can help me understand the best way to approach this:
I need to programmatically create a spec based on external data (specifically, a GraphQL schema/query). Is there any good examples out there of how to approach this?#2018-09-2017:01taylorI think two current options depending on type of spec are either use spec's internals or eval
, and I think spec-tools lib has some features for this#2018-09-2017:38favilaWe've done this with code-generation#2018-09-2017:39favilabuild up quoted lists of code then write it out to a clj file#2018-09-2017:39favila(pprinted)#2018-09-2017:40favilaif you have a deterministic order of evaluation (e.g. sorting the specs by name or topology or something) you will also see diffs clearly#2018-09-2115:00hjrnunesHi,
I have a map for which there are several possible keys the values of which I'd like to use the same spec, for.
So ideally I'd do something like
(s/def :key/spec (some spec...))
(s/def :key/spec2 :key/spec)
(s/def :map/spec (s/keys :opt-un [:key/spec :key/spec2])
But I don't seem to be able to do this (s/def :key/spec2 :key/spec)
.
Ideas?#2018-09-2115:01taylorI think I've done this same thing before and it worked#2018-09-2115:05hjrnuneshmm, then maybe it's something else. I'll double check, thanks#2018-09-2115:18Alex Miller (Clojure team)should work. you say ādonāt seem to be able to do thisā - in what way?#2018-09-2115:57lilactownI'm real confused. I'm trying to spec a function, and this is the error I'm seeing in my tests now:
-- Spec failed --------------------
Function arguments
:asur
should satisfy
(cljs.spec.alpha/* any?)
#2018-09-2115:57lilactownI'm real confused. I'm trying to spec a function, and this is the error I'm seeing in my tests now:
-- Spec failed --------------------
Function arguments
:asur
should satisfy
(cljs.spec.alpha/* any?)
#2018-09-2116:00taylorI think it's complaining that :asur
isn't a sequence#2018-09-2116:01lilactownthis is my fdef:
(spec/fdef get-of :args (spec/* any?))
#2018-09-2116:01tayloris get-of
variadic?#2018-09-2116:01lilactownyes#2018-09-2116:01taylor(s/conform (s/* any?) :asur)
=> :clojure.spec.alpha/invalid
#2018-09-2116:02lilactownI tried this in a CLJ REPL with a slightly stricter spec:
user=> (spec/fdef get-of
#_=> :args (spec/cat
#_=> :key keyword?
#_=> :path (spec/* keyword?)))
user/get-of
user=> (defn get-of [key & things] key)
#'user/get-of
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (stest/instrument)
[user/get-of]
user=> (get-of :asur)
:asur
user=> (get-of :asur :bsh)
:asur
#2018-09-2116:03taylor#2018-09-2116:03lilactownwell that exact spec fails the same way in my tests#2018-09-2116:05lilactown-- Spec failed --------------------
Function arguments
:asur
should satisfy
(cljs.spec.alpha/cat
:key
keyword?
:path
(cljs.spec.alpha/* any?))
#2018-09-2116:09taylorhmm, not sure. this works for me:
(defn foo [& xs] (prn xs))
(s/fdef foo :args (s/* any?))
(st/instrument `foo)
(foo 1 2 3)
#2018-09-2116:09lilactownright. hence my WTF#2018-09-2116:11taylorin times like these, I restart REPL/computer/get back in bed#2018-09-2116:11lilactownhahaha#2018-09-2116:11lilactownI have restarted my REPL and everything. bed is tempting#2018-09-2116:11lilactownI wonder if there's something weird going on with CLJS instrumentation#2018-09-2115:58lilactownthe test output/stacktrace doesn't give the line # or file or anything. but I'm confused WHY the arguments wouldn't satisfy that spec??#2018-09-2116:21lilactownoh for crying out loud: https://dev.clojure.org/jira/browse/CLJS-2793#2018-09-2117:03Alex Miller (Clojure team)doh!#2018-09-2117:03Alex Miller (Clojure team)sorry, I was unaware of that one#2018-09-2117:00jrychterI'm writing up my thoughts on clojure.spec, having tried to use it for several months now in an application. I'm banging my head against multiple issues. So far it seems like most of them are related to data conversion: if I removed all conformers and :into
from my specs and used separate explicit functions for data adjustments, I think my life with spec would be easier. But most writeups and presentations about spec tantalizingly mention conforming data as a major advantage...#2018-09-2117:01Alex Miller (Clojure team)conforming != coercion#2018-09-2117:01jrychterThat is something that I think I missed (I don't think I'm the only one).#2018-09-2117:02Alex Miller (Clojure team)conforming was always intended to be an advanced tool to use in writing spec implementations, not for data transformation#2018-09-2117:02Alex Miller (Clojure team)it is intentionally absent from the spec guide#2018-09-2117:02jrychterAnd yet there is :into
(in the guide, too).#2018-09-2117:03Alex Miller (Clojure team)yes, and is useful for some use cases (+ for gen), but that doesnāt make it a generic transformation tool#2018-09-2117:03Alex Miller (Clojure team)https://github.com/wilkerlucio/spec-coerce is imo a pretty good approach to leveraging specās for the purpose of coercion#2018-09-2117:13jrychterHmm. I think there is confusion around the issue (see for example https://stackoverflow.com/questions/45188850/is-use-of-clojure-spec-for-coercion-idiomatic). I think it's worth mentioning explicitly in the guide. I also found that there is not a lot of guides for spec. The "Spec Guide" is great, but I see it as more of a walktrough. I've been looking on tips on how to use spec in apps (e.g. do I use s/*
or s/coll-of
?). Most online presentations or tutorials are introductory material only.#2018-09-2117:14jrychter@alexmiller I have a file with my notes on spec (basically documenting the holes I've been falling into). I could E-mail it to you, would you like it? It would let you know what my (erroneous) thinking was.#2018-09-2117:17Alex Miller (Clojure team)the answers on that stackoverflow thread seem pretty straightforward and correct to me :)#2018-09-2117:17jrychterYes.#2018-09-2117:17Alex Miller (Clojure team)youāre welcome to email, but I am just about to enter the Strange Loop black hole and wonāt be looking at anything for Clojure for the next week and a half#2018-09-2117:18jrychterThat's fine. I don't want you to answer that E-mail anyway. I'm not looking for support. I just thought that if I was the author of the spec guide, I'd want to know what some misconceptions are in readers' minds.#2018-09-2117:19Alex Miller (Clojure team)happy to hear those - in fact, an issue on https://github.com/clojure/clojure-site would be just as good#2018-09-2117:19Alex Miller (Clojure team)at the moment, Iām not looking to invest a lot of time in additional spec docs because we are starting to work on some spec changes that are likely to change some of whatever advice we would give#2018-09-2117:23jrychterAn issue it will be, then. Perhaps the holes I'm falling into will influence some of the thinking behind spec changes š#2018-09-2117:25Alex Miller (Clojure team)I do have a full day of spec materials Iāve taught several times as a course now#2018-09-2117:25Alex Miller (Clojure team)given time, could be turned into some useful advice#2018-09-2117:33seancorfield@jrychter FWIW, we do use spec for some very limited coercion but, as noted in one of those SO answers, we also have two types of spec: "API (coercing) specs" and "domain specs" (non-coercing). All of our internal specs are non-coercing. And even in the API specs, we only do very limited coercion: we accept strings-that-can-be-converted-to-<T> and produce values of type T, for longs, doubles, Booleans, and date/time values. That's it.#2018-09-2117:33roklenarcicis it possible to create a generator from a specced function (i.e. one with a fdef
)?#2018-09-2117:35seancorfieldWe've found that to be extremely convenient because if you have a form field that should be a long, it's going to come in as a string so you either have a spec that attempts coercion to long but still conforms to the string and then you need an actual coercion as well, or you have a layer of coercion first followed by specs for whatever successfully coerces -- and then you have two layers in your error handling which makes for more complicated code.#2018-09-2117:35jrychter@seancorfield Well, my coercions were very limited, too. Strings to keywords and collection types, basically (to keep a collection as a set or a sorted set). But even then I'm falling into traps. An example is that I though that s/valid?
tells you if your data is valid according to the spec. It doesn't. It tells you if the data will be valid if you pass it through s/conform
.#2018-09-2117:37seancorfieldRight, well the pattern we use is (let [params (s/conform ::api-spec input-params)]
(if (s/invalid? params)
(respond-with-error (s/explain-data ::api-spec input-params))
(happy-path params)))
#2018-09-2117:37jrychterI'd argue that parsing strings to longs is not really coercion, it's parsing (my form code does it).#2018-09-2117:39jrychter@seancorfield Yes, I discovered this the hard way. Problem is that if you spec your functions, then data that is not valid (but conformable) will reach your function code.#2018-09-2117:39seancorfieldWe don't spec many functions -- we mostly spec data structures.#2018-09-2117:40seancorfieldAnd, like I say, these are specs at the outer boundary of our system. Inside our system we only have non-coercing specs.#2018-09-2117:40jrychterI will try to remove all conformers (`s/conformer` and :into
) from my specs and use explicit coercion. I'd love to be able to "attach" information about coercions to the specs, otherwise I have to manually define coercion functions and deal with nested coercions.#2018-09-2117:41seancorfieldAnd our use of spec is almost all explicit and part of our production code. So we don't rely on instrumentation of functions that way.#2018-09-2117:42jrychter@seancorfield That's how I see it, too, but I'm still considering instrumenting functions, and looking for possible uses of spec.#2018-09-2117:44jrychterSo, at a first glance, that spec-coerce
library tries to do too much. I can see parsing from strings to doubles or integers. I'd rather have a way to attach custom coercion fns to specs and a way to walk a spec and apply all coercion fns that I specified.#2018-09-2117:45seancorfieldInstrument (and check) are purely part of testing for us -- as well as using exercise to produce random conforming data for use in some example-based tests.#2018-09-2117:45jrychterI think I will also remove all uses of s/coll-of ... :into
and convert those into explicit (s/and (s/coll-of ::something) sorted? set?)
or similar. That will let me use s/valid?
to catch instances of data where something isn't a sorted set.#2018-09-2117:45seancorfieldAs an FYI, from our Clojure codebase: Clojure build/config 48 files 2538 total loc
Clojure source 268 files 63902 total loc,
3331 fns, 658 of which are private,
403 vars, 42 macros, 71 atoms,
478 specs, 19 function specs.
Clojure tests 149 files 20002 total loc,
23 specs, 1 function specs.
#2018-09-2117:47jrychterI have ~45k lines of code in my app right now, so not nearly there, but close.#2018-09-2117:49Alex Miller (Clojure team)I think itās useful to think of the purpose of conform as āwhy does this value conform to the spec?ā and not ātransform this value into some target valueā#2018-09-2117:50jrychterI do think that s/valid?
is not a good name, though.#2018-09-2117:51Alex Miller (Clojure team)becauseā¦#2018-09-2117:52jrychterBecause it doesn't tell me if my data is valid.#2018-09-2117:52Alex Miller (Clojure team)sorry, donāt understand#2018-09-2117:53roklenarcic64k of lines of clojure is a lot of clojure š#2018-09-2117:55jrychter(s/def ::stuff (s/coll-of int? :into #{}))
(s/valid? ::stuff [3 2 1])
(s/def ::other-stuff (s/and (s/conformer keyword) keyword?))
(s/valid? ::other-stuff "not-a-keyword-at-all")
#2018-09-2117:56jrychterBoth s/valid?
will return true
, even though the data, as supplied to s/valid?
, will cause problems in functions that expect a set and a keyword, respectively.#2018-09-2117:58bbrinckFor the first spec, would :kind set?
be more accurate?#2018-09-2118:00jrychterYes, I should remove all conformers and :into
from my specs. My understanding right now is: s/valid?
tells you if the data will be valid when conformed, so do not use s/valid?
to check if the data is valid if you use conformers or :into
anywhere in your specs.#2018-09-2118:00bbrinckYes, the examples seems like concrete cases where using s/conformer
for coercion creates headaches#2018-09-2118:01jrychterHence my point about s/valid?
not telling me if my data is valid.#2018-09-2118:01favilavalidity is only testable after conforming#2018-09-2118:01favilaconforming isn't something separate that exists outside of an abstract expression of the desired result shape#2018-09-2118:02favilait's a predicate that also transforms#2018-09-2118:04bbrinckCorrect, if you use conformer
to do coercion (which is not recommended), the results of valid?
are confusing. Also, some details of explain-data
get confusing because the paths to data can change. This is one reason why coercion is not recommended, AIUI#2018-09-2118:04favilaconforming is super-handy though#2018-09-2118:05favilabut using it means you are indirectly specing the "input"#2018-09-2118:05favilathe input must be something-that-i-can-make-into-something-else#2018-09-2118:05favilamaybe think of conform as a normalization step#2018-09-2118:06favilarather than coersion or parsing#2018-09-2118:06favilathat said I have written parsers using conform#2018-09-2118:06Alex Miller (Clojure team)conform is not normalization or coercion or parsing#2018-09-2118:06favilawhat's left?#2018-09-2118:07Alex Miller (Clojure team)destructuring#2018-09-2118:07favila?#2018-09-2118:08Alex Miller (Clojure team)if the value matches the spec, it will destructure the input value to tell you which alternative was chosen in specs with alternates and which parts of the data matched which components of the spec#2018-09-2118:08jrychterThat's how I understand it now, too.#2018-09-2118:09favilaok, I'm talking about conformers#2018-09-2118:09Alex Miller (Clojure team)tags exist in s/or, s/alt, s/cat so that parts and choices can be labeled in the conformed (destructured values)#2018-09-2118:09favilanevermind we are talking past each other#2018-09-2118:09Alex Miller (Clojure team)conformers exist to build complex composite spec implementations from existing pieces#2018-09-2118:12favilawhat would be an example of that? s/keys* does not appear to use conformers#2018-09-2118:15Alex Miller (Clojure team)oh, I might have called out the wrong example (s/keys* uses s/&). might be remembering s/nilable (which actually has been rewritten since for performance)#2018-09-2118:17favilaah but the & predicate is a conformer#2018-09-2118:18favilait appears to be the only use in spec#2018-09-2118:19favilaso I think you are saying that the intended use of conformers is to act as glue to chain specs together#2018-09-2118:19favilait's not meant to make s/conform visible alternations#2018-09-2118:21Alex Miller (Clojure team)yes#2018-09-2118:09Alex Miller (Clojure team)like building s/keys* from s/keys#2018-09-2118:10Alex Miller (Clojure team)conform is not transformation, conformers are not coercion#2018-09-2118:11jrychterSo, I'm looking for suggestions: what is the simplest way to add my own coercion functions to certain specs and then walk a spec calling my functions? Metosin's spec-tools do this by wrapping specs, which I really do not like. Is that the only way?#2018-09-2118:11Alex Miller (Clojure team)https://github.com/wilkerlucio/spec-coerce#2018-09-2118:11Alex Miller (Clojure team)^^ is vastly preferable to spec-tools imo#2018-09-2118:12jrychter@alexmiller At a first glance, this seems to try to do too much. I don't want to parse strings to integers by default. I only want to walk the spec and call specific coercion functions that I provided. But perhaps I'm missing something, I'll look deeper.#2018-09-2118:13favila@jrychter spec really pushes you in the direction of using different keys#2018-09-2118:14favilai.e. if you care about both the "raw" data and the "cleaned-up" (slightly-parsed, whatever) data as independent specs, you need separate keys#2018-09-2118:14jrychterIt seems that with spec-coerce
you have to use coerce-structure
to walk a complex (nested) spec, duplicating your data structure in the call.#2018-09-2118:19jrychterBasically, I'm looking for something that would let me do this:
(s/def ::kw (s/and keyword? (s/coerce-fn keyword)))
(s/def ::nested (s/keys :req-un [::kw]))
(s/def ::data (s/keys ::req-un [::nested]))
(coerce ::data {:nested {:kw "x"}}) => {:nested {:kw :x}}
ā¦without touching anything that doesn't have a coerce-fn
defined.#2018-09-2118:25favilayeah spec-coerce seems backward; I would want to opt-in to coercion, not have universal coersion predicates#2018-09-2118:26bbrinckSeems like a sensible suggestion to have the option to turn off the default coercions#2018-09-2118:26bbrinck(a suggestion for spec-coerce project, I mean)#2018-09-2118:27favilaI'll reiterate though that spec really seems to urge in the direction of more specs#2018-09-2118:27favilai.e. a separate "edge" spec vs an internal spec#2018-09-2118:28favilaso in your example "::kw as a string encoding a keyword" is different from "::kw as a kw"#2018-09-2118:28favilaand then rename keys everywhere#2018-09-2118:29favila(if using s/keys with namespaced maps)#2018-09-2118:36bbrinck@jrychter A challenging thing in your example above is that the coerce-fn is tied to something global (the qualified kw), but your data {:nested {:kw "x"}}
is presumably figuring out that :kw
means ::kw
based on context?#2018-09-2118:36jrychterI need a way to walk a spec, which doesn't seem to exist. I thought about defining multimethods dispatching on spec keys, but that won't work for unqualified keys.#2018-09-2118:37jrychter@bbrinck I'm passing all information about the context as the first argument to coerce
: the ::data
spec knows everything.#2018-09-2118:37bbrinckright, but youād have to walk the spec to know what spec :kw
means#2018-09-2118:38bbrincklike you said above, that adds a requirement to walk the spec to know where you are in the spec#2018-09-2118:40jrychterYes, but I thought that is what spec does anyway (when, say, conforming).#2018-09-2118:41bbrinckcorrect, youād need to duplicate this process. itād be an interesting project, Iād need to dig more into the spec impl to understand how viable itād be to duplicate#2018-09-2118:42faviladoesn't spec-tools have a spec visitor?#2018-09-2118:42bbrinckOTOH, if you could assume that the name unqualified keyword always used the same coercion, the problem is simpler#2018-09-2118:43bbrinckwhich doesnāt seem entirely crazy. Perhaps naively, Iād think that if you want to, say, convert :zip-code
from string->int in one place, you probably want to do the same coercion elsewhere#2018-09-2118:43bbrinckmaybe thatās oversimplifying the real world case š#2018-09-2118:44jrychterNot necessarily, but I could stick with that for a while as a workaround. That would mean I could define a multimethod coercer and walk the structures myself. But then spec provides me very little value in the end.#2018-09-2118:45bbrinckHm, well spec would still let you validate the result of coercion (and conform to use destructuring)#2018-09-2118:45bbrinckhttps://gist.github.com/bhb/7fde08abc1f3f5d06b05481b4e200614#2018-09-2118:46favilaif you used the "option" keys in a consistent way, s/conform could perform the tagging for you (except for s/keys req-un and opt-un)#2018-09-2118:46bbrinckNote that if you donāt to have to call two different implementations of def
for :kw
, then you could just make a macro that does both e.g.#2018-09-2118:47bbrinckhttps://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L930-L940#2018-09-2118:47bbrinck@jrychter (an example of a def
macro that calls the normal def
but also registers a message. In your case, youād be registering a spec + coercer)#2018-09-2118:51bbrinck@jrychter whoops, I messed up the example in that gist. Please reload, Iāve updated. The example should have been: (coerce-structure {:nested {:kw "x"}}) ;; => {:nested {:kw :x}}
#2018-09-2118:51jrychterThat is interesting. It's not as good as clojure.spec
calling my coercions (clojure.spec would know exactly which function to call), but I guess it is a workaround.#2018-09-2119:01jrychter@bbrinck Thank you for that gist. I am trying to implement that right now to see how it goes.#2018-09-2119:03bbrinck@jrychter np. Good luck! Definitely a workaround, but hopefully covers the 80% case. Let me know how it goes š#2018-09-2119:11jrychter@alexmiller Can we hope that this use case (explicit coercion using user-registered coercion functions) will be considered in future spec work?#2018-09-2119:12Alex Miller (Clojure team)you can hope for anything you like :)#2018-09-2119:12Alex Miller (Clojure team)not something weāre working on currently#2018-09-2119:18jrychterLet's narrow it down a bit: at least a way to walk a spec, calling a function with the keyword and value at each point. That would still mean maintaining a separate registry, but that's fine.#2018-09-2119:27jrychter@bbrinck I'm discovering more limitations as I go ā for example, a certain spec might be redefined as s/nilable
in some places under the same (when unqualified) keyword.#2018-09-2119:30bbrinck@jrychter Right, itās probably worth making all coercing functions a little bit defensive i.e. if the pre-coercion type isnāt what you expect, just return the existing value#2018-09-2119:30bbrinckso, untested, but something like (if (string? x) (keyword x) x))
#2018-09-2119:31bbrinckKeep in mind the role of coercion functions is not to validate the incoming data, but rather do a best-effort attempt to get the fields into a format that will be valid according the spec#2018-09-2119:32jrychterOf course. And the suggestion about being defensive is a very good one.#2018-09-2119:55jrychterAnd another limitation (as I'm going through my code): if a spec is redefined under a different name, coercion needs to be redefined for that new name, too. It won't carry over.#2018-09-2120:05bbrinckVery true, itās per name, not per spec#2018-09-2120:06bbrinckYou could do some magic looking at s/form
for a spec and then look up that spec ā¦ but itād get complicated š#2018-09-2118:22josh.freckletonIs spec the right tool for validating data from JSON and db results (if so, any docs or blogs about it?), or should I stick with Schema?#2018-09-2118:24Alex Miller (Clojure team)spec is somewhat cumbersome right now for validating maps with unqualified keys (which tends to include cases like JSON)#2018-09-2118:24Alex Miller (Clojure team)the next round of spec changes will have some improvements in this area#2018-09-2118:26roklenarcicIs there some way to refer to args spec of a function?#2018-09-2118:29seancorfield@roklenarcic Can you provide more context? What problem are you trying to solve?#2018-09-2118:31roklenarcicI'm trying to generate arguments to a function that is specced#2018-09-2118:31roklenarcicbut without calling it#2018-09-2118:32roklenarcicso I have a function and a bunch of fdefs#2018-09-2118:32roklenarcicI mean one fdef for this particular one#2018-09-2118:32roklenarcicand I want to generate vectors of arguments#2018-09-2118:35Alex Miller (Clojure team)yes, if you get a function spec, it implements key lookup so you can grab :args, :ret, :fn out of it#2018-09-2118:36Alex Miller (Clojure team)(-> a-sym s/get-spec :args)#2018-09-2118:50roklenarciccool#2018-09-2118:51roklenarcicthat's what I was looking for š#2018-09-2119:23ikitommihave you @alexmiller used either of spec-coerce or spec-tools?#2018-09-2119:25jrychterNo, but I took a look at both. Both do something different, not quite what I want, and both are way too complex.#2018-09-2119:27ikitommiI feel both are hacks (sorry @wilkerlucio), because how spec is designed.#2018-09-2119:30ikitommiletās look at an example:#2018-09-2119:30ikitommi(require '[clojure.spec.alpha :as s])
(s/def :db/ident qualified-keyword?)
(s/def :db/valueType (s/and keyword? #{:uuid :string}))
(s/def :db/unique (s/and keyword? #{:identity :value}))
(s/def :db/cardinality (s/and keyword? #{:one :many}))
(s/def :db/doc string?)
(s/def :simple/field
(s/cat
:db/ident :db/ident
:db/valueType :db/valueType
:db/cardinality (s/? :db/cardinality)
:db/unique (s/? :db/unique)
:db/doc (s/? :db/doc)))
(s/def :simple/entity
(s/+ (s/spec :simple/field)))
(def value
[[:product/id :uuid :one :identity "id"]
[:product/name :string "name"]])
(s/valid?
:simple/entity
value)
; true
#2018-09-2119:31ikitommiI think spec really shines here (the regex parts).#2018-09-2119:31ikitommibut same over json:#2018-09-2119:31ikitommi(def json-value
[["product/id" "uuid" "one" "identity" "id"]
["product/name" "string" "name"]])
(s/valid?
:simple/entity
json-value)
; false
#2018-09-2119:33ikitommiout of luck. To make a standalone transformer outside of spec, one would need to be able to parse the spec and basically rebuild the s/conform
to understand what branches are being used.#2018-09-2119:36ikitommispec-tools uses the s/conform
and makes the spec do all the heavy lifting. but sadly, the leaf specs need to be wrapped into āconforming predicatesā.#2018-09-2119:39ikitommispec-coerce reads the spec forms, which works nicely for most/simple forms.#2018-09-2119:47ikitommihopefully there will be a solution for this in the upcoming spec releases, I think it would be bad for the clojure / spec story not to support (or enable libs to fully support) coercion. one s/walk
method on Specs for libs to use? protocol dispatch would be clean and much faster than parsing the forms or using dynamic binding like the current coercion libs do.#2018-09-2119:56jrychter@ikitommi It seems that that is exactly what I have been asking for in that thread (a way to walk a spec).#2018-09-2119:58Alex Miller (Clojure team)that is something we have talked about providing, but itās a ways down the list#2018-09-2119:59jrychterIt would seem that this is something that people with large apps with complex data structures and JSON interop encounter all the time.#2018-09-2120:02seancorfieldWe haven't encountered that yet š#2018-09-2120:03seancorfield(but maybe we're deliberately keeping our data structures simple enough?)#2018-09-2120:03jrychter@seancorfield You seem to be using conformers for that, judging from the code you've shown?#2018-09-2120:04seancorfieldOnly right at the edge -- where I think it's acceptable.#2018-09-2120:04jrychterBut that proves my point. Coercion is needed.#2018-09-2120:04seancorfieldNot "needed". Convenient.#2018-09-2120:05seancorfieldAnd spec provides what we need already.#2018-09-2120:05jrychterI'd argue that it doesn't. Or my expectations are too high: I expected to be able to "spec" my data structures only once, and reuse the resulting structure for coercion. I can't do that right now.#2018-09-2120:06jrychterI can see how this could be outside the scope of spec, but this is why I'm asking (and @ikitommi seems to be, too) for a way to walk the spec data, so that we can extend it and reuse the definitions.#2018-09-2120:23ikitommi@jrychter there is CLJ-2251#2018-09-2120:13Alex Miller (Clojure team)does it seem weird to you to have a āspecā that defines your data, but then require coercion that does not conform to that spec? (it seems weird to me)#2018-09-2120:19ikitommiweb is weird. we expect a valid edn data but get JSON instead ;)#2018-09-2120:23Alex Miller (Clojure team)it seems conceptually cleaner to me to separate the transformation of the crap that you get into the format you want from validating that transformed data#2018-09-2120:24Alex Miller (Clojure team)and I get that youāve already defined the structure and it seems convenient to use that structure to learn about target expectations while doing the transformation#2018-09-2120:24jrychter@alexmiller In theory, yes. But in practice I'm dealing with data from a JSON database, nested data structures. What you are proposing means that I will be duplicating data structure definitions: once for spec, and again for coercion. Seems like a waste.#2018-09-2120:24Alex Miller (Clojure team)and so I think requests that deal with using specs as data in that way make sense#2018-09-2120:25Alex Miller (Clojure team)asking spec to do the work seems weird to me (but a good case for a lib on top)#2018-09-2120:25wilkerlucio@jrychter had you tried using coerce-structure
from spec-coerce? because thats the indented feature for it, you pass it a structure and it will use the leaf attribute specs to do the coercion#2018-09-2120:26jrychter@alexmiller Actually, I do not want to involve "target expectations" in this. All I want is to reuse the structure from spec and be able to hang my own data (coercion functions in this case) on it.#2018-09-2120:26Alex Miller (Clojure team)well that would potentially be handled by meta support, which weāre in favor of (itās just tricky to implement well)#2018-09-2120:27jrychter@wilkerlucio Yes. I am using a variant of that now, suggested by @bbrinck. But this has a number of drawbacks.#2018-09-2120:27Alex Miller (Clojure team)as a stop gap, itās not hard to build a second registry, also keyed by spec name that had that info#2018-09-2120:27ikitommi@alexmiller agrer that it should be made on top. But we need help from spec to make this possible. E.g. the walk#2018-09-2120:27Alex Miller (Clojure team)yes#2018-09-2120:27jrychter@alexmiller That is exactly what I'm doing right now, and there are a number of limitations, especially with non-namespaced keys.#2018-09-2120:27Alex Miller (Clojure team)that may become clearer soon#2018-09-2120:27wilkerluciospec-coerce also supports a secondary registry to specify, what I would like from spec is to be easier to traverse the spec definitions, I have to do some things that I feel are not very stable to get that#2018-09-2120:28wilkerluciobut I guess specs on the specs will solve this#2018-09-2120:28Alex Miller (Clojure team)maybe#2018-09-2120:29Alex Miller (Clojure team)this is all helpful, in particular Iām trying to clarify the problem statement so I can talk well to Rich about it#2018-09-2120:29Alex Miller (Clojure team)spec walking has come up in several contexts and is one potentially useful generic feature, meta is another#2018-09-2120:30jrychter@alexmiller It seems to me that you think much of this is unnecessary, because invalid data points to deficiencies in transport or db. That is true in general. But please consider that even using EDN you will not get sorted sets by reading. Something needs to coerce sets into sorted sets.#2018-09-2120:30Alex Miller (Clojure team)and better support for unnamespaced attributes#2018-09-2120:30Alex Miller (Clojure team)I didnāt say it was unnecessary. Iāve built my share of apps and I get it.#2018-09-2120:31Alex Miller (Clojure team)I just donāt think that spec necessarily needs to be the thing providing ācoercionā#2018-09-2120:31jrychterOh, absolutely. It's just that it makes sense to re-use the resulting structure.#2018-09-2120:32Alex Miller (Clojure team)Iām agreeing with you on that#2018-09-2120:33Alex Miller (Clojure team)using conformers for this stuff is what Rich calls the āmeat grinderā approach - I see that turning the crank gets me from A to B#2018-09-2120:34ghadiPart of the challenge is that 99% of other libraries provide "coercion" of some sort (plumatic/schema and everything in Python) and there are Real Problems with that stuff. I think it has psychologically primed us to want to tangle it together... some caution is advisable and focus on the problems#2018-09-2120:34Alex Miller (Clojure team)but it misses that the grinder is not the point of it#2018-09-2120:35jrychterHere's a practical example from my screen right now:
(ns partsbox.build-quantity
(:require [clojure.spec.alpha :as s]
[partsbox.coerce :as c]))
;; A build quantity is an integer that is always greater than 0.
(s/def ::quantity pos-int?)
;; A non-empty sorted set of quantities used for build quantities and pricing quantities.
(s/def ::quantities (s/and (s/coll-of ::quantity :min-count 1) sorted? set?))
(c/defcoercion ::quantities (fn [data] (into (sorted-set) data)))
(def +default-quantities+ (s/conform ::quantities (c/coerce ::quantities [1 10 25 50 100 250 500 1000])))
This is all fine (a separate registry, another line of code for defining the coercion). I would just want to be able to write a general c/coerce
that would walk the spec correctly. I can only have a deficient implementation right now.#2018-09-2308:39dottedmagI'd like to instrument a function taking an argument spec'ed in another namespace. When I do (s/fdef my-func :args (s/cat :arg1 :alias/myspec))
it fails to resolve spec via alias. (s/fdef my-func :args (s/cat :arg1 :))
works. Is it intended?#2018-09-2311:36taylorfor ::alias/myspec
to work I think you need to require/alias that namespace e.g. (require '[some.ns :as alias])
#2018-09-2313:52dottedmag@U3DAE8HMG Yep, thanks. I was using wrong syntax (with a single :
).#2018-09-2308:40dottedmagOh, :alias/myspec
vs ::alias/myspec
.#2018-09-2501:46bbrinckIām confused about the behavior of explain
for keys*
specs#2018-09-2501:46bbrinckFor a cat
spec, the in
refers to the to the position of the bad data before conformance#2018-09-2501:47bbrinckbut for a keys*
spec, it seems to refer to the position of the bad data after conformance.#2018-09-2501:47bbrinckWhy is it different? https://gist.github.com/bhb/a7ba32e6965bebecc94020f10751ec58#2018-09-2501:54taylorI wonder if it's because of this https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1787#2018-09-2501:55taylor>takes the same arguments as spec/keys and returns a regex op that matches sequences of key/values, converts them into a map, and conforms that map#2018-09-2515:29bbrinckAh, thanks! Although that explains the mechanism of why the two are different, Iām still not sure on the reasoning š#2018-09-2515:31bbrincki.e. why the difference in behavior is desirable#2018-09-2515:32taylorsame but that's out of my depth š I only noticed the keys*
impl. would seem to cause this behavior#2018-09-2515:33bbrinckYep, I appreciate the link š It does indeed explain the difference. Thx!#2018-09-2515:33taylorI suppose it may be just to make the final conform step easier, when the input has already been conformed to a map :man-shrugging:#2018-09-2515:36bbrinckYeah, maybe! The downside is that when I want to locate the source of the bad data, the in
value is inconsistent in itās format. In particular, itās hard to make a custom printer like Expound be able to locate the bad data#2018-09-2501:47bbrinckIs this a known issue? Or am I misunderstanding the intent here?#2018-09-2519:15roklenarcicHm if I define a spec, it requires that components are already defined. So if I have mutually recursive spec, do I just define one of the components at the top with bogus spec?#2018-09-2519:46taylorI've run into this issue before with a recursive spec, and I think it involved a call to s/spec
inside the recursive spec definition#2018-09-2519:23donaldballIām not sure what you mean by it requiring that components are already defined. Certainly something recursive like this works fine:
(s/def ::json
(s/or :string string?
:list ::json-list
:map ::json-map))
(s/def ::json-list
(s/coll-of ::json :kind vector?))
(s/def ::json-map
(s/map-of ::json ::json))
#2018-09-2520:18nenadalmnote that in some cases it doesn't work in cljs. e.g.: https://github.com/metosin/reitit/issues/127#2018-09-2521:34richiardiandreaHello folks I am having a hard time reading this spec error, I think I need some help in understanding what is wrong:
#### [{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha23581], :clojure.test.check/ret {:shrunk {:total-nodes-visited 0, :depth 0, :pass? false, :result #error {:message Call to #'lambda.response/error-map did not conform to spec:
val: "Invalid body" fails at: [:args] predicate: (cat :message string? :data (? (nilable map?)))
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha23551]
:cljs.spec.alpha/value "Invalid body"
:cljs.spec.alpha/args "Invalid body"
:cljs.spec.alpha/failure :instrument
, :data {:cljs.spec.alpha/problems [{:path [:args], :pred (cljs.spec.alpha/cat :message cljs.core/string? :data (cljs.spec.alpha/? (cljs.spec.alpha/nilable cljs.core/map?))), :val Invalid body, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha23551], :cljs.spec.alpha/value Invalid body, :cljs.spec.alpha/args Invalid body, :cljs.spec.alpha/failure :instrument}}, :result-data {:clojure.test.check.properties/error #error {:message Call to #'lambda.response/error-map did not conform to spec:
#2018-09-2521:36taylorthat spec looks like it may be for a variadic function, so maybe this is related https://dev.clojure.org/jira/browse/CLJS-2793#2018-09-2522:24richiardiandreaLet me see if that's the issue#2018-09-2522:34richiardiandreaWell it seems like things are working with just a non-variadic function, however I am not sure my bug is related to the above#2018-09-2523:16richiardiandreaYep it is relate, I can reproduce it.#2018-09-2521:40guyval: "Invalid body" fails at: [:args] predicate: (cat :message string? :data (? (nilable map?)))
#2018-09-2521:41guyto me it looks like you have a value "invalid body"
#2018-09-2521:41guyand a predicate#2018-09-2521:41guy(cat :message string? :data (? (nilable map?)))
#2018-09-2521:42guySo maybe your predicate / spec is wrong?#2018-09-2521:44guy{:path [:args], :pred (cljs.spec.alpha/cat :message cljs.core/string? :data (cljs.spec.alpha/? (cljs.spec.alpha/nilable cljs.core/map?)))
#2018-09-2521:45guyThis is saying that the args (i guess ur function args)
are being given the value Invalid body
and failing your spec#2018-09-2521:48guyhttps://clojuredocs.org/clojure.spec.alpha/cat
Tbh I donāt really understand how this works#2018-09-2521:49guy@richiardiandrea sorry iām not much help haha#2018-09-2521:49richiardiandreayeah#2018-09-2521:49richiardiandreathat predicate is supposed to match one arguments as string and optionally a data field#2018-09-2521:49guywhat does ur (s/fdef ..)
look like?#2018-09-2521:49guyah right#2018-09-2521:50richiardiandrea(s/fdef error-map
:args (s/cat :message string? :data (s/? (s/nilable map?)))
:ret :tools.lambda.response/error-map)
#2018-09-2523:16taylorwhat's your defn error-map
arg list look like?#2018-09-2523:37richiardiandreait was [message & [data]]
#2018-09-2523:38taylorit could be related to that JIRA I mentioned above about CLJS, instrumentations, and variadic functions#2018-09-2523:56richiardiandreaYep left a comment there, I can reproduce it here...#2018-09-2607:21steveb8nquestion : Iāve heard about context specific specs quite a few times i.e. same entity but different uses e.g. reading vs inserting a map from/to db. another might be reading entities in a list vs individually. I can guess how these specs might look but was hoping I could learn from looking at others. does anyone know of public codebases with examples of this technique?#2018-09-2612:11jaihindhreddy@steveb8n I would be interested to see this too.
But what you describe feels like something spec is not meant to do. Spec specs data, not entities or identities. ::entity-while-reading
, ::entity-while-inserting
might be better as two different specs. Maybe metosin/spec-tools
can do this.#2018-09-2612:14steveb8nagreed. I think each context is probably itās own spec.#2018-09-2612:59favilaYeh spec doesnāt support this, not sure where you heard about that. If you find out Iām interested#2018-09-2613:03favilaThere are two cases that hurt where I want this. One is dealing with datomic data. Because itās a graph each ref key could be (in the widest possible case) an entity id, a string, a lookup ref, or a nested vector or map, again with unknown keys ( depends on what was pulled)#2018-09-2613:05favilaAt certain spots in the program I want to allow only a few of those possibilities. Eg On read I often want to guarantee that a certain set of sub-keys was pulled#2018-09-2613:06favilaThe other case is dealing with data where the same key is used but it may have a narrower spec depending on the ākindā of map it is in#2018-09-2613:10favilaEg a key could have values 1 2 or 3, but that key must have value 1 when some other key has a certain value#2018-09-2613:11favilaMulti-spec and predicates can do this but itās awkward and verbose and doesnāt gen well#2018-09-2613:11favilaMulti-spec and predicates can do this but itās awkward and verbose and doesnāt gen well#2018-09-2616:59pvillegas12Interested about this, I would like to be able to express that a restriction on a composed spec. If spec1 has value :a
within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another spec)#2018-09-2620:55pvillegas12To do this, one can s/and
an arbitrary function at the compound spec level to make this happen defined as the second operand (for gen
support to be there)#2018-09-2621:17favilas/and of s/keys works very badly#2018-09-2621:17favilait's unlikely a useable generator will result#2018-09-2621:17favilas/and maps to gen/such-that#2018-09-2621:18favilathis is why s/merge exists#2018-09-2621:22pvillegas12In my case I donāt want to merge two sets of specs. I have a spec that is the composition of 5 specs (with s/keys
) and I wanted two ensure that two of those did not have the same value#2018-09-2621:23pvillegas12This works with gen because the set of possibilities for those specs is 2 š (a really small set)#2018-09-2621:23favila5 s/keys specs?#2018-09-2621:23pvillegas12(s/def ::integration (s/and
(s/keys :req [:m/name
:m/doc-type
:m/origin
:m/target
:m/rules])
distinct-origin-target))
#2018-09-2621:24favilaoh that's not at all what I run in to#2018-09-2621:24pvillegas12what do you run into?#2018-09-2621:24pvillegas12(I can see how distinct-origin-target
can bump gen
if the resulting restriction is too big a set)#2018-09-2621:25favilasomething like if :m/doc-type is "x", :m/origin must be some narrower range of values that is normally allowed#2018-09-2621:25favilavs if :m/doc-type were "y" or "z"#2018-09-2621:26pvillegas12Right, if m/origin
for example is restricted to a given string
matching a regex it will destroy gen
ās ability to generate example data#2018-09-2621:27pvillegas12not sure how with-gen
would work with that compound spec though#2018-09-2621:27pvillegas12(thatās how I make sure gen
continues to work)#2018-09-2621:39favilas/keys+ works like s/keys, except you can add a :conf map from spec->override#2018-09-2621:39favilathe spec is checked all the time#2018-09-2621:40favilabut the override is also checked, and is used for gen#2018-09-2621:44favilathis is really just contravariance/covariance at the end of the day. The trouble is spec doesn't have a way to structurally use the same map key and spec that key two different ways#2018-09-2613:13favilaI wrote a hacky keys+ macro to try and do this. It allows an inline redef of a keyās spec+generator, but it will still conform both the ānaturalā and the redef-ed spec of the key#2018-09-2617:03justinleeIn my reagent app on cljs, I use a spec validator on my appās state atom. This works well, but is unfortunately a bit slow in some cases. Is there anything clever one can do to do validations in this way but skip parts of the state tree that havenāt changed? The answer might be just to use more than one atom.#2018-09-2617:07noisesmiththe validation function receives the two states as args, so at least hypothetically you could check only changed subtrees using cheap identity checks#2018-09-2617:08noisesmithI'd imagine a classic recursive tree walk, either short-circuiting if identity is true, or checking if identity was false, for each branch new/old#2018-09-2617:09noisesmiththen again you'd have to also walk subtrees of the spec to really make it work...#2018-09-2617:09noisesmithsounds messy#2018-09-2617:11dadairIs the state atom flat or deep? If its flat you could have a spec for each of the main subtrees, and only validate the main subtrees that have changed?#2018-09-2617:12noisesmiththat is probably the best plan#2018-09-2617:16justinleeyea thatās probably the right approach. actually i already have specs for each of the main subtrees#2018-09-2617:19noisesmithand I bet using identical?
instead of =
performs better for checking each subtree for changes, but that should be straightforward to swap out and test#2018-09-2617:29justinleethat works beautifully. the obvious thing i didnāt realize is that the validator of course is called before the atom is changed so you can just compare it against the old atom#2018-09-2617:32pvillegas12I would like to be able to express that a restriction on a composed spec. If spec1 has value :a
within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another specād map)#2018-09-2617:32pvillegas12Is there some way to achieve this?#2018-09-2617:36noisesmith@lee.justin.m one of the args to the validator function is the old state, one is the new state#2018-09-2617:40mishaassuming the state tree is a map, and if there are no cross-keys validations (if k1 is this, k2 should be that), you can top level diff new and old values, and just test it against empty (s/keys) @lee.justin.m#2018-09-2617:41mishaeg: old {:a 1 :b 2}, new {:a 1 :b 3 :c 4}, diff {:b 3 :c 4}, (s/valid? (s/keys) diff)#2018-09-2617:42mishamight save you some milliseconds, if state is huge, but diff is just few keys at a time#2018-09-2617:44justinlee@noisesmith I was using https://clojuredocs.org/clojure.core/set-validator! which says `validator-fn must be nil or a
side-effect-free fn of one argument, which will be passed the intended
new state on any state change`#2018-09-2617:44justinleeis there something else I could be using?#2018-09-2617:44justinlee@misha yea there arenāt that many top level keys and most of them wonāt change, so I think I can just test them with identical?
#2018-09-2617:45mishathat's the validator, yes. I don't know about other ones either. I think @noisesmith meant usual ref-callback with [key ref old new] signature#2018-09-2617:47dadairthatās using add-watch
#2018-09-2617:48noisesmith@dadair yeah that's the one I was thinking of#2018-09-2617:48dadairit mostly depends on how you want to handle the spec failure#2018-09-2617:48noisesmithI got the two functionalities confused#2018-09-2617:55justinleethe only thing Iād really like to do is have the validator throw an error synchronously so that i get a stack trace from the offending function, though this is straying from the channel topic#2018-09-2619:36justinleeare specs queryable once theyāve been made? itād be cool to ask for the spec corresponding to a key in a map dynamically#2018-09-2619:37dadairhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/get-spec#2018-09-2621:05Alex Miller (Clojure team)then s/form#2018-09-2621:21favila#2018-09-2622:23ChrisIs there a known issue with s/or and not working with s/keys?#2018-09-2622:25ChrisFor example, this works fine:
(s/conform (s/or :success (s/keys :req [::email/email-address])
:not-found nil?) {::email/email-address "
but if I have additional keys in the value, i get invalid:
(s/conform (s/or :success (s/keys :req [::email/email-address])
:not-found nil?) {::email/email-address "
#2018-09-2623:18seancorfield@chris547 If you call s/explain
(instead of s/conform
) in that second case, what does it say? I suspect you have a spec for ::email/creation-timestamp
and your string is not valid for that.#2018-09-2623:18ChrisIt tells me that creation-timestamp doesn't match the :not-found predicate#2018-09-2623:19ChrisIn: [:io.cloudrepo.signup.email/creation-timestamp] val: "2018-09-26T21:47:57.304Z" fails spec: :io.cloudrepo.signup.email/creation-timestamp at: [:success :io.cloudrepo.signup.email/creation-timestamp] predicate: :clojure.spec.alpha/unknown
val: #:io.cloudrepo.signup.email{:email-address "
#2018-09-2623:20ChrisIt's super weird#2018-09-2623:20Chrisbecause all it should care about is if ::email/email-address conforms, since that's the only key mentioned in the :success condition of the s/or#2018-09-2700:01seancorfieldYou're only reading the last part of the failure -- the first part says "2018-09-26T21:47:57.304Z" fails spec: :io.cloudrepo.signup.email/creation-timestamp
#2018-09-2700:01seancorfieldSo you have an ::email/creation-timestamp
spec and that string fails to conform to it.#2018-09-2700:02seancorfieldSince that fails, the whole :success
part fails and the the whole hashmap fails the :not-found
part -- the val
in explain is the whole hash map, not just the creation timestamp string.#2018-09-2700:03seancorfields/keys
will conform/validate any keys that are provided that have a corresponding spec, not just the listed :req
keys.#2018-09-2700:03seancorfield^ @chris547#2018-09-2700:06ChrisAh, that makes sense now#2018-09-2700:06ChrisI didn't even check to see if the timestamp was okay because I didn't care about it#2018-09-2700:06Chrisbut I guess that's good because it will catch where you create bad data at the first possible point#2018-09-2700:07Chrisrather than when you explicitly check, right?#2018-09-2700:12seancorfieldYup. The assumption is that specs should apply wherever those uniquely-name keys appear.#2018-09-2700:12seancorfieldIf you're using unqualified keys, you won't run into that -- since only the :req-un
and :opt-un
keys will map to specs.#2018-09-2700:15ChrisOkay, thanks Sean - you're always super helpful!#2018-09-2722:02lilactownI have this spec:
(spec/cat :op #{:jdbc/insert}
:table keyword?
:row map?)
for a data structure like [:jdbc/insert :foo {:bar "baz"}]
. how would I add an optional 4th element at the end?#2018-09-2723:07seancorfield:options (s/? ::opt-spec)
#2018-09-2723:08seancorfield(assuming you wanted to optionally pass in an options map of some sort @lilactown)#2018-09-2809:50manutter51Iām trying to run (stest/instrument)
in a clojurescript repl, but I canāt even get that far: dev:cljs.user=> (require '[cljs.spec.test.alpha :as stest])
---- Could not Analyze ----
No such namespace: clojure.test.check, could not locate clojure/test/check.cljs, clojure/test/check.cljc, or JavaScript source providing "clojure.test.check" in file file:/Users/mark/.m2/repository/org/clojure/clojurescript/1.10.339/clojurescript-1.10.339.jar!/cljs/spec/test/alpha.cljs
---- Analysis Error ----
#2018-09-2809:51manutter51:dependencies [[org.clojure/clojure "1.9.0"]
[org.clojure/clojurescript "1.10.339"]
[org.clojure/spec.alpha "0.2.176"]
#2018-09-2809:52manutter51Am I missing a step somewhere?#2018-09-2809:55manutter51I just added the explicit dependency on spec.alpha 0.2.176 this morning, but it makes no difference, I get the same error either way.#2018-09-2809:56valeraukoi thought you needed to depend on clojure.test for generating etc#2018-09-2810:23manutter51You ought to be able to load the namespace though, right?#2018-09-2810:24manutter51Iām spinning up a brand new REPL, and the first thing I type is the require, and it wonāt load the namespace.#2018-09-2810:28manutter51Hmm, something odd in lein deps :tree
too: [org.clojure/clojure "1.9.0"]
[org.clojure/core.specs.alpha "0.1.24"]
āspecsā plural? Whatās that?#2018-09-2811:10mgrbyteI think they are the specs that have been added for clojure language itself.#2018-09-2811:22manutter51Ah, that would make sense.#2018-09-2811:23manutter51So is anybody else having this problem? Canāt require
the cljs.spec.test.alpha
ns in a freshly-started REPL?#2018-09-2811:24manutter51It does seem suspicious that itās looking for it specifically inside the clojurescript jar when I have a perfectly good clojure jar on the same classpath.#2018-09-2811:49athosProbably that can be fixed by adding test.check
https://github.com/clojure/test.check to your dependencies.#2018-09-2822:51manutter51I didnāt get to it on my lunch break, but I just now added the dependency to test.check, and it works great! Thanks a ton! š#2018-09-2812:01manutter51Ok, thanks, I'll try that on my lunch break; got to start day job now.#2018-09-2812:25urzdsHi! I just tried to update to Clojure 1.9 and get weird errors when building my code: :cause Call to clojure.core/refer-clojure did not conform to spec:
Sadly this error message is so convoluted that I cannot even figure out which file is causing this. Any idea what to look for?#2018-09-2812:26urzds(We do not use spec ourselves.)#2018-09-2813:01mpenetyou likely have malformed ns declaration(s)#2018-09-2813:01mpenetor one of your deps#2018-09-2813:07favilaUnfortunately the error will say it is coming from the requiring ns instead of the file it is in#2018-09-2813:08favilaThe ns form isnāt been evaluated yet (because it has an error) so the current ns ( as shown in the message) is still the requiring one#2018-09-2815:01urzds@mpenet, @favila: So the namespace that requires the broken namespace is the one mentioned in the LOAD FAILURE for <ns>
line?#2018-09-2815:01favilathat's been my experience, yes#2018-09-2815:07urzdsThanks.#2018-09-2815:07urzdsAny idea what this spec problem report might be hinting at? https://gist.github.com/urzds/880cd51851a248230861b178c7d6b986#2018-09-2815:08urzdsAlso, in the backtrace it mentions a different file from the namespace it mentions in the LOAD FAILURE for ...
line.#2018-09-2815:09urzdsSo my guess would be that the file it mentions in the backtrace actually contains something funny. But I have no idea what I am looking for...#2018-09-2815:14dadairLooks like it is complaining about the ā:as coreā part#2018-09-2815:30seancorfieldYeah, the syntax is (:refer-clojure :exclude [...])
-- that :as core
part is what is blowing up.#2018-09-2815:31seancorfield@urzds if you post a Gist of your dependencies and/or the full stacktrace, we can probably narrow it down for you.#2018-09-2815:32seancorfieldAlso @urzds check out this page and see if any of your dependencies are listed there https://dev.clojure.org/display/design/Errors+found+with+core+specs#2018-09-2815:35urzds@seancorfield Thanks!#2018-09-2815:38urzdsDifferent question (I am trying to stay on 1.8 and first make sure all my deps are still working): Is it possible to hot patch a function into clojure.core
? Lacinia Pedestal is broken on Clojure 1.8, because it uses a function from 1.9: https://github.com/walmartlabs/lacinia-pedestal/issues/80#2018-09-2816:39seancorfield@urzds Is there a pressing reason to stay on 1.8?#2018-09-2816:41seancorfield(I think, yes, you can define your own clojure.core/pos-int?
but you'd need to do it somehow before that Lacinia namespace was even loaded so...)#2018-09-2816:46urzds@seancorfield No pressing need. But before I try to upgrade to Clojure 1.9, I want to make sure everything still works after upgrading all my deps to versions that should in theory be capable of supporting 1.9.#2018-09-2816:52seancorfieldI suspect you'll be caught between a rock and a hard place there with anything that already uses clojure.spec
since such code may assume a number of things present in 1.9 -- although I thought clojure-future-spec
was supposed to patch those up as well?#2018-09-2816:56seancorfieldAh, I see... it expects you to (:require [clojure.future :refer :all])
into the spec-using namespace but doesn't add anything to clojure.core
directly. So it only helps you use clojure.spec
in your own code -- that's not going to help with any other code that already assumes 1.9+.#2018-09-2816:57urzdsAh, but that's nice. I am right now writing my own shim... š#2018-09-2816:58urzdsSo I'll just patch up lacinia-pedestal with clojure.future instead of my own shim.#2018-09-2816:59seancorfieldYeah, I think you'll have to fork lacinia-pedestal
for now, patched with clojure.future
, and then revert to the official version when you're ready to move to 1.9.#2018-09-2817:00seancorfieldBut, frankly, I would consider that a waste of effort and just go ahead with the 1.9 upgrade and do any necessary work there instead.#2018-09-2817:00urzdsYes, but they even claim to support Clojure 1.8, so it's actually a bugfix. I'll just send them a PR.#2018-09-2817:01seancorfieldBut on 1.8, doesn't it just not use ? That's how most of the libraries seem to handle 1.8 compatibility when they have specs?#2018-09-2817:01urzdsNot this one... š#2018-09-2817:02seancorfieldSo it already assumes clojure-future-spec
is being used? That's kinda odd...#2018-09-2817:02urzdsYes, that's what they suggest to use: https://lacinia.readthedocs.io/en/latest/clojure.html#2018-09-2817:03seancorfieldAh... and then clearly don't test against that setup š#2018-09-2817:05seancorfieldOverall tho', I still suspect you'd have a lot less pain trying Clojure 1.9 directly instead of all this working-around-spec on 1.8 š#2018-09-2817:07seancorfieldProbably a good idea to create an issue (or send them a PR) to add a :1.8
alias to project.clj
with Clojure 1.8 and clojure-future-spec
so it's easy to test with in the future (and they could auto-test against both 1.8 and 1.9 on CircleCI).#2018-09-2817:22urzds@seancorfield Thanks for all your help! I don't really know what you refer to with the last line, though.#2018-09-2817:26urzdsNow the spec violations vanished. That's weird...#2018-09-2817:28urzdsProbably it was good that I tinkered with the different dependencies for so long. Maybe cheshire or commons-codec or commons-io or org.clojure/tools.reader were causing it, because now I :exclude them from deps that request older versions and my code suddenly compiles...#2018-09-2817:56lilactownI have a map of var-quoted functions that I'm using as a lacinia resolver map. After add fspecs for all of them and turning on instrumentation, I'm getting this error:
com.fasterxml.jackson.core.JsonGenerationException: Cannot JSON encode object of class: class clojure.spec.alpha$regex_spec_impl$reify__2436:
#2018-09-2913:02bbrinckItās a bit esoteric, but I put together a quick guide on how to understand the :in
path in spec https://gist.github.com/bhb/462c3ef97058d669a448aa85e7db5998 . Questions/feedback welcome!#2018-09-3012:10misha@bbrinck how is it even reproducible in case 4? https://gist.github.com/bhb/462c3ef97058d669a448aa85e7db5998#file-ex-txt-L47#2018-09-3012:11misha"it" = "same keys order in map"#2018-09-3013:59bbrinck@misha I may be misunderstanding your question, but the order of a map is consistent, (i.e. for the same map m
, (first m)
will always return the same thing, since maps are seqable?
).#2018-09-3014:41jumblemuddleIs the only difference between using spec/valid?
in the pre/post map of a function and spec/fdef
that spec/fdef
can be specified elsewhere?#2018-09-3014:56tayloryou have to instrument
fdef
ād functions, and that wonāt check the function return value like :post
does#2018-09-3014:57taylorpre/post checks will run by default, but instrumentation is opt-in#2018-09-3014:59jumblemuddleI'm noticing that instrument-all
is no longer available. Is it common to just instrument a single function while testing it nowadays?#2018-09-3015:00taylorcalling instrument
with no args will instrument everything thatās been loaded#2018-09-3015:00jumblemuddleOh, nice š#2018-09-3015:00jumblemuddleThanks#2018-09-3015:01jumblemuddlefdef
doesn't verify the output from the :ret
parameter though?#2018-09-3015:01jumblemuddleAt least while using instrument
?#2018-09-3015:01taylorright, instrumented functions will only assert the fdef :args
spec#2018-09-3015:02jumblemuddleHuh, is there anything that actually uses the :ret
then?#2018-09-3015:02taylorthereās a lib called Orchestra with another version of instrument
that adds return value checking#2018-09-3015:02jumblemuddleš#2018-09-3015:02jumblemuddleThanks#2018-09-3015:02tayloryeah, if you check
an fdef
function it uses the :ret
spec (and :fn
if it exists)#2018-09-3015:03taylorcheck towards the bottom of the clojure.spec guide for check
#2018-09-3015:03jumblemuddleAh, ok#2018-09-3015:03taylorI wrote some examples here too https://blog.taylorwood.io/2017/10/15/fspec.html#2018-09-3016:55andy.fingerhut@bbrinck I do not know your use case exactly, but as long as you are only relying on that for the same identical map m
, not two different maps m1
and m2
where (= m1 m2)
. If you try to rely on (= (first m1) (first m2))
you will often be disappointed. But yes, you can rely on (= (seq m) (seq m))
being true.#2018-09-3017:02bbrinck@andy.fingerhut good point! Yes, this is for the identical map - the āinā path returned by spec is only valid for the data you originally check, NOT other data that happens to be equal to the original data. #2018-10-0112:55jumblemuddleDuring development, does it makes sense to use fdef
and instrument
for all my functions or sprinkle my functions with assert
and old enable assertions during development?
Perhaps a combination of the two?#2018-10-0112:59jumblemuddleI guess the benefit of fdef
is that it can be in a separate namespace; is that recommended?#2018-10-0113:09taylorI think this depends on personal preference and whether youāre writing something intended to run on Clojure 1.8. I tend to put fdef
next to the function definitions, but keep other specs in spec-specific namespaces#2018-10-0113:10jumblemuddleHmm, ok. Thanks#2018-10-0113:11taylorthis lib keeps everything separate so it can be used w/pre-1.9 https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2018-10-0113:12jumblemuddleAssuming pre-1.9 compatibility isn't a concern, do you think having them together is ideal?#2018-10-0113:12taylorpersonally I like to see the fdef
right next to the function itself :man-shrugging:#2018-10-0113:13jumblemuddleThanks :+1::skin-tone-2: #2018-10-0114:50bbrinckYou may already know this ,but itās perfectly fine to put the fdef
definition above the function definition. I find this much more readable than putting it below the function.#2018-10-0114:50bbrinck(sometimes people donāt realize this is a valid way to organize their specs)#2018-10-0114:53jumblemuddleAh, interesting. :) #2018-10-0113:04jumblemuddleIt seems like that would defeat some of the namespace'd keyword designs of spec though.#2018-10-0115:08mike_ananev@alexmiller hi! We found strange flawing bug for clojure.spec lib in multithreaded environment. We use latest clojure.spec versions ("0.2.176" "0.2.44") for request validation. Every http request we translate to clojure map, then we call (s/valid? spec input-request-map) for input validation. We have 300 000 sample requests (saved in edn file) for testing (constant requests). If we use one thread on JMeter to send request then we always have result true for (s/valid? spec input-request-map) for every 300 000 constant requests. if we use 2 threads or more on JMeter to send requests then sometimes we have false result. If we save false map to a file and then try to validate it (s/valid? spec input-request-map) then we see that it is always true. If we validate twice in multithreaded env (or (s/valid? spec input-request-map) (s/valid? spec input-request-map) ) then it works (but logs tell us that one of s-exps inside or sometimes false). All map is standard clojure maps (immutable). The code (or (s/valid? spec input-request-map) (s/valid? spec input-request-map) ) works for us as workaround but we can't watch on it without crying. It's Clojure, but we met undefined behaviour like in mutable world.#2018-10-0116:34Alex Miller (Clojure team)Sounds unexpected to me. :) Instead of using valid?, could you try using s/explain so youād get a print to the console when it goes wrong? How is JMeter getting the data to send? Is it possible you have two threads reading from the same reader and thus getting something invalid? If you could isolate this down to something reproducible, would be great to have a jira for it.#2018-10-0115:38mike_ananevAny advice?#2018-10-0117:04andy.fingerhut@djtango If you are using Leiningen, the command lein deps :tree
can be useful. With tools.deps
there is clj -Stree
. Maybe you were already aware of those and looking for something more, though.#2018-10-0117:05djtangothanks for this - wasn't really aware of using lein deps or using it in this way#2018-10-0117:05djtangoat this point any and all suggestions are welcome!#2018-10-0117:31seancorfield"lein pedantic" is something to look into as it will flag the conflicts for you. Boot has a similar feature on its show
task.#2018-10-0117:31seancorfieldI'm not sure if there's an equivalent yet for clj
...#2018-10-0209:50djtangothank you!#2018-10-0119:14jumblemuddleSo, I added (clojure.spec.test.alpha/instrument)
to the bottom of my core.cljs
file, however I have a namespace alpha
which defines a symbol (def global (beta/example ...))
. The problem being that core
depends on alpha
which uses beta/example
prior to it being instrumented. Do I have to add the instrument
to every namespace to avoid this?#2018-10-0119:32taylorif you want that call to beta/example
to get instrumented and you want to keep def global
(instead of making it a function or wrapping it in a delay) then yeah I think you'd have to instrument it separately in this case#2018-10-0119:34jumblemuddleMakes sense š
In this case, it was easy enough to not define the global, though I could see scenarios where that's not really a solution.#2018-10-0119:34jumblemuddleThanks#2018-10-0119:15jumblemuddleHow do I spec and instrument functions that will be used during initialization?#2018-10-0119:36jaihindhreddyHow would you spec standards based things.
Country codes for example: https://en.wikipedia.org/wiki/ISO_3166-1_numeric#2018-10-0119:39jaihindhreddyThis is where I am so far.
(s/def :iso/country-code (s/and s/string? #(= 3 (count %))))
#2018-10-0119:40taylorI'd probably scrape the values, put them in a set, and use that set as the spec#2018-10-0119:41bbrinckIf you can get a big set of valid sample data and convert it into valid EDN, you could try using https://github.com/stathissideris/spec-provider to infer the spec#2018-10-0119:43jrychter@jaihindh.reddy in these kinds of cases I enumerate all the possibilities in a set and use that set in the spec.#2018-10-0119:43jaihindhreddy@taylor ISO charges money for downloads of data. But it is freely available on their website. Scraping that would be awkward to say the least.#2018-10-0119:44jrychtere.g. (s/and string? countries/all-country-codes)
where all-country-codes
is a set.#2018-10-0119:44taylorin your example all the ISO codes are on the Wiki page#2018-10-0119:44jrychterI generate from https://github.com/lukes/ISO-3166-Countries-with-Regional-Codes#2018-10-0119:44jrychterā¦which also makes it somewhat maintainable in the longer term.#2018-10-0119:45taylorare you concerned with keeping the list fresh as time passes?#2018-10-0119:45jaihindhreddyIdeally I would love to do something like (into #{} (slurp url-for-all-country-codes))
#2018-10-0119:45tayloryeah, that's a separate problem from clojure.spec#2018-10-0119:45jaihindhreddyCountry codes should be relatively stable. Dont mind manually updating it#2018-10-0120:00jrychterWell, manually is fine, but I would plan on doing it every once in a while. The list of countries is actually surprisingly unstable.#2018-10-0121:55jaihindhreddyCan I spec protocols and multimethods?#2018-10-0122:10seancorfieldSee https://dev.clojure.org/jira/browse/CLJ-2109#2018-10-0122:12seancorfieldI think you're OK with multimethods tho' (I saw a CLJS ticket that indicated s/instrument
was fixed to work with multimethods, so I assume it works in CLJ too).#2018-10-0122:13jaihindhreddyOn that note, when should I use one or the other?#2018-10-0122:14jaihindhreddyCurrently I think protocols should be the default, and multimethod if you think you need the arbitrary dispatch.#2018-10-0122:22seancorfield@jaihindh.reddy I think this is still a good set of guidelines https://cemerick.com/2011/07/05/flowchart-for-choosing-the-right-clojure-type-definition-form/#2018-10-0313:50fabraoHello all, can I ask newbe question here? š IĀ“m trying to figure out how to use spec in my projects.#2018-10-0313:51fabraoIs that in repl development process?#2018-10-0313:53fabraoLike I saw in this article https://blog.taylorwood.io/2017/10/15/fspec.html , so defn
-> s/fdef
-> stest/instrument
-> stest/check
?#2018-10-0313:54fabraouse as replacement of unit test?#2018-10-0314:02taylorIMO they should supplement/augment tests, but not totally replace them#2018-10-0313:56fabraohow do you use in your workflow daily?#2018-10-0313:56taylorhey @fabrao, an example of how you might use it: have some defn
s you want to spec, write some specs/`fdef`s for those, then in a test namespace you might use check
as part of a test#2018-10-0313:57taylorand the specs/`fdef`s can be in the same file as the functions or separate, up to you#2018-10-0313:57fabrao@taylor by the way, was you that wrote the artice?#2018-10-0313:59tayloryes, thanks for reading š#2018-10-0314:01fabraoIt was very clear about the propose of spec#2018-10-0314:01fabraoBut you use it in your testing workflow?#2018-10-0314:01tayloryes, I tend to use instrument
more than check
#2018-10-0314:02fabraobut, have you already keep it on in production scenario?#2018-10-0314:03fabraofor example to validade an csv file?#2018-10-0314:03taylorno, instrument
has a performance penalty, and anywhere I need to use spec "all the time" then I use the explicit conform
/`valid?` kind of functions#2018-10-0314:04taylors/explain-data
etc.#2018-10-0314:04fabraohow do you use it in Exception scenarios?#2018-10-0314:05taylorlike if I wanted to throw an exception because of invalid input? Probably use ex-info
and maybe put some of the s/explain-data
into the info map#2018-10-0314:05tayloryou should also look into the expound library for spec#2018-10-0314:06taylorthat can help beautify spec's description of invalid inputs#2018-10-0314:07fabraoI got it, but you mix real code with spec codes?#2018-10-0314:07taylorin some cases yeah I might call s/conform
or s/explain-data
or s/assert
from my "normal" code#2018-10-0314:08taylorfor always-on types of checks (asserts aside)#2018-10-0314:08fabraoin case of using valid?
, you use it inside your functions?#2018-10-0314:09taylorprobably not as often as the others, because I'm usually also interested in how/why the input was invalid#2018-10-0314:10fabraoYes, I donĀ“t want to polute my code with spec[ing] stuffs#2018-10-0314:10fabraoI want to be non intrusive checking#2018-10-0314:11tayloryeah, overall I don't have many places in my projects where I'm explicitly interacting with spec in my app code/logic#2018-10-0314:12fabraoyes, thank you about your help, I was afraid beeing specing everthing š#2018-10-0314:13taylorno problem, one great thing about spec is you can use it as little as you like š#2018-10-0314:15fabraoYes, I saw many videos about it, but only shows something using valid?
and nothing about the use like in your article#2018-10-0314:19fabraodo you think is it too intrusive doing this? https://gist.github.com/borkdude/8d69f326b3d363a0b34e7b949b039992#2018-10-0314:20fabraoin this way, itĀ“s better using typed-clojure than that š#2018-10-0314:23taylorI think that's fine (and it's cool that it's even possible) if that's your workflow, although personally I would probably just use regular defn
and fdef
s#2018-10-0314:24taylorit's really up to how you want to use spec and structure your code :man-shrugging: try some different approaches and see what you like#2018-10-0314:31fabraoYes, I understood, thanks for your time. IĀ“d appreciate all the subjects of your blog, all subject that I sometimes worked with#2018-10-0314:33fabraoIĀ“m using Instaparse in production already#2018-10-0314:33fabraothat you can build your own language#2018-10-0314:34taylorI've used Instaparse in a production project too, it's great#2018-10-0313:58tayloryou might choose to enable instrumentation for particular tests by calling instrument
, or maybe in a dev profile so everything is instrumented at dev time#2018-10-0317:16seancorfield@fabrao The other thing to bear in mind with spec is that there are two related but separate types of spec: data specs and function specs. At work, we rely heavily on the former in production code, where we call s/conform
(mostly) and s/valid?
on those specs and some data. We use function specs to support testing, either with st/instrument
while we're developing or as part of a "unit test", or with st/check
mostly while we're developing but also in small, limited tests (generative testing can take a while so it's not really designed as part of your short, fast "unit test" cycle).#2018-10-0317:17seancorfieldWe've been using spec in production pretty much since the first alpha dropped, back in the early 1.9 days. We love it!#2018-10-0317:18taylorthere's a Clojure 1.8 backport of clojure.spec I've used in production haha#2018-10-0317:34seancorfieldThat appeared long after we were already on 1.9 and using spec in production tho' š#2018-10-0317:34seancorfieldWe're currently on 1.10 Alpha 8 in production.#2018-10-0317:26fabrao@seancorfield I saw that we can use s/conform
to contruct maps from data in more easy way#2018-10-0317:28fabraoin that video explain lots of stuffs#2018-10-0317:28fabraohttps://www.youtube.com/watch?v=TD7VGSSZ3ng#2018-10-0317:32seancorfield@fabrao Be careful about coercions and conformers in spec tho' -- consider those "very advanced" usage until you're comfortable with the rest of it in your workflow.#2018-10-0317:33seancorfieldIf you bake a coercion into your spec, you are "forcing" that coercion on all clients of the spec -- and it may have consequences for generators / generative testing.#2018-10-0317:33seancorfieldBut, yeah, overall spec is awesome!#2018-10-0322:07ScotWhere's my error? What am I doing wrong?#2018-10-0322:08ScotI'm sure I am doing something stupid, but I can't find any answers#2018-10-0322:08noisesmithto verify an fdef I think you need to use instrument to explicitly turn on verification#2018-10-0322:09noisesmithalso beware of instrumenting functions that take function arguments - the spec is checked by passing various generated data to the function that was passed in#2018-10-0322:10ScotSo the only way to get runtime verification is via asserts?#2018-10-0322:10noisesmithinstrument turns on the validation of the function's spec, it's something you have to explicitly ask for#2018-10-0322:11ScotThanks#2018-10-0322:13noisesmithif you want to ensure that a specific arg is always checked, you can use s/valid? as a predicate, or s/conform if you want an error for non-matching data#2018-10-0408:10misha@scot-brown
1) https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/fdef
Note that :fn specs require the presence of :args and :ret specs to
conform values, and so :fn specs will be ignored if :args or :ret
are missing.
2) https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument
does not validate against :ret and :fn fdef's spec-components, see: https://groups.google.com/d/msg/clojure/JU6EmjtbRiQ/uND70kAFBgAJ
The official way to test against :ret and :fn of s/fdef - is test.check.
Folks who accept performance price also use https://github.com/jeaye/orchestra instead of vanilla stest/instrument#2018-10-0416:48noisesmith@misha thanks for the clarification#2018-10-0416:48noisesmithso instrument
only validates the arglist and not the return value#2018-10-0416:52Alex Miller (Clojure team)Yes- it instruments functions to ensure you are invoking them correctly#2018-10-0416:54trissIf I have a lot of functions all with the same spec how can I share it between them? Iām guessing I can use an fspec
with fdef
somehow? I really want these specs to show up in the doc strings#2018-10-0416:55Alex Miller (Clojure team)Sure, name the common spec and use the name#2018-10-0416:56Alex Miller (Clojure team)fdef is really a wrapper around fspec then def#2018-10-0416:57trissoh nice. Thanks @alexmiller#2018-10-0416:57Alex Miller (Clojure team)So you can (s/def foo ::a)
where ::a is an s/fspec
#2018-10-0416:57Alex Miller (Clojure team)There were some bugs around this but I think those are all fixed now#2018-10-0417:48fabraowhat is the right use? [clojure.spec.alpha :as s]
or [clojure.spec :as s]
. I tried [clojure.spec :as s]
but it seems no lib in this path#2018-10-0417:48fabrao?#2018-10-0417:51seancorfieldSpec is still alpha right now.#2018-10-0417:51seancorfield[clojure.spec.alpha :as s]
#2018-10-0417:52favilaalways the namespace with "alpha". Clojure 1.9-alpha had a brief period where spec was in clojure core (not a separate lib). That is why you may see non-alpha namespaces sometimes.#2018-10-0417:52seancorfieldAll of the Spec namespaces end in .alpha
-- until it stops being alpha.#2018-10-0417:52seancorfield@favila Ah, yeah, that must have been a fairly brief period š#2018-10-0417:53fabraoyes, I saw it https://www.youtube.com/watch?v=TD7VGSSZ3ng video#2018-10-0417:53fabraoThatĀ“s because I was confuse#2018-10-0417:54the2bearsThe great thing about REPL development is you can try these things very quickly and see which one works.#2018-10-0417:54seancorfieldThere's a comment on that video, dated a year ago, that says "Good introduction! But something changed in the meantime: starting from 1.9.0-alpha16 you have to include clojure.spec.alpha" @fabrao#2018-10-0417:54the2bearsThough Lein setup with the project.clj file can slow that down a bit.#2018-10-0417:55seancorfield@the2bears Aye, I always have a REPL open and I try stuff in it all the time.#2018-10-0417:55favilahttps://clojure.org/community/devchangelog clojure 1.9.0-alpha16 is when it split. So from May 24 2016 to april 26, 2017 "clojure.spec" was the correct namespace#2018-10-0417:55fabraoI left using lein for boot because of @seancorfield#2018-10-0417:56seancorfieldLonger than I realized -- thanks @favila for that detective work!#2018-10-0417:56seancorfieldBoot REPL is nice because you can easily add new dependencies without restarting the REPL!#2018-10-0417:56fabraothatĀ“s the main part I changed#2018-10-0417:57fabraoC-c C-e in set-env!
and thatĀ“s it#2018-10-0417:57the2bears@seancorfield my problem is I lose track of my REPL's state from time to time š . But I have 2 or 3 open at any one time, usually, main project then libraries used in it.#2018-10-0417:58fabraoIĀ“ll do cider-connect and start it from command line#2018-10-0417:59fabraoI tried to use Proto-REPL with Atom but I canĀ“t leave Emacs anymore#2018-10-0419:17fabraoIn this case, I got Assert failed
. Is there any way to custom error message?
(defn stringer-bell
"Prints a string and rings bell."
[s]
{:pre [(s/valid? (s/nilable string?) s)]}
(println s "\007"))
#2018-10-0419:19taylormaybe relevant thread here about custom pre/post messages https://groups.google.com/forum/#!msg/clojure/xHrFyDcPS9A/gXiNY6pmAwAJ#2018-10-0419:19taylorone suggestion is to use clojure.test/is in the assertion#2018-10-0419:20taylor(or something similar of your own)#2018-10-0419:20fabraothanks again taylor, as you see I little lost about using it#2018-10-0419:20taylorhaha np!#2018-10-0419:21fabraoI tried
(if (not (s/valid? ::campos campo))
(throw (Exception. (str "Tipo do campo incorreto. VƔlidos = #{" (reduce #(str %1 %2 " ") "" campos) "}"))))
#2018-10-0419:21fabraobut is too much code, donĀ“t you think?#2018-10-0419:22taylorI think that reduce could be replaced with clojure.string/join
maybe#2018-10-0419:23tayloralso, if you think you need this type of check a lot, you could easily define a function that takes a spec, an input, and maybe an error message or explain-data-to-string function; and then wherever you call it should be pretty terse?#2018-10-0419:24fabraoIĀ“m thinking use https://github.com/bhb/expound to be more explicity#2018-10-0419:24tayloralso check out https://github.com/bhb/expound if you haven't already, and I think there's a similar lib that's more geared towards user-facing error messages#2018-10-0419:24fabraoš#2018-10-0420:32misha@fabrao offtopic: have a look at https://clojuredocs.org/clojure.core/when-not#2018-10-0420:32mishaand https://clojuredocs.org/clojure.core/ex-info#2018-10-0420:33fabraothanks#2018-10-0420:36mishaand (reduce #(str %1 %2 " ") "" campos)
looks like (clojure.string/join " " campos)
#2018-10-0420:37mishasuddenly, not too much code
opieop#2018-10-0423:38agif you are writing a s/fdef foo
but want to place it in a separate ns (not the same where foo
is), should you require the ns where the function is in ns where the spec is? But what if you need to use specs say for validation (in the ns where s/fdef is implemented) how do you avoid circular dependency?#2018-10-0423:40agor should you move higher order specs to a separate ns, but keep s/fdef foo
where foo
is implemented?#2018-10-0423:41noisesmithisn't some of this simplified by the fact that the spec is looked up by a keyword? you can use a namespaced keyword before the namespace it nominally refers to exists#2018-10-0423:42noisesmithby the time the spec is actually checked, you need the ns that defined the spec for that keyword to be loaded, of course#2018-10-0502:04Alex Miller (Clojure team)you donāt have to require the namespace#2018-10-0502:05Alex Miller (Clojure team)fdef creates an entry in the spec registry keyed by the namespaced symbol, but you donāt need to load that namespace at that point#2018-10-0519:59aisamuWrapping a cat
with an and
is breaking my spec in interesting ways.
What am I missing?
(s/def ::pair
(s/cat :first (s/nilable number?)
:second (s/nilable number?)))
(s/conform ::pair [1 2])
;; => {:first 1, :second 2}
(s/conform (s/? ::pair) [1 2])
;; => {:first 1, :second 2}
(s/def ::ordered-pair
(s/and ::pair
#(<= (:first %) (:second %))))
(s/conform ::ordered-pair [1 2])
;; => {:first 1, :second 2}
(s/conform (s/? ::ordered-pair) [1 2])
;; => :clojure.spec.alpha/invalid
(s/explain-str (s/? ::ordered-pair) [1 2])
;; => "In: [0] val: 1 fails spec: :apij.unit.specs.models.rate/pair predicate: (cat :first (nilable number?) :second (nilable number?))\n"
#2018-10-0520:02taylorthe s/?
isn't "flattening" the regex spec (`s/cat`) because it's wrapped in s/and
. I think those last tests would conform [[1 2]]
#2018-10-0520:03taylorit should also conform []
#2018-10-0520:04taylor(s/? ::ordered-pair)
is becoming "match a sequence of zero-or-one ::ordered-pairs"#2018-10-0520:05taylorif you wanted a something-or-nothing spec you could try (s/nilable ::ordered-pair)
too but I'm not sure of your use case#2018-10-0520:09aisamuYup, both cases conform correctly! I'm mostly trying to understand the non-flattening, though. Is there an alternative construct to use the ordering function without using s/and
?#2018-10-0520:10aisamu(and thanks, @U3DAE8HMG!)#2018-10-0520:19taylorhmm what would multiple pair inputs look like for you? one big sequence like [1 2 3 4]
or a sequence of pair seqs like [[1 2] [3 4]]
#2018-10-0520:28Alex Miller (Clojure team)s/and converts a regex spec to a non-regex. s/& is a regex version#2018-10-0520:28Alex Miller (Clojure team)will still continue operating at the same ālevelā of sequence#2018-10-0520:30taylorah yeah, you can just replace s/and
with s/&
in ::ordered-pair
if you want to conform inputs like [1 2 3 4]
#2018-10-0520:53aisamuOh,facepalm
Not only it makes perfect sense, but it's pretty obvious in retrospect!
Thanks @U064X3EF3 and @U3DAE8HMG!#2018-10-0800:49kingcodeWhat is the fastest way to tell whether one vector is a subsequence of another?#2018-10-0812:12roklenarcicProbably the same algo than the one for substring search.#2018-10-0812:19roklenarcicKnuth-Morris-Pratt or Boyer-Moore#2018-10-0800:50kingcodesorry..wrong channel (I was thinking of using specsā coll regex)#2018-10-0805:18tianshuHi, how can I know if there is a spec on a given keyword or not? seems s/spec?
only detect spec object.#2018-10-0805:31seancorfield(s/get-spec ::k)
will return nil
if no spec is registered (else will return the spec itself).#2018-10-0806:00tianshu@seancorfield thanks!#2018-10-0916:41jrychterI wish there was a book I could buy, titled "Data Modeling with clojure.spec".#2018-10-0916:44jrychterHere's an example of a practical question. Should I do (a) or (b)?
;; A:
(ns testing.spec
(:require [testing.id :as id]))
(s/def ::id ::id/id)
(s/def ::data-structure (s/keys :req [::id]))
;; B:
(ns testing.spec
(:require [testing.id :as id]))
(s/def ::data-structure (s/keys :req [::id/id]))
#2018-10-0916:45Alex Miller (Clojure team)You should do whichever one matches your data#2018-10-0916:46jrychterHence my wish for the book š#2018-10-0918:05mattlypersonally I would define the spec for the id in the same ns as the id#2018-10-0918:06mattlyunless the testing.spec ns is the only place you plan to refer to it#2018-10-0918:19jrychter::
is a spec for an UUID in a certain form, this gets used all over the place. Question is, do I alias it locally using ::id
? In other words, who "owns" the ::id
definition?#2018-10-0918:23jrychterAnd Alex's Zen Master answer is actually spot on. My current thinking is that I should use local ::id
, since this is really a property of the data that I'm defining locally, and only coincidentally an UUID whose properties are defined elsewhere.#2018-10-0918:23Alex Miller (Clojure team)I think about it as two levels - itās useful to have specs that capture common type/shapes/formats domain things - phone numbers, invoice numbers, etc. then there are entities that collect attributes and those may have their own names. Like you have a :foo.domain/phone and then you have :foo.company/phone#2018-10-0918:23Alex Miller (Clojure team)(and the latter would just be an alias for the former)#2018-10-0918:24Alex Miller (Clojure team)that adds weight (and there are some bugs around overriding generators for aliased specs) so it may or may not be worth it#2018-10-0918:24Alex Miller (Clojure team)phone number is probably something with extra constraints, but if you have something adequately represented as int?
then I would not pull it out as a domain type as that doesnāt seem worth it#2018-10-0918:25Alex Miller (Clojure team)some day weāll consider doing a 2nd edition of Clojure Applied and think about some of this stuff :)#2018-10-1013:57orestisIād love a 2nd edition of Clojure Applied ā or perhaps a 2nd volume? š#2018-10-1014:02Alex Miller (Clojure team)based on my time availability, I donāt think itās happening anytime soon#2018-10-1014:04orestisUnderstandable, of course. Thanks for the first edition in any case!#2018-10-0918:25jrychterRight. This is similar to what I've been thinking, too.#2018-10-0918:28jrychterA book would be very welcome. I would really like to learn about how people approach modeling data, preferably from people who actually build real systems. Spec seems very flexible, but I found that if you deviate from a certain path, you'll be swimming against the current (for example, if you insist on using unqualified keys).#2018-10-0918:30jrychterBTW, I solved my coercion problems for the time being by judiciously removing every s/conform
from my code and writing a tiny piece of code that maintains registries of coercions (multiple registries, as it turns out: one for converting to the domain model, and one for converting to db form). This seems to work very well and apart from limitations of unqualified keys in specs solves the problem for me.#2018-10-1016:02ScottStackelhouseHi all, I donāt slack much. I have a case where I model data as a map, some keys required, some not. I already use one key as a ātypeā or ākindā indicator. The problem I have is I have a set of keys where all are optional, but at least one of them must be present. #2018-10-1016:02ScottStackelhouseI have not had luck figuring out how to spec that#2018-10-1016:02ScottStackelhouseAny thoughts or links to point me at?#2018-10-1016:04Alex Miller (Clojure team)use s/and to combine a predicate that checks that condition#2018-10-1016:05ScottStackelhouseMostly the problems I run ino with specāing like that is it seems to imply/expect more hierarchy than o have#2018-10-1016:06ScottStackelhouseIām sure itās user error, but examples are hard to come by#2018-10-1016:06piotr-yuxuanSorry for disrupting your chat. Iām wondering how to programmatically register a spec.
(defmacro draft-make-spec
[spec-name spec-pred]
(eval `(list 'spec/def ~spec-name ~spec-pred)))
(def spec-name ::g)
(def spec-pred int?)
(draft-make-spec spec-name spec-pred)
(spec/valid? ::g 4)
;; works correctly
;; => true
(let [spec-name ::h]
(draft-make-spec spec-name int?)
(spec/valid? ::h 4))
;; => throws in java.lang.InstantiationException
#2018-10-1016:07Alex Miller (Clojure team)@scottstack (s/and (s/keys :opt [::a ::b ::c]) #(some #{::a ::b ::c} (keys %)))
#2018-10-1016:09Alex Miller (Clojure team)@piotr2b youāre doing too much there I think#2018-10-1016:09ScottStackelhouseThatās interesting. I tried something similar with s/alt but didnāt quite get there. #2018-10-1016:10ScottStackelhouseThanks @alexmiller , Iāll see where I get with that. It at least tells me Iām heading in the right direction. #2018-10-1016:14piotr-yuxuan@alexmiller, at first I thought it would as easy as this:
(defmacro draft-make-spec
[spec-name spec-pred]
`(spec/def ~spec-name ~spec-pred))
but it doesnāt work:
(let [spec-name ::h]
(draft-make-spec spec-name int?)
(spec/valid? ::h 4))
;; => CompilerException java.lang.Exception: Unable to resolve spec: ::h
š any tiny help from you would be greatly appreciated! šŖ#2018-10-1016:16piotr-yuxuanI was wondering how I could write a small library to generate specs from avro schema, so I need to call spec/def in a let and I only know the spec name at runtime.#2018-10-1016:30ScottStackelhouse@piotr2b have you looked at the macro expansions of your macro and of s/def? I did a quick try and it didnāt come out how I expected. #2018-10-1016:31Alex Miller (Clojure team)Iāve done some stuff like this elsewhere but canāt put my finger on it right now and I need to log off, sorry#2018-10-1016:36ScottStackelhouseI put the macro expansion of the s/def form into the let in place of the macro call, ie (s/def-impl āspec-name āint? int?) and it fails an assertion that ākā is a keyword... #2018-10-1016:37ScottStackelhouseNomenclature fails me but it is like spec-name in your macro needs a double deref #2018-10-1016:53ScottStackelhouse@piotr2b If you take the expansion of s/def, and use that in your draft-make-spec macro, it works#2018-10-1016:53piotr-yuxuanwow, wait!#2018-10-1016:53piotr-yuxuanCould you show me that? š#2018-10-1016:54piotr-yuxuanItās funny because weāve swapped problems#2018-10-1016:54piotr-yuxuanI was working on solving yours#2018-10-1016:54ScottStackelhouseYeah, on my phone tho#2018-10-1016:54ScottStackelhouseHeh#2018-10-1016:54piotr-yuxuanHere is what Iāve got so far:
(spec/def :entity/kind #{:entity/car :entity/person})
(spec/def :person/name string?)
(spec/def :person/age pos-int?)
(spec/def :car/model-name string?)
(spec/def :car/age pos-int?)
(spec/def ::scottstack-spec
(and (spec/keys :opt-un [:entity/kind
:person/name
:person/age
:car/model-name
:car/age])
#(condp = (:entity/kind %)
:entity/car (every? (set (keys %)) #{:car/model-name :car/age})
:entity/person (every? (set (keys %)) #{:person/name :person/age})
false)))
(spec/valid? ::scottstack-spec {:entity/kind :entity/car
:person/name "scottstack"
:person/age 21})
;; => false
(spec/valid? ::scottstack-spec {:entity/kind :entity/person
:person/name "scottstack"
:person/age 21})
;; => true
#2018-10-1016:54piotr-yuxuanDoes it suit your need?#2018-10-1016:55ScottStackelhouseI will have to check#2018-10-1016:55piotr-yuxuanBasically Iām a bit new with macros, so if you could write down the code in addition to your previous explanations, that would be wonderful š#2018-10-1016:55ScottStackelhouseLet me switch to something with a keyboard #2018-10-1016:55piotr-yuxuan^^#2018-10-1016:58piotr-yuxuan@alexmiller how sad! if you could find it back it would be awesome. Anyway, thanks for everything you do for this amazing language š#2018-10-1016:59piotr-yuxuan@scottstack, I fixed a typo, itās better with :opt-un
#2018-10-1017:00piotr-yuxuanAnd the last test case:
(spec/valid? ::scottstack-spec {:entity/kind :entity/person
:person/name "scottstack"})
;; => false
#2018-10-1017:02ScottStackelhouse(defmacro draft-make-spec [spec-name spec-pred]
`(s/def-impl spec-name āspec-pred ~spec-pred))
(let [sname ::something]
(draft-make-spec sname int?)
(s/valid? ::something 4)
#2018-10-1017:03ScottStackelhouseI am very good with macros either. I depend heavily on cider's macroexpansion #2018-10-1017:05ScottStackelhouseS/def would expand to (s/def-impl 'sname 'int? int?)
#2018-10-1017:06ScottStackelhouseAnd that would fail an assertion that the first arg be a keyword#2018-10-1017:06ScottStackelhouseOh#2018-10-1017:06ScottStackelhouseI see I goofed up, maybe it still doesn't work#2018-10-1017:07ScottStackelhouseNevermind, I just confused myself momentarily... I think it is ok#2018-10-1019:00firstclassfuncAfternoon, is anyone aware of any projects to transform JSON Schema into Clojure Spec?#2018-10-1019:22lilactownI don't know of any projects for doing JSON Schema => clojure.spec#2018-10-1019:23lilactownspec-tools
can go the other way: clojure.spec => JSON Schema#2018-10-1019:24firstclassfunc@lilactown Yea tks.. Do people think that would be valuable? There are lots of standards
defined in terms of JSON Schema and would be beneficial to auto-generate specs so they can be consumes without the tedious nature of crafting data structures from scratch#2018-10-1019:53lilactownI think it could be pretty useful to help bootstrap some of the commonly needed specs, yeah š#2018-10-1021:57devnThis may be a test.check question more than a spec question, but here goes:
I have a map named foo
:
{:a 1 :b nil :c 32}
If a
is present, then b
should be nil
, and c
should be a pos-int?
.
{:a nil :b true :c nil}
When a
is nil, c
must also be nil
, and b
must hold a value.
I want to be able to generate examples of both types of maps.#2018-10-1021:57devnPerhaps I should be tagging them as :type :a
and :type :b
and using a multi-spec
?#2018-10-1022:00gfredericksAs a pure t.c question it's easy - use gen/one-of; not sure about making that more spectomatic#2018-10-1022:00gfredericksmulti-spec sounds plausible but I am not a dentist#2018-10-1022:07devnThe way I explained it above is kind of crappy:
(s/def :my/a #{1 2 nil})
(s/def :my/b (s/or :has-b pos-int? :no-b nil?))
(s/def ::thing (s/keys :req-un [:my/a :my/b])
example:
(let [a-thing {:a 1 :b nil}
(when (and (:a a-thing) (not (:b a-thing))) (println "it's a foo")
(when (and (not (:a a-thing)) (:b a thing)) (println "it's a bar")))
I think I need some such-that
magic maybe?#2018-10-1022:10devnSo what I want is if I then ran (gen/sample (s/gen ::thing))
, I'd never get back records which have both :a
and :b
populated, only one or the other.#2018-10-1022:12gfredericksHave you tried writing a map spec for each case and using s/or?#2018-10-1022:13devnno but that makes perfect sense#2018-10-1022:13devnthanks @gfredericks#2018-10-1022:19devnjust to be clear, what I think you're suggesting is this:
(s/def :myA/a #{1 2})
(s/def :myA/b nil?)
(s/def :myB/a nil?)
(s/def :myB/b pos-int?)
(s/def ::thing-a (s/keys :req-un [:myA/a :myA/b])
(s/def ::thing-b (s/keys :req-un [:myB/a :myB/b])
(s/def ::either-thing (s/or ::thing-a ::thing-b))
#2018-10-1022:25devnfollow on question is then perhaps, how to tune the distribution of thing-a and thing-b's generated#2018-10-1022:28bbrinckhavenāt tried it, but my guess in a custom generator (See s/with-gen
) + https://clojure.github.io/test.check/clojure.test.check.generators.html#var-frequency#2018-10-1022:28devn(gen/sample (gen/frequency [[2 (s/gen ::thing-a)]
[2 (s/gen ::thing-b)]])
10)
perhaps?#2018-10-1022:28bbrinckha#2018-10-1022:28bbrinckyeah#2018-10-1022:29devnthough you may be on to something with with-gen#2018-10-1022:30devnit would be nice to attach the frequency to the spec#2018-10-1022:30bbrinckuntested, but then you can do something like (s/def ::either-thing (s/with-gen (s/or ::thing-a ::thing-b) #(gen/frequency [[2 (s/gen ::thing-a)] [2 (s/gen ::thing-b)]]) ))
#2018-10-1022:31devnthat doesn't seem to do it#2018-10-1022:32bbrinckhm, probably a bug in my code, but in general with-gen
should override the default generator. remember the function needs to return a generator#2018-10-1022:33bbrinckIOW, the second arg canāt be a generator#2018-10-1022:34devnyes, which is indeed what your code does there#2018-10-1022:34devnmaybe gary's previous suggestion to use one-of is worth trying here#2018-10-1022:35bbrinckwhat doesnāt work with frequencies?#2018-10-1022:36devni tried changing the frequency from 2 to 0 for one of them, and it doesn't vary the samples#2018-10-1022:36bbrinckah#2018-10-1022:36devnthey all are ::thing-a
-like things#2018-10-1022:36devnwhereas the thing I posted above does work#2018-10-1022:37bbrinckwell, if you put 0 for thing-b, then everything should be thing-a, yes?#2018-10-1022:37devnyes, but i tried it both ways, and the same result#2018-10-1022:37bbrinckah#2018-10-1022:39devntested again to make sure im not crazy#2018-10-1022:39devnExceptionInfo Couldn't satisfy such-that predicate after 100 tries.
#2018-10-1022:40devnis what I wind up with when I try to make it return only one of the types, so perhaps something else is amiss here#2018-10-1022:44bbrinckhmmm, yes, iām trying a simpler example and not seeing what i expect, so Iām clearly missing something#2018-10-1022:46devn@bbrinck here's a thing: if i switch the order of the s/or
, it works in one direction, but not the other#2018-10-1022:49devnbah, ok, im an idiot#2018-10-1022:49devn(s/or ::thing-a ::thing-b)
!= (s/or :a ::thing-a :b ::thing-b)
#2018-10-1022:49bbrinckhaha wow I missed that too#2018-10-1022:50bbrinckjust trying to figure out why my simple example wasnāt even conforming#2018-10-1022:51bbrinckgood catch#2018-10-1022:54bbrinckok, now with-gen
+ frequency
is working again for my simple example.
(s/def ::name string?)
(s/def ::age int?)
(s/def ::either (s/with-gen (s/or :name ::name :age ::age) #(gen/frequency [[4 (s/gen ::name)] [1 (s/gen ::age)]])))
(s/exercise ::either)
#2018-10-1022:54bbrinckis your example working now?#2018-10-1022:54devnyeah, works good#2018-10-1022:54devnchef kisses fingers#2018-10-1022:55devnthanks for your help @bbrinck#2018-10-1022:55bbrincknp#2018-10-1022:56devn(my/or :name ::name 1 :age ::age 2)
or somesuch#2018-10-1022:57bbrinckmaybe not easy, but easier if you define the syntax with a regexp spec and conform š#2018-10-1023:06devnok i spent 10min on it and the sugar ain't worth it š#2018-10-1023:18devn(defmacro myor [& key-pred-freq-forms]
`(s/with-gen (s/or
or something, but :man-shrugging:#2018-10-1116:35domkmHas anyone written a generator for keys
that always chooses to generate optional entries until *recursion-limit*
is reached?#2018-10-1118:09domkmIt seems like this may not be possible currently because whenever a generator-creating function is allowed (`with-gen`, (keys :gen (fn [] ...))
) it is takes 0 args while the default generator-creating functions take 3 args (`[overrides path rmap]`). Why do custom generators not have access to the current length of recursion?#2018-10-1121:36bastiIāve got a pretty specific question regarding clojure spec-toolsās dataspecs -
iāve got those two data specs, how do I āorā them together?
{:status (s/spec #{200})
:success boolean?
:body {:data [{:id string?}]}}
{:status (s/spec #{500})
:success boolean?
:body empty?}
#2018-10-1205:11ikitommi@basti there are two options: use ds/or
or convert data-specs into normal clojure.specs and use s/or
:
(ds/or
{:200 {:status (s/spec #{200})
:success boolean?
:body {:data [{:id string?}]}}
:500 {:status (s/spec #{500})
:success boolean?
:body empty?}})
(s/or :200 (ds/spec
{:name ::200
:spec {:status (s/spec #{200})
:success boolean?
:body {:data [{:id string?}]}}})
:500 (ds/spec
{:name ::500
:spec {:status (s/spec #{500})
:success boolean?
:body empty?}}))
#2018-10-1215:13bastithat awesome thanks for your help --appreciated!#2018-10-1212:14Charles FourdrignierAfter solving Infix Calculator on 4Clojure (http://www.4clojure.com/problem/135), I would like to write some spec for the args.
(s/def ::infix-args
(s/cat :a int?
:rest (s/*
(s/cat :f #{+ - / *} :b int?))))
Do you see a better way to do that ?#2018-10-1212:18taylorlooks fine to me#2018-10-1212:19taylorI suppose you could use number?
instead of int?
#2018-10-1212:22Charles FourdrignierOf course !#2018-10-1410:40vemvWondering if there's a precise definition of :in
and :at
within the context of clojure.spec
failures:
...
in: [:lines 1]
at: [:lines]
...
#2018-10-1410:42vemv(this comes from CIDER so perhaps an extra key is being added)#2018-10-1417:05domkmIs there a recommended way of debugging built-in spec generators? I have a large graph spec in which all leaf nodes are clojure.core functions like any?
, string?
, boolean?
, etc. and all collections are either sequential collections or maps with only optional keys, though some keys are recursive. It halts when generating data, eventually erroring (I think it runs out of memory).#2018-10-1418:27misha> though some keys are recursive
(s/def ::foo (s/or :my-non-recursive #{:a} :my-recursive (s/coll-of ::foo)))
(binding [s/*recursion-limit* 2]
(s/exercise ::foo))
;=> some data
(binding [s/*recursion-limit* 10]
(s/exercise ::foo))
;=> OutOfMemoryError GC overhead limit exceeded clojure.lang.PersistentVector$ChunkedSeq.next (PersistentVector.java:422)
@domkm#2018-10-1418:27mishadefault is 4#2018-10-1418:32mishahowever, s/*recursion-limit*
requires recursive specs to have a non-recursive branch available to be able to stop at the limit.
but you will probably get StackOverflow much earlier, if you "explicitly" omit nonrecursive branch:
(s/def ::foo (s/coll-of ::foo))
(binding [s/*recursion-limit* 10]
(s/exercise ::foo))
;=> CompilerException java.lang.StackOverflowError, compiling:(... .clj:691:1)
#2018-10-1418:36domkm@misha Hmm, I hadn't considered that it would halt before reaching the s/*recursion-limit*
. I assumed that since all keys are optional it would choose a non-recursive option. I'll try messing around with s/*recursion-limit*
and see what happens. Thanks#2018-10-1418:37mishayou might get surprised how often limit=4 runs out of memory#2018-10-1418:38mishatry limit=1 and see how deep of a structure s/exercise returns, to at least somewhat calibrate your intuition#2018-10-1421:22alzaHi there, I have a small project to transform one xml format into another (call them fmt-a and fmt-b). I thought Clojure Spec might be useful to define the shape of the data at each stage of the transformation and check the transformation works correctly for expected inputs. Then I found spec-tools and it's transformers, so I thought I'd investigate that. Although I noted that although it supports JSON it doesn't seem to support XML yet.
Anyway this was going to be my approach with spec-tools:
1. parse fmt-a-xml from xml file
2. transform fmt-a-xml-data (i.e. parsed {:tag :attributes :content structure}
) to fmt-a (i.e. {:fmt-a-key some-val}
)
3. transform fmt-a to fmt-b (i.e. {:fmt-b-key some-val}
)
4. transform fmt-b to fmt-b-xml-data (i.e. {:tag :attributes :content structure}
)
5. format fmt-b-xml-data to xml file
Does this sound sensible? I'm not sure how to achieve all of the above with spec-tools, but I was going to start experimenting with step 3, the core data transformation.
note: there's no xml schema available for fmt-a or fmt-b, if that matters.#2018-10-1605:59ikitommiOf coercion: tested the two approaches: conforming-based (spec-tools) and form-parsing based (spec-coerce). Both have big issues. This just canāt be resolved with the current spec architecture. One CLJ-2251 please š#2018-10-1606:04ikitommiwill most likely merge the two approaches into spec-tools, so the st/decode
will work without wrapping of specs for many simple forms, fallbacking to conforming-based approach to support all specs (regex included)#2018-10-1606:04ikitommihttps://github.com/metosin/spec-tools/pull/139#2018-10-1616:40hmaurerHi! What would be the canonical way to force a map to have a :type key with a specific value with Spec? Would you have:
(s/def ::person (s/and map? #(= (:type %) ::person) (s/keys ...))
#2018-10-1617:57taylorcould you use a spec for the :type
key like (s/def ::type #{::person})
#2018-10-1616:41hmaureror is there a better/neater way?#2018-10-1617:03jrychterDepends on what you want to do ā sometimes you want a multi-spec, where you dispatch on :type
.#2018-10-1617:04jrychterAs a side note, I found that prefer to use boolean ::person?
keys rather than a general :type
. I'm converting my old code as I go.#2018-10-1617:25hmaurer@jrychter oh, why the boolean approach?#2018-10-1617:26hmaurerand under that approach, how would you suggest I handle a hierarchy like this:
https://puu.sh/BM1sE/7aaf4c8e7b.png
?#2018-10-1617:26hmaurereach of these represent maps of a certain ātypeā with certain properties#2018-10-1617:26jrychterLess complexity, basically. And I don't know, but I suspect that using a single data structure for all of these might not be the best approach.#2018-10-1617:27hmaurerhow so?#2018-10-1617:44Nolanhey everyone, i may be missing something obvious, but is there a concise way to use spec
to perform what is essentially a select-keys according to a spec? e.g.
(def composite-thing { ... })
(s/def ::some-component ...)
(s/select-keys ::some-component composite-thing)
;; => map with keys of composite-thing relevant to ::some-component
#2018-10-1617:45Nolanor maybe read the relevant keys from a specās definition at runtime?#2018-10-1617:53taylor(s/def ::my-map
(s/keys :req-un [::id]
:opt-un [::parent-id ::children]))
(->> (s/get-spec ::my-map)
(s/describe) ;; get abbrev. form of original spec
(rest) ;; drop `keys` symbol
(apply hash-map)) ;; put kwargs into map
=> {:req-un [:sandbox/id]
:opt-un [:sandbox/parent-id :sandbox/children]}
#2018-10-1617:59seancorfieldI don't think you need (s/get-spec ::my-map)
there, you can just do (s/describe ::my-map)
(since s/describe
calls s/form
).#2018-10-1618:06ikitommiremember to walk over special and
and or
s.#2018-10-1619:33jrychterThis is something I'd really like to see. I know that Rich said that spec is not about limiting what you can do, but there are legitimate use cases where you want to enforce the set of keys and limit it only to what is in your spec (think for example API endpoints).#2018-10-1619:38seancorfield@jrychter It's easy enough (in most cases) to pull the set of possible keys out of the form for the spec and use that -- treating the spec as the "source of truth" and deriving key sets from it.#2018-10-1621:06trissIām building my first really big interconnected system and I have to admit immutablity is driving me crazy!#2018-10-1621:07trissI just canāt used to passing paths around for my āobjectsā all the time.#2018-10-1621:07trissWhat tare the other functional approaches to this?#2018-10-1621:08trissIām really missing OO/imperative systems being happy re: passing around references to objects all the time.#2018-10-1621:09the2bearsI don't miss that at all š
#2018-10-1621:10trissI wish I didnāt!#2018-10-1621:10trissIām passing around the paths in to a nested map all the timeā¦#2018-10-1621:14the2bearsNot sure what you're trying to do, sounds like you have a big nested state, and what about the paths you're passing around?#2018-10-1621:15trissIām porting an old AI project to Clojure for nowā¦. the whole system seems thoroughly object orientated. Objects sending messages to others all over the place.#2018-10-1621:16trissIām pretty sure Core-async would make things a bit easier since I could just pass the chanell around all the time.#2018-10-1621:16trissBut obviously this is a very different solution.#2018-10-1622:04jrychterIt sounds like this approach deserves a good refactoring. Core.async would be just swiping the dust under the carpet. Break things down into smaller functions that do one thing only, and are testable.#2018-10-1622:18the2bearsI was actually asking about your paths, that you're "passing around". I don't know what you mean by that. I can imagine a collection of functions that act upon something flowing through your program. They could be "wired" by a state machine, or a wiring of channels. Why do you think you might need to pass channels around? It's certainly possible, but it might also make sense to push data to a channel that triggers a flow through your functions and something comes out the other end, transformed.
That's what I was asking about, with the "what you're trying to do" comment. A description of your data/logic flow in a bit of high-level detail rather than just "porting an old AI project" which doesn't tell much š#2018-10-1622:18the2bearsie. What does the old AI project do?#2018-10-1712:10Ho0manHi, Is there a way to make s/cat
to generate vector instead of list (other than using s/with-gen
) ?#2018-10-1712:12Alex Miller (Clojure team)No, and that is a known issue#2018-10-1714:05hmaurerIs there any way I can define a generator for a spec like this?
(s/def ::hello #(isa? % :im/base))
#2018-10-1714:05hmaurerI would want it to pick at random from (descendants :im/base)
#2018-10-1714:06hmaurer(it has to pick from the descendants at the point when the generator is called, not at the point where the spec/generator is defined)#2018-10-1714:08hmaurernevermind, thatās just a simple application of with-gen
!#2018-10-1715:46strickinatoHi - Iām pretty new to clojure, Iām playing around with spec and ran into some trouble:
Iām trying to spec a Person that has a last-contacted
key and Iām using clj-time
, such that I expect the field last-contacted
to be a DateTimeZone. Iām running into problems though and Iām not sure how Iām supposed to be using the spec from their library (or generally the best way to proceed.
### Thought 1:
Iād like to access the ::date-time
spec defined in clj-time.spec
since my understanding is specs are āregistered globallyā
Unfortunately, it appears I canāt access it. Trying to (s/exercise ::date-time)
yields and āUnable to resolve specā error and qualifying it like (s/exercise clj-time.spec/::date-time)
gives a parse error
Question: How do I use specs defined in libraries?
### Thought 2:
If the above did work, how could I even use it with spec/keys
. From what I can tell, the name of the spec has to match the name of the key. eg:
(s/def ::person (s/keys :req [::name, ::date-time]))
Question: Is there syntax for doing something like the following invented code:
(s/def ::person (s/keys :req [::name, [:last-contacted ::date-time]]))
where a key with a different name gets a spec run on it?#2018-10-1715:55dadair::key is syntactic sugar that expands to :the.current.ns/key#2018-10-1715:56dadairSo if you see the use of ::key in some libraryās namespace ācom.fooā, you need to use :com.foo/key. You will also need to require com.foo so that the ns is evaluated and the specs are registered#2018-10-1715:48taylorre: #1 you just need a slightly different syntax: :clj-time.spec/date-time
, or if you've aliased clj-time.spec e.g. [clj-time.spec :as ts]
then you could do ::ts/date-time
#2018-10-1715:56strickinatowhoa - very unexpected, but totally works!! Thank you!#2018-10-1715:49taylorre: #2 yes the name of the spec must match the name of the key, however you can alias the original spec to whatever name you like e.g. (s/def ::my-date-time :clj-time.spec/date-time)
#2018-10-1715:50taylor(then use ::my-date-time
in your keys
spec)#2018-10-1715:57strickinatoThank you so much!! All answers I needed!#2018-10-1716:13Nolanwhere would be the best place to expand a bit on a question from yesterday? i have a somewhat longer scenario id like to get some input on, but dont want to spam the channel. is there an active mailing list for spec?#2018-10-1716:25taylorI wouldn't worry about spamming the channel :man-shrugging: There's clojureverse, stack overflow (code review section?), there's a google group/mailing list for Clojure but not sure if there's a spec-specific one#2018-10-1716:33Nolanmuch appreciated. still thinking on it a bit#2018-10-1718:12manutter51This seems like an easy one: how do I specify the :args
for this fn when I do s/fdef
? (defn demo [opts & [label]]
{:opts opts
:label label})
#2018-10-1718:14manutter51Iāve got this: (s/def :key/a string?)
(s/def :key/b boolean?)
(s/def :key/c vector?)
(s/def :demo/label string?)
(s/fdef demo
:args (s/cat :opts (s/keys :req [:key/a]
:opt [:key/b :key/c])
:label (s/? :demo/label))
:ret map?)
#2018-10-1718:14manutter51which seems to pass: (s/explain (s/get-spec 'demo) '(demo {:key/a "foo"}))
;; ==> Success!
#2018-10-1718:15manutter51but when I instrument it and try to run it, it blows up. (demo {:key/a "foo"})
;; #error {:message "Call to #'cljs.user/demo did not conform to spec:\nIn: [0] val: ([:key/a \"foo\"]) fails at: [:args] predicate: (cat :opts (? :demo/opts) :label (? :demo/label)), Extra input\n:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378]\n:cljs.spec.alpha/value {:key/a \"foo\"}\n:cljs.spec.alpha/args {:key/a \"foo\"}\n:cljs.spec.alpha/failure :instrument\n", :data {:cljs.spec.alpha/problems [{:path [:args], :reason "Extra input", :pred (cljs.spec.alpha/cat :opts (cljs.spec.alpha/? :demo/opts) :label (cljs.spec.alpha/? :demo/label)), :val ([:key/a "foo"]), :via [], :in [0]}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378], :cljs.spec.alpha/value {:key/a "foo"}, :cljs.spec.alpha/args {:key/a "foo"}, :cljs.spec.alpha/failure :instrument}}
;; Error: Call to #'cljs.user/demo did not conform to spec:
;; In: [0] val: ([:key/a "foo"]) fails at: [:args] predicate: (cat :opts (? :demo/opts) :label (? :demo/label)), Extra input
;; :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378]
;; :cljs.spec.alpha/value {:key/a "foo"}
;; :cljs.spec.alpha/args {:key/a "foo"}
;; :cljs.spec.alpha/failure :instrument
#2018-10-1718:19taylorhere's another take:
(s/fdef demo
:args (s/cat :opts (s/keys :req [:key/a]
:opt [:key/b :key/c])
:rest (s/? (s/cat :label :demo/label)))
:ret map?)
#2018-10-1718:30manutter51Ok, I get app:cljs.user=> (demo {:key/a "foo"})
#error {:message "Call to #'cljs.user/demo did not conform to spec:\nIn: [0] val: [:key/a \"foo\"] fails at: [:args :opts] predicate: map?\n:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378]\n:cljs.spec.alpha/value {:key/a \"foo\"}\n:cljs.spec.alpha/args {:key/a \"foo\"}\n:cljs.spec.alpha/failure :instrument\n", :data {:cljs.spec.alpha/problems [{:path [:args :opts], :pred map?, :val [:key/a "foo"], :via [], :in [0]}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378], :cljs.spec.alpha/value {:key/a "foo"}, :cljs.spec.alpha/args {:key/a "foo"}, :cljs.spec.alpha/failure :instrument}}
Error: Call to #'cljs.user/demo did not conform to spec:
In: [0] val: [:key/a "foo"] fails at: [:args :opts] predicate: map?
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378]
:cljs.spec.alpha/value {:key/a "foo"}
:cljs.spec.alpha/args {:key/a "foo"}
:cljs.spec.alpha/failure :instrument
#2018-10-1718:34manutter51wait, for some reason you just made me think of the caveat about nested specs, I wonder if thatās itā¦#2018-10-1718:54manutter51Nope, no joy in Specville#2018-10-1718:54manutter51I should mention this is CLJS-specific ā seems to work ok in CLJ#2018-10-1719:46taylorhmm yeah I tested in CLJ and it works, and I feel like I've seen a JIRA ticket for this issue or similar for CLJS#2018-10-1719:50Nolanfinally got around to posting a longer question on SO. hopefully it makes sense, but let me know how it could be improved for clarity and whatnot, or if im just missing basic concepts here. in any case its been a fun (s/)exercise
https://stackoverflow.com/questions/52862471/using-clojure-spec-to-decompose-a-map#2018-10-1721:07Nolan@taylor epic response! really appreciate it. i had played with using a separate s/def
for the keys, but i kept going down a path where i was trying to use multi-specs there, too, to no avail. i think ill ultimately use something like the get-spec-keys
way. the eval isnāt something i had thought about but thats a cool one too. thanks again man!#2018-10-1817:45Nolanpretty amped about how this turned out. ended up dispatching using s/or
, and it will work with arbitrarily merged multi-specs (as long as the s/keys
are defined by the convention you suggested, which im ok with, since this is sort of a hack anyway). but its pretty sweet. it works pretty much just like select-keys
i.e.
(select-keys-spec m ::many-level-merged-spec)
#2018-10-1816:19Alex Miller (Clojure team)ādoesnāt workā == ?#2018-10-1816:20lwhortondisregard, being stupid again. š#2018-10-1816:20lwhortoni was getting compiler errors and it was mostly unrelated (missing spec definition further up the chain)#2018-10-1816:21favila:foo/1
won't read; using a bare number as the name part in a keyword literal is a bad idea#2018-10-1816:21favilathat it works at all is an historical accident, and it wasn't fixed because it was noticed too late#2018-10-1816:22lwhortonis ::1
equally a bad idea?#2018-10-1816:22favilayes, so is :1
#2018-10-1816:23favilathink "would this be a readable symbol if I chopped off the initial :
s?"#2018-10-1816:24lwhortonwell, isnāt {:1 true}
a valid map?#2018-10-1816:24favilait is#2018-10-1816:24favilaI'm just talking about literals#2018-10-1816:25lwhortonhm, i have to digest this because itās not making sense. are you saying that ::1
ultimately just becomes my.namespace/1
and thus {1 true}
?#2018-10-1816:25favilano#2018-10-1816:27favilahttps://dev.clojure.org/jira/browse/CLJ-1527#2018-10-1816:27favilahttps://dev.clojure.org/jira/browse/CLJ-1286#2018-10-1816:27lwhortonš thanks#2018-10-1816:27favilahttps://dev.clojure.org/jira/browse/CLJS-677#2018-10-1816:29favilaI am saying clojure does not have a strict formal grammar for keywords and symbols, only human descriptions and implementations; using numbers is a greyer, edgier case that risks you falling into implementation-specific behavior#2018-10-1816:30favilaI think this is the heart of this particular reader ambiguity: https://dev.clojure.org/jira/browse/CLJS-677?focusedCommentId=35025&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-35025#2018-10-1816:30favilabut that's just my opinion#2018-10-1816:30favilano core member has confirmed or clarified to my knowledge#2018-10-1816:31taylorClojure 1.9.0
user=> {::1 1}
#:user{:1 1}
user=> {:1 1}
{:1 1}
user=> #:user{:1 1}
#:user{:1 1}
user=> {:user/1 1}
RuntimeException Invalid token: :user/1 clojure.lang.Util.runtimeException (Util.java:221)
1
RuntimeException Unmatched delimiter: } clojure.lang.Util.runtimeException (Util.java:221)
#2018-10-1816:32favilaoh also https://dev.clojure.org/jira/browse/CLJ-1252#2018-10-1816:32favilaI think this started it all#2018-10-1816:33Alex Miller (Clojure team)well Iām a core member and I wrote that ticket and patch#2018-10-1816:33Alex Miller (Clojure team)the original intent was that keywords follow the same constraints as symbols#2018-10-1816:33Alex Miller (Clojure team)symbols cannot start with a leading digit#2018-10-1816:34Alex Miller (Clojure team)the regex for keywords is subtly incorrect and allows keywords to have a leading digit#2018-10-1816:34Alex Miller (Clojure team)by āfixingā that we discovered that people were using such things (at the time java.jdbc made result set keywords like that for example)#2018-10-1816:35Alex Miller (Clojure team)there did not seem to be any good reason to break peopleās existing working code so we rolled that back#2018-10-1816:36Alex Miller (Clojure team)so itās possible to read and use :1
and ::1
and we have no plans to make that stop working#2018-10-1816:36favilait's deliberately not specifically allowed, however?#2018-10-1816:36favilaby specifications#2018-10-1816:36Alex Miller (Clojure team)I would say thatās never been resolved#2018-10-1816:37Alex Miller (Clojure team)CLJ-1527 is the ticket to do so#2018-10-1816:37favilafrom https://clojure.org/reference/reader#_literals#2018-10-1816:38favila> Keywords - Keywords are like symbols, except:
> They can and must begin with a colon, e.g. :fred.
> They cannot contain '.' or name classes.#2018-10-1816:38Alex Miller (Clojure team)āresolvingā means making that reference page and the code in sync#2018-10-1816:38Alex Miller (Clojure team)but the first question is what should be allowed#2018-10-1816:40Alex Miller (Clojure team)there are plenty of other open areas for characters that are allowed in the reader but not explicitly allowed in the reference docs#2018-10-1816:41favilatwo things: 1) "are like symbols" seems to cause confusion about whether what follows includes or does not include the initial colons. that was what my comment in cljs-667. 2) keywords can't have '.'? even in the ns part? (I know putting them in the name part is discouraged because that looks like class names). That's definitely wrong?#2018-10-1816:44Alex Miller (Clojure team)keywords can definitely have dots in the namespace part#2018-10-1816:45favilaany way to fix those docs short of a jira patch?#2018-10-1816:45favila"They cannot contain '.' after the first '/' or name classes" suggested edit#2018-10-1816:46Alex Miller (Clojure team)It says āor in namespace namesā ?#2018-10-1816:46favila"keywords are like symbols, except: They cannot contain '.' or name classes." is what it says now#2018-10-1816:47Alex Miller (Clojure team)oh, sorry I was looking at the symbol part#2018-10-1816:48favila"are like symbols" is the handwavy part that I think causes most of the confusion#2018-10-1816:48Alex Miller (Clojure team)Iāll look at an edit to that, but the intent again is to be similar to symbols and allow dots in namespaces#2018-10-1816:49Alex Miller (Clojure team)agreed - the question is whether to describe two things as similar and then indicate differences or to restate, obscuring the differences#2018-10-1816:50Alex Miller (Clojure team)I donāt feel that there is any ambiguity in intent that keyword namespaces should allow .#2018-10-1816:50favilaspecifically "Symbols begin with a non-numeric character" combined with "keywords are like symbols" causes one to wonder, "does a keyword begin with : or the char after that"?#2018-10-1816:50Alex Miller (Clojure team)well that is exactly the bug in the regex#2018-10-1816:50Alex Miller (Clojure team)but I would say after the colon#2018-10-1816:51favilathe intent of the spec being: "expand ::, chop off the first :, and you should have a valid symbol literal"#2018-10-1816:51Alex Miller (Clojure team)if you ask for the namespace and name parts of a keyword, you donāt see a colon. the colon is syntactical#2018-10-1816:52Alex Miller (Clojure team)I donāt think thatās a semantically meaningful operation, so I wouldnāt describe it that way#2018-10-1816:53favilawhat I mean is the intuitive sense of a valid keyword was intended to be "does it look like a symbol if the colons that make it a keyword were removed"#2018-10-1816:53favilaor at least thats how I am interpreting what you are saying.#2018-10-1816:53Alex Miller (Clojure team)I understand what you mean. I just wouldnāt describe it that way#2018-10-1816:54favilaI think that's a little better than "keywords are like symbols," but still inappropriate for a capital-S "specification"#2018-10-1816:55favilabut I am glad I've discovered the authorial intent behind that phrase#2018-10-1816:55favilathank you#2018-10-1816:59Alex Miller (Clojure team)https://github.com/clojure/clojure-site/commit/5481163d24491ec2ebc5863bc2d0876d36aacc5a#2018-10-1817:01favilaThat :a/b.c
reads is an accident of history/implementation?#2018-10-1817:10dominicmThat isn't supposed to work?! #2018-10-1817:18Alex Miller (Clojure team)Iād say thatās not supported in either symbols or keywords. whether it reads is a separate question#2018-10-1817:52borkdudeexpound question:
I try to make a custom print function:
(alter-var-root #'s/*explain-out*
(expound/custom-printer
{:show-valid-values? false
:value-str-fn my-value-str
:print-specs? true}))
but I get:
Unhandled clojure.lang.ExceptionInfo
Unknown data:
{:data #function[clojure.spec.test.alpha/spec-checking-fn/fn--3026]}
what up?#2018-10-1817:58borkdudehmm, no matter what options I give, custom-printer doesnāt work#2018-10-1817:59borkdudeah got it#2018-10-1817:59borkdudeconstantly
#2018-10-1817:59bbrinckYep š#2018-10-1817:59borkdudethanks š#2018-10-1818:00bbrinckha, I didnāt really do anything š#2018-10-1818:00bbrinckJust keep in mind that alter-var-root
will appear to not work if you are within a clojure REPL (youāll want set!
in that case)#2018-10-1818:05borkdudeit appears that expound is working, but I donāt see my function being used in the output:
(alter-var-root #'s/*explain-out*
(constantly (expound/custom-printer
{:show-valid-values? false
:value-str-fn my-value-str
:print-specs? true
}) #_expound/printer)
)
#2018-10-1818:06borkdudeoh wait, it does#2018-10-1818:06borkdudesorry š#2018-10-1818:06bbrincknp!#2018-10-1818:20borkdude@bbrinck in my custom function I want to say: if this is a com.stuartsierra component map, I want to print only the keys (else Iām flooded with pages of information), else I want to do whatever is the default in expound#2018-10-1818:20borkdude@bbrinck can I do the else branch in a nice way?#2018-10-1818:31bbrinck@borkdude Hm, unfortunately itās not super easy to do the else
part right now, but if you create an issue Iāll think more about how to make it nicer. Right now, I think the best you could do is to use the private function value-in-context
or grab the value of the private var *value-str-fn*
and call that. https://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L85#2018-10-1818:31bbrinckNot exactly pretty š#2018-10-1818:32borkdudeWhatās the best way to make clojure.spec.alpha not print the entire component system map. We are using (s/assert ::system system). Even when making a custom expound printing function, clojure.spec still barfs my entire console with output.#2018-10-1818:34bbrinckIām not sure exactly what you mean. Can you provide an example of the output?#2018-10-1818:40borkdude@bbrinck https://gist.github.com/borkdude/fe495b05e157fd129628aa3a6e489b01#2018-10-1818:40bbrinckIām curious to know which part of the output is showing the entire map - I would think if you provide a custom printer, you could in principle hide it entirely (or show as much as you like), but maybe itās printing somewhere else?#2018-10-1818:41borkdudeuntil line 124 itās ok, there I only print the keys of the system map. afterwards, still the whole system is printed.#2018-10-1818:44bbrinck@borkdude Ah, right, so this isnāt actually spec - the issue is that the program is printing out all the exception data from the assertion#2018-10-1818:44borkduderight#2018-10-1818:44bbrinckIs this running in boot? I wonder if the error printer is configurable? I can see why this is the default (you want to see the data for an assertion exception)#2018-10-1818:45bbrinckbut in the context of spec, it will put all of the detailed explain-data
into the exception#2018-10-1818:45borkdudeyes, itās boot#2018-10-1818:46bbrinckI can see why thatās how boot works and I can see why spec adds a lot of info to the exception (might be useful), but when they work together, itās definitely a lot of output.#2018-10-1818:47borkdudeI also tried this:
(try (rr/reset)
(catch Exception e
(throw (component/ex-without-components e))))
#2018-10-1818:47borkdudebut that doesnāt work, because the assert exception already has the entire system in it :-s#2018-10-1818:48bbrinckhm, not sure what you mean. Why canāt you alter the exception data in ex-without-components
?#2018-10-1818:48bbrinckis it already in the string message?#2018-10-1818:49bbrinckif itās just in the data, I presume you could just dissoc it#2018-10-1818:50bbrinckor just construct a new error that has less info, using the message from the exception#2018-10-1818:50borkdudehmm, I have to inspect the exception in the REPL to see what to dissoc, but good idea#2018-10-1818:51bbrinckIf that doesnāt work, the code for assert
isnāt too long, might be possible to just use a custom one: https://github.com/clojure/spec.alpha/blob/f23ea614b3cb658cff0044a027cacdd76831edcf/src/main/clojure/clojure/spec/alpha.clj#L1959-L1968#2018-10-1818:51bbrinckbasically just omit line 1964 and 1965#2018-10-1818:52borkdudeah, right, thatās easier š#2018-10-1819:40borkdude@bbrinck so what I wanted was actually not a custom :value-str-fn
function, but a way to transform my value before itās printed by expound. would it make sense to have a :transform (fn [v] ...)
option in expound on the custom printer?#2018-10-1819:46bbrinckHmm, good question.#2018-10-1819:47bbrinckCan you talk more about why a transform fn would be preferable to customizing the way it prints?#2018-10-1819:48bbrinckMaybe we should move this to #expound since weāre talking about potential features#2018-10-2010:28slipsetThis is probably working as expected, but why?#2018-10-2010:28slipsethttps://gist.github.com/slipset/645b1a785a0d6e23097e03b837b6a18b#2018-10-2010:29slipsetSo, Iāve created a spec for a function like#2018-10-2010:29slipset(s/def ::reducing-fn (s/fspec :args (s/cat :accumulator any? :elem any?)))
#2018-10-2010:29slipsetThen, when I ask if a given fn is valid:#2018-10-2010:30slipset(s/valid? ::reducing-fn (fn [x y]) ;; takes forever and ends with a out of heap memory
#2018-10-2010:31slipsetSo it seems (because test.check
needs to be a dep for this to run) that spec is using generative testing to verify if the spec is valid.#2018-10-2010:31slipsetHmm, do I actually have a question? No, probably not, I just wanted to state that I was surprised by this, but itās probably just the way it has to be.#2018-10-2010:33hmaurerIs there a spec which forces a certain value? same as #(= % some-value)
but with a built-in generator, etc#2018-10-2010:43taylorYou can use a set with the value in it#2018-10-2010:59hmaurer@U3DAE8HMG how so?#2018-10-2010:59hmaurer@U3DAE8HMG oh yes sorry, nevermind#2018-10-2010:59hmaurerthanks!#2018-10-2014:18misha#{:foo}#2018-10-2014:18mishaslowpoke#2018-10-2020:03borkdudeSeems like some macros/special forms have different ways of catching/reporting insufficient input
user=> (let)
Syntax error macroexpanding clojure.core/let at (4:1).
() - failed: Insufficient input at: [:bindings] spec: :clojure.core.specs.alpha/bindings
user=> (defn)
Syntax error macroexpanding clojure.core/defn at (5:1).
() - failed: Insufficient input at: [:fn-name] spec: :clojure.core.specs.alpha/defn-args
user=> (fn)
Syntax error macroexpanding clojure.core/fn at (6:1).
() - failed: Insufficient input at: [:fn-tail]
user=> (def)
Syntax error compiling def at (7:1).
Too few arguments to def
user=> (if)
Syntax error compiling if at (1:1).
Too few arguments to if
#2018-10-2022:21gfredericksyour examples are distinguished by whether it's a special form or not#2018-10-2022:21gfredericksI get different results on version 1.10.0-alpha6#2018-10-2022:21gfredericksespecially interestingly, (fn*)
compiles#2018-10-2022:22gfredericksit seems to produce a function that throws an arity exception on any call#2018-10-2121:23borkdude@gfredericks yikes, in RC1 as well. maybe a bug? EDIT: it was valid in 1.8.0 as well#2018-10-2121:25borkdudeitās probably not meant as a public function anyway?#2018-10-2121:30gfredericksI hadn't considered that; I'm not sure#2018-10-2121:43borkdudenot listed here: https://clojure.github.io/clojure/clojure.core-api.html#2018-10-2122:00Alex Miller (Clojure team)Thatās the compiler hook under fn#2018-10-2122:01Alex Miller (Clojure team)Certainly not the preferred entry point#2018-10-2122:06borkdudeWeāre writing some specs for clojure.core functions. I notice things can get a bit slower on big codebases with instrumenting all core functions. Is there a way to let spec know to not check when a function is called from clojure itself? so only external calls. I hoped direct linking would solve this, but I donāt think it is now#2018-10-2123:08Alex Miller (Clojure team)I would not expect calls between core functions to get checked#2018-10-2123:18andy.fingerhutI would be very surprised, if you are compiling Clojure with the default option of direct linking enabled, that calls between Clojure functions within Clojure are being checked, if you are using instrument. If you believe you are seeing otherwise, I would love to see an example of that.#2018-10-2123:20andy.fingerhutinstrument works by re-def'ing Vars. direct linking means that the current value of the Var is ignored, instead being embedded in the calling function at compile time.#2018-10-2123:22andy.fingerhutFYI, there is one function in clojure.data/diff's call tree that calls a function in clojure.set (I think clojure.set/difference) with non-set arguments. I have tried writing a spec for clojure.set/difference that causes errors if you call it with anything other than a set, and the only way I could make the error be caught is to disable direct linking when compiling Clojure.#2018-10-2123:23andy.fingerhutSome details here: https://dev.clojure.org/jira/browse/CLJ-2287#2018-10-2123:26andy.fingerhutAnd if you want to try compiling Clojure with direct linking disabled, and thus likely see instrumented runs get even slower, there are some instructions near the bottom of this page, with the heading "Building Clojure without direct linking": https://dev.clojure.org/display/community/Developing+Patches#2018-10-2206:51borkdudeThanks. Then my understanding of direct linking is correct and I need to do more research to see how to speed up things.#2018-10-2123:05hmaureris there a shortcut for a Spec s/or
which only contain spec keywords and want to tag them with the same keyword? i.e. (s/or ::foo ::foo ::bar ::bar ...)
#2018-10-2123:07Alex Miller (Clojure team)Not in spec, no#2018-10-2123:09hmaurerThanks @alexmiller. Other question: I read that specs are discouraged to be used as runtime because they can be slow. With that in mind, would it still make sense to use specs at runtime to branch on code based on the shape of data. i.e. check if data conforms to some spec to know if itās of some ākindā, branch on that, etc#2018-10-2123:10hmaureruse it as duck typing for dynamic dispatch#2018-10-2123:11hmaureri.e. use s/valid?
and s/conform
pretty extensively in runtime code#2018-10-2123:11Alex Miller (Clojure team)Only you can answer whether itās fast enough for you#2018-10-2123:11hmaurerItās definitely fast enough for me; Iām just wondering if it goes against some deeper philosophy that Iām missing and could come bite me in the back down the line (beyond performance)#2018-10-2123:44hmaurerIāve another question on Clojure spec for whomever is around: how do you avoid circular namespace dependencies? It seems preferable to put every spec in a namespace mapping to the namespace part of the spec keyword to be able to then require and alias that namespace wherever the spec is used. i.e.
(ns my-app.schema
(:require [my-app.schema.person :as person]))
(s/def ::person (s/keys :req [::person/first-name]))
(ns my-app.schema.person)
(s/def ::first-name string?)
The simplest example of the issue Iām running into is, then, what if I want to add a friends
attribute on a person. i.e.
(ns my-app.schema.person
(:require [my-app.schema :as schema])) ; oops, circular dependency!
(s/def ::first-name string?)
(s/def ::friends (s/coll-of ::schema/person))
#2018-10-2123:47hmaurerThe only solution I can think of it to still create the my-app.schema.person
namespace but to move the attribute specs for person
in the my-app.schema
namespace, like this:
(ns my-app.schema)
(s/def :my-app.schema.person (s/keys :req [:my-app.schema.person/first-name :my-app.schema.person/friends]))
(s/def :my-app.schema.person/first-name string?)
(s/def :my-app.schema.person/friends (s/coll-of :my-app.schema/person))
#2018-10-2123:48hmaureractually I could still use the namespace alias here, but I would have to define the attribute specs in my-app.schema
namespace and essentially just create an almost blank file for the my-app.schema.person
namespace#2018-10-2123:54misha> It seems preferable to put every spec in a namespace mapping to the namespace part of the spec keyword
false opieop
actually, "it really depends..."#2018-10-2123:56mishaimagine random map flowing through your app code. Do you really want to look for :my-app.schema.person/first-name
in it? or is something more readable would be better and enough? (and I bet :my-app
is much longer, and you'll end up with keys violating PEP8's 80 chars length in no time)#2018-10-2200:00mishaalso, I feel like you split namespaces too much (with regards to spec, at least.)#2018-10-2200:01misha@hmaurer#2018-10-2200:03hmaurer@misha can you make it much shorter though? you could skip the my-app.schema
part, sure#2018-10-2200:06mishathe spec kw length you have in example - is ok as is I think, but: I'd used that length for something more descriptive/convenient, so you would not feel the nee to alias it away, and avoid entire issue with dependencies.#2018-10-2200:08mishaand when you don't need to use aliases, you just need to make sure all specs are :required at the app entry points. no need to import those all over the place, as those are accessible from global registry, which you can get away initing just once.#2018-10-2200:11mishait is helpful to think about spec-kw namespaces as db table names, not as code namespaces those happen to be defined in. Otherwise specs would not survive first mild refactoring#2018-10-2200:13hmaurer@misha that makes sense; thanks a lot š#2018-10-2200:13hmaureralthough I was going to put those spec in a schema
namespace, not next to code, so there isnāt a big difference there between that and the ātable namesā approach youāre suggesting#2018-10-2200:13hmaurerbeyond the extra verbosity of the prefix#2018-10-2200:14mishayou receive map as rest response. it contains "...schema.../..." keys. what a waste of electricity opieop#2018-10-2200:15hmaurerš#2018-10-2200:16mishathen, imagine, you now need to move them around in the code. ā you either loose all your aliases, or break backward compatibility. ouch#2018-10-2200:17andy.fingerhutI mean, isn't one of the proposals of namespaces in keys that you don't rename them, unless you are planning to make a breaking change in a published API, and you should weigh consequences of that pretty seriously?#2018-10-2200:17hmaurerso I guess it would still be fine to namespace-qualify collection specs (i.e. (s/def ::person (s/keys ...
), but you would avoid such long namespace qualification for key specs#2018-10-2200:17hmaurer?#2018-10-2200:17andy.fingerhutAnd gzip and things like Transit reduce the waste of electricity somewhat.#2018-10-2200:18mishathere is an ugly workaround, (create-ns 'foo.bar.baz) (alias 'fbb 'foo.bar.baz), but it is meh.#2018-10-2200:18hmaurerbecause collection specs are essentially a codebase-level notion, whereas keys may be sent over the network, stored, etc#2018-10-2200:18hmaurerI think?#2018-10-2200:18misha@andy.fingerhut imagine "...schema..." suffix in every sql column or table name#2018-10-2200:19andy.fingerhutMy mind isn't reeling yet, but I haven't lived and breathed it day in and out, either. Six of one, half dozen of another.#2018-10-2200:19mishaand yes, you should not rename those, if they are public outside the app code. therefore if you suddenly need to move definitions elsewhere - file ns and spec ns do not match anymore.#2018-10-2200:21misha@hmaurer nested maps is the collection spec over the wire right there.#2018-10-2200:22hmaurermmh yeah#2018-10-2200:22hmaurerbut the keyword does not appear on the wire#2018-10-2200:22mishaanyway. the best way to appreciate this - is to experience pain yourself kappa#2018-10-2200:24mishatry to model your app your way, and give it week or two#2018-10-2200:26mishawhy don't they appear?
{:my-app.schema/persons [{:my-app.schema.person/first-name "joe"}]}
#2018-10-2200:27hmaurerah yep nevermind, youāre right#2018-10-2200:27mishaalso, if you don't preserve ns for js clients - you are missing out as well: "my-app.schema.person/first-name"
e.g. cheshire prints qualified keywords this way by default#2018-10-2200:27mishaand in js/python/java/xpath/etc. you can just obj["my-app.schema.person/first-name"]
#2018-10-2200:28hmaurerso, i.e., in this case, would you name things as:
(ns my-app.schema)
(s/def :schema/person (s/keys :req [:person/first-name :person/friends]))
(s/def :person/first-name string?)
(s/def :person/friends (s/coll-of :schema/person))
?#2018-10-2200:30mishawell, if you are confident you will not have external person - it might be ok. otherwise, prepending with a short project codename would not hurt.
but I'd think twice before adding anything between codename and entity name#2018-10-2200:32mishaif you are in a corporate setting, you are probably required to add at least full org name instead or in addition to project name: :com.cognitect/person
:com.cognitect.project/person
#2018-10-2207:34borkdudeIs it possible to have clojure.spec not throw exceptions on fdef violations, but only print āfdef error, line so and soā and then continue as always?#2018-10-2213:08Alex Miller (Clojure team)no#2018-10-2213:31hmaureris there a way to conform a value to a spec only one level deep?#2018-10-2214:04Alex Miller (Clojure team)no#2018-10-2222:50hmaurerthanks#2018-10-2222:45hmaurerhttps://github.com/funcool/cats/blob/master/src/cats/monad/either.cljc#L48
Is there anyway I can spec such a type (`Either left right`) on a per-function basis? So for example, I might want to add a spec to a function saying it returns either a string?
or a number?
, etc#2018-10-2222:53Alex Miller (Clojure team)Always good to remember that specs are not types. You can use s/or to spec an alternative but thatās not going to be generic.#2018-10-2222:53hmaurer(I realise this sounds similar to what the s/or
spec does, but I need to work with records here, as defined in cats)#2018-10-2222:53hmaurerah#2018-10-2222:53hmaureris there any way I can do something generic in this case @alexmiller?#2018-10-2222:54Alex Miller (Clojure team)You can treat records as maps with unqualified keys and use :req-un to specify specs on keys#2018-10-2222:55Alex Miller (Clojure team)But, not generically#2018-10-2222:56Alex Miller (Clojure team)For specific functions, you could define a :fn spec that checks a relationship between args and ret#2018-10-2222:56hmaurer@alexmiller but then Iāll have to make qualified keys for every possible way there is to use the Either record. As far as I know there is no way to directly define an unqualified map spec, right? i.e. nothing like (s/foo :left string? :right number?)
which would accept a map with keys :left and :right matching the specs#2018-10-2223:12misha@hmaurer
(s/valid?
(s/cat
:left (s/tuple #{:left} string?)
:right (s/tuple #{:right} number?))
{:left "a" :right 1})
=> true
kappa#2018-10-2223:20Alex Miller (Clojure team)this s/cat assumes maps are ordered, which they are not. in fact, in latest spec, you will get false here.#2018-10-2223:20Alex Miller (Clojure team)s/cat can only be used with sequential? collections now#2018-10-2223:29misha@hmaurer
there is also more legal way, where map-of will permit only 1 map-entry:
(s/or
:left (s/map-of #{:left} string?)
:right (s/map-of #{:right} number?))
#2018-10-2223:15hmaurerO_o#2018-10-2223:17mishadon't do it#2018-10-2223:17misha(s/def :cats/left (s/or :string string? :exception #(instance? Exception %)))
(s/def :cats/right any?)
(s/def :cats/monad (s/keys :req-un [(or :cats/left :cats/right)]))
(s/valid? :cats/monad {:left (ex-info "foo" {})})
(s/valid? :cats/monad {:left "a"})
(s/valid? :cats/monad {:right [:bar]})
=> true
=> true
=> true
#2018-10-2223:18mishayou then can (s/merge)
with :more.specific/right
#2018-10-2223:19mishabut I'd rewrite all using exceptions :D#2018-10-2223:19hmaurerRight, but then for every potential use of Either i need to define a keyword spec, i.e. :some.specific/right
#2018-10-2223:20hmaurersame for left#2018-10-2223:20hmaurerinstead of being able to just specify it inline in the return spec of a function#2018-10-2223:20mishawell, the same for any return value, no?#2018-10-2223:21mishaor, if you will have specific spec for wrapped value anyway, you can unpack in custom predicate, and s/and with that#2018-10-2223:21mishabut then you'll lose generators for free opieop#2018-10-2223:21mishaconsider exceptions kappa#2018-10-2223:24mishaor, keep lifting separate from business-logic#2018-10-2223:30favila@hmaurer I had this problem continually, where I have a spec key but I need to narrow it further in specific contexts without changing the key name#2018-10-2223:30favilaI ended up making a terrible macro, keys+ to do it#2018-10-2223:31favila(s/def :either/either (s/keys ::req-un [:either/left :either/right]))
is the basic notion of "either"#2018-10-2223:33favila(s/spec (s/merge :either/either (keys+ :req-un [:either/left :either/right] :conf {:either/left number? :either/right string?}))
#2018-10-2223:34favilathat would be "this is an :either/either, but some of the keys use a different generator and predicate"#2018-10-2223:35favilahttps://gist.github.com/favila/ab03ba63e6854a449d64d509aae74618 if you are interested#2018-10-2223:35misha(defmacro speceither [left-spec right-spec]
`(s/or
:left (s/map-of #{:left} ~left-spec :min-count 1)
:right (s/map-of #{:right} ~right-spec :min-count 1)))
(s/valid?
(speceither string? (s/coll-of number?))
{:right [1 2 3]})
=> true
conformed values are garbage though, but exercise works just fine#2018-10-2223:42hmaurerthanks @favila! Iāll read this up. Does this spec give nice errors / generators / conforming?#2018-10-2223:45favilaThe only thing you are allowed to do is use :conf
to override the normal spec/predicate/generator for that key#2018-10-2223:45favilaāOverrideā is maybe too strong because the registered spec is checked too#2018-10-2223:57hmaurerAh right, great. Out of curiosity do you use Either / cats in your codebase @favila? Or did you write this for another use-case?#2018-10-2300:18favilaOther use case#2018-10-2300:18favilaBut itās the same basic problem#2018-10-2300:19favilaThereās a general structure and spec, but also more specific specs which must nevertheless maintain the structure#2018-10-2300:19favilaUsing predicates all the time became unbearable #2018-10-2313:39j0niHey folks, I'm having a tough time trying to figure out how multi-specs interact with unqualified keys - here's some example code which is a trivial version of what I'm trying to achieve: (def mode? #{:dog :cat})
(s/def ::cat string?)
(s/def ::dog string?)
(s/def ::mode mode?)
(s/def ::dog-o (s/keys :req-un [::mode
::dog]))
(s/def ::cat-o (s/keys :req-un [::mode
::cat]))
(defmulti animode ::mode)
(defmethod animode :dog [_] ::dog-o)
(defmethod animode :cat [_] ::cat-o)
(s/def ::animal (s/multi-spec animode ::mode))
then some repl: user> (s/explain ::animal {:cat "edward" :mode :cat})
val: {:cat "edward", :mode :cat} fails spec: :user/animal at: [nil] predicate: animode, no method
=> nil
user> (s/explain ::animal {:cat "edward" ::mode :cat})
val: {:cat "edward", :user/mode :cat} fails spec: :user/cat-o at: [:cat] predicate: (contains? % :mode)
=> nil
any guidance as to how I square this apparent circle?#2018-10-2314:01seancorfield@j0ni Your defmulti
and s/multi-spec
should have :mode
, not ::mode
. You want a function that operates on your data and returns the discriminant. That's the keyword :mode
, not the spec ::mode
.#2018-10-2314:02seancorfield(def mode? #{:dog :cat})
(s/def ::cat string?)
(s/def ::dog string?)
(s/def ::mode mode?)
(s/def ::dog-o (s/keys :req-un [::mode
::dog]))
(s/def ::cat-o (s/keys :req-un [::mode
::cat]))
(defmulti animode :mode)
(defmethod animode :dog [_] ::dog-o)
(defmethod animode :cat [_] ::cat-o)
(s/def ::animal (s/multi-spec animode :mode))
then user=> (s/explain ::animal {:cat "edward" :mode :cat})
Success!
nil
#2018-10-2314:09j0niwow, thanks @seancorfield! I had been through this permutation, but apparently evaluating the ns is insufficient, I needed to reload the namespace completely#2018-10-2314:10seancorfieldA trick with defmulti
is to put (def animode nil)
above it and then it should work with just a re-eval of the ns.#2018-10-2314:10j0niooh nice š#2018-10-2314:10j0niwell, useful, if actually a little gross š but I'll take it#2018-10-2315:41adamfreyI was surprised by this behavior in spec. Is there a reason that I shouldn't expect this to work?
(gen/generate (s/gen #{:works})) => :works
(gen/generate (s/gen #{nil})) => clojure.lang.ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.
I know I can do (gen/generate (s/gen nil?))
to get around this, but I found this while writing some code for handling generic generator overrides#2018-10-2315:48adamfreyI guess I can take generators out of the equation and get to the crux of the issue:
(s/valid? #{nil} nil) => false
(s/valid? #{false} false) => false
the spec just calls the set as a function and (#{nil} nil) => nil
in clojure and spec is looking for the function truthiness. The only way to make this work would be to change the way spec deals with sets as specs where spec could call them with (contains? #{nil} nil)
instead. But that sort of special casing might be undesirable.#2018-10-2316:42bmaddyI'm trying to come up with a spec for nested associative structures:
(s/def ::nested-associative
(s/or :coll (s/coll-of (s/or :branch ::nested-associative
:leaf string?)
:min-count 0
:max-count 1
:gen-max 1)
:map (s/map-of keyword? (s/or :branch ::nested-associative
:leaf string?)
:min-count 0
:max-count 1
:gen-max 1)))
;; in the repl:
> (gen/sample (s/gen ::nested-associative) 100)
clojure.lang.ExceptionInfo: Unable to construct gen at: [:map 1 :branch :map 1 :branch :coll :branch :coll :branch] for: :user/nested-associative
Does anyone know why this would be crashing randomly? Alternatively, is there maybe a better way to do this?#2018-10-2316:52bmaddyInteresting, it works if I just generate the s/coll-of
form. It must be something to do with s/or
.#2018-10-2401:57mishaI think it fails to create generator within s/*recursion-limit*
#2018-10-2401:59misha(dotimes [_ 100]
(binding [s/*recursion-limit* 10]
(count (gen/sample (s/gen ::nested-associative) 100))))
=> nil
(dotimes [_ 100]
(binding [s/*recursion-limit* 1]
(count (gen/sample (s/gen ::nested-associative) 100))))
=>
CompilerException clojure.lang.ExceptionInfo: Unable to construct gen at: [:coll :branch :map 1 :branch] for: :scratch/nested-associative #:clojure.spec.alpha{:path [:coll :branch :map 1 :branch], :form :scratch/nested-associative, :failure :no-gen}
#2018-10-2402:09mishatried 10000 times, limit=9 threw in a ~20 seconds, limit=10 ran for minutes, had to kill it opieop#2018-10-2402:14mishaso, you need to add non-recursive clause to s/or, so it can be chosen when recursion-limit is reached:
(s/def ::nested-associative
(s/or
:leaf string?
:coll (s/coll-of ,,,)
:map (s/map-of ,,,)))
#2018-10-2402:14misha@bmaddy#2018-10-2402:26taylor@bmaddy Iām able to get that to reliably generate with s/*recursion-limit* 1
by pulling the recursive s/or
branch into its own spec:
(s/def ::branch
(s/or :leaf string?
:branch ::nested-associative))
(s/def ::nested-associative
(s/or :coll (s/coll-of ::branch
:min-count 0
:max-count 1
:gen-max 1)
:map (s/map-of keyword? ::branch
:min-count 0
:max-count 1
:gen-max 1)))
but Iām not sure why that is, maybe itās the way that recursive depth is tracked internally and the āinlineā recursive s/or
specs lose some context on recursion?#2018-10-2402:27taylor(dotimes [_ 1000]
(binding [clojure.spec.alpha/*recursion-limit* 1]
(dorun (gen/sample (s/gen ::nested-associative) 100))))
#2018-10-2403:03mishathis is the first thing I did, and it still failed for me numerous times, @taylor#2018-10-2403:53taylor:man-shrugging: works on my machine#2018-10-2404:16bmaddyOh, I get it now. I'd tried setting s/*recursion-limit*
to a low number before to help avoid StackOverflow issues, but that's actually the problem here. When the recursion limit is hit, it must throw that error. Thanks @misha and @taylor!#2018-10-2415:40bastiIs there a strict validation mode in spec?
Whereas if you provided more data then fail?#2018-10-2415:52dadairNo; closed specs are generally discouraged. You can do your own strictness using s/and, however.#2018-10-2419:31bbrinck@basti spell-spec gives you options for either keeping the map open (but reporting likely misspelled keys) or complete closed maps (not recommended) https://github.com/bhauman/spell-spec#2018-10-2500:23jrychterI really do not like this canned answer ("closed specs are discouraged"). It gets repeated a lot, and yet there are valid scenarios where you do want to restrict the data to exactly what is specified in the spec without manually duplicating lists of keys. Think about API endpoints, or anything that receives data from an untrusted source in general.#2018-10-2500:34seancorfieldIf you ignore the additional parameters, why do you care whether they are passed in?#2018-10-2500:36seancorfieldOne valid case I can see for closed specs is when you're about to store data into a database (and you'll get an error if you try to store columns/keys that do not exist in the table). At that point tho', you can extract the list of valid keys from the spec itself and use select-keys
to get just the keys you care about. Again, you don't necessarily care that more data was present -- you just need to ensure you don't send it to JDBC.#2018-10-2500:37seancorfield(and of course you could use select-keys
on the API input data too, if you wanted)#2018-10-2500:37seancorfieldThe reality is that most APIs out there silently ignore extra form and query parameters.#2018-10-2500:38seancorfield(which is all a long way of saying I think the canned answer is correct)#2018-10-2500:59taylorI agree there are cases were you need to restrict inputs but I think those are usually near the boundaries of apps, and spec doesnāt prevent this. Other than that I donāt think (in Clojure) your code should ābreakā if passed a map with some data it doesnāt care about, and I think most projects have much more code on that side of the fence than the other#2018-10-2500:59jrychterIf your database is a document database (like mine) and stores your data pretty much in model form (like mine does), and your API serves your ClojureScript app which keeps the same data model, then you very much care about additional parameters. You can pass data pretty much intact between client code and the database, but you do need to make sure that you don't store any extra data that just happens to arrive over the network.#2018-10-2501:01jrychterThe manual select-keys
solution seems obvious and results in error-prone code (been there done that). A more reasonable approach is to get the list of allowed keys from the spec.#2018-10-2503:30seancorfieldWhen I do select-keys
I derive the list of keys to be selected directly from the spec.#2018-10-2503:41jrychterā¦and that is exactly what I'll be doing.#2018-10-2501:02jrychterI'd also submit that "most APIs" describes "most traditional APIs". My APIs speak EDN and receive Clojure data structures which I verify using spec.#2018-10-2501:04csmpossibly s/conform
should remove keys not in the spec? Iām not sure of the implications of that, though.#2018-10-2501:05taylorthereās no need to change spec, you can easily get this restrictive behavior if you need it#2018-10-2501:06bbrinck@jrychter Does https://github.com/bhauman/spell-spec solve your problem?#2018-10-2501:07bbrinckYou donāt need to duplicate keys w/ spell-spec#2018-10-2501:09csmmaybe I should just ask, though: why doesnāt conform
remove keys not in a map spec?#2018-10-2501:09jrychterspell-spec
solves a different problem and has a different goal. But I am dealing with the problem myself, I just wanted to point out that whenever people repeat what Rich said about specs intended to be open, there should be a footnote saying "except in cases where you do need to restrict your data, such as in APIs".#2018-10-2501:10bbrinckAFAICT, spell-spec (specifically strict-keys
https://github.com/bhauman/spell-spec#spell-specalphastrict-keys ) does solve the problem you describe above. Can you talk more about why it doesnāt?#2018-10-2501:11bbrinck@csm AIUI, conform
is useful for destructuring, so presumably there isnāt a need to remove keys.#2018-10-2501:12bbrinck(because you would usually know the keys you want to grab as part of destructuring)#2018-10-2501:12csmš Iām just thinking out loud if there are approaches that would satisfy use cases like the above, but without making specs closed#2018-10-2501:13bbrinckAh, I see#2018-10-2501:14bbrinckremember that conform also changes the shape of data, so may not apply here if youāre trying to write the input to, say, a document DB#2018-10-2501:14bbrinck(I suppose you could conform/unform if it did remove unknown keys, but that gets a little strange š )#2018-10-2501:15csmyeah, I thought about that; maybe a fn that does ācheck this spec, then select-keysā would be an OK addition#2018-10-2501:15andy.fingerhutDoesn't the word "discouraged" in "closed specs are discouraged", vs. e.g. "closed specs are prohibited", convey to you that there might be occasional valid use cases for them?#2018-10-2501:15bbrinckanyway, I am curious why strict-keys
from spell-spec doesnāt work here#2018-10-2501:17andy.fingerhutAnd sure, people can repeat things without knowing the reasons for them, nor the subtler points of when they are good ideas, and when they are not. That is a more general habit of some people to be second-handed in what they say, rather than understanding themselves.#2018-10-2501:17andy.fingerhutnot restricted to computing at all.#2018-10-2501:17bbrinckYeah, I think ādiscouragedā is an accurate term, especially since Iāve seen a few people jump to use closed specs by default. It turned out that open specs were a better fit.#2018-10-2501:17bbrinckBut yes, there are exceptions#2018-10-2501:18csmI, for one, wish spec was mature before I started using schema instead. Iām not a fan of closed specs for APIs#2018-10-2501:25jrychterI do agree that open specs are a good idea.#2018-10-2501:27andy.fingerhut@jrychter I am not trying to jump down your throat. I am asking because I would actually like to learn more about your use case where you want to disallow unrecognized keys. You say you may get these extraneous keys over the network at some document database service, and want to detect this as an error?#2018-10-2501:28jrychter@andy.fingerhut my API receives EDN data, in model form. I want to pass this data to my model code (where extra keys might not matter), and often directly to a document database (RethinkDB in this case). I do not want to store extra keys, for obvious reasons.#2018-10-2501:29jrychterI know that if one uses an SQL database, there is code that needs to re-pack the data, but in my case there is often no such code.#2018-10-2501:31andy.fingerhutBear with me if I ask something foolish for lack of experience in these kinds of applications, but if the data model is extended in the future, a reason commonly given for open specs is that unless you can somehow arrange for both participants to be upgraded simultaneously (e.g. by shutting down the service while everyone upgrades), then you can't upgrade without lots of errors flying around during the partially updated state, can you?#2018-10-2501:32jrychterAn invariant in my case is that the spec describes the data model.#2018-10-2501:32andy.fingerhutIn all processes on all systems where the code is running, all of the time, even during updates to the data model?#2018-10-2501:33andy.fingerhutIf yes, then it sounds like partially updated systems are never an issue in your use case. That certainly simplifies implementing such things, if so.#2018-10-2501:36jrychterAt the moment I mostly use atomic "stop-the-world" migrations, but in general the assumption has been that the spec needs to accomodate partially updated systems. But again: I do agree that open specs are a good idea, I just think that people take Rich's "discouragement" a bit too far and see closed specs as "a bad idea". They are not a bad idea.#2018-10-2501:38andy.fingerhutIf open specs are a good idea, then there must be some situations where closed specs are a bad idea.#2018-10-2501:39andy.fingerhutIt sounds like Rich & team's experience in consulting is that they have seen multiple scenarios where people wanted to close their specs, and later regretted it.#2018-10-2501:40andy.fingerhute.g. the first occurrence of the word "consulting" in his talk on spec here: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureSpec.md#2018-10-2501:42andy.fingerhutI mean, I know people can read/hear that and think "Rich means that for 100% of all software everywhere", but I doubt that. But he is speaking in context of an industry where I think there is an over-tendency to make brittle inextensible systems.#2018-10-2501:42taylorspec isnāt particularly unusual in encouraging openness/forward-compatibility either, itās a concern that has to be addressed in things like protobuf, thrift, etc. too#2018-10-2501:44andy.fingerhutI know I'm in the minority working at Cisco in embedded software that is far too much C code for my taste, but these systems are incredibly brittle. Changing an int to a long is often developer-weeks worth of effort, if it is in a message spread over 3 processes.#2018-10-2503:11the2bearsThis current discussion on closed specs sounds a little too much like using or relying on spec to clean your data. To be fair, though, I don't have any use cases for a closed spec myself and might be missing the real picture. Just seems better to clean the data with a different set of functions and tools.#2018-10-2503:14mishahow about not using tool-designed-not-to-solve-A to solve A?#2018-10-2503:15mishaif it is such a bummer, just implement closed keys with 1-2 macros yourself, or take one of many available in the libs already.#2018-10-2503:16mishaimplementing select-spec-keys is trivial, especially when you know what specifically you prefer to do if extra stuff present ā ignore, warn, blow up, etc.#2018-10-2506:10orestisI implement this select-spec-keys just yesterday. Itās very straightforward to write and I can deal with assumptions about namespaces, blow up or warm etc etc. #2018-10-2513:25jaihindhreddyHow can I add test.check
to my deps.edn
?#2018-10-2513:29jaihindhreddyBecause it doesn't have any dependencies, will this work?
{:deps
{clojure/test.check {:git/url ""
:sha "f01e3d8ad3317e6dfbeee29332a6122b1a6c12bc"}}}
#2018-10-2513:30gfredericksYou can use it with the mvn/version just like clojure#2018-10-2513:33jaihindhreddy{org.clojure/test.check {:mvn/version "0.10.0-alpha3"}}
then?#2018-10-2513:35gfredericksSounds right#2018-10-2513:38jaihindhreddyAh, so because test.check publishes artifacts (on maven) I don't need to worry about the fact that it doesn't have a deps.edn#2018-10-2513:41gfredericksRight
I guess I should add an empty one#2018-10-2513:41gfredericksI'll do that soon#2018-10-2514:49Alex Miller (Clojure team)you can force the dep type without having a deps.edn#2018-10-2514:51Alex Miller (Clojure team){:deps
{clojure/test.check {:git/url ""
:sha "f01e3d8ad3317e6dfbeee29332a6122b1a6c12bc"
:deps/manifest :deps}}}
#2018-10-2514:53Alex Miller (Clojure team)that overrides the manifest detection and forces it to be a deps.edn project, which is tolerant to a missing deps.edn#2018-10-2514:53Alex Miller (Clojure team)and just treats it as a project with no deps#2018-10-2516:15seancorfieldA useful trick -- is that documented in the CLI/tools.deps Reference? (specifically that it can be used to treat an arbitrary git-based project "as a project with no deps")#2018-10-2516:37Alex Miller (Clojure team)Not currently#2018-10-2600:38bbrinckIf youāve run into āCannot convert pathā errors with Expound (which happens if you use a spec with a conformer), please try out 0.7.2-SNAPSHOT and let me know what you think#2018-10-2602:02richiardiandreaWill try that out thanks a lot for the fix!#2018-10-2608:55borkdude(s/fdef foo
:args (s/cat :n number?)
:ret string?)
(defn foo [n]
1)
(s/conform (:ret (s/get-spec `foo)) 1) ;; :clojure.spec.alpha/invalid
(s/assert (:ret (s/get-spec `foo)) 1) ;; 1
#2018-10-2608:55borkdudewhy does the assert not throw there?#2018-10-2609:12mpenetyou prolly need to enable asserts#2018-10-2609:14mpenet(s/check-asserts true)#2018-10-2609:15mpenetby default they are compiled in but not enabled#2018-10-2609:18mpenetfor debugging (and because I am lazy) I sometimes use s/assert*, it bypasses the various flags#2018-10-2609:18mpenetbut that's hairy territory#2018-10-2611:27gfrederickswhy does stest/check
take symbols instead of vars? it doesn't require
code for you does it?#2018-10-2623:50asilvermanHi there clojurians, I am looking for some help regarding the use of clojure.spec.alpha. Basically I would like to verify that a map of key-value pairs i receive as an argument is a sub set of a well defined group. For example let the group of valid keys be :key1 :key2. I would like my spec to conform if I get a map with en empty amount of keys, or with either :key1 exclusively or :key2 exclusively or both :key1 and :key2 exclusively. Iām confused about how to specify this#2018-10-2701:15misha@ariel.silverman what does "exclusively" mean here exactly? only those keys, and no others?
are those qualified keys? do you have value specs too, or is it just keys?#2018-10-2916:13asilvermanThank you misha for the very thorough explanation. The code snipped was extremely helpful, as @U04V70XH6 mentioned, I also asked this question in the #beginners forum. The idea was to limit an API endpoint query parameters to a known set and error out when anything but a valid sub-set of them is passed down. Thank you again!#2018-10-2701:22misha#2018-10-2701:24mishaand a due disclaimer "clojure.spec is explicitly designed to discourage closed maps ("no keys except these")"#2018-10-2701:43seancorfield(it was answered in #beginners )#2018-10-2713:59souenzzo#datomic query returns a java.util.HashSet
, that prints like a set
but it's not a coll?
. So I cant use s/coll-of
on it.
I changed my spec from (s/coll-of ::stuff)
to seqable?
Should spec have a s/seq-of
that use seqable?
or somethink like that?#2018-10-2715:52borkdudehttps://github.com/slipset/speculative#test-tools#2018-10-2716:12dangercoderIs running (spec/explain ::foo data-structure) in the beginning of a function bad practise?#2018-10-2716:16dangercoderI'm using spec/fdef right now and turning on instrument in dev-environment.#2018-10-2716:36borkdudeNothingās bad, just depends what you want#2018-10-2716:51dangercoderTrue, I was thinking if there can be some kind of noticeable performance loss if having instrument on in production, but i'll have to measure and see later on.#2018-10-2717:22borkdudeyes, this definitely affects performance#2018-10-2717:48andy.fingerhutI don't have any measurements of a real application with instrument on vs. off, but I do have measurements for clojure.set functions with instrumentation on vs. off, for a simple spec that just checks whether the args satisfy set? or not. It is partway down the README on this page: https://github.com/jafingerhut/funjible#wait-isnt-this-what-clojurespec-is-for#2018-10-2717:49borkdudeI had enabled speculative that has some amount of core specs. My backend + frontend became significantly slower.#2018-10-2717:49andy.fingerhutinstrument in that case is significantly slower than the original functions when their run time is short. The percentage overhead obviously gets smaller as the run time of the instrumented function increases.#2018-10-2717:51borkdudeI think how may want to use āinstrument allā is when I have some weird bug whose stack trace I find uninformative. Then instrument all, run again, find the problem, turn it off.#2018-10-2717:51dangercoder@borkdude sounds like a solid approach#2018-10-2717:52borkdudesome specs I do want on all the time in dev, especially at boundaries like DB insert queries#2018-10-2717:53borkdudeitās a tweaking process I think#2018-10-2718:57seancorfieldAs another data point for the overhead of instrument
: clojure.java.jdbc
has its specs in a separate namespace and the time difference between running the tests with and without instrumentation -- just on the public API functions -- is huge. I would be extremely careful about turning on instrumentation in production.#2018-10-2719:00seancorfieldAnother thing to watch for: if you have a higher-order function instrumented, you'll invoke generative testing on the (function) argument being passed in -- which you almost certainly don't want in production (you probably wouldn't even want org.clojure/test.check
included as a dependency in production?).#2018-10-2719:13dominicmWe've been using s/assert to have dev and production operate differently. #2018-10-2818:17domkmWhy is s/nonconforming
not in the wiki? https://github.com/clojure/spec.alpha/blob/b2b5db433cad9f1ad56bd28f584136cef2bae73e/src/main/clojure/clojure/spec/alpha.clj#L1811#2018-10-2820:27Alex Miller (Clojure team)Undetermined whether it will remain in the api#2018-10-2820:55domkm@alexmiller I'm finding it to be extremely useful, for what it's worth#2018-10-2820:59Alex Miller (Clojure team)Are you using it for anything other than or?#2018-10-2821:00Alex Miller (Clojure team)I think itās definitely useful, but wonder whether just having a nonconforming flag on or would solve the 95% case#2018-10-2821:11domkm@alexmiller It does look like I am only using it for or
currently. I still think having general nonconforming
is useful and shouldn't be removed. We don't know what third-party specs we might want to utilize without conforming.#2018-10-2821:12Alex Miller (Clojure team)So far, I donāt think Iāve ever seen anyone use it for anything but s/or#2018-10-2821:00borkdudespeaking of undocumented APIs, is there a way to ask spec what functions are currently instrumented? not which functions are instrumentable?#2018-10-2821:01borkdudealternatively, asking wether a function is instrumented would also be fine#2018-10-2821:01Alex Miller (Clojure team)Ask the registry #2018-10-2821:02Alex Miller (Clojure team)Well I guess thatās not telling you whatās instrumented#2018-10-2821:02Alex Miller (Clojure team)Iād look at the impl of unstrument#2018-10-2821:02borkdudeIāll tell you the use case.#2018-10-2821:03borkdudehttps://github.com/slipset/speculative/blob/master/src/speculative/test.cljc#L28#2018-10-2821:03borkdudemaybe thereās a better way to do it.#2018-10-2821:03borkdudeIām now using the private atom: https://github.com/slipset/speculative/blob/master/src/speculative/test.cljc#L19#2018-10-2821:05borkdudethe with-unstrumented
macro is problematic in clojurescript though, because part of instrumenting/unstrumenting happens at macro compilation time and it uses two separate atoms to bookkeep instrumented functionsā¦#2018-10-2821:38borkdudeif (stest/instrument 'clojure.core/reduce)
gave back [cljs.core/reduce]
only when it wasnāt already instrumented, I could leverage that.#2018-10-2821:40borkdudebut of course I could call (s/unstrument 'clojure.core/reduce)
first, to check if it was already instrumentedā¦#2018-10-2821:40borkdudethis way I wouldnāt have to use something private#2018-10-2910:13borkdudefixed.#2018-10-2912:34borkdudehttps://twitter.com/stuarthalloway/status/1056881838927568896#2018-10-2916:33jaihindhreddyAlex Miller commented on a podcast about making specs more data oriented. Can't wait to see what's cookin' at Cognitect!#2018-10-2913:03domkmHas there been any thought/discussion about clojure.spec for directed graph validation? The problem we're running into is that some edges are required and that causes validation to never terminate due to cyclic validations. It would be interesting if there were a way to make required validation respect s/*recursion-limit*
.#2018-10-3013:17borkdudeI have an interesting case where I call stest/check
from the REPL (lein + cider nrepl) and it doesnāt finish. Anyone familiar with that one?#2018-10-3013:18borkdudeit does terminate when I run the tests from the command line#2018-10-3013:19Alex Miller (Clojure team)how do you know it wonāt finish? did you solve the halting problem!?#2018-10-3013:26borkdudeIām a human, I use heuristics.#2018-10-3013:28borkdudeseems to work with clj repl
#2018-10-3013:30borkdude$ clj -A:test:repl
user=> (stest/check `str)
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2524 0x3a94964 "
#2018-10-3013:33gfredericksHave you tried at least a few times from the command line?
Tests that probabilistically take a Long Time can be misleading like that with just a few data points#2018-10-3015:05borkdude@gfredericks trying.
hmm, now clj
lingers after printing the result of stests:
$ clj -A:test -e "(require,'[speculative.core])" -e "(require,'[clojure.spec.test.alpha,:as,stest])" -e "(stest/check \`str)"
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2524 0x3e14c16d "
#2018-10-3015:05borkdudeit just takes a long time before it returns to the cmd line. maybe garbage collecting?#2018-10-3015:06gfredericksis it exactly 60s?#2018-10-3015:09borkdudethis is the output with time
:
$ time clj -A:test -e "(require,'[speculative.core])" -e "(require,'[clojure.spec.test.alpha,:as,stest])" -e "(stest/check \`str)"
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2524 0x3e14c16d "
#2018-10-3015:13gfredericksI meant 60s following the end of the tests
if so, then adding (shutdown-agents)
after the stest/check
call should help#2018-10-3015:17borkdudeRepro:
;; run with:
;; clj -Srepro -Sdeps '{:deps {org.clojure/clojure {:mvn/version "RELEASE"} org.clojure/test.check {:mvn/version "RELEASE"}}}' -i stest_time.clj
(ns stest-time
(:require
[clojure.test :as t]
[clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest]))
(s/fdef clojure.core/str
:args (s/* any?)
:ret string?)
(stest/check `str)
(println "after stest")
(shutdown-agents)
(println "after shutdown agents")
Still takes 13s after printing the last message.#2018-10-3015:18borkdudemaybe any?
just generates loads of garbage?#2018-10-3015:18gfredericksthat sounds very strange#2018-10-3015:18borkdudeyou can try the scriptā¦#2018-10-3015:18gfredericksI mean it does, but I don't think a JVM needs to GC on shutdown#2018-10-3015:18gfredericksyou just burn it all down and let the OS sort it out#2018-10-3015:19borkdudemaybe a System/exit helps#2018-10-3015:19borkdudeyeah it doesā¦ but for a test namespace thatās not an option maybe#2018-10-3015:19taylormaybe use jstack
while it's "hung" to see what it's doing at the moment#2018-10-3015:20gfrederickskill -3
says there's an agent thread running generator stuff#2018-10-3015:21gfredericksso that code launches a future somehow#2018-10-3015:22gfredericksI think shutdown-agents
might not be synchronous in its effect#2018-10-3015:22gfredericksso the fact that it waits longer when there's a future in the background that hasn't completed isn't surprising#2018-10-3015:22gfredericksbut why there's a future in this case I don't know#2018-10-3015:23gfredericks(I haven't used clojure.spec.test.alpha
much)#2018-10-3015:26taylorat clojure.spec.test.alpha$quick_check.invokeStatic(alpha.clj:310)
at clojure.spec.test.alpha$quick_check.invoke(alpha.clj:303)
at clojure.spec.test.alpha$check_1.invokeStatic(alpha.clj:336)
at clojure.spec.test.alpha$check_1.invoke(alpha.clj:324)
at clojure.spec.test.alpha$check$fn__3088.invoke(alpha.clj:412)
at clojure.core$pmap$fn__8342$fn__8343.invoke(core.clj:6994)
š¤#2018-10-3015:27Alex Miller (Clojure team)check uses pmap#2018-10-3015:27taylorhttps://github.com/clojure/spec.alpha/blob/f23ea614b3cb658cff0044a027cacdd76831edcf/src/main/clojure/clojure/spec/test/alpha.clj#L411#2018-10-3015:27gfredericksah well there you go then#2018-10-3015:28gfredericksgotta spend 15 seconds generating a few last giant piles of garbage in case somebody wants them#2018-10-3015:30gfredericks@borkdude the most recent alpha of test.check might be a bit better about this. I can't remember, and also don't know what version you get by specifying RELEASE
#2018-10-3015:31taylorlooks like org/clojure/test.check/0.10.0-alpha3
#2018-10-3015:31gfredericksoh; nevermind then#2018-10-3015:31gfredericksfor this kind of test it's probably best to use a smaller max-size for the gen/any
#2018-10-3015:32borkdudeIāll try that, thanks.#2018-10-3015:38gfredericksI wonder if gen/any
should just have a smaller default size universally š¤#2018-10-3015:38borkdudeany snippet handy that I could re-use?#2018-10-3015:38gfredericksno, I'm not sure of the exact syntax for sizing control in spec#2018-10-3015:39borkdudek, Iāll figure something out#2018-10-3015:39gfredericksif you don't mind affecting the whole generator, I think the stest/check
call lets you pass max size#2018-10-3015:39borkdudeoh that would be nice#2018-10-3015:39gfredericksfor finer grained control you'd probably set an override specific to the any?
spec#2018-11-0209:53borkdudeAny idea how to do this? I tried by using a :gen
override for :clojure.core/any?
but that doesnāt work, because any?
is a builtin I think:
https://github.com/clojure/spec.alpha/blob/31165fec69ff86129a1ada8b3f50864922dfc88a/src/main/clojure/clojure/spec/gen/alpha.clj#L118#2018-11-0212:58gfredericksI wouldn't have expected that, but assuming you're right and you can't override builtins, another option would be having your own (def ::any any?)
that you can override#2018-11-0212:58gfredericksunfortunate if you have to do that, but should work at least#2018-10-3015:46borkdude@alexmiller if you want I can post that script in an issue if you think itās something that should be improved over time#2018-10-3015:49Alex Miller (Clojure team)Hmm?#2018-10-3015:50borkdudehttps://clojurians.slack.com/archives/C1B1BB2Q3/p1540912650053300#2018-10-3015:51Alex Miller (Clojure team)I donāt understand what weāre talking about#2018-10-3015:51Alex Miller (Clojure team)check returns a lazy seq - you need to consume it#2018-10-3015:52Alex Miller (Clojure team)As noted in the doc string#2018-10-3015:52Alex Miller (Clojure team)So wrap a doall around the call to check#2018-10-3015:52borkdudeaah, wrapping it in a doall
works#2018-10-3015:53borkdudethank you#2018-10-3015:53borkdudeissue solved (see above thread).#2018-10-3015:54borkdudedidnāt realize that (pun intended)#2018-10-3021:15andy.fingerhutI slightly envy your pun-creation skills.#2018-10-3018:00martinklepschI'm not sure if this is a bad idea or something but I'd love to always have stest/instrument
switched on during development.
Is there some way to keep functions instrumented after re-defining them in the REPL?#2018-10-3018:09taylorI see there's (defonce ^:private instrumented-vars (atom {}))
in spec.alpha, but I don't see it publicly exposed. Something that came to mind was using that to re-instrument things that were previously instrumented?#2018-10-3020:39dadairIf you are using something like Component or Integrant you can wrap their system reset functions with calls to instrument#2018-10-3020:40dadairIāve done that with our duct-based project at work#2018-10-3022:52martinklepschIāve considered that but often Iām just working in one namespace in the REPL and restarting the system seems like overkill in a way #2018-10-3022:53martinklepschI guess I should just bind stest/Instrument to a hotkey#2018-10-3018:45borkdude@martinklepsch I hook up stest/instrument with component. So when I restart the system, my new fns are instrumented.#2018-10-3023:10borkdudesomehow this also lingers after the test has executed, even with doall
:
(ns spec-keys-test
(:require
[clojure.test :as t :refer [deftest testing is]]
[clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest]))
(defn foo [n] n)
(s/fdef foo
:args (s/cat :n number?)
:ret number?)
(deftest repro-test
(testing "output of check"
(let [r (doall (stest/check `foo))
c (count r)]
(println "count" c)
(is (= c (count (keep :spec r))))
(is (= c (count (keep :sym r))))
(is (= c (count (keep :clojure.spec.test.check/ret r)))))))
(t/run-tests)
#2018-10-3023:56andy.fingerhutI think shutdown-agents
was mentioned before, but wasn't sure whether you had tried it. If your code calls future
directly (or indirectly through one of several other functions), it can cause a 60-second pause at the end before the JVM exits: http://clojuredocs.org/clojure.core/future#2018-10-3108:18borkdudeRight, thatās it. So doall
+ shutdown-agents
it is.#2018-10-3023:56andy.fingerhute.g. pmap
clojure.java.shell/sh
#2018-10-3100:45misha@domkm I think you can use your own predicate in root graph spec (s/and asyclic? (s/keys ...))
, which internally uses s/*recursion-limit*
, but that would traverse graph twice#2018-10-3100:47domkm@U051HUZLD Thanks for the suggestion. I think I wasn't clear. The graph is cyclic. I just only want to validate to N levels and stop validating after that point.#2018-10-3100:52mishawell, I got nothing then, except writing your own validator, and forfeiting all the for-free generators, etc.#2018-10-3100:53mishaor, use 1 spec to generate, and custom walking predicate to validate#2018-10-3100:54mishapredicate can somewhat reuse specs for nodes and edges, for keys, but it is still not pretty#2018-10-3109:18magnusdkHas anyone found a nice workaround for this: https://dev.clojure.org/jira/browse/CLJ-2067?
[spec] (s/def ::a ::b) throws unable to resolve error if ::b is not defined
#2018-10-3109:19magnusdkWe hack-fixed it by defining this macro:
(defmacro def*
[name spec]
`(do (when (and (qualified-keyword? ~spec)
(not (#'s/maybe-spec ~spec)))
(s/def ~spec
#(throw (ex-info "Spec is declared, but not defined!"
{:spec-name ~name
:spec-alias ~spec
:args %&}))))
(s/def ~name ~spec)))
(def* ::foo ::NaS) ;;=> :my.spec/abc
(s/valid? ::foo "something") ;;=> clojure.lang.ExceptionInfo: Spec is declared, but not defined!
(def* ::NaS string?) ;;=> :my.spec/NaS
(s/valid? ::foo "something") ;;=> true
And doing something similar with s/fdef
, but this broke down when we started using orchestra for instrumentation.#2018-10-3113:44domkmIf you can't rearrange the order of your s/def
s, perhaps you can (s/def ::b any?)
first and then redef it later?#2018-10-3115:27magnusdkedit: just some more context
Using a similar macro as the one above we specced a functionās :ret
using a spec that hadnāt yet been defined, but we knew that it would be defined after all the namespaces had been loaded. However, when we started instrumenting the function (using orchestra) the spec was still throwing our Ā«declared, but not definedĀ» exception. It seems that the spec wasnāt overwritten, but describing the specs in the REPL displayed the expected results.#2018-10-3115:34magnusdkYour suggestion gets rid of the exceptions, but the spec is still not redefed š¤ The function can now return anything because the any?
spec still stands#2018-10-3117:32bmaddyThis seems like a silly question, but is there a standard way of checking specs in tests? I suspect it uses (-> (stest/enumerate-namespace 'user) stest/check)
and looks something like this:
(deftest specs
(...
(-> (stest/enumerate-namespace 'user) stest/check)))
#2018-10-3117:53seancorfield@bmaddy Generative tests can be long-running so having them in "unit tests" that you run all the time is a bit of an anti-pattern.#2018-10-3117:57jaihindhreddyHow do you guys do unit testing? Just what percentage of tests are manually written example based ones?#2018-10-3118:13bmaddyWe mostly do classic unit tests. We have one file where we use clojure.test.check.clojure-test/defspec
, but I think that was before spec came out. We're about 97% unit tests.#2018-10-3117:58bmaddyHmm, so do people just remember to run a stest/check
function that's in a comment after the function every time they update it? That's what I've been doing so far, but I figured there was a better way.#2018-10-3118:10bbrinckI think check
can be useful when writing the function/spec, but for tests, I tend to use test.chuck and write my properties in the test#2018-10-3118:11bbrinckthen I can use chuck/times
to control the number of generative tests (low number during development, higher number on CI)#2018-10-3118:13bbrinckbut I donāt tend to write a lot of functions that have super interesting :fn
specs, I just check :args
and :ret
specs in tests#2018-10-3118:13bbrinckMy tests will instrument my functions, then for any input I can just use s/gen
to get the generator, use that with test.chuck
#2018-10-3118:22jaihindhreddyI thought test.chuck
was a typo. š#2018-10-3118:23bbrinckheh, no, itās a really useful lib for generative testing š#2018-10-3118:27seancorfieldThere's also clojure.test.check.clojure-test/defspec
for property-based tests... https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/clojure_test.cljc#L74#2018-10-3118:27seancorfieldAnd, yeah, big +1 for test.chuck
-- we love the regex string generator in it!#2018-10-3118:30seancorfield@bmaddy I think a good option is to have some scripts that run stest/check
and summarize/assert the results are good, and then run those scripts directly as part of either periodic manual or full-suite testing (rather than automated/unit test level stuff).#2018-10-3118:32seancorfieldAt work we have a few generative tests that run in our "mainline" automated/unit tests, and then we have some more extensive generative/property tests that are in (comment ...)
forms in various files that we currently run manually, from time-to-time.#2018-10-3118:33seancorfield("Rich Comment Forms" -- per Stu's Strange Loop talk)#2018-10-3119:43bmaddyThanks @seancorfield. I'd be interested to hear how others do this as well if they feel like sharing.#2018-10-3121:35borkdudeam I right that the stc
namespace for options can only be aliased like:
#?(:clj (alias 'stc 'clojure.spec.test.check))
in cljc code for clojure?
Putting it in the ns
form didnāt compile.#2018-10-3121:35borkdudeI think because it doesnāt exist as a file, but is created when loading clojure.spec.test.alpha
#2018-10-3121:36gfredericksthat sounds plausible#2018-10-3121:36gfredericksI didn't know it did that#2018-10-3121:36borkdudeitās a bit awkward. trying to think about this: https://dev.clojure.org/jira/browse/CLJS-2952#2018-10-3121:37borkdudeI think I had a nice solution, but now my ns looks like:
(ns spec-keys-test
(:require
[clojure.test :as t :refer [deftest testing is]]
[clojure.spec.alpha :as s]
[clojure.test.check]
[clojure.spec.test.alpha :as stest]
#?(:cljs [clojure.spec.test.check :as stc])
))
#?(:clj (alias 'stc 'clojure.spec.test.check))
#2018-10-3121:38borkdudemaybe changing the clojure version of the alias stc
to clojure.test.check
makes more sense than the in-ns
thing in clojure.spec.test.alpha
#2018-10-3122:02borkdudeAh I see why that isnāt the case, because clojure.test.check is lazily loaded (optional dep).#2018-10-3122:04borkdudeWould it be OK if clojure.spec.alpha created an empty ns in a file, instead of the in-ns
?
Then both requires for cljs and clj could look the same. This would be nice in .cljc.#2018-10-3122:23borkdudePosted an issue about this at JIRA#2018-11-0115:22borkdudeIs nil considered valid input for Clojure set functions? 0 arity returns empty set, but 1 arity with nil returns nil#2018-11-0115:22borkdudeIn other words should a spec for those functions account for nil input or throw#2018-11-0115:45seancorfield@borkdude Could you show some code? I'm not sure what you're asking.#2018-11-0115:50borkdude(set/union) vs (set/union nil), they differ in return type#2018-11-0115:55seancorfieldset/union
is only defined for arguments that are sets tho', right?#2018-11-0115:56seancorfield(and nil
is not a set -- so it's "garbage-in, garbage-out" here?)#2018-11-0115:58seancorfieldSo I guess the answer to your question is "No, a spec for those functions should not allow nil
as input".#2018-11-0116:16borkdudeIf we can agree for union
etc. that 1) :ret
should be set?
then we must either 2) reject nil
inputs in the :args
spec, or 3) assert that the implementation is wrong#2018-11-0116:18borkdudemaybe @alexmiller can say something on this#2018-11-0116:54Alex Miller (Clojure team)donāt know#2018-11-0116:55borkdudewould we find it useful for fdefs detect nil (as an instance of non-sets) inputs for set fns?#2018-11-0116:57borkdudeor can we say this is undefined territory and assume the simpler spec#2018-11-0116:57Alex Miller (Clojure team)I think right now I would treat both inputs and output as nilable#2018-11-0116:57Alex Miller (Clojure team)as existing code may be relying on the behavior of those things#2018-11-0116:58borkdudeif we donāt allow nil, we will find out š#2018-11-0116:59Alex Miller (Clojure team)if your spec fails on working existing code, I think your spec is wrong#2018-11-0116:59borkdudetrue, but I mean, we can actually discover if people use this in the wild and then adapt the spec accordingly#2018-11-0117:00Alex Miller (Clojure team)well, disagree#2018-11-0117:01borkdudeyesā¦?#2018-11-0117:01Alex Miller (Clojure team)I donāt know how else to say it#2018-11-0117:02Alex Miller (Clojure team)you asked for my opinion. My opinion is that you should spec the inputs and output as nilable.#2018-11-0117:03borkdudeyes, I wondered why you disagree with adapting the spec gradually when we discover that people actually use those fns like that#2018-11-0117:03borkdudebut maybe we can rightaway assume people do this#2018-11-0117:04Alex Miller (Clojure team)nils are often used interchangeably with empty collections. it seems unlikely to me that there is not some code relying on this either for input or output#2018-11-0117:04borkduderight.#2018-11-0117:08borkdudenext caseā¦
(clojure.set/union 3)
#2018-11-0117:09borkdudethe identity
functionā¦.#2018-11-0117:09borkdudeI think itās safe to assume people do not use the 1-arity as the identity function š#2018-11-0117:09Alex Miller (Clojure team)I think you can figure that one out#2018-11-0117:11borkdudethanks for the replies#2018-11-0120:39danielcomptonIs there a recommended way of running stest/check
in a deftest
? I looked around and couldn't see anything official, just lots of different ways people did it#2018-11-0120:40borkdude@danielcompton if there is Iād like to know, because Iām doing that right nw#2018-11-0120:45danielcomptonhttps://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite#2018-11-0120:48borkdude@danielcompton Iām trying to write tests for clojure, cljs and self-hosted cljs all within one .cljc file. Guess what, all three environment expect and return different keys for the clojure.test.check opts/rets.#2018-11-0120:48danielcomptonOuch#2018-11-0120:53bmaddy@borkdude, @danielcompton There was a short discussion about that yesterday. People suggested using test.chuck (not a typo) or separating your property based tests out and run them separately.#2018-11-0120:53bmaddyThis is mainly because I'm curious, but does anyone now why the args passed to :fn
in s/fdef
are conformed values? I'd like to use the actual value in there, but dealing with the conformed value is tricky.#2018-11-0120:54borkdude@bmaddy does test.chuck support .cljc?#2018-11-0120:54bmaddyI've never used it, but the readme says this in the Acknowledgements section: @nberger for adapting to cljc format for ClojureScript, and general maintenance help
#2018-11-0120:55andy.fingerhut@borkdude Fine detail nit on your earlier questions -- in the clojure.data/diff implementation there are calls at least clojure.set/union, and perhaps a couple of other clojure.set functions, with sequences as arguments (the return value of the clojure.core/keys
function IIRC), so not a set and not nil. I believe the implementation of clojure.set/union for the ways they are called from always return correct results (i.e. duplicates in the return value are only harmful for performance, not correctness of the clojure.data/diff results). Not sure if that is considered a bug or something that an 'official' spec for clojure.set/union should allow. https://dev.clojure.org/jira/browse/CLJ-1087 I am an interested observer of such detailed questions, too, not a policy maker.#2018-11-0120:57borkdude@andy.fingerhut Please include these notes at https://github.com/slipset/speculative/issues/70#2018-11-0120:59andy.fingerhutOK, I've just added a comment to that issue with a copy/paste of what I just said above.#2018-11-0214:00borkdudeThis works:
(set/map-invert (java.util.HashMap. {:foo :bar}))
This also works:
(set/map-invert [[:a 1] [:b 2]])
Even this works:
(set/map-invert ["a1" "b2"])
{\1 \a, \2 \b}
Any reducible coll of nth-able pairs will do. How far should we go with the spec for set/map-invert
?#2018-11-0214:00borkdudeThe first case is used in Datomic#2018-11-0214:01borkdudeSee: https://github.com/slipset/speculative/issues/70#2018-11-0219:59borkdude@danielcompton I added some test helpers here: https://github.com/slipset/speculative/blob/master/src/speculative/test.cljc
(`success?` and gentest
, combined into this test util, also named gentest
: https://github.com/slipset/speculative/blob/master/test/speculative/test_utils.cljc#L22)#2018-11-0221:08danielcomptonNice :+1: #2018-11-0415:09roklenarcicI'm writing a macro where I want to turn the passed in symbol/keyword into a spec. How do I do that?#2018-11-0415:09roklenarcicI'm writing a macro where I want to turn the passed in symbol/keyword into a spec. How do I do that?#2018-11-0415:11taylors/get-spec
if youāre trying to look up an existing spec by symbol/keyword#2018-11-0415:12roklenarcicyeah but that doesn't work for predicates#2018-11-0415:12roklenarciclike someone passes in string?
#2018-11-0415:12roklenarcics/get-spec
of course says that no such spec has been registered#2018-11-0415:14borkdudeDoes creating a ::string spec work? @U08EKSQMS just made a PR to speculative for a lot of those#2018-11-0415:17taylorit sounds like you want to check if the symbol resolves to a function (predicate), and then make a spec out of that function?#2018-11-0415:17roklenarcicThe thing is that it's a library so I don't know what people will be passing in.#2018-11-0415:17roklenarcicYes something like that#2018-11-0415:18roklenarciccreating spec out of a symbol has the benefit of having that symbol show up in description of the spec failure#2018-11-0415:33taylorthis doesnāt really help, just an observation re: names/descriptions in failures:
(s/explain string? 0)
val: 0 fails predicate: :clojure.spec.alpha/unknown
=> nil
(s/explain (s/every string?) [0])
In: [0] val: 0 fails predicate: string?
=> nil
#2018-11-0415:37taylor(s/explain (s/spec string?) 0)
val: 0 fails predicate: string?
=> nil
#2018-11-0415:40roklenarcicI see#2018-11-0415:42taylor(defmacro ->spec [k]
`(or (s/get-spec ~k) (s/spec ~k)))
(defn foo? [x] (= "foo" x))
=> #'?
(s/explain (->spec foo?) "foo")
Success!
=> nil
(s/explain (->spec foo?) "bar")
val: "bar" fails predicate: foo?
=> nil
(s/def ::foo #{"foo"})
=> :
(s/explain (->spec ::foo) "foo")
Success!
=> nil
(s/explain (->spec ::foo) "bar")
val: "bar" fails spec: : predicate: #{"foo"}
=> nil
#2018-11-0415:43taylormaybe something like that would work#2018-11-0415:43roklenarciccool I'll check it out#2018-11-0417:52mishawhy do you need it in the first place? to have named things in explain's out? @U66G3SGP5#2018-11-0418:30roklenarcicyes š#2018-11-0423:47didibusI'm confused, sounds like you just want to call (s/def). It takes a keyword or symbol and registers a spec for it.#2018-11-0611:38roklenarcicis there a way to spec map's key+value, but the key isn't specific#2018-11-0611:38roklenarciclike how do I say: map key is symbol or keyword, if key is symbol then value must be string but if key is keyword then value must be integer#2018-11-0611:39roklenarcicI tried s/coll-of :kind map?
but that only conforms value#2018-11-0611:51guyIs this useful to you perhaps? https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/map-of#2018-11-0611:51guy@roklenarcic#2018-11-0612:07borkdude@roklenarcic can the map also have combinations of these key/val pairs?#2018-11-0612:08borkdudein that case I would make a spec for the map-entry#2018-11-0612:08Alex Miller (Clojure team)You can use s/every with an s/or of s/tuple for entry combinations#2018-11-0612:10borkdudeSelf-hosted cljs with spec in the browser: https://twitter.com/borkdude/status/1059758785613479937#2018-11-0612:12roklenarcicthank you alex, I'll try that#2018-11-0612:35roklenarcichm well every doesn't conform#2018-11-0612:37roklenarcichm... when using coll-of, if I specify kind as map? and if I don't specify kind, conforming works differently š#2018-11-0612:39borkdude@roklenarcic specify it as a seq of tuples?#2018-11-0612:40borkdudeso something like (coll-of ::my-tuple-spec)#2018-11-0612:40roklenarcictuple spec is no better than map-of spec... each item is independent#2018-11-0612:41borkdude::my-tuple-spec can spec independent things?#2018-11-0612:41roklenarcic(s/tuple pred1 pred2)
#2018-11-0612:41roklenarcicyou can't link type of 1 and type of 2#2018-11-0612:42borkdude(s/or (s/tuple symbol? string?) (s/tuple s/keyword? int?))#2018-11-0612:42roklenarcicso s/coll-of that?#2018-11-0612:42borkdudeyes#2018-11-0612:44borkdudeand I guess :kind could optionally also be set to map?, itās just an extra check#2018-11-0612:44roklenarcic(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3})
=> {:s1 [a "A"], :s2 [:b 3]}
#2018-11-0612:44borkdudebut often a seq of map-entries can be used in places where maps are used#2018-11-0612:44roklenarcicthis is completely different to how I would expect things to go#2018-11-0612:44roklenarcicI would expect a collection of tuples#2018-11-0612:44borkdude@roklenarcic you mean the conformed value?#2018-11-0612:44roklenarcicyes#2018-11-0612:45roklenarcicthis looks completely wrong and unexpected to me#2018-11-0612:46borkdudeit isnāt wrong, itās conformed. else you get ::s/invalid#2018-11-0612:46roklenarcic(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"})
=> {:s1 [c "B"], :s2 [:b 3]}
#2018-11-0612:46roklenarcicsee... now I lost a term#2018-11-0612:46roklenarcicinput map has 3 key-values#2018-11-0612:46roklenarcicresult has two items#2018-11-0612:46borkdudeah I see#2018-11-0612:47borkdudeso maybe every will help there?#2018-11-0612:48roklenarcicnah, I'll just switch back#2018-11-0612:48roklenarcicto :kind map?#2018-11-0612:48roklenarcicand take vals#2018-11-0612:48roklenarcicproblem solved#2018-11-0612:48borkdudeif I had time I would help you figure it out, but I have to go now#2018-11-0612:48roklenarcicdon't worry, I have a solution, I was just looking for something more elegant#2018-11-0612:50borkdudeyes, (s/conform (s/every (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"})
works#2018-11-0612:51roklenarcicevery doesn't do any conforming at all FYI#2018-11-0612:51roklenarcicat least I think so#2018-11-0612:51borkdudeoh you want conforming#2018-11-0612:51borkdudesorry š#2018-11-0612:51roklenarcic(s/coll-of x :kind map?)
, conforms key+value into value#2018-11-0612:52roklenarcicso you just have to take (vals )
of value in the map then#2018-11-0612:52borkdude@roklenarcic
(s/conform (s/cat :map-entries (s/* (s/alt :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?)))) '{a "A" :b 3 c "B"})
#2018-11-0614:00Alex Miller (Clojure team)I donāt think this approach will work on latest spec - regex specs now only work on sequential collections (too confusingly dangerous to do sequential specāing on unordered collections)#2018-11-0614:02Alex Miller (Clojure team)(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"})
#2018-11-0614:02Alex Miller (Clojure team)would be better#2018-11-0614:04borkdudethat was already proposed by me, but that didnāt work, because the result would be: {:s1 [c "B"], :s2 [:b 3]}
and you lose map-entries#2018-11-0614:04borkdudeI guess you can call seq on the map first if it doesnāt work with the newest spec#2018-11-0614:07borkdudeevery also works, but that doesnāt conform and the OP wanted that#2018-11-0614:12Alex Miller (Clojure team)(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?)) :into []) '{a "A" :b 3 c "B"})
#2018-11-0614:13Alex Miller (Clojure team)=> [[:s1 [a "A"]] [:s2 [:b 3]] [:s1 [c "B"]]]
#2018-11-0614:13Alex Miller (Clojure team)user=> (s/conform (s/coll-of (s/nonconforming (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) :into []) '{a "A" :b 3 c "B"})
[[a "A"] [:b 3] [c "B"]]
#2018-11-0614:14Alex Miller (Clojure team)if you want to hide the or tag#2018-11-0614:18borkdudeaah, :into []
, ok#2018-11-0612:58roklenarcicwell that's clever š#2018-11-0613:45martinklepschI'm having some problems adding :args
specs to a function, I think it kind of boils down to this:
(spec/def ::title string?)
(spec/def ::entry-x (spec/cat :title ::title))
(spec/valid? ::entry-x ["Title"]) ; => true
(spec/valid? (spec/cat :entry ::entry-x) ["Title"]) ; => true
(spec/valid? (spec/cat :entry ::entry-x) [["Title"]]) ; => false
I would expect the additional cat
to require additional wrapping but it appears that ::entrty-x
and (spec/cat :entry ::entry-x)
are identical for some reason.#2018-11-0613:46borkdudeI guess you can nest s/cat?#2018-11-0613:48borkdude(s/def ::foo (s/cat :a int? :bcat (s/cat :b int?)))
(s/conform ::foo [1 2]) ;; {:a 1, :bcat {:b 2}}
#2018-11-0613:55Alex Miller (Clojure team)regex ops nest to describe the same structured collection#2018-11-0613:55Alex Miller (Clojure team)if you need a new ālevelā, insert s/spec#2018-11-0613:55martinklepschInteresting I would have expected that spec to match [1 [2]]
I guess#2018-11-0613:55Alex Miller (Clojure team)this is covered in the spec guide#2018-11-0613:56borkdudeit behaves the same as s/alt, s/*, etc. they consume the same seq ālevelā#2018-11-0613:58martinklepschOk, s/spec
did the trick. Also found the section in the spec guide now#2018-11-0614:19borkdude@roklenarcic please watch the thread, Alex gave some advice there#2018-11-0614:28borkdudebtw, handy link if you quickly want to check examples posted here: http://app.klipse.tech/?cljs_in=(require%20%27[clojure.spec.alpha%20:as%20s])#2018-11-0614:36borkdudeis this feature in clojure 1.10 related to upcoming changes in clojure.spec maybe? https://twitter.com/timbaldridge/status/1059815352106795008#2018-11-0618:05Alex Miller (Clojure team)no, itās a genericization of stuff from datafy#2018-11-0618:05Alex Miller (Clojure team)although datafy is potentially something we will do in spec, still tbd#2018-11-0618:12borkdude:+1:#2018-11-0614:46kirill.salykinwhat is instance based protocol polymorphism?#2018-11-0614:46mpenettype vs instance#2018-11-0614:46mpenetnumber vs 42#2018-11-0614:47mpenetin short you could have a custom impl of some protocol fn bound to a value (via metadata) instead of a type#2018-11-0614:47ghadiin clojurescript this is like specify!
#2018-11-0614:47mpenetwell the number mention is not a good example š no meta#2018-11-0614:48kirill.salykincant you dispatch now based on meta?#2018-11-0614:48kirill.salykinsorry for being annoying, but I am very curriuos about it š#2018-11-0614:48mpenet(defprotocol Foo (foo [x])) (foo (with-meta [42] {`foo (fn [x] :boo)})) -> :boo
#2018-11-0614:49mpenethttps://github.com/clojure/clojure/blob/master/changes.md#22-protocol-extension-by-metadata#2018-11-0614:49kirill.salykini will have a look, thanks!#2018-11-0614:50mpenetsuper cool feature (not really related to spec tho, as far as we know)#2018-11-0614:56borkdudewell, I thought, maybe this way predicates could be bound to keywords and spec could use that#2018-11-0614:57mpenetkeywords do not support meta I believe#2018-11-0614:57borkdudeah#2018-11-0614:58bronsathey don't, this is ~ specify
limited to IObj
s (modulo the different method resolution rules around direct impl)#2018-11-0615:49jumarsatisfies?
doesn't work with the new "instance-based polymorphism"?#2018-11-0615:49mpenetapparently not#2018-11-0615:50bronsano, I was looking at a related fix for CLJ-1814 but I think somebody should make a ticket just for this specific issue#2018-11-0615:51ghadi^^ yes, please.#2018-11-0615:53jumarOk, I can do it.#2018-11-0616:04jumarHere's the thing: https://dev.clojure.org/jira/browse/CLJ-2426
It's my first ticket so I hope it's clear.#2018-11-0616:07ghadiit is, thanks @U06BE1L6T#2018-11-0618:06Alex Miller (Clojure team)yep, thx#2018-11-0617:27eoliphantHi, I'm trying to get a feel for where to 'draw the line' on more dynamic behavior in specs. I've created an EDN based data description language for one of my projects. I have it spec'ed out pretty thoroughly, but I have a situation where I have say :data/type
that could be one of a fixed set of types (:string :long, etc) or a type that's defined elsewhere in the larger data structure I'm validating. my code is already doing some additional validation based on core.logic etc once the definitions are loaded. So just wondering if it's better to defer this type of check to that point#2018-11-0618:20mathpunkIs there an equivalent of ns-unmap
for a symbol you used spec/def
on? (I suspect this question will reveal an error in my mental model of how symbols and specs relate)#2018-11-0618:21didibusJust s/def it to nil#2018-11-0618:21mathpunkš thanks!#2018-11-0618:21didibusIf I remember correctly#2018-11-0619:15Alex Miller (Clojure team)yes#2018-11-0713:38danielstocktonNot sure how to achieve something. If I want to check a map for :key
, but define the spec with ::some-type-of-key
(different spec name) in (s/keys :req-un [::key])
?#2018-11-0713:39danielstocktonMore concretely, I have a :type
key on multiple things, and the spec is slightly different for each.#2018-11-0713:39danielstocktonMore concretely, I have a :type
key on multiple things, and the spec is slightly different for each.#2018-11-0713:46jumarYou should probably use namespace-qualified keys in that case (that is :req
)
You can also use a level of indirection and define specs like this:
(s/def :a/type int?)
(s/def ::a-type (s/keys :req-un [:a/type]))
(s/def :b/type string?)
(s/def ::b-type (s/keys :req-un [:b/type]))
(def int {:type 10})
(def b {:type "number"})
(s/valid? ::a-type a)
;; => true
(s/valid? ::a-type b)
;; => false
(s/valid? ::b-type b)
;; => true
(s/valid? ::b-type a)
;; => false
#2018-11-0713:51danielstocktonNamespaced keys works, thanks#2018-11-0719:35shaun-mahood@alexmiller: I'm doing my first really rigorous spec of something with a lot of data, and this morning I'm on my 4th or 5th time understanding why a certain design decision makes more sense than I realized with either smaller data or less complete specs. It's pretty great to be using something with so much depth of thought and experience behind it!#2018-11-0809:51danielstocktonCan anyone suggest how i might spec that a vector should contain maps in a particular sort order? (based on a key)#2018-11-0809:53danielstocktonIt might contain any number of maps, but they should be in a particular order based on a :type
key.#2018-11-0810:37borkdude@danielcompton use this predicate?
(defn sorted-by? [coll k]
(and (reduce (fn [prev m]
(if (pos? (compare (get prev k) (get m k)))
(reduced false)
m)) coll)
true))
#2018-11-0811:08danielstocktonWrong @ š#2018-11-0811:09borkdudeah sorry š#2018-11-0810:40borkdudeIām not sure if predicates could give more information to spec to present a more helpful error than ::s/invalid#2018-11-0810:41danielstocktonYeah, i'll need a different comparator but the basic idea would work. Thanks!#2018-11-0810:42borkdude(sorted-by? [{:type 2} {:type 3} {:type 3}] :type) ;; true
(sorted-by? [{:type 2} {:type 3} {:type 2}] :type) ;; false
#2018-11-0810:43danielstocktonType is a string and the order is quite specific. But I can easily work that part out, thanks.#2018-11-0810:44danielstocktonI'll use a higher order function like
(defn correctly-sorted? [sort-order k]
(fn [coll] ...))
#2018-11-0810:59danielstockton(defn correctly-sorted? [sort-order k]
(fn [coll]
(reduce
(fn [prev m]
(if (> (.indexOf sort-order (get prev k))
(.indexOf sort-order (get m k)))
(reduced false)
m))
coll)))
Does it matter if the returned value is simply truthy rather than true?#2018-11-0811:00borkdudedepends what you want. itās a convention to return booleans from ?
functions.#2018-11-0811:01borkdudenote that youāre using >
which doesnāt work for the case when you have equal elements with regards to k#2018-11-0811:01borkdudestylistically: correctly- is a bit redundant, itās either sorted, or not#2018-11-0811:03danielstocktonThanks, you're right on all fronts..#2018-11-0811:04borkdudehaha, didnāt mean to be pedantic, but couldnāt help š#2018-11-0811:04danielstocktonProblem is sorted?
is a core function#2018-11-0811:04borkdudesorted-by?
isnāt#2018-11-0811:05borkdudebut this is just naming, choose whatever you want š#2018-11-0811:05danielstocktonsorted-accordingly?
š
sorted-by?
is just fine#2018-11-0811:18Matt ButlerHi, I'm trying to spec a list of different maps, based on the docs I think I'm supposed to use a multi-spec, which i've done in the past without trouble. However I want to dispatch on more than one key/value, which I haven't seen an example of. Is this supported?
I'm doing this, and valid?
works correctly (returns true).
But explain
returns something unexpected so wanted to check im not doing something wrong
(defmulti foo
(fn [x]
[(:bar x) (:baz x)]))
(defmethod foo [1 2] [_]
#{{:bar 1 :baz 2 :biff 3}})
(defmethod foo [:a :b] [_]
#{{:bar :a :baz :b :biff :c}})
(s/explain (s/multi-spec foo :foo/type) {:bar :a :baz :b :biff :c}) => val: {:bar :a, :baz :b, :biff :c} fails at: [[:a :b]] predicate: foo
(s/valid? (s/multi-spec foo :foo/type) {:bar :a :baz :b :biff :c}) => true
I suspect its because I've just put something random :foo/type
in the retag arg.#2018-11-0812:36Matt ButlerSo it appears that multispec doesn't conform the set to a spec for you, which was causing the inconsistency.
Returning (s/spec #{{:bar :a :baz :b :biff :c}})
from the methods, now keeps both explain and valid happy.#2018-11-0811:22Matt ButlerOn a slightly unrelated note, is it considered back practice to transform data before checking it against a spec?
I have the same data but in 2 different formats and i'd prefer to not to spec each one individually.
I think I can write some fairly simple code to transform 1 into the other and then check it against a single spec. [a b c] => {:foo a :bar b :baz c}
#2018-11-0813:16Alex Miller (Clojure team)Doesnāt seem inherently bad to me#2018-11-0917:22Lyn HeadleyIs it the case that cljs.spec.gen.alpha does not support recursive-gen? I am seeing: ^--- Use of undeclared Var cljs.spec.gen.alpha/recursive-gen#2018-11-0920:50hiredmanrecursive-gen is a thing that test.check has, but I don't think spec exposes a wrapped version of it#2018-11-0920:51hiredman(in cljs or clj)#2018-11-1006:39tianshuI have a question, is it s/conformer
not work on s/map-of
? is it designed?#2018-11-1016:57schmeeIām very confused by the behavior of instrument and fspec#2018-11-1016:57schmeelong story short: how do I turn off generative checking of fspec?#2018-11-1016:58schmee*fspec-iterations*
doesnāt cut the mustard since it will try to create generators even though you set it to 0#2018-11-1016:59schmeemy workaround now is to copy-paste the spec in question and replace the fspecs with any?
and pass that to the :spec
arg to instrument
, but that is not a viable solution#2018-11-1017:03schmeeis it an acceptable solution to never call validate-fn
at all if *fspec-iterations*
is zero or nil? https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1756#2018-11-1023:52didibusAre you saying a call to instrument forces the gen to be generated?#2018-11-1023:55schmeethat seems to be the case#2018-11-1023:57didibusHum, let me check if I have that behavior#2018-11-1100:04didibusI don't get that behavior#2018-11-1100:05didibus#2018-11-1100:06didibusThis works fine for example. Even though if you run: (sgen/generate (s/gen `add))
you get ExceptionInfo Unable to construct gen at: [:a] for: (= (type %) java.lang.Long) clojure.core/ex-info (core.clj:4739)
#2018-11-1100:13Alex Miller (Clojure team)when an instrumented fdef with an fspec is checked, the :args spec of the fspec is generated and then the fspec function is invoked to verify it conforms to its spec#2018-11-1100:14Alex Miller (Clojure team)many people find this surprising and I expect we will ultimately change it#2018-11-1100:19schmeewhat do you think of my suggestion above as a temporary measure (to not call validate-fn
when *fspec-iterations*
is 0 or nil)?#2018-11-1100:20schmeealso, I appreciate that you are considering options for this š#2018-11-1100:23didibusOh I see, you mean if you spec a higher order function on an fdef. Interesting, I don't think I ever specced one before.#2018-11-1219:23asilverman#clojure-spec - hi all, could you help me understand why
(inst? (t/date-time 1986 10 14))
=> false
#2018-11-1219:24schmeeI guess t/
is clj-time/
?#2018-11-1219:35asilvermanYes, t -> https://github.com/clj-time/clj-time#2018-11-1219:26ghadi@ariel.silverman (class (t/date-time 1986 10 14))
would be some third-party Joda time thing, which isn't in scope of the inst?
predicate#2018-11-1219:38asilverman@ferossgp @schmee - fair enough. How do I sample a joda time thing then?#2018-11-1219:54asilverman#clojure-spec - I was hoping to ask for some help generating samples for clj-time/date-time since I am currently interested in testing my spec and using the spec as a generator for unit tests. Can anyone spare a couple minutes to help or point me out to some resource that can assist me in this goal?#2018-11-1220:02schmeesomething like this should get you going:
(ns foo
(:require [clojure.spec :as s]
[clojure.spec.gen :as gen]
[clojure.test.check.generators :as tg]))
[clj-time.coerce :as c]
(def epoch-2000-01-01 946684800)
(def epoch-2100-01-01 4102444800)
(def date-range
(tg/large-integer* {:min epoch-2000-01-01 :max epoch-2100-01-01}))
(s/def ::timestamp
(s/spec nat-int?
:gen #(tg/fmap #(c/from-long %) (tg/large-integer* {:min epoch-2000-01-01 :max epoch-2100-01-01})))
#2018-11-1220:34asilverman@schmee Thank you!
Actually I was looking into leveraging the specs specified here as part of my spec definition: https://github.com/clj-time/clj-time/blob/master/src/clj_time/spec.clj#2018-11-1220:35asilvermanI was able to require the namespace clj-time.spec and now I am trying to figure out how to do the dynamic binding of *period*
#2018-11-1219:59schmee@ariel.silverman I have a one around for that, secā¦#2018-11-1320:29DormoDo people tend to keep spec instrumentation on in testing environments? I don't mean automated testing, but environments for manual/internal testing before being deployed to production#2018-11-1320:30Dormoi guess "staging environments" would be a good term#2018-11-1321:56souenzzoI use just on automated test (lein test)#2018-11-1321:56souenzzoThe "develop" jar should behave the same as "prod" jar.#2018-11-1511:01stijnHow can I create a generator that either generates something of a type or nil? (e.g. (spec/gen (spec/inst-in start end))
or nil
)#2018-11-1511:04borkdude@stijn the either part you can do with s/or
or s/alt
#2018-11-1511:04borkdudeor use nilable
#2018-11-1511:05stijnoh, of course š#2018-11-1511:05stijnthanks!#2018-11-1514:43borkdudewhatās the minimum clojure version in which spec can be used, 1.9.0?#2018-11-1514:51mpenet1.8 if you use clojure-future-spec#2018-11-1514:52mpenetprolly lower too, but we ran on 1.8 + clojure-future-spec smoothly for a while on some services#2018-11-1515:06borkdudeah yes, now that you remind me, we did too#2018-11-1516:19Wilson VelezHi all, Where can I start to learn spec?#2018-11-1516:19samedhihttps://clojure.org/guides/spec#2018-11-1516:19samedhihttps://clojure.github.io/spec.alpha/clojure.spec.gen.alpha-api.html#clojure.spec.gen.alpha/fmap#2018-11-1516:47eoliphanthi one of my devs is playing with spec, and ran into this issue when composing a couple
(s/def ::even-or-odd (s/or
:even even?
:odd odd?))
(s/def ::small-even-odd (s/and ::even-or-odd #(< % 100)))
(s/valid? ::small-even-odd 1)
; causes
=> ClassCastException clojure.lang.MapEntry cannot be cast to java.lang.Number (Numbers.java:226)
It seems like the anon function is getting a KV pair instead of the value thatās being checked.#2018-11-1516:51borkdude@eoliphant whatās the result of (s/conform ::even-or-odd 1)
?#2018-11-1516:52eoliphantthatās what I was thinking, but i didnāt realize that that would ācarry forwardā into the next spec#2018-11-1516:52borkdudemaybe try nonconforming
around the spec#2018-11-1516:52borkdudehttps://clojuredocs.org/clojure.spec.alpha/nonconforming#2018-11-1516:57borkdudehttps://clojurians-log.clojureverse.org/clojure-spec/2017-08-12/1502573905.650871#2018-11-1516:57borkdudewonder what the current state of that is#2018-11-1516:59eoliphantthanks#2018-11-1517:00eoliphantyeah to your point itās a tuple [:odd 1]
#2018-11-1518:12jaihindhreddyWell, you could flip the order. (s/def ::small-even-odd (s/and #(< % 100) ::even-or-odd))
#2018-11-1518:37Alex Miller (Clojure team)wonāt gen then#2018-11-1518:38Alex Miller (Clojure team)actually, I guess it wonāt gen regardless#2018-11-1518:38Alex Miller (Clojure team)you need an int?
at the beginning or something#2018-11-1519:31devthif i have two functions in the same namespace that i want to spec with req-un
with the same named arg e.g. match
but different shapes, must I put those specs in other namespaces in order to specify the different shapes?#2018-11-1519:34devthsince the spec must be named match
and I can't specify two specs with the same name, e.g.:
(s/def ::match string?)
...
(s/def ::match number?)
#2018-11-1519:35noisesmithif they are req-un why not just give them different namespaces?#2018-11-1519:36devthinstead of ::
you mean?#2018-11-1519:36devthlike :fun1/match
:fun2/match
?#2018-11-1519:36noisesmithright - ::
is a shorthand for the current namespace#2018-11-1519:37devthok. guess i wasn't sure if it's appropriate to abandon the ::
convention since all my other specs in that namespace are using it š¤#2018-11-1519:37devthalmost seems like that makes them private/internal specs#2018-11-1519:38Alex Miller (Clojure team)itās ok and fine to do that#2018-11-1519:38devthgot it. thanks :+1:#2018-11-1519:38Alex Miller (Clojure team)fyi, changes coming in this area for next version of spec that should make it less weird#2018-11-1519:38noisesmithI've never seen any assumption or convention that specs that don't use the current namespace are private#2018-11-1519:42borkdudeanother way to mark them private is to move them to an foo/impl.clj namespace š#2018-11-1519:42devthis that a java thing? š#2018-11-1519:43borkdudepretty general convention#2018-11-1519:43noisesmithit's a convention in many clojure projects, if the ns has impl as part of its path you know you aren't expected to rely on it as a consumer#2018-11-1519:43Alex Miller (Clojure team)core.async is a good example#2018-11-1519:43devthah, cool#2018-11-1519:44seancorfield@alexmiller Are you aware that clojure.spec-alpha2.test/check
is "broken" at the moment?#2018-11-1519:45Alex Miller (Clojure team)yes#2018-11-1519:45Alex Miller (Clojure team)and I will fix it as soon as I finish getting the next Clojure released :)#2018-11-1519:45Alex Miller (Clojure team)beta that is#2018-11-1519:45seancorfield'kthx#2018-11-1519:46seancorfieldI tried to fix it locally but I haven't quiet gotten my head around the path through the code to figure out exactly what needs changing... yet...#2018-11-1519:47Alex Miller (Clojure team)I did not intentionally change anything around this, so itās something I (unintentionally) broke, but havenāt looked yet#2018-11-1519:50seancorfieldI think it's because stest/check
still, ultimately, calls s/spec
(a macro) and this line in check-1
no longer behaves the same as before: specd (s/spec spec)]
#2018-11-1519:51seancorfield(I just haven't quite figured out the correct invocation to replace that with)#2018-11-1520:05Alex Miller (Clojure team)prob a bug in the s/spec impl#2018-11-1521:20seancorfieldFYI, I tried switching our code over to clojure.spec-alpha2
because, why not? I ran into two speed bumps (so far)...#2018-11-1521:21seancorfield1. we reuse :clojure.core.specs.alpha/binding
in one of our specs (I got around that by using eval
to s/def
that spec into the new spec's registry).#2018-11-1521:22seancorfield2. we use A spec that uses test.chuck
fails with Caused by: java.lang.IllegalArgumentException: No method in multimethod 'create-spec' for dispatch value: ws.domain.member/bounded-string
#2018-11-1521:31seancorfieldNot yet sure what causes that.#2018-11-1521:32seancorfield(s/def ::email (s/with-gen (s/and (bounded-string 5 128)
wstr/valid-email?)
(wgen/fn-string-from-regex wstr/email-regex)))
#2018-11-1521:32seancorfield(defn bounded-string
"Given min/max lengths, return a spec for a string within
those lengths (inclusive).)"
[from to]
(s/and string? #(<= from (count %) to)))
#2018-11-1521:36seancorfieldI have a feeling this is related to the stest/check
failure since it seems to be due to an assumption that a form like (a-fn ...)
can be dispatched on the first argument, rather than evaluating it and dispatching on the result?#2018-11-1521:38Alex Miller (Clojure team)yeah, might be missing a case there#2018-11-1521:55seancorfieldIf it's any help, I tried this (defn bounded-string
"Given min/max lengths, return a spec for a string within
those lengths (inclusive).)"
[from to]
(s/and string? #(<= from (count %) to)))
(def ^:private email-bounded-string? (bounded-string 5 128))
and it failed with Exception in thread "main" Syntax error compiling at (ws/domain/member.clj:87:1).
Unable to resolve symbol: from in this context
so at this point I'll give up š#2018-11-1613:51stijnis there a way to do something like this (s/def ::string-or-number (apply s/or [:string string? :number number?]))
or is it macros to the rescue for e.g. creating a combined spec from a list of regexes?#2018-11-1613:55mpenetyou can do this with eval#2018-11-1613:56mpenet(eval `(s/or
#2018-11-1613:56mpenetetc#2018-11-1613:57stijn@mpenet thanks!#2018-11-1613:59borkdude@stijn apparently (s/def ::string-or-number (s/or #
also works#2018-11-1614:00borkdude(ab)using reader conditionals#2018-11-1614:08Alex Miller (Clojure team)thatās gross#2018-11-1614:08Alex Miller (Clojure team)this stuff is better in spec-alpha2#2018-11-1614:08Alex Miller (Clojure team)but donāt have time to talk about it#2018-11-1614:09mpenetyep hairy#2018-11-1614:09mpeneteager to see the new stuff#2018-11-1614:25mhuebertWhen using fdef
, are the arguments validated by :args
supposed to be affected by the destructuring forms present in the function argslist?#2018-11-2815:16favilaso spec ::foo with the widest spec#2018-11-2815:17favilaand contextually in certain s/keys specs, say it must conform to ::foo-with-bar or ::foo-with-baz also#2018-11-2815:17favilaI was dealing with data whose shape I could not control, and the alternative was renaming keys everywhere#2018-11-2815:17favila#2018-11-2815:18favilaso your example would look like (keys+ :req [::foo] :conf {::foo ::foo-with-bar})
#2018-11-2815:19favilathe value in ::foo must validate against both ::foo and ::foo-with-bar#2018-11-2815:19favilaso ::foo is (s/keys :opt [::bar ::baz])
#2018-11-2815:19favilabut you can see from the gnarliness of this code that spec really doesn't even want this to be allowed#2018-11-2815:20favilaI don't even necessarily recommend that you use this#2018-11-2815:36eoliphanthi iām working on an fspec for a macro. The macro itself, takes an argument that will become a spec basically something like
(s/def spec-key spec-pred)` in the macro body. Trying to come up with a valid spec for :spec-pred
since it could be a function or say (s/or ā¦ etc. etc.)
. So far I have this (s/or :fn fn? :spec s/spec?)
#2018-11-2815:42favilayou might need ifn? too (to catch stuff like #{1 2 3}
#2018-11-2815:49Alex Miller (Clojure team)You might want to look at CLJ-2112#2018-11-2815:49Alex Miller (Clojure team)Specifically the patch in there#2018-11-2815:56urzds@favila Thanks!#2018-11-2816:01eoliphantthanks guys#2018-11-2818:25eoliphanthaving a peculiar issue . Iām working on a spec for a variadic function, the ārestā should be members of a set. In my case the set members are maps, and the test always fails. replacing them with say keywords works fine.
(defmacro test-mac
[one & many]
`[~one
#2018-11-2819:52taylorthe problem is spec can't see what the value of test-a
is during expansion: https://clojure.org/guides/spec#_macros#2018-11-2819:53taylor(test-mac "a" {:a :a})
works, for example#2018-11-2915:12eoliphantah.. crap.. right.. any ideas on how to perhaps work around this?#2018-11-2915:13taylorcould maybe move some functionality out of the macro into a function (and spec that), or reconsider whether you need a macro at all#2018-11-2916:12eoliphantyeah itās a bit of a chicken/egg problem. Iām working on a little DSL to create data to define a domain, so want to be able to assert stuff about say :user/name
including itās spec. so I need to pass in the spec, as well as other meta data so needed to use macros#2018-11-2916:12eoliphantbut going to play around with it#2018-11-3013:59benzapSo Rich said they're releasing a new clojure.spec, is it going to be radically different, or backwards compatible?#2018-11-3014:00mpenethe mentioned being careful about backward compat in the talk. Then again it's an alpha, so breakage might very well happen#2018-11-3014:34Alex Miller (Clojure team)The plan is that it will be as backwards-compatible as possible#2018-11-3014:34Alex Miller (Clojure team)Things like s/keys may continue to exist but be deprecated for example#2018-11-3014:35gklijsFrom what I got, instead of having to split into required and optional argument sin the spec. it's moved to something different, and you can specify on the function witch fields you need/expect#2018-11-3014:36gklijsIt it already in spec alpha-2, or do we need to wait a bit?#2018-11-3014:45Alex Miller (Clojure team)No, itās not available yet#2018-11-3014:47Alex Miller (Clojure team)We are doing some reorg of the impl in spec-alpha2 at the moment#2018-11-3014:47Alex Miller (Clojure team)And then we will be diving into the new stuff#2018-11-3014:49benzapjust finishing up the talk, so spec stuff will be baked into defn
, or is that pseudo-code?#2018-11-3014:49benzapor rather spec/defn#2018-11-3014:52benzapman, i'm really excited for these changes#2018-11-3014:53benzapI was hesitant when using spec in my work's project, but I feel like it would really benefit from these changes, especially when defining specs between different data layers#2018-11-3015:02orestisJust a few days ago I suggested to someone to just use optional keys for his specs and validate presence of keys in some other way. (GraphQL context). Glad to hear thereās going to be a better way. #2018-11-3016:02Alex Miller (Clojure team)@benzap itās possible it will get integrated more into defn - currently TBD but Rich is thinking about it#2018-11-3016:03benzapInteresting! I can definitely see the tradeoffs on both sides#2018-11-3016:04Alex Miller (Clojure team)that of course wouldnāt happen till Clojure 1.11, if at all#2018-11-3016:05benzapThat's good to know, i'm thinking i'll use the current clojure.spec until then#2018-12-0206:23fossifoonice talk/announcement. really looking forward to it, especially the "Better programmatic manipulation of specs" part š#2018-12-0220:57djtangoDoes anyone know why this is crazy slow / never terminates?
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])
(defn my-reverse [c]
(reverse c))
(s/def ::reverse-args (s/cat :c (s/coll-of any?)))
(s/def ::reverse-ret (s/coll-of any?))
(s/exercise ::reverse-args)
(s/exercise ::reverse-ret)
(s/def ::reverse-fn
(fn [{:keys [args ret]}]
(let [input-c (:c args)]
(= input-c
(my-reverse ret)))))
(s/fdef my-reverse
:args ::reverse-args
:ret ::reverse-ret
:fn ::reverse-fn)
(stest/check `my-reverse)
#2018-12-0220:57djtango^ (you should be able to copy paste that straight into your REPL)#2018-12-0220:58djtangooddly - if I swap out (s/coll-of any?)
for coll?
I see the same behaviour but if I use (s/coll-of integer?)
it terminates nearly instantly (and successfully)#2018-12-0220:59lilactownit looks like there are many cases to generate š#2018-12-0220:59djtangodid it work for you?#2018-12-0220:59jaihindhreddyany?
generates pretty hairy stuff#2018-12-0220:59lilactownI didnāt try it, but I can tell that generating cases for (s/coll-of any?)
is going to cause test.check to really go crazy#2018-12-0221:00djtangoah I see#2018-12-0221:00jaihindhreddyAnd is generally not a good fit for generative testing#2018-12-0221:00lilactownyou can specify an upper limit to generation#2018-12-0221:00djtangohave I unwittingly stumbled onto a bad example of how to use spec / gen testing?#2018-12-0221:01jaihindhreddyYou might wanna customize the generator so the tests will be quick without changing the semantics of validation that your spec gives#2018-12-0221:01djtangoreverse felt like a nice simple function for illustrating fn specs#2018-12-0221:02jaihindhreddyIt is, and I think you wanna keep any?
in your spec but customize the generator to improve the testing#2018-12-0221:02lilactownclojure.spec/coll-of takes a :gen-max argument#2018-12-0221:03lilactownyou could limit it to like 5 or 100 or something that at least finishes š#2018-12-0221:07jaihindhreddyThat is an option too.#2018-12-0221:08jaihindhreddyThis is what I was talking about: (s/def ::reverse-ret (s/with-gen (s/coll-of any?) #(s/gen (s/coll-of int?))))
#2018-12-0221:08jaihindhreddy@djtango ^#2018-12-0221:09djtangothanks @jaihindh.reddy#2018-12-0303:19kvltDoes anyone know of a tool where I can give it a datastructure, and it will generate a spec for me based upon that datastructure?#2018-12-0303:20taylorhttps://github.com/stathissideris/spec-provider#2018-12-0303:28kvltThats wonderful, thank you!#2018-12-0318:20aviI feel like Iām a little rusty with spec, and getting tripped up by somethingā¦
Can anyone see what Iām doing wrong here?
(s/fdef probably-diagram-yaml?
:args (s/cat :v (s/or :is-diagram :ss/diagram-yaml-str
:is-not-diagram string?))
:ret boolean?)
When I test this function with stest/check
Iām seeing values generated via string?
being validated against :ss/diagram-yaml-str
ā is that whatās supposed to happen? If soā¦ why? (This is probably just PEBKAC)#2018-12-0318:24seancorfieldWhat's the generator for :ss/diagram-yaml-str
look like?#2018-12-0318:25seancorfield(I would expect that to also have string?
in it based on its name)#2018-12-0318:29aviItās a little crazy#2018-12-0318:30aviThe thing is, whenever I exercise
that spec it succeeds just fine. Andā¦ hmm, well, I hope that exercise
does validate the generated values against the spec š
#2018-12-0318:33aviGenerator:
#(gen/fmap
(fn [diagram]
(str (sometimes (str seyaml/default-front-matter doc-separator))
(seyaml/stringify diagram)))
(s/gen ::st/diagram))
and sometimes
is:
(defmacro sometimes [body]
`(when (< (rand) 0.5)
~body))
#2018-12-0318:38taylorFYI youāll probably want to replace that sometimes
with one of test.checkās generators, otherwise itās an unbound/uncontrolled source of randomness to test.check#2018-12-0318:38taylorhttps://github.com/clojure/test.check/blob/master/doc/generator-examples.md#an-integer-90-of-the-time-nil-10#2018-12-0318:39aviGreat tip ā thanks so much!#2018-12-0318:39taylormaybe something like (gen/frequency [[1 (gen/return "thing")] [1 (gen/return nil)]])
#2018-12-0318:40aviah thatād work? equal frequencies means equally likely?#2018-12-0318:40avi(I didnāt look at the docstring for frequency
yet)#2018-12-0318:41taylorthatās my untested assumption š and there might be an even more appropriate function in test.check Iām forgetting/donāt know about#2018-12-0318:41avibased on the link you provided, looks like I should use (gen/return nil)
rather than just nil
#2018-12-0318:41tayloroh yeah, sorryā¦ my example had concrete values instead of generators, fixing#2018-12-0318:42aviš#2018-12-0318:43taylorI suppose you could use gen/elements
too, if your values are static/not-generated. I suppose elements
given a two-element coll would choose between them equally?#2018-12-0318:44aviI dunno what the distribution is#2018-12-0318:44aviIIRC some test.check generators donāt use even distributions#2018-12-0318:48aviI wonder if this is on the right track:
#(gen/fmap
(fn [diagram]
(str
(gen/frequency
[[1 (gen/return nil)]
[1 (gen/return (str seyaml/default-front-matter doc-separator))]])
(seyaml/stringify diagram)))
(s/gen ::st/diagram))))
#2018-12-0318:59taylorlooks pretty reasonable to me. I still wonder if elements
would be better suited in this case b/c it'd definitely be more concise#2018-12-0318:59aviprobably#2018-12-0318:59avithanks for helping me improve my code!#2018-12-0318:35aviAh yes, peeked at the source of exercise
and it calls conform
on all generated values. (Itās mentioned in the docstring too.)#2018-12-0318:57aviI think this is starting to dawn on meā¦ I think maybe (Iād like to think because Iāve been rusty) Iāve just been not quite grokking s/or
ā¦ when a value is conformed against an s/or
spec it will quite naturally be conformed against each scenario in order. And after a value is generated via a generator, I guess thereās no mechanism at that point to conform it straight off against its original scenario specā¦ in other words, s/or
isnāt like pattern matching, which I guess was for some reason how I was trying to use it.#2018-12-0319:03taylorI think it's pretty conceptually similar to pattern matching, but I'm not sure I follow this issue:
>And after a value is generated via a generator, I guess thereās no mechanism at that point to conform it straight off against its original scenario spec#2018-12-0319:06taylorBTW one way I think s/or
differs from pattern matching like in ML languages is that s/or
clauses don't have to be exhaustive by default. Don't have to be non-overlapping either#2018-12-0319:35avier, yeah, sorry. I think it was just my rusty thinking.
ā¢ An or
spec is a spec, and (I suppose) most functions that work with specs will treat/use an or
spec as an opaque spec, like any other spec.
ā¢ Accordingly, the workflows of conform
and stest/check
have no affordances for taking values generated from the or
spec and influencing how those values are conformed
against the or
spec and its internal scenarios
ā¢ Accordingly, values generated by any given scenario of an or
spec might be conformed against any other given scenario of an or
spec, because thatās how an or
spec works ā values are conformed against its scenarios one by one in order
ā¢ The only reason I got caught up on this is because one of my specs would sometimes throw an exception when passed an invalid value#2018-12-0319:36aviI hope that might begin to approach making some kind of sense#2018-12-0319:37taylorYeah Iāve been bitten using a predicate in an or that wasnāt nil safe. Order matters in some cases#2018-12-0319:38aviyes, exactly#2018-12-0319:38avimy spec calls a function that throws if passed an empty string#2018-12-0319:38avirelated to a regex search#2018-12-0318:57aviMaybe#2018-12-0318:57aviI feel like I need a couch to lie down on and someone with a pipe to nod and doodle on a notepad#2018-12-0318:57avisorry for the noise!#2018-12-0320:31prozzhmm, have a strange question: how to actually spec that something is a sentence? or write a generator for a good sentence? (let's say lorem ipsum language with limited number of words is enough)#2018-12-0320:33prozzjust realized im specing stuff as string? but would like my generators to be clever and generate usernames as random strings but description as some sort of lorem ipsum with spaces. if this question is too beginner like for this channel, let me know pls.#2018-12-0320:44andy.fingerhutIf you mean actual grammatically correct English, or some other natural language, then you need some kind of NLP library or a product of machine learning for that. If by "sentence" you mean something else, please clarify.#2018-12-0320:45noisesmithand spec explicitly isn't a string parsing library - anything for differentiating one string from another will be wrapped up and just be another function as far as spec is concerned#2018-12-0320:46andy.fingerhutSorry, you did mention "some sort of lorem ipsum with spaces", but still might need some clarification of exactly what kind of strings you want to include vs. exclude. A regular expression could be written over strings that matches most lorem ipsum like text, but rejects things that contain weird special characters.#2018-12-0320:47andy.fingerhutIf you did that, and then wanted to generate random examples for testing, you would likely only be able to do so by writing a custom string generator, because otherwise the default test.check string generator would, with very high probability, generate strings that do not satisfy the spec.#2018-12-0320:57gfrederickstest.chuck (different lib) has a regex -> generator thingamabob#2018-12-0320:58andy.fingerhutcool. I probably saw that mentioned before and forgot about it. Such a thingamabob sounds pretty non-trivial, unless you found a library that transforms regexes into state machines for you.#2018-12-0320:59gfredericksit was nontrivial#2018-12-0320:59gfredericksthe most nontrivial parts were punted on (esp. lookahead, lookbehind)#2018-12-0321:00andy.fingerhutMakes sense. Common regex libraries include so many features like that, that complicate analyzing them (and IMO, understanding what they do)#2018-12-0321:01gfredericksone big hard part was parsing the regexes correctly, and the other one was handling characters correctly#2018-12-0321:06andy.fingerhutI saw a quote similar to the following (paraphrasing from memory) in the early 1990s in someone's signature on a public Internet forum and loved it: "Ain't nothin's easy when you're doing it fer real! -- a gunnery sergeant in the U.S. Army"#2018-12-0321:08gfrederickswell some things are, but they're not very interesting to talk about#2018-12-0321:33prozzthanks for explanation, you all above confirmed what i thought. need to dive deeper into generators, check possibilities and decide what i really wanna do š basically i want a fixtures generator for particular domain. hence need for sentences.#2018-12-0323:19prozzis there an existing generator for normal distribution from given range?#2018-12-0400:14Alex Miller (Clojure team)No#2018-12-0400:14Alex Miller (Clojure team)Well, not in spec#2018-12-0400:15Alex Miller (Clojure team)Maybe there is in test.check#2018-12-0400:15gfredericksNope#2018-12-0400:16gfredericksDon't have any continuous stat distributions#2018-12-0400:16gfredericksYou could make them using fmap from something we do have#2018-12-0400:16prozzyep, there is frequencies#2018-12-0400:17gfredericksUnclear what kind of shrinking and growth you'd want in many cases#2018-12-0402:49andy.fingerhutI think it is fairly straightforward to take a uniform random number generator for a float in the range [0, 1] and turn it into a generator for any distribution that has a function whose CDF you know how to evaluate. This page has what it claims is a very accurate approximation of the normal distribution CDF: https://web.archive.org/web/20151030215612/http://home.online.no/~pjacklam/notes/invnorm/#Pseudo_code_for_rational_approximation#2018-12-0402:51andy.fingerhutI haven't checked out their implementation but kixi stats lib claims to implement this and many other distributions: https://github.com/MastodonC/kixi.stats#2018-12-0416:52jaihindhreddyWill the decomplecting of schema/structure from selections/context be there in 1.11, or will it get in earlier?#2018-12-0416:55seancorfield@jaihindh.reddy The whole point of clojure.spec
being developed and delivered as a separate library is that it is no longer tied to specific Clojure versions.#2018-12-0416:58jaihindhreddyWhile that is true, spec
does come in the clojure jar if I'm not mistaken, hence my doubt.#2018-12-0417:10Alex Miller (Clojure team)no, it does not#2018-12-0417:10Alex Miller (Clojure team)itās a dependency, which you can independently update#2018-12-0417:10Alex Miller (Clojure team)weāve updated spec two or three times since 1.9 for example#2018-12-0417:11Alex Miller (Clojure team)I expect that the new work will be available before 1.11#2018-12-0417:11Alex Miller (Clojure team)but itās likely to be in a different namespace so will be a little trickier to use - weāve spent some time thinking through that, and probably will require some more#2018-12-0421:23urzdsIs there some valid-or-explain
function that I can easily use in my function's {:pre [(valid-or-explain ::spec %)]}
context?#2018-12-0421:23urzdsHaving s/valid?
in there as suggested in the guide is nice, but if I cannot see what it is complaining about, it does not help much... š#2018-12-0421:37seancorfields/assert
#2018-12-0421:39taylorgotta be careful using s/assert
in pre/post-conditions because if a nil input is valid, it'll return it (nil), causing the pre/post-condition to fail#2018-12-0421:39taylor(because nil ā false)#2018-12-0421:40taylorand also have to be mindful of s/*compile-asserts*
#2018-12-0421:41taylor@urzds if you want this behavior in pre/post-conditions it should be very easy to write a small function that does it#2018-12-0421:41seancorfieldWell :pre
/ :post
only take effect if *assert*
is true...#2018-12-0421:42taylor*assert*
is unrelated to s/*compile-asserts*
AFAICT#2018-12-0421:43seancorfieldYes, but the same caveat essentially applies -- you don't get the check in all cases.#2018-12-0421:45seancorfieldYour point about nil
is a good one -- I hadn't thought about that.#2018-12-0421:53taylorit was a learned lesson š#2018-12-0421:48seancorfieldIf you want the check always performed, write it as code in the body of the function. If s/assert
works for you (not nil
values, (s/check-asserts true)
has been run), that's probably a reasonable approach -- otherwise you'll have to write your own variant of s/assert
. Or perhaps write an fspec
for the function and instrument
it?#2018-12-0508:14jumarYou can also set it via system property clojure.spec.check-asserts
#2018-12-0421:53urzdsThanks!#2018-12-0421:59urzdsWhat would be a good way to check whether a collection contains certain elements? I.e. I have [{:name "a"}{:name "b"}{:name "c"}]
and I want to ensure that the collection always contains at least one element with :name "a"
and one with :name "b"
.#2018-12-0422:00urzdsAdditionally, I would like to ensure that each :name
only occurs once.#2018-12-0422:01tayloryou could use s/and
with an extra predicate for those conditions#2018-12-0422:01urzds(Also note: The element's maps contain more keys than just :name
, so I cannot use :distinct
.)#2018-12-0422:03dchelimskyWould it be reasonable to change the structure to a map where name moves up to keys?#2018-12-0422:04dchelimskyI donāt assume it is reasonable. Just asking if it is.#2018-12-0422:04urzdsWas thinking about that initially, but I want to expose the data structure using GraphQL, and that can afaik only query for predefined keys.#2018-12-0422:05dchelimskyIāve run into that recently :)#2018-12-0422:05taylormaybe something like this for your uniqueness constraint, and easy to add your other constraint
(s/def ::names
(s/and (s/coll-of (s/keys :req-un [::name]))
#(apply distinct? (map :name %))))
(s/explain ::names [{:name "a"} {:name "b"}])
(s/explain ::names [{:name "a"} {:name "a"}])
#2018-12-0422:06urzds@taylor: (s/and #(some (fn [x] (= (:name x) "a")) %) #(some (fn [x] (= (:name x) "b")) %))
?#2018-12-0422:09taylor(s/def ::names
(s/and (s/coll-of (s/keys :req-un [::name]))
#(let [names (map :name %)]
(and (apply distinct? (map :name %))
(clojure.set/superset? (set names) #{"a" "b"})))))
here's another way, not necessarily any better though#2018-12-0422:09urzdsAt least it becomes more clear what we're trying to do.#2018-12-0422:10taylorand you may prefer to break each condition into its own predicate like your first example#2018-12-0422:10tayloras opposed to packing multiple conditions into one predicate#2018-12-0422:11urzdsSo I can combine the ::specs instead of having the big inline function?#2018-12-0422:11tayloryep#2018-12-0422:12taylorthe explain
output for invalid inputs will likely be easier to understand if the predicates are separated#2018-12-0422:14taylori.e. if they're all combined the explain
output isn't going to tell you exactly which condition failed#2018-12-0422:16urzdsThanks!#2018-12-0422:17urzdsI guess there is no way to reuse names
in both specs?#2018-12-0422:07urzdsLooks horribly complicated, I guess I'm missing the easy syntax...#2018-12-0422:49Alex Miller (Clojure team)coll-of has a :distinct option fyi#2018-12-0422:49Alex Miller (Clojure team)oh, I guess youāre a level off of that#2018-12-0514:27domkmA :distinct-by
option would be helpful. For coll-of
, I've rarely needed :distinct
but often needed :distinct-by
.#2018-12-0521:19lilactownsorry if this is a tired question, but is there any upcoming plans for supporting dynamic creation of specs other than eval
-ing the macros? e.g. I want to stick a spec in a DB and use it to check some data later#2018-12-0521:36shaun-mahoodWould https://github.com/metosin/spec-tools#data-specs work for what you need?#2018-12-0521:40richhickey@lilactown work already in progress https://github.com/clojure/spec-alpha2#2018-12-0521:43lilactownthx! I'll wait for docs - it's not clear me to from the commits what it's taking shape as yet.#2018-12-0617:16borkdudewhat would be a good name for a lib with some testing utils around specs? mostly things like instrumenting and unstrumenting in the scope of a body kind of macros, cross platform. my gut feeling says that spec-test will be too confusing#2018-12-0617:22shaun-mahoodPretty sure checkulative should be a word#2018-12-0617:27borkdudehaha#2018-12-0621:36Marc O'Morainchickity check yo self#2018-12-0621:36Marc O'Morainhttps://www.youtube.com/watch?v=bueFTrwHFEs#2018-12-0621:37Marc O'MorainChickity-check your specs before you wreck your specs.#2018-12-0617:16taylorspec-too-the-speckoning#2018-12-0617:17borkdudethese functions will be in it: https://github.com/slipset/speculative/blob/master/doc/test.md#2018-12-0617:49Alex Miller (Clojure team)I actually wrote a with-instrument at one point, not sure if I put it in a ticket or not#2018-12-0618:00borkdudethis one also works with clojurescript#2018-12-0618:02Alex Miller (Clojure team)actually, there is a ticket, but not from me - https://dev.clojure.org/jira/browse/CLJ-2015#2018-12-0619:15lilactownI'm having a silly thought: clojure-spec-as-a-service.
We have a few different clients in our system, some of which are Clojure(Script) and some aren't. For those that aren't, exposing common specs for our domain is a bit more troublesome then distributing a library.
One solution might be to create a service that accepts (data, spec-name)
and responds with true
/`false` if it conforms. Could also do generation of fake data that conforms.
Anyone done this before? I'm not sure if this is a good idea yet, architecturally#2018-12-0619:17noisesmith@lilactown I'd be tempted to copy the way kafka works with avro - the schema is versioned and serialized and there's a rest endpoint to look up the schema, then you can deserialize, cache and reuse it#2018-12-0619:18noisesmithand naturally you can use post to add a new spec or new version of a spec, with backward compat enforcement#2018-12-0619:21lilactown> look up the schema, then you can deserialize, cache and reuse it
practically this means that I need to have a common language and engine for working with those schemas in all my clients#2018-12-0619:22lilactownI think the problem I have right now, is "I have all these specs; how do I reuse them across my JS web & mobile clients?"#2018-12-0619:23lilactowndo I build a language on top of spec and build an engine for each platform? Do I adopt something else other than clojure.spec that I more of my clients will understand?#2018-12-0619:24rapskalian@lilactown that's an interesting idea...would part of the spec endpoint also do format conversion? For example, would it be necessary to validate JSON data (convert it to EDN, run through spec/valid?, return result)?#2018-12-0619:24lilactownyes, I am thinking that JSON with be the format it primarily accepts#2018-12-0619:24lilactownso additional complexity#2018-12-0619:33rapskalianOne issue I can think of would be semantic parity between environments...JSON is sort of inherently less expressive then, say, EDN, so you'd have to be careful that they were validating "apples to apples" so to speak.#2018-12-0619:34noisesmithtransit could be useful for this - it's expressly designed for interchange (unlike EDN) and it has implementations for the bigger languages, and it can handle all the standard clojure data types#2018-12-0619:38rapskalianSpec reuse is definitely the right idea. It seems like there'd be quite a few "translation layers" no matter how you tackled it. For example, I've spoken with people who are using Datascript as a sort of "lingua franca" of domain modeling, and then generating specs from that representation. I wonder if "specs" could be generated for other runtimes in their respective native form.#2018-12-0619:38lilactowna lot of the data we'd be verifying would be JSON in the first place - e.g. we get a response from our GraphQL API, and then want to see if it conforms to certain rules: "Does this user data have <product> associated with them?" "Are they allowed to use <capability>?"#2018-12-0619:40lilactownbut there could be other context, unrelated to user data, of course#2018-12-0619:41noisesmithoh - also if you are doing permission checks there's actually an advantage to checking on the central server instead of telling the client how to validate#2018-12-0619:41noisesmith(eg. keeping changes in sync without pushing...)#2018-12-0619:43lilactownyeah permissions isn't primarily what it would be used for but I feel it could grow out of this service organically#2018-12-0622:34borkdudehttps://github.com/borkdude/respeced#2018-12-0700:21ro6I'm starting to try using ::keyword
style keywords more, but running into a limitation and wondering if it's a misconception on my part. I tried using the namespace as a prefix on a longer keyword, eg ::more.prefix/name
, but that's an "Invalid token". Is there something about the Clojure impl (eg around namespaces or interning?) that would break if that were allowed? Is there a "good idea" that's being enforced here to save me from a "bad idea"? I was hoping it would expand to :the.ns.more.prefix/name
.#2018-12-0700:22seancorfield::foo/bar
will expand the alias foo
into the full namespace.#2018-12-0700:23seancorfielduser=> (alias 'foo.bar (create-ns 'long.ns.foo.bar))
nil
user=> ::foo.bar/quux
:long.ns.foo.bar/quux
user=>
#2018-12-0700:23seancorfield(that's essentially the same as (require '[long.ns.foo.bar :as foo.bar])
#2018-12-0700:24ro6right, at the usage site. I'm inside the defining namespace and conceptually wanting to "group" some things without creating a whole new ns#2018-12-0700:24seancorfield::
can only be used with an alias.#2018-12-0700:24seancorfield(or a bare keyword -- the "alias" is implicitly the current namespace)#2018-12-0700:24ro6right#2018-12-0700:27ro6I'm just wondering if there's a deeper "why" for that.#2018-12-0700:28seancorfieldI'm not sure what your question is...?#2018-12-0700:28noisesmith:: is a keyword reading feature that works with aliases#2018-12-0700:28seancorfield::more.prefix/name
means "look up more.prefix
as an alias and resolve it (to some namespace) and then construct a keyword with that namespace and /name
"#2018-12-0700:28seancorfieldThat is the semantics of ::
#2018-12-0700:31ro6I think what's going on in my mind is that as I imagine these ns-qualified keywords escaping my Clojure system, I want to narrow them to remove ambiguity. In other words, I'm trying to name things so they still make sense with fewer contextual assumptions.#2018-12-0700:32seancorfieldNot much outside Clojure is going to be able to accept qualified keywords... Or am I misunderstanding you?#2018-12-0700:34ro6No, I think you're right on.#2018-12-0700:35ro6I guess this line of thought is more about self-documenting keywords for human consumption than machine disambiguation.#2018-12-0700:35seancorfieldIf you want an alias, to avoid typing, you can always do this inside your the.ns
namespace: (alias 'more.prefix (create-ns 'the.ns.more.prefix))
... ::more.prefix/name ...
#2018-12-0700:37ro6interesting compromise. That would let me organize my code into files independently of how I'm organizing my names/concepts.#2018-12-0700:37seancorfield(there's talk of an easier way to create aliases being added to Clojure at some point but that's what we have right now)#2018-12-0700:37seancorfieldThere's no need for the "namespace" portion of a qualified keyword to be an actual namespace with code.#2018-12-0700:38seancorfieldWe have several qualified keywords in our code base that do not map to namespaces.#2018-12-0700:39ro6ok. Thanks for the info.#2018-12-0700:40ro6I also just realized that dot-separated segments after the /
are possible too#2018-12-0700:46ro6I guess at runtime what separates keywords (ns-qualified or not) from strings is mostly faster equality checks due to interning.#2018-12-0700:46Alex Miller (Clojure team)But not recommended#2018-12-0700:46ro6haha, thanks.#2018-12-0707:34macDoes anyone know of a (stateful) generator of locally unique integers for use with spec?#2018-12-0716:27gfrederickswhat does "locally unique" mean?#2018-12-0716:27gfredericks@mac ^^#2018-12-0717:11ro6(guessing) Non-repeating relative to some local scope (eg the generator itself, the running process, ....)?#2018-12-0717:11ro6Not sure about the "stateful" part#2018-12-0719:09mac@gfredericks Just means that for the run of the test no two ints produced will be the same.#2018-12-0719:12gfredericks@mac depends what "for the run of the test" means in this context; is this the generative test that spec does with spec/check
or whatever it is?#2018-12-0719:32gfredericksI'll probably just end up recommending https://www.youtube.com/watch?v=F4VZPxLZUdA#2018-12-0719:34taylormaybe (gen/vector-distinct gen/int)
would be useful... somewhere#2018-12-0720:45mac@gfredericks Yes it is "the generative test that spec does with spec/check
"#2018-12-0720:46mac@robert.mather.rmm Stateful because I cannot imagine being able to guarantee uniqueness without maintaining state.#2018-12-0720:47gfredericksthe standard way to do it would be to write your own generator at the lowest level that encompasses the whole relevant scope of uniqueness#2018-12-0720:47gfredericksyou could take the default generator and gen/fmap
it to something that uniqueifies the ints#2018-12-0720:47gfredericksan alternative, if you don't care how big the ints are, is to just generate really big integers and not allow them to shrink#2018-12-0720:51gfredericks(you'd need to take care to make the generator something like uniform-distribution, so it doesn't start small either)#2018-12-0720:52mac@gfredericks thanks#2018-12-0720:56ro6I don't know if it can be done lazily/immutably, but if you're worried about performance and you know how many random+unique ints you need up front, there's a cool algorithm for that. #2018-12-1220:09domkmApologies if this has been asked previously but why are the Alpha1 and Alpha2 namespaces inconsistent? I would expect clojure.spec.alpha2
instead of clojure.spec-alpha2
?#2018-12-1220:12Alex Miller (Clojure team)because we found having all the namespaces end in āalphaā was annoying#2018-12-1220:43domkmI see#2018-12-1222:35seancorfield@alexmiller On that subject, should I open JIRA issues for the bugs I ran into with spec-alpha2
or isn't it time yet?#2018-12-1222:56Alex Miller (Clojure team)I would wait right now#2018-12-1222:56Alex Miller (Clojure team)I have another giant refactor pending#2018-12-1305:43nopromptThereās also the greek letter beta. troll#2018-12-1305:44nopromptWhy should beta be reserved only for use-at-your-own-risk software I ask? š#2018-12-1310:29jumarLet's say I have following config spec:
(s/def ::my-config-spec
(s/keys :req-un [::host
::port
]))
(s/explain-data ::my-config-spec {:host ""})
;; => #:clojure.spec.alpha{:problems (
;; {:path [], :pred (clojure.core/fn [%] (clojure.core/contains? % :port)),
;; :val {:host ""}, :via [:cacsremoteservice.config/my-config-spec], :in []}), :spec :cacsremoteservice.config/my-config-spec, :value {:host ""}}
I used explain-data and returned :path
key to indicate which keys are in the config are invalid/missing.
What I didn't realize is that when the key is missing then the path is empty (instead of pointing to the particular key).
Is there a nice way how to get the name of a missing required key?#2018-12-1313:41seancorfieldNot nice but you can pattern match :pred
with destructuring since it has that particular shape and extract the missing field name from the contains?
call. #2018-12-1313:56jumarThanks for the idea!
I'm sure there's a better way but here's my quick solution:
(defn extract-key [spec-problem]
(let [[_fn _args [_contains _map req-key]] (:pred spec-problem)]
req-key))
(let [ed (s/explain-data ::my-config-spec {:host ""})]
(->> ed :clojure.spec.alpha/problems
(map extract-key)))
#2018-12-1313:42seancorfield(And it might change in a future version of spec)#2018-12-1323:42thedavidmeister@simon223#2018-12-1400:27aengelbergwhy doesn't clojure.spec.gen.alpha
have recursive-gen
?#2018-12-2020:21wilkerluciothis seems more a generation concern than a spec concern, so I think makes sense been part of test.check / test.chuck or something else#2018-12-1613:12ro6I'm surprised there aren't more examples of specs for common stuff like URLs using s/cat
. Has anyone worked on something like that? I've seen http://conan.is/blogging/a-spec-for-urls-in-clojure.html and https://github.com/SparkFund/useful-specs, but they are using opaque url validators. I'm hoping for something where explain
would actually be instructive.#2018-12-1707:07flyboarderHello everyone, im getting a strange error Unable to resolve spec: :clojure.core.specs.alpha/args+body
anyone know whats up?#2018-12-1707:15flyboarder^I figured it out - renamed alpha spec#2018-12-1819:07Keith HarperThought the patter was interesting, so I figured Iād share. Data points generated with:
(s/def ::latitude (s/double-in :min 32 :max 34 :NaN false :infinite? false))
(s/def ::longitude (s/double-in :min -84.9 :max -82 :NaN false :infinite? false))
#2018-12-1819:17rapskalianCool! Thanks for sharing. It looks like the double generator is a bit "quantum biased", for lack of a better word.#2018-12-1901:48bbrinckIf you previously couldnāt use Expound because your specs used a custom conformer, try Expound 0.7.2 - it may not be able to give a precise error, but at least it wonāt throw a āCannot convert pathā error#2018-12-1919:56ro6Question about evolving specs: I realize the whole idea with specs is "don't change them in breaking ways", so I'm thinking about how to plan for that.
If I define a spec with a single definition, then later realize there's another valid representation, so I want to change it to (s/or ....)
, can that be a non-breaking change to consumers?#2018-12-1919:58ro6I guess if the original definition was one of the options under the s/or
it would still validate the same, but the conformed value would change right? I guess generally the conformed value is more likely to be depended-upon by internal code rather than API consumers...#2018-12-1920:15Alex Miller (Clojure team)adding additional valid things means old stuff should continue to be valid#2018-12-1920:15Alex Miller (Clojure team)I donāt know that we would expect forward-consistent stuff out of conform though#2018-12-2017:38jaihindhreddyWill there be a way to s/select
into a subset of dispatch keys of a s/multispec
?#2018-12-2017:44jaihindhreddySay I have a polymorphic map called ::payment-info
which is polymorphic, like this: (s/def ::payment-info (s/multi-spec pi ::type))
where (s/def ::type #{::credit-card ::debit-card ::net-banking ::bitcoin})
Because cards are processed one way, and each of the other types another way,
is s/select
ing into ::credit-card
and ::debit-card
a good idea?
Was just wondering about this, s/select
isn't even a thing yet so these are potential future questions š#2018-12-2018:12Alex Miller (Clojure team)itās not real enough yet that I canāt answer those questions#2018-12-2018:55ShaneLesterHow would I write a spec to say- in this map I expect to have X Y and Z keys (which have their own specs) and then I donāt care about the rest of the map? Does specifying those 3 in the :req of the keys function assume that I donāt care about the rest?#2018-12-2018:56tayloryeah the s/keys
specs are "open" by default#2018-12-2018:57ShaneLesterOkay cool, thanks#2018-12-2020:53ShaneLesterSo I have these 3 specs hereā¦ I am trying to say, hey there is going to be any number of inputs here in this input-type. But apparently something like {:inputs [:email {:value nil}]}
doesnāt meet this spec? Itās saying it should satisfy map?
but it seems to me that that IS a map. Anyone have a clue what Iām doing wrong here?#2018-12-2021:10taylor::inputs
is expecting a collection of ::input-type
s which are maps/`s/keys` specs, but your input is a single map#2018-12-2021:11ShaneLesterohhh I see. thank you#2018-12-2021:12taylor[{:value nil}]
would conform to your existing spec#2018-12-2021:13tayloryou may have some other issues judging from the vector [:email {:value nil}]
, with the spec as it is#2018-12-2021:14ShaneLesterRightā¦ yeah I may be doing some things in not the best way š time to re-evaluate#2018-12-2021:15taylorfeel free to post some other sample inputs in here and I could help write some sample specs for them#2018-12-2021:17ShaneLesterCool, I appreciate that#2018-12-2022:10agcan someone throw an example of recursive specā¦ e.g. directory structure?#2018-12-2022:11agI want something like:
(s/def :folder/type #{"folder" "file"})
(s/def :folder/id (s/and string? (partial re-matches #"\d*") #(< 8 (count %))))
(s/def :folder/name string?)
(s/def ::entries
(s/coll-of ::folder))
(s/def ::folder
(s/keys :req-un [:folder/type :folder/name ::entries]))
but obviously that doesnāt work#2018-12-2022:11agI want something like:
(s/def :folder/type #{"folder" "file"})
(s/def :folder/id (s/and string? (partial re-matches #"\d*") #(< 8 (count %))))
(s/def :folder/name string?)
(s/def ::entries
(s/coll-of ::folder))
(s/def ::folder
(s/keys :req-un [:folder/type :folder/name ::entries]))
but obviously that doesnāt work#2018-12-2022:15taylorthis will work with (s/def ::entries (s/* ::folder))
#2018-12-2022:16taylor(binding [clojure.spec.alpha/*recursion-limit* 2]
(gen/sample (s/gen ::folder)))
...
{:type "file",
:name "8y4FOn",
:entries ({:type "folder",
:name "",
:entries ({:type "folder", :name "x2RcQC", :entries []}
{:type "folder", :name "6on7", :entries []}
{:type "folder", :name "", :entries []}
{:type "folder", :name "aakQc3q", :entries []}
{:type "folder", :name "6g4748z", :entries []})}
{:type "file",
:name "1f",
:entries ({:type "folder", :name "j52GS", :entries []} {:type "folder", :name "e223Z", :entries []})}
{:type "file",
:name "DUuXT6",
:entries ({:type "folder", :name "t", :entries []}
{:type "folder", :name "720S2cy", :entries []}
{:type "file", :name "", :entries []}
{:type "folder", :name "", :entries []})}
{:type "folder",
:name "18EZK",
:entries ({:type "folder", :name "27", :entries []} {:type "file", :name "Q", :entries []})}
{:type "file",
:name "2K8w",
:entries ({:type "folder", :name "Pv", :entries []} {:type "file", :name "xl4346Q", :entries []})}
{:type "folder",
:name "j",
:entries ({:type "file", :name "21C", :entries []}
{:type "file", :name "3", :entries []}
{:type "folder", :name "2", :entries []}
{:type "folder", :name "1dFvly", :entries []}
{:type "folder", :name "zczGQf", :entries []})})}
...
#2018-12-2022:16agright.. rightā¦ I vaguely remembered, but I couldnāt figure it outā¦ Thanks!#2018-12-2022:30agnow I wonder if there is a way to make binding [clojure.spec.alpha/*recursion-limit* 2]
part of (s/with-gen
?#2018-12-2022:31agso it never generates more than 2 levels deep?#2018-12-2100:19agis it possible to run count
of (s/def #{"a" "b" "c"})
?#2018-12-2101:59tayloris that s/def
missing a name?#2018-12-2101:59taylore.g. (s/def ::foo #{1 2 3})
#2018-12-2102:03tayloranyway if your spec is a literal set, you could do
(s/def ::foo #{1 2 3})
(s/form ::foo)
=> #{1 3 2}
#2018-12-2102:03tayloror you could define the set separately, and reference it from the s/def
and wherever you need to count
it#2018-12-2100:20aghow can I find out how many elements there?#2018-12-2101:57ro6Is there a way to abstract over whether a key in a map is namespaced or not? In a certain context, as long as the value conforms, I want to accept both with the same spec.#2018-12-2102:48Alex Miller (Clojure team)If youāre using s/keys, the :req-un and :opt-un will do that#2018-12-2117:08urzdsCan I spec record methods just like any ordinary function?#2018-12-2117:09urzdsThe guide does not really go into details on records, except that one can spec their fields/attributes: https://clojure.org/guides/spec#2018-12-2117:15Alex Miller (Clojure team)records donāt really have methods#2018-12-2117:15Alex Miller (Clojure team)they implement interfaces/protocols which define methods#2018-12-2117:15Alex Miller (Clojure team)currently, you canāt spec protocols or interface methods#2018-12-2117:16Alex Miller (Clojure team)and because of the calling implementation itās not possible to instrument them (and probably not something weāre ever going to really do because of that)#2018-12-2117:16Alex Miller (Clojure team)but I wouldnāt completely rule it out#2018-12-2117:21urzds@alexmiller So I can spec neither the protocol for all its implementations, nor every record implementation individually?#2018-12-2117:23Alex Miller (Clojure team)no#2018-12-2117:23urzdsBut I could place the implementation in a function, which I could spec, and then from the record implementation just call that function? I.e. insert one step of indirection?#2018-12-2117:23Alex Miller (Clojure team)yes, but I wouldnāt do that just to be able to spec it#2018-12-2117:24urzdsWell, right now my implementations look like (s/assert ...args...) (s/assert ...body...)
, which is hardly better...#2018-12-2117:25Alex Miller (Clojure team)this is a case where the impl of protocols (designed to tap into the highly optimized java/jvm calling semantics) is at odds with the dynamic indirection possible in Clojure via vars#2018-12-2117:26Alex Miller (Clojure team)I guess maybe thereās some possible future where newer Java capabilities like method handles could be used to implement this kind of thing#2018-12-2117:26Alex Miller (Clojure team)retaining most of the speed but also giving you the dev time instrumentation#2018-12-2117:33urzdsSo what do you do when during development you want to ensure that the maps returned from a method implementation of the record conform to a spec? And that all callers call it properly? Insert (s/assert arg-spec arg) (s/assert ...body...)
, like I did?#2018-12-2117:36urzdsI don't really care if the call slows down by an order of magnitude, if it helps me in debugging where that borked data came from.#2018-12-2117:37urzdsOr do you just split up the code in such small pieces that the method implementation in the record does not really do any significant work anymore, so you have enough functions that you can properly instrument?#2018-12-2117:38Alex Miller (Clojure team)I donāt think there is one canonical answer to that question, it depends on the code#2018-12-2117:39Alex Miller (Clojure team)splitting up code into smaller pieces is usually a good idea though#2018-12-2117:39urzdsThanks, I'll start with that then, maybe it already alleviates the problem.#2018-12-2122:06dominicmIs there any work yet on determining whether a spec is a subset or a superset of another? I think Rich has hinted at this a lot. #2018-12-2122:34Alex Miller (Clojure team)No#2018-12-2123:27ro6is anyone using (s/cat ...)
to spec strings? I realize I could just delegate to a traditional string regex with (re-matches ...)
, but then I give up on high-fidelity explain
results no?#2018-12-2215:39dustingetzHow do I spec inside a reagent reaction or other container type that isn't a seq#2018-12-2215:40dustingetzI just googled and found https://stackoverflow.com/questions/37972074/how-to-clojure-spec-a-reference-type-like-atom which says "Don't", okay i get it but that is not compatible with performant UI programming#2018-12-2215:41dustingetzHas this been debated before?#2018-12-2216:23didibusWell, you're not supposed to spec everything#2018-12-2216:25didibusThe same way you don't necessarily put comments on everything#2018-12-2216:26didibusYou need to ask, is this hard to understand, do people wonder what the shape and structure of this thing is, if so, a spec can be a great way to make that easier#2018-12-2216:28didibusOr, if you specifically want to perform generative tests on a particular function that would benefit from it#2018-12-2216:31didibusOr if you need to validate user input, or validate data you're about to persist, etc.#2018-12-2216:33didibusBut if its obvious what the things are, you don't need a spec.#2018-12-2216:34didibusNow to spec an atom, I mean, specs are just predicates. You can easilly do: #(s/int? @%)
#2018-12-2216:35didibusNow you do have to make sure that the deref won't trigger unintended effects#2018-12-2216:35didibusLike in the case of a future or delay#2018-12-2216:37didibusAnd for atom, the other challenge is mutation. Your spec needs to either be the union of all possible valid values the atom will ever contain, throughout its many mutations#2018-12-2216:38didibusOr it has to be more generic, like validates that you expect the thing to be an atom and that's all#2018-12-2216:38didibusOr you need a way to know the particular context and time and what in that context and time the atom is supposed to contain#2018-12-2216:39didibusAnd spec relative to that#2018-12-2216:40didibusThe validator idea is good also. Validators were designed to validate the data being set on an atom. So you can leverage spec to make the validation. #2018-12-2216:40didibusSince spec is pretty much a DSL for easy validation#2018-12-2217:58dustingetzIt's the ratom which is the incidental complexity, it is spiritually just data. For better or worse, Reagent UIs pass things that are data as reactions and it is what it is. But I was unable to get it to work in a nontrivial case#2018-12-2223:33didibusCan you show an example? Also, the ratom isn't a normal atom, and its reactive nature has a different hook for when the data would be updated and need to be validated. At least from what I know, I actually havn't used reagent#2018-12-2221:41tedcushmanI have encountered errors when trying to use s/and
with s/or
as the first argument:
=> (s/explain
(s/and (s/or :int int? :double double?)
pos?)
3)
ClassCastException clojure.lang.MapEntry cannot be cast to java.base/java.lang.Number clojure.lang.Numbers.isPos (Numbers.java:96)
#2018-12-2221:43tedcushmanUsing s/or
after the first argument appears to work though:
(s/explain
(s/and pos?
(s/or :int int? :double double?))
3)
Success!
#2018-12-2221:44tedcushmanUnfortunately, if you are trying to write (s/and (s/or ...) (s/or ...))
there doesnāt seem to be a workaround.#2018-12-2221:58Alex Miller (Clojure team)s/and will flow the conformed result so the and will be receiving a value like [:int pos?]
#2018-12-2221:59Alex Miller (Clojure team)prob the best option here is to (s/or :int (s/and int? pos?) :double (s/and double? pos?))
#2018-12-2222:59tedcushmanok, now that I read the doc, I see the part about āsuccessively conformed valuesā.#2018-12-2718:33thomasHi All, I have been trying to write a spec recently and I can't quite describe what I want...
I have a map where two sets of keys depend on each other like this:#2018-12-2718:35thomas{:a "string"
:b :value-b
:c :value-c}
or
{:a "string"
:b :value-not-b
:c :value-not-c}
and the :b
and :c
values depend on each other.#2018-12-2718:35manutter51( try three backticks to format that without emojis)#2018-12-2718:37thomasthank you @manutter51#2018-12-2718:37thomasso I either have to first version of the second one.#2018-12-2718:39thomasI assume I need something with an or
and an and
but how exactly?#2018-12-2718:41manutter51Iām fairly shaky on spec still, but Iām thinking maybe you could take advantage of namespaced vs. unnamespaced keys, let me see if I can type up what Iām thinking#2018-12-2718:43thomasthank you... I don't get it either yet I'm 'fraid#2018-12-2718:45manutter51(s/def :any/a int?)
(s/def :is-value/b #{1 2 3})
(s/def :is-not-value/b #{4 5 6})
(s/def :is-value/c #{1 2 3})
(s/def :is-not-value/c #{4 5 6})
(s/def :is-or-not/my-map
(s/or :is (s/keys :req-un [:any/a :is-value/b :is-value/c])
:is-not (s/keys :req-un [:any/a :is-not-value/b :is-not-value/c])))
#2018-12-2718:46manutter51Iāve got a suspicion that wonāt work, because maps are specāed by key, not by value, but a perverse part of my brain is saying maybe it would, because the keys are defined to take two mutually exclusive sets.#2018-12-2718:46thomaslet me give that a try @manutter51#2018-12-2718:48thomasI just did a few generates on that and that looks like it does work!#2018-12-2718:53manutter51Generates seem like theyād be more likely to succeed, but Iām not as confident about conforms with ābadā data.#2018-12-2718:53manutter51maybe, though? :crossed_fingers:#2018-12-2718:55thomaswell... I assume if it generates the correct data, then the other way round should work as well.#2018-12-2718:56thomasbut let me try...#2018-12-2718:58thomasyes, validating works as well as expected.#2018-12-2718:59thomasThank you again @manutter51!!!#2018-12-2718:59manutter51Thatās pretty wild, Iāll have to remember that one myself.#2018-12-2718:59thomasa good trick to use the namespaces for that... I wouldn't have thought of that.#2018-12-2718:59thomasI suspect it is something that happens more often.#2018-12-2719:01taylorI think you could also combine your s/keys
spec using s/and
with another predicate, and do any "custom" assertions in the extra predicate#2018-12-2809:52borkdudeI tried to write a generator for arguments to assoc
. Not sure if this is the best way to get overrides for the generators:
https://github.com/slipset/speculative/blob/master/src/speculative/core.cljc#L66#2018-12-2821:38hmaurerHey! How can I define mutually recursive specs? I tried (s/def :my/spec nil)
but it doesnāt seem to work#2018-12-2821:39taylorI think (in most cases?) it should work without you needing to register nil/dummy specs. Can you post a bigger example?#2018-12-2821:41hmaurer@U3DAE8HMG sure!
(s/def :elogic.negation/operand :elogic/formula)
(s/def :elogic/negation (s/keys :req [:elogic.negation/operand]))
(where a formula is later defined by referencing :elogic/negation)#2018-12-2821:42hmaureractually sorry, I just realised in this specific case the formula definition can be hoisted above the negation definition, even thought it references it#2018-12-2822:41jaihindhreddyHere's a good example of mutually recursive specs by @U0FR82FU1
#2019-12-3018:13bedershello spec-fans,
is there a way I can describe a map with spec that uses uuids as keys?
It seems that this is outside the scope of spec and thus makes spec unsuitable for describing general purpose data schemas. Am I missing something here?#2019-12-3018:18mpenet(s/map-of uuid? any?)? Unless you mean mapping specific uuid values#2019-12-3018:30bedersThank you! That was what I was looking for !#2019-12-3022:38borkdudeI found the following workaround for the problem that (s/valid? any? ::s/invalid)
is false
:
https://dev.clojure.org/jira/browse/CLJ-1966?focusedCommentId=51067&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-51067
Iām not sure about the methods where I filled in nil
, so feedback welcome#2019-12-3112:20hmaurerHow would you run https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check as part of a deftest
?#2019-12-3112:21hmaurer(I want to write a test that runs check
on a function, passes if all cases pass, fails otherwise and reports the failing case)#2019-12-3112:31borkdude@hmaurer respeced has a thing for it: https://cljdoc.org/d/respeced/respeced/0.0.1/doc/readme#2019-12-3112:31borkdude@hmaurer respeced has a thing for it: https://cljdoc.org/d/respeced/respeced/0.0.1/doc/readme#2019-12-3115:11hmaurerI am getting Exception: java.util.concurrent.ExecutionException: Syntax error compiling at (clojure/test/check/clojure_test.cljc:95:1).
with:
(deftest foo
(is (successful? (stest/check `term->ast {}))))
any idea why?#2019-01-0108:52borkdudeDonāt know. Can you paste the full exception somewhere?#2019-01-0109:31borkdude@hmaurer
Clojure 1.10.0
user=> (require '[respeced.test :refer [successful?]])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (require '[clojure.test :as t])
nil
user=> (t/deftest foo (is (successful? (stest/check `term->ast {}))))
Syntax error compiling at (REPL:1:17).
Unable to resolve symbol: is in this context
user=> (t/deftest foo (t/is (successful? (stest/check `term->ast {}))))
#'user/foo
user=>
#2019-01-0119:35hmaurer@borkdude the test definition works fine for me as well; itās at the point when itās run that the problem arises š¤#2019-01-0119:35hmaurer@borkdude actually weirdly enough calling it manually worksā¦ but not as part of the test suite. SOmething must be fucked in my setup#2019-01-0119:40borkdude(require '[respeced.test :refer [successful?]])
(require '[clojure.spec.test.alpha :as stest])
(require '[clojure.test :as t])
(t/deftest foo (t/is (successful? (stest/check `term->ast {}))))
(require '[clojure.spec.alpha :as s])
(s/fdef term->ast :args (s/cat :x int?))
(defn term->ast [x] x)
(foo)
#2019-01-0119:42borkdudeif you donāt wrap the call to stest/check in successful? but in a doall, does it work?#2019-01-0119:43hmaurer@borkdude yep that works in the repl for me as well, but when running lein test
it throws.. š I donāt think the issue is with your library at all; I replaced the successful?
function with a function that always returns false
and got the same error#2019-01-0119:43borkdudeand if you remove the call to stest/check, do you still get the error?#2019-01-0119:44hmaurer@borkdude with doall
, same error#2019-01-0119:44borkdudeit might be that you need to include the dependency clojure.test.check, but it depends on your error of course. I havenāt seen the full stacktrace#2019-01-0119:44hmaurer@borkdude nope, if I remove it I donāt get the error#2019-01-0119:45hmaurer@borkdude hereās the full stack trace: https://gist.github.com/hmaurer/ab61d9f1e42cc961a5028fb847409e3a#2019-01-0119:45hmaurer(with lein test
; I normally use kaocha
)#2019-01-0119:46borkdudeit might be your spec?
I see this in the error:
Caused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn#2019-01-0119:47hmaurer@borkdude well if I run stest/check
in the REPL it works fine (it generates inputs, etc, and the output conforms to the :ret spec)#2019-01-0119:47hmaurerš¤#2019-01-0119:48hmaurerhell, I even ran clojure.test/run-all-tests
in the repl and it worked#2019-01-0119:48borkdudemaybe remove the target folder?#2019-01-0119:49borkdudedo you run the repl with the same dependencies and build system as the tests?#2019-01-0119:50hmaurer@borkdude removing the target folder didnāt fix it, and yes I do. lein repl
and lein test
respectively#2019-01-0119:51hmaurer@borkdude the dependencies are almost the same except for respeced
and fulcrologic/fulcro-spec
with have :scope "test"
set on them#2019-01-0119:51borkdudeshould work. maybe make the smallest possible repro and put it in a separate repo#2019-01-0119:51borkdudeusing clojure.spec and not my lib#2019-01-0120:02hmaurer@borkdude ok I reproduced the error, pushing#2019-01-0120:03hmaurer@borkdude https://github.com/hmaurer/clojure-spec-check-bug#2019-01-0120:05borkdudeI get the same error when running lein test. Iāll have a look#2019-01-0120:05hmaurer@borkdude Thank you š#2019-01-0120:11borkdudeI think you need to include :monkeypatch-clojure-test false
in project.clj. then it works for me.#2019-01-0120:11borkdudeI have no idea how it works, but I found it in this issue: https://dev.clojure.org/jira/browse/TCHECK-113#2019-01-0120:13hmaurer@borkdude oh, interesting. I want to use kaocha
though, so Iāll have to figure out what their equivalent is. Thatās still a solution though; thanks a lot š#2019-01-0120:16borkdudeIām also using this: https://github.com/cognitect-labs/test-runner in some projects#2019-01-0120:17borkdudeoh but thatās not for lein sorry#2019-01-0120:23hmaurer@borkdude switching to test.check v`0.10.0-alpha3` fixed it for me š®#2019-01-0120:24hmaurer(as pointed out in the issue you linked)#2019-01-0120:48borkdudecool#2019-12-3112:31borkdude(is (successful? (check `foo {} {:num-tests 10})))
#2019-12-3113:01hmaurer@borkdude fantastic, thank you!#2019-12-3113:02eoliphantdo specs support multimethods yet?#2019-12-3113:05hmaurer@borkdude out of curiosity, if the check fails, will this report the failing case as part of the failed test report?#2019-12-3113:07borkdude@hmaurer yes, you can see an example here: https://circleci.com/gh/borkdude/speculative/17#2019-12-3113:08borkdude(look at the script/test step)#2019-12-3123:31richiardiandreasay I have a password key in a map that is speced...and some other key does not validate, this will throw an exception...is there a way to tell spec NOT to include that sensitive key in the ex-data at all?#2019-01-0100:33seancorfield@richiardiandrea Isn't the concern there more than you need to not display ex-data
values "as-is" to anyone?#2019-01-0100:34seancorfieldIt would be the same as any other ex-info
you throw -- it could well include sensitive data in some context...#2019-01-0100:34richiardiandreathat's a tough one, because then my display thing needs to know about what is displaying#2019-01-0100:35richiardiandreaaka, I need to call dissoc
on things, maybe it should#2019-01-0100:35seancorfieldI wouldn't expect production code to just display any spec failure as is -- those exceptions are intended for code, not humans, and certainly not end users.#2019-01-0100:36seancorfieldFWIW, our application error logging/reporting code has always contained logic to strip known, sensitive fields from any data logged or reported, long before spec was a thing...#2019-01-0100:38richiardiandreaok yeah maybe that's something I would need to do anyways#2019-01-0100:39richiardiandreaI wonder though if things change ...say the string for password changes now I need to change spec AND dissoc
#2019-01-0102:12seancorfield@richiardiandrea That's a good reason to use a globally-unique qualified key name š#2019-01-0309:48y.khmelevskiiHi everyone! Just curious, does anybody know when clojure/clojurescript with spec/schema
and spec/select
will be released? I like this idea and want to try it š#2019-01-0313:34Alex Miller (Clojure team)We have not started developing it yet, so not soon :)#2019-01-0313:34Alex Miller (Clojure team)We have not started developing it yet, so not soon :)#2019-01-0320:18y.khmelevskiithank you for info š#2019-01-0316:25borkdudewhat is generally preferred in specs?
(s/nilable (s/or ...)
vs (s/or :nil nil? ā¦)
when enumerating alternatives#2019-01-0316:25Alex Miller (Clojure team)the first is perf optimized#2019-01-0316:25borkdudethanks#2019-01-0316:26borkdudeit might also be more robust, since some predicates can crash on nil#2019-01-0316:26Alex Miller (Clojure team)the generator for nilable is also intentionally unbalanced so it only produces nils 10% of the time#2019-01-0316:26Alex Miller (Clojure team)whereas the latter will be 1/n where n = # of options#2019-01-0316:28borkdudecan something similar be said about s/alt vs s/or in a regex?#2019-01-0316:29Alex Miller (Clojure team)not sure what you mean. do you mean nilable vs alt on nil?#2019-01-0316:29Alex Miller (Clojure team)if so, then thatās a different situation as s/nilable is not a regex and wonāt compose in the same way as s/alt with other regex ops#2019-01-0316:29borkdudeno sorry, (s/cat :foo int? :x (s/alt ...))
vs s/or instead of alt#2019-01-0316:30borkdudeso independent of nil#2019-01-0316:30borkdudeIām looking for an argument other than āalt belongs in regexā#2019-01-0316:30Alex Miller (Clojure team)s/alt is a regex op and composes with other regex ops to describe a single sequential structure#2019-01-0316:30Alex Miller (Clojure team)s/or is not a regex op#2019-01-0316:31Alex Miller (Clojure team)so acts as an independent value#2019-01-0316:31borkdudesometimes the spec for :x can be taken apart and then it only makes sense to do it with or to make it re-usable on a single value#2019-01-0316:31Alex Miller (Clojure team)it depends here on what ā¦ is#2019-01-0316:32borkduderight, ā¦ can contain other regex specs#2019-01-0316:32Alex Miller (Clojure team)in some cases, there is no perceptible difference#2019-01-0316:32borkdudeso, it depends then#2019-01-0316:32Alex Miller (Clojure team)there are cases where you might want s/alt and some where you might want s/or#2019-01-0316:33Alex Miller (Clojure team)if youāre describing alternatives of the sequential structure, then probably s/alt#2019-01-0316:33Alex Miller (Clojure team)if youāre describing alternate value sets that occur at a particular point in the structure, then probably s/or#2019-01-0316:34Alex Miller (Clojure team)usually the s/alt case will contain more regex ops inside the ā¦ whereas the s/or will not#2019-01-0316:34borkdudeclear, thanks!#2019-01-0417:44dacopareWhat's the best way to integrate spec with clojure.test
? I want to run my spectest/check
s as part of the test suite.#2019-01-0418:33taylorI've used this pattern before, not sure if it's good or not:
(deftest foo-test
(is (= 1 (-> (st/check `foo)
(st/summarize-results)
:check-passed))))
#2019-01-0418:34taylordoesn't give meaningful output on failure#2019-01-0418:20rapskalian@dacopare Clojure.test.check has the clojure.test.check.clojure-test/defspec
macro for that purpose.
https://github.com/clojure/test.check#2019-01-0418:29Alex Miller (Clojure team)thatās not going to help you here#2019-01-0418:38borkdude@dacopare Iām using respeced (a lib I wrote). Hereās an example how I use it: https://github.com/borkdude/speculative/blob/master/test/speculative/core_test.cljc#2019-01-0419:29aviIāve got a few primitive functions to glue specās check
and clojure.test
together, along with expound; I originally copy-pastaād them from a gist and then massaged mangled them over time: https://github.com/FundingCircle/fc4-framework/blob/master/tool/src/test_utils/fc4/test_utils.clj#2019-01-0419:30avi(Oh cool, that snippet includes the entire file if you expand it. Too bad it doesnāt include syntax highlighting.)#2019-01-0505:37dacopareThank you, @borkdude and @aviflax. I'll have a look at both.#2019-01-0517:29kuzmin_mIs it possible to relax requirements in s/keys
?
s/keys
check input value with map?
predicate.
I try to spec datascript entity, but it implements only clojure.lang.Associative
.
https://github.com/tonsky/datascript/blob/master/src/datascript/impl/entity.cljc#L128
https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L857#2019-01-0520:20mishaentities are lazy. validating/conforming with s/keys
would imply realizing and walking the entity graph (friends of friends of friends ...)#2019-01-0604:21mathpunkThis doesn't look right at all. Am I using instrument
incorrectly?#2019-01-0604:22lilactown@mathpunk :args
needs to be, at the very least, a collection#2019-01-0604:22lilactownyou can think of it as, youāre verifying the [s]
collection#2019-01-0604:24lilactownusually I use s/cat
. e.g.:
(s/fdef friendlier
:args (s/cat :s string?))
#2019-01-0604:24mathpunkohhhhh#2019-01-0604:24mathpunkok i think I see this#2019-01-0604:25lilactown(s/cat :s string?)
checks if the first element in the sequence is a string#2019-01-0604:25mathpunkand we're verifying an args /vector/, which happens to have one element#2019-01-0604:26lilactownright#2019-01-0604:26lilactownitās actually a sequence, I think. you can see itās printed as ("World")
#2019-01-0604:27mathpunkok this makes sense, thank you#2019-01-0604:28mathpunkand yeah it's right there but i guess i thought those were decorative parens š#2019-01-0604:28lilactownyeah š
trust me I went through the same confusion the first time I tried to use fdef#2019-01-0714:50urzdsAre logs of this channel available somewhere?#2019-01-0714:51kuzmin_mhttps://clojurians-log.clojureverse.org/clojure-spec#2019-01-0721:22kennyIs there a way to limit the size of the a map generated from (gen/generate (s/gen map?))
?#2019-01-0721:25taylorgenerate
can take another argument size
e.g. (gen/generate (s/gen map?) 10)
#2019-01-0721:25taylorI think the general range of sizes is 0-200 maybe?#2019-01-0721:26taylorlooks like it uses 30
by default#2019-01-0721:24Alex Miller (Clojure team)no, but you can use something like (s/map-of any? any? :gen-max 5)
#2019-01-0721:35borkdudeis there a better way of making a transducer spec than ifn?
?
e.g. this works, but it shouldnāt:
(s/conform (:ret (s/get-spec `map)) [1 2 3])
[:transducer [1 2 3]]
#2019-01-0721:41Alex Miller (Clojure team)Iāve looked at it, and generally Iād say no#2019-01-0721:42Alex Miller (Clojure team)this is well outside the sweet spot for spec#2019-01-0721:52kennyI'm seeing strange behavior when running clojure.spec.test.alpha/check
on an fdef
'ed function. I have a prn
as the first line in my function that prints out the size of the args. When the generative tests first start running, I get print statements out rapidly. After running for a few seconds, the print statements slow down to once every second. After 10-20s more, they speed up again. This goes on for about 10 mins until check
returns. At first I thought it was due to test.check generating large maps so I changed the :args
in my fdef
to use (s/map-of any? any? :gen-max 5)
. Then I thought it was a JVM memory issue but (.maxMemory (Runtime/getRuntime))
says the JVM has over 7gb available. I haven't seen this behavior with spec check
before. Any ideas on what could cause this?#2019-01-0721:58Alex Miller (Clojure team)sounds awfully like GC churn to me#2019-01-0721:59Alex Miller (Clojure team)do you have an fspec anywhere?#2019-01-0722:00kennyNot in this function.#2019-01-0722:01Alex Miller (Clojure team)you can add -verbose:gc to watch the gc#2019-01-0722:02Alex Miller (Clojure team)you could isolate just the gen on the args if you suspect that itās it#2019-01-0722:04Alex Miller (Clojure team)something like
(gen/sample (s/gen (:args (s/get-spec `foo))) 1000)
#2019-01-0722:48kennyGenerating the args like you have there takes about 30s. I'll add the gc flag to see if it sheds any light.#2019-01-0722:50kennyAdded the gc flag. I get a message that looks like this [GC (Allocation Failure) 2477246K->1068161K(3133952K), 0.1575999 secs]
printed out every second or so.#2019-01-0723:06kennyThis is what the output with the -verbose:gc flag looks like https://pastebin.com/raw/XCxMykgq. The numbers are the count
on the two args this function takes.#2019-01-0723:08kennyUpdated to test.check 0.10.0-alpha3
and the problem goes away.#2019-01-0723:53lilactownI have a map like:
{::kind :bool
::default true}
that, given a different ::kind
, might need a different predicate for ::default
. E.g.:
{::kind :string
::default "foo"}
What's the best way to model this in spec? I thought multi-specs would work, but I'm still struggling with the fact that the name of the spec I pass into s/keys needs to match the key, which makes overloading ::default
difficult#2019-01-0723:57mattly@lilactown I've done this with s/or
specs#2019-01-0723:57mattlyI can give you an example in our work slack#2019-01-0800:14lilactownAFAICT there's no way to do this without using :req-un
š#2019-01-0800:19favilahttps://gist.github.com/favila/ab03ba63e6854a449d64d509aae74618 is a hack I wrote a while ago that will add an additional thing to conform to for some of the named keys#2019-01-0800:19favila#2019-01-0800:20favilaspec is super duper opinionated on this point though#2019-01-0800:20favilahence the complexity of the hack#2019-01-0800:21favilause like (keys+ :req [::whatever] :conf {::whatever narrower-specish-thing})
#2019-01-0800:23favila::whatever
value will be asked to validate against both its own spec and narrower-specish-thing
(so ::whatever
should be the widest possible spec you could have for that key)#2019-01-0800:23favilabut narrower-specish-thing
will be used for conforming and generators#2019-01-0800:25favilaThe idea is that contextually (in a specific map) a key may have a narrower spec than normal#2019-01-0800:26favilawhich for some reason happens to me all the time and made spec very painful#2019-01-0800:26favilathe alternative is s/or with more predicates, and with-gen to adjust the generator#2019-01-0800:26mattlyyou could also just forgo using s/keys
and do it manually (spec/def :my-union/shape (fn [thing] (case (::kind thing) :bool (if (boolean? (::default thing)) true ::s/invalid) ::s/invalid)))
#2019-01-0801:09lilactownhm. yeah, I think I settled on:
(defmulti parameter-kind ::kind)
(defmethod parameter-kind :bool [m]
#(if (boolean? (::default %))
true
false))
(s/def ::parameters (s/map-of keyword?
(s/and
(s/keys :req [::kind])
(s/multi-spec parameter-kind ::kind))))
#2019-01-0803:26Alex Miller (Clojure team)s/multi-spec
is kind of designed to use different specs based on data#2019-01-0803:27Alex Miller (Clojure team)youād need to use it with s/keys and :req-un here though since you have the same attribute name with different specs apparently#2019-01-0814:28urzdsBack in December I asked whether it was possible to spec protocol methods, where the answer was "no" and "because of the implementation that is targetted at performance". I am wondering whether I could instead use multi-methods instead of protocols and methods and spec them. (My code should allow replacing the record with a map and the protocol methods with multi-methods.) Can I just use s/fdef
on the multifn
and spec :args
and :ret
that have to be valid for all implementations? An alternative would be to use pre/post conditions, but I cannot see anything resembling a pre-post-map
(as is present for defn
) in the docs for defmulti
or defmethod
.#2019-01-0814:37Alex Miller (Clojure team)Currently, I do not believe that works, but I think it could be made to work#2019-01-0814:54urzdsI just tried the following code and got no error, which I guess suggests that it does indeed not work:
(require '[clojure.spec.alpha :as s])
(defmulti testfn :type)
(defmethod testfn :atype [m] 1)
(s/fdef testfn :args (s/keys :req-un [::type ::does-not-exist]))
(testfn {:type :atype})
; => 1
What would be the path forward from here? Should I open an issue / feature request for https://github.com/clojure/spec.alpha ?#2019-01-0815:34urzds@alexmiller ^^#2019-01-0815:51Alex Miller (Clojure team)we handle spec issues in the main CLJ jira system and I think there already is one for this#2019-01-0815:53Alex Miller (Clojure team)you didnāt call stest/instrument in the example above so that at least is a missing step#2019-01-0815:53Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2450 is one issue#2019-01-0816:00urzdsYou're right, the following code at least throws an exception:
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])
(defmulti testfn :type)
(defmethod testfn :atype [m] 1)
(s/fdef testfn :args (s/keys :req-un [::type ::does-not-exist]))
(stest/instrument `testfn)
(testfn {:type :atype})
=> clojure.lang.ExceptionInfo: Call to #'user/testfn did not conform to spec.
But it also throws this exception for arguments that should conform:
(testfn {:type :atype :does-not-exist 1})
=> clojure.lang.ExceptionInfo: Call to #'user/testfn did not conform to spec.
Sadly there is no explanation why...#2019-01-0816:06Alex Miller (Clojure team)I donāt actually see a CLJ issue for just āmultimethods canāt be instrumentedā but would be ok to make one if you like! Like I said, I think this is something that is fixable.#2019-01-0816:06Alex Miller (Clojure team)actually I think your spec is wrong#2019-01-0816:07Alex Miller (Clojure team)youāre missing the top level args sequence#2019-01-0816:07Alex Miller (Clojure team)(s/fdef testfn :args (s/cat :m (s/keys :req-un [::type ::does-not-exist])))
#2019-01-0816:08Alex Miller (Clojure team)that works for me#2019-01-0816:09urzdsOups#2019-01-0816:12urzdsCan specs be redefined? I just tried to execute (s/fdef testfn :args (s/cat :m (s/keys :req-un [::type ::does-not-exist])))
in the same REPL session, which appeared to be successful, but spec would still throw an exception even when I passed in the correct arguments. Only restarting the process fixed that.#2019-01-0816:15Alex Miller (Clojure team)you need to instrument again#2019-01-0816:16Alex Miller (Clojure team)or possibly unstrument / instrument (although I think either will work)#2019-01-0816:16urzdsyes, just calling stest/instrument
again worked.#2019-01-0816:23borkdude@urzds if you use something like component, you can hook up re-instrumentation with the start/stop lifecycles#2019-01-0816:24urzds@borkdude Thanks!#2019-01-0816:28urzdsBTW, I also found the request for specs for protocol methods (my original question): https://dev.clojure.org/jira/browse/CLJ-2109#2019-01-0817:12urzds@alexmiller Could you please expand on the rationale behind this? https://dev.clojure.org/jira/browse/CLJ-2378?focusedCommentId=49601&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-49601
In https://groups.google.com/d/msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ you refer to the changelog, but that does not appear to include history data: https://github.com/clojure/spec.alpha/blob/master/CHANGES.md#2019-01-0817:16Alex Miller (Clojure team)So just repeating my comment in the jira, the idea behind instrument
is to check whether a function has been correctly invoked#2019-01-0817:17Alex Miller (Clojure team)The idea behind check
is to verify that a function produces proper outputs in response to valid inputs#2019-01-0817:31urzdsHm, I got that, but I would also like to see, during development, that a function is being invoked correctly and it responds correctly in live workloads. I.e. in my use-case I want more than to just check whether it was invoked correctly, but I also am not running the function in a test where I could use check
. How is my use-case to be handled?#2019-01-0817:32urzdsWhat I did not understand is the technical necessity for instrument
not to also check the return value. I assumed knowing why you decided against that might help me understand the context better.#2019-01-0817:42borkdude@urzds instrument can take a performance hit when you have a lot of fdefs. if you check the return values, you will often check those twice, since they are arguments for another function. so I think itās a sane default#2019-01-0817:43borkdudethis may not the reason that core decided to do this, but I have come to appreciate it for this reason#2019-01-0817:44urzdsIn one of my cases these functions are GraphQL resolvers, invoked by Lacinia. So there really is nothing coming afterwards in my own code where I could check the return value... (And GraphQL schemas are not as expressive as Clojure Spec.)#2019-01-0817:45borkdude@urzds you can try to write a generative test for it.#2019-01-0817:45urzdsMaybe my understanding of Spec is wrong, though. I thought I should spec everything that goes into my system and everything that goes out of it, in order to ensure that it behaves nicely with others.#2019-01-0817:46Alex Miller (Clojure team)instrument has functionality to do stubs and mocking too, which can be used in combination with check#2019-01-0817:47borkdude@urzds some examples of generative tests of fdefs: https://github.com/borkdude/speculative/blob/master/test/speculative/core_test.cljc#2019-01-0817:47borkdudea ret and fn spec is really useful to check if your implementation is correct in combination with generative testing#2019-01-0817:49borkdude@urzds you can always plug in the ret spec checking manually, if you want. just separate out the spec and call valid?#2019-01-0817:49borkdudeor use :pre
and :post
#2019-01-0817:50urzdsHm, maybe I should find ways to split up those functions better in order to allow generative testing. Seems a bit difficult right now, because they rely on input from external services. So I just have unit tests for my internal functions, and wanted to rely on spec throwing exceptions when the interaction with the outside world shows signs of problems.#2019-01-0817:50borkdudeyou can also use spec/assert
#2019-01-0817:50borkdudelots of options#2019-01-0817:51urzds:pre
and :post
do not work here, because the functions are actually protocol methods (could be changed to multi-methods, but those also do not support :pre
and :post
).#2019-01-0817:51borkdudespec/assert can also be elided with compile time options.#2019-01-0817:52urzdsspec/assert
is what I am using right now. The whole body of the function is wrapped in spec/assert
, which looks a bit ugly TBH and the commit introducing this will cause a large amount of code to be reformatted for indention, which is also undesirable.#2019-01-0817:53borkdudethen donāt wrap. you donāt have to. and when you elide the call, youāll be left with an empty function#2019-01-0817:56urzdsHow would I not wrap and still check the return value, in the absence of :post
?#2019-01-0818:40borkdude@urzds
(fn [args]
(s/assert args-spec args)
(let [res (calc args)]
(s/assert ret-spec res)
res))
#2019-01-0911:42urzdsAnother recommendation I received was something along these lines:
(defn wrap-fn-with-spec [f s-args s-reg]
(fn [& args]
(s/assert s-args args)
(let [ret (apply f args)]
(s/assert s-ret ret)
ret)))
#2019-01-0911:43borkdudeyeah same idea. you could also do it with a macro to receive better line errors#2019-01-0911:45urzdsIs there a way to retrieve the specs attached to the symbol using s/fdef
? That might make the above function integrate a bit better with the rest of spec.#2019-01-0911:46borkdudeyes, (s/get-spec
foo)` and then call :args
or :ret
on it.#2019-01-0911:46borkdudebut another option is to spec the args and ret spec separately, so you can write a more specific generator for it#2019-01-0911:48urzdsSo like this?
(defn wrap-fn-with-spec [f]
(let [s-fn (s/get-spec f)
s-args (:args s-fn)
s-ret (:ret s)]
(fn [& args]
(s/assert s-args args)
(let [ret (apply f args)]
(s/assert s-ret ret)
ret)))
#2019-01-0911:48borkdudeIām not sure if s/get-spec works with a function, I donāt think so. so youāll need the symbol for the function too#2019-01-0911:49borkdudejust try it from the REPL and youāll see.#2019-01-0911:49borkdudeah, it works with a var:
https://clojuredocs.org/clojure.spec.alpha/get-spec#2019-01-0911:49borkdudeso youāre better off passing the var then#2019-01-0911:49urzds> but another option is to spec the args and ret spec separately, so you can write a more specific generator for it
How do you mean? Doesn't s/fdef
have the args and ret spec separately already?#2019-01-0911:50borkdudeyes, but they donāt have a specific name, so you cannot generate specific combinations of arguments. hereās an example for assoc:
https://github.com/borkdude/speculative/blob/master/test/speculative/core_test.cljc#L107#2019-01-0911:55borkdudeIām not saying you have to, but it gives options#2019-01-0911:56urzdsThanks!#2019-01-0817:53borkdudegotta go#2019-01-0817:57urzdsGuess I gotta read more docs about this subject.#2019-01-0818:39dustingetzIs it possible to spec this: #<Track: [nil :fiddle/links 1234]
ā a tuple inside a deftype#2019-01-0819:39Alex Miller (Clojure team)deftypes do not expose their structure (other than the type)#2019-01-0819:39Alex Miller (Clojure team)whether you can spec it in some other ways depends on what interfaces/protocols you implement and which predicates you want to spec it with#2019-01-0819:40Alex Miller (Clojure team)so without more info, I would default to: no :)#2019-01-0916:07ShaneLesterWhatās the normal way I should be handling speccing functions that have optional arguments?#2019-01-0916:12Alex Miller (Clojure team)generally you should use regex ops (like s/cat) to spec args and ops like s/? to handle optional arguments within that#2019-01-0916:13ShaneLesterCool. Thanks š#2019-01-0920:17jstewIs there a way to match an exact number of items in a collection or sequence? Letās say I want to spec a seq or collection to have 3 and only 3 items in it?#2019-01-0920:19jstewI suppose I could always assert that in the spec for the fn that uses the collection.#2019-01-0920:23tangrammerAs spec guide states š ā¦
(s/def ::point (s/tuple double? double? double?))
(s/conform ::point [1.5 2.5 -0.5])
=> [1.5 2.5 -0.5]
https://clojure.org/guides/spec#_collections#2019-01-0920:23jstewtuple! Thank you.#2019-01-0920:35Alex Miller (Clojure team)s/tuple is good for heterogeneous fixed-size (āslottedā) vectors#2019-01-0920:35Alex Miller (Clojure team)s/coll-of with the :count option is good for homogenous (all items match the same pred) collections#2019-01-0920:36Alex Miller (Clojure team)like (s/coll-of double? :count 3)
#2019-01-0920:36jstewThat fits this use case better. Thanks, Alex.#2019-01-0920:36Alex Miller (Clojure team)either will work and they conform and gen pretty similarly but I think one or the other usually has a better match on intent#2019-01-0921:07ShaneLesterIām attemping to make a spec for something that can either be any number of vectors, or 0 or 1 strings. currently I have (spc/or :vectortype (spc/* vector?) :stringtype (spc/? string?))
but that doesnāt really seem to be working. (I know this is whacky but, using it to learn the nuances of spec#2019-01-0921:08ShaneLesterAnyone see if I am doing something obviously wrong there?#2019-01-0921:09manutter51Looks fair to me, what are you getting from that that makes you say itās not working?#2019-01-0921:10ShaneLesterGetting a message in-console saying that the function call does not match the spec when the argument that that spec covers is just a string#2019-01-0921:13manutter51Function arguments need to be inside (spc/cat ...)
maybe youāre missing that?#2019-01-0921:14ShaneLesterI guess this would have been more useful- here is the whole spec for the function currently (spc/fdef typog :args (spc/cat :props map? :children (spc/or :vect (spc/* vector?) :stringt (spc/? string?))))
#2019-01-0921:14ShaneLesterso I do have cat. but not sure if im using it right#2019-01-0921:16manutter51Do you have the :ret
key in your fdef too? Iāve seen odd results when that was missing.#2019-01-0921:16ShaneLesterHmm I dont, ill try adding that#2019-01-0921:25manutter51Everything else looks good to me, with the caveat that Iām still getting comfortable with spec myself, so I might be missing something.#2019-01-0921:30ShaneLesterI just threw a :ret any?
in there, but same results sadly. Yeah. It seems to logically make sense to me, and to others Iāve showed it to. Something strange going on perhapsā¦#2019-01-0921:30ShaneLesterI appreciate your help š#2019-01-0921:43Alex Miller (Clojure team)the s/or is not a regex op so inserts a new ālevelā of nesting#2019-01-0921:45Alex Miller (Clojure team)so that is always expecting 2 args with the first a map and the second either a collection of 0 or more vector or an empty collection or a collection containing one string#2019-01-0921:45Alex Miller (Clojure team)I donāt think thatās what you want#2019-01-0921:46Alex Miller (Clojure team)I suspect changing the or
to alt
is probably closer, would at least match the string case you want#2019-01-0921:46Alex Miller (Clojure team)but depends what you mean by ā0 or 1 stringsā#2019-01-0921:47Alex Miller (Clojure team)if you mean āno 2nd argā for ā0 stringsā, then that case is already covered by (* vector?), which can be 0 vectors#2019-01-0922:02ShaneLesterAhh it was alt! That is what I needed. I did gloss over that in the api but apparently that did not make enough sense for me. haha. Thank you alex.#2019-01-0922:19ShaneLesterAlso good point about the case of the arg not being there handled by the *, that is definitely true.#2019-01-1114:35mpingHi#2019-01-1114:35mpinghow can I create a spec for a map like {"2018-01-01": {:value 10}, "2018-03-22: {:value 20}}
#2019-01-1114:35mpingmap where keys are dates#2019-01-1114:37Alex Miller (Clojure team)write a function that decides whether that is a valid string - that function is your predicate#2019-01-1114:39Alex Miller (Clojure team)(s/def ::date valid-date?)
(s/def ::value int?)
(s/def ::data (s/keys :req-un [::value]))
(s/def ::m (map-of ::date ::data))
#2019-01-1114:52mpingtks Alex, pretty much what I had, problem must be somewhere else.#2019-01-1115:03ikitommi@mping you might want to add :conform-keys true
to map-of
#2019-01-1115:07Alex Miller (Clojure team)with a pred, the keys will will just conform to themselves, so thatās just making it slower#2019-01-1115:23mping@alexmiller in what cases would conform-keys be needed?#2019-01-1115:23mpingcore specs?#2019-01-1115:26Alex Miller (Clojure team)cases where the keys need to be conformed#2019-01-1115:27Alex Miller (Clojure team):)#2019-01-1115:27Alex Miller (Clojure team)most maps have keys that are strings, numbers, keywords, uuids, etc - all of which conform to themselves, so there is no reason to conform the map keys#2019-01-1115:28Alex Miller (Clojure team)but if you had a map whose keys conformed to a value other than themselves (a regex spec, an s/or, a nested map, etc) then you would want to consider it#2019-01-1115:29Alex Miller (Clojure team)but even in that case, you have to think carefully to make sure the conformed values donāt create a case where two map keys have the same conformed value. if so, they will collide in the conformed map.#2019-01-1115:30mpingI see.#2019-01-1115:31mpingthanks guys#2019-01-1200:08blanceIs it idiomatic to use spec to validate data with remote service through http request?#2019-01-1200:09seancorfield@blance I think it's idiomatic to use spec for validating data in a lot of cases. Can you be a bit more specific?#2019-01-1200:10seancorfieldFor example, we use to spec to validate (and conform) input arguments to our REST APIs and in some other places either before or after interaction with an external system.#2019-01-1200:11blancesay I have a user id that needs to be an int, normally i would just validate it using int?
. i'm just curious does it make sense to also do (def ::user (s/and int? user-exist?))
#2019-01-1200:11blancewhere user-exist
would have to call to an external service to check if user exists#2019-01-1200:16seancorfieldI'd be careful about using predicates that rely on outside systems for your specs. That feels wrong to me.#2019-01-1200:16lilactownif you try and use ::user
with generators, are you going to be upset if it sends hundreds of thousands of requests to your external service?#2019-01-1200:17seancorfieldYup. I would say it's worth trying to stick with specs that can be used in generators.#2019-01-1200:17lilactownsometimes it's surprising which spec functions use generators too. e.g. instrumentation#2019-01-1200:19blancei haven't really start looking at generative testing yet#2019-01-1200:19blancei suppose http request can be mocked during testing?#2019-01-1200:25blancereading more on the doc, A key design constraint of spec is that all specs are also designed to act as generators of sample data that conforms to the spec
sounds like I shouldn't be using any non trivial pred that can't easily generate good sample data#2019-01-1212:37mishaYou can use non-trivial predicates (and, I think, it is one of the value props of spec), but if you end up using it for generative testing you will have to help that predicate with a custom generator to minimize "Couldn't satisfy such-that predicate after 100 tries" errors (https://clojure.org/guides/spec#_custom_generators)
You can write predicates with side-effects, but as with usual code ā try to isolate them from the pure ones, and know all the contexts where those might be used. So checking if user exists is perfectly fine, but probably should be done at a very specific points of the system, separate from checking hypothetical user's structure.
Also: "not everything needs to be a spec", as well as "not everything needs to be specced". @blance#2019-01-1223:51blancethat make perfect sense, thanks!#2019-01-1220:38trevorI'm struggling on how to create a generator that returns a constant value#2019-01-1220:38trevormy example is UUID or constant id#2019-01-1220:39trevor(gen/one-of [(s/gen ::uuid) ...])
#2019-01-1220:39trevorlet's say the constant is 0
#2019-01-1220:45Alex Miller (Clojure team)(s/gen #{::uuid})
#2019-01-1220:46trevorI was looking for gen/return
#2019-01-1311:45borkdudeI have a spec of map-entry and seqable-of-map-entry. Sometimes I get this exception while generating values:
(require '[clojure.spec.gen.alpha :as gen])
(require '[clojure.spec.alpha :as s])
(s/def ::map-entry
(s/with-gen map-entry?
(fn []
(gen/fmap first
(s/gen (s/and map? seq))))))
(s/def ::seqable-of-map-entry
(s/coll-of ::map-entry :kind seqable?))
(gen/sample (s/gen ::seqable-of-map-entry))
Error printing return value (ClassCastException) at clojure.core/conj (core.clj:82).
java.lang.String cannot be cast to clojure.lang.IPersistentCollection
#2019-01-1311:46borkdudeI wonder why it tries to call conj at all here#2019-01-1311:49borkdudeI can imagine it tries to build a seqable, so it starts with an empty string, and then it tries to conj map-entries to it:
(conj "" (first {:a 1}))
#2019-01-1311:51borkdudeif this is a bug, I would be happy to file it in JIRA#2019-01-1311:52borkdudefor now I can use this workaround:
(s/def ::seqable-of-map-entry
(s/with-gen (s/coll-of ::map-entry :kind seqable?)
(fn []
(s/gen (s/coll-of ::map-entry :kind list?)))))
#2019-01-1312:01borkdudeI could also add :into []
but that would exclude lazy seqs#2019-01-1313:51Alex Miller (Clojure team)coll-of always gens a collection, never a lazy seq#2019-01-1313:59borkdudeYes, I meant, if I add the into, it would generate correctly but it would realize lazy seqs #2019-01-1312:03borkdudemaybe something like this? (doesnāt work yet)
(s/def ::seqable-of-map-entry
(s/coll-of ::map-entry :kind (s/with-gen seqable?
#(s/gen vector?))))
#2019-01-1312:25borkdudeso kind must be a predicate and cannot be a spec, but it must also generate. in other words, you can only use pre-defined predicates?#2019-01-1312:33borkdudethis seems to work:
(defn seqable-of
"Prevents generating strings and therefore Exceptions during generation"
[elt-spec]
(s/with-gen (s/coll-of elt-spec :kind seqable?)
#(s/gen (s/coll-of elt-spec :kind vector?))))
#2019-01-1313:52Alex Miller (Clojure team):kind seqable? does not make sense#2019-01-1313:52Alex Miller (Clojure team)coll-of always gens a collection#2019-01-1313:53Alex Miller (Clojure team)Itās a collection spec#2019-01-1314:00borkdudeWhatās the recommended way of specāing a seqable of something then?#2019-01-1314:07borkdudeMaybe seqable and only checking the first value?#2019-01-1314:57borkdudeMaybe just coll-of would work, since
(coll? (seq {:a 1 :b -1 :c 1 :d -1}))
(coll? (filter (comp pos? val) {:a 1 :b -1 :c 1 :d -1}))
are both true#2019-01-1315:04borkdudenope, it really should be a seqable, since (java.util.HashMap. {:a 1})
is also supposed to work.#2019-01-1315:08borkdudeevery might be the one I should use then#2019-01-1315:17borkdudethatās it. every also only checks a maximum number of elts, so a lazy infinite seq would still be supported. thanks. :duck:#2019-01-1315:19borkdudeisnāt this a bit inconsistent, since nil puns as an empty sequence?
user=> (s/conform (s/every string?) '())
()
user=> (s/conform (s/every string?) nil)
:clojure.spec.alpha/invalid
user=> (s/conform (s/every string? :min-count 0) nil)
:clojure.spec.alpha/invalid
user=> (count nil)
0
#2019-01-1315:58borkdude(s/conform (s/every string? :kind seqable?) nil)
works though, but then Iām back into the same problem where I started:
user=> (gen/sample (s/gen (s/every string? :kind seqable?)))
Error printing return value (ClassCastException) at clojure.core/conj (core.clj:82).
java.lang.String cannot be cast to clojure.lang.IPersistentCollection
#2019-01-1315:59borkdudeso maybe (s/nilable (s/every ::map-entry))
is best then#2019-01-1316:12Alex Miller (Clojure team):kind seqable?
is just not right here. every
(like coll-of
) is a spec for collections (not seqables). you are using a broader predicate for :kind than the spec is intended for.#2019-01-1316:15borkdudewhy is (s/nilable (s/every ::map-entry))
not the right fit for seqables? the implementation uses seq
on the input and then checks every element up to a limit. so if this is not it, whatās the alternative?#2019-01-1316:19borkdudeif you donāt specify the kind to every, whatās the default?#2019-01-1316:42Alex Miller (Clojure team)Iām saying seqable? does not make sense with coll-of/every because some seqables are not collections#2019-01-1316:43Alex Miller (Clojure team)The default is vector iirc#2019-01-1318:39borkdude@alexmiller would this be OK?
(defn seqable-of [spec]
(s/with-gen (s/and seqable?
(s/or :empty empty?
:seq (s/and (s/conformer seq)
(s/every spec))))
#(s/gen (s/nilable (s/every spec :kind coll?)))))
#2019-01-1406:25devthi wonder why aren't specs first class things we can define literally and pass around, and why they need their own special registry when we already have namespaces and vars? š¤
i'm playing with building up specs in an automated way (e.g. reducing a data structure into a spec representation). looks like i'm gonna have to s/def
things along the way. isn't that some kind of PLOP? maybe this is something that will be improved in future version š#2019-01-1409:34mpenetThe latter it seems#2019-01-1409:36mpenetWhy another registry, I believe to not clutter vars/nses more and make their potential evolution separate?#2019-01-1415:33devththen why not have a registry for all atom
s? and another for all fn
s? and on and on. š#2019-01-1415:35mpenetThere are already issues with vars initialization at startup right now, I guess that's not to make it worse among other thing. Then it's a "private" thing, we never get exposed to the fact it's separate#2019-01-1415:36mpenet-> https://dev.clojure.org/display/design/Lazy+var+loading#2019-01-1415:36devthi see. š#2019-01-1409:38mpenetWe ll see with alpha2 I guess. The posts from @alexmiller about the ongoing work on this are quite interesting #2019-01-1409:40mpenetthey don't reveal much about these 2 questions in particular, but soon enough it might#2019-01-1409:51borkdude@devth if you can give an example, we can see if it can be improved using the current version of spec?#2019-01-1416:20devthtried to explain the problem in detail at https://clojurians.slack.com/archives/C1B1BB2Q3/p1547482812312100#2019-01-1411:04misha@devth there is s/spec
which gives you spec w/o being registered, which might be useful for you if you build specs dynamically, use, and throw away right away.#2019-01-1413:58Alex Miller (Clojure team)there are no plans to change the registry aspect of spec#2019-01-1413:59Alex Miller (Clojure team)all of the programmatic construction aspects will be different in spec 2#2019-01-1415:45flyboarder@alexmiller could you elaborate more on that? Will existing dynamic specs break?#2019-01-1415:56Alex Miller (Clojure team)hard to say yet#2019-01-1416:20devth#2019-01-1416:31rapskalianDoes there exist a flavor of s/merge
for disjunctions, i.e. s/or
? I have a base set of disjunctions (A or B) that Iād like to be able to extend in certain contexts (base or C or D). #2019-01-1416:45borkdude@devth Isnāt this the flaw of current spec that Rich spoke about in his recent Clojure Conj talk?#2019-01-1416:45borkdude@devth Isnāt this the flaw of current spec that Rich spoke about in his recent Clojure Conj talk?#2019-01-1416:49devthi'm not sure i've seen his latest conj talk. i'll have to look it up#2019-01-1416:58Alex Miller (Clojure team)https://www.youtube.com/watch?v=YR5WdGrpoug#2019-01-1416:59devthoh, i have seen part of that one.#2019-01-1417:54devthstill curious about the place vs value oriented nature of spec š¤ any thoughts?#2019-01-1417:54borkdudeI recommend watching the talk.#2019-01-1417:54devthk, i'm 30 min in. still watching š#2019-01-1417:55borkdudelet it sink in on the hammock š#2019-01-1418:01devthtaking notes#2019-01-1500:54devthBig ideas:
1. separating out schema (shape) and selection (optionality). cool stuff!
- it's still not "just data". why the departure? everything in Clojure is just data, and for good reason. the entire core lib is built around manipulating data. it's the best part of clj.
- it's still place oriented, at least from what i gather so far š
2. Better programmatic manipulation of specs. great! but again: why not make it data? then programmatic manipulation comes with.
Imagine if the schema for:
{:weather {:weatherbitio {:key "xoxb" :default {:zip "98104"}}}
:command {:prefix "!"}}
Was simply:
{:weather {:weatherbitio {:key string? :default {:zip string?}}}
:command {:prefix string?}}
There's elegance in mirroring the data structure you're specifying, kinda like how Datomic Pull queries mirror the data you get get back.
š¤#2019-01-1416:46borkdudeor do you mean āwhy do I have to gives names to each sub-spec, canāt you inline them?ā ?#2019-01-1416:49borkdudeit may be related to each other#2019-01-1416:49devthyeah, both i think. i want values#2019-01-1416:50borkdudefuture spec will allow you (I think!) to spec the whole schema and then define selections on them, so you donāt have to name each selection.#2019-01-1416:50borkdudeIām not entirely sure if this maps to your issue with spec, but just my 2cts#2019-01-1416:50devthpretty sure that'd help!#2019-01-1416:51devthany word on timeline for the next version of spec?#2019-01-1416:54rapskalianWhenever Iāve seen this asked, the short answer is āwhen itās readyā š#2019-01-1416:55devthhaha. fair#2019-01-1416:56Alex Miller (Clojure team)Iām working on it every day#2019-01-1416:58Alex Miller (Clojure team)hard to say when the next point will be where itās useful to look at but it wonāt be āaā next version - itās going to be a series of releases#2019-01-1417:00devthawesome#2019-01-1417:00devthso to be clear: currently there isn't necessarily a better way to achieve what i'm doing? i have to dynamically s/def
a bunch of stuff?#2019-01-1417:01borkdude@devth there is a lib called spec-tools which may help you accomplish this, but I expect breaking changes when new spec comes out#2019-01-1417:01borkdudeI havenāt used it in anger myself#2019-01-1417:02devthlooks interesting, thanks.#2019-01-1417:02borkdude@devth https://github.com/metosin/spec-tools#transforming-nested-specs applies to your situation I think#2019-01-1417:03borkdudeyou may lose the ability to generate data this way, Iām not sure#2019-01-1417:04borkdude@devth o wait, maybe it was this: https://github.com/metosin/spec-tools#data-specs (the README is so long ;))#2019-01-1417:04devth> Just data, no macros
šÆ#2019-01-1417:04devthlong READMEs are nice š#2019-01-1417:06Alex Miller (Clojure team)most of things spec-tools is built on are going away, but there will be one or maybe even two alternative paths to this goal#2019-01-1417:06borkdudeIām not sure if I agree. Do one thing well also has benefits š#2019-01-1417:07devthnot familiar enough with spec-tools to have an opinion on whether it should have been multiple libs. but long searchable READMEs with lots of examples are always nice.#2019-01-1417:07devthi think i'll play with spec-tools for now. if things change i can always port to the new way#2019-01-1417:08borkdudeglad I could help š#2019-01-1417:13rapskalian(create-ns 'my.really.long.ns)
(alias 'mrln 'my.really.long.ns)
::mrln/attribute
Is this trick for shorter namespaced keywords likely to get me scolded by experienced clojure spec devs? Is it better to formally create the ns file even if itās largely empty?#2019-01-1417:14borkdude@cjsauer there is something like this in spec itself, so I think itās ok#2019-01-1417:15borkdudehttps://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L19#2019-01-1417:15bronsasee also https://dev.clojure.org/jira/browse/CLJ-2123#2019-01-1417:15borkdude@cjsauer if youāre writing portable code, be aware that this may not work on CLJS#2019-01-1417:17borkdude@cjsauer I tried moving this to a proper ns, so CLJS could use the alias as a namespaced keyword as well, but that was rejected:
https://dev.clojure.org/jira/browse/CLJ-2421#2019-01-1417:19rapskalianAh interesting, thank you. in-ns
is a bit better. Which part of this does cljs not like? I am hoping to use these specs on both server and client.#2019-01-1417:19borkdude@cjsauer in-ns simply doesnāt work in CLJS.#2019-01-1417:20borkdudeit does, but only in the REPL.#2019-01-1417:20rapskalianBummerā¦does the same go for create-ns
?#2019-01-1417:20borkdudeyes#2019-01-1417:20borkdude(I think so, gonna check now)#2019-01-1417:23borkdude(ns dude)
(in-ns 'foo)
(defn foo [x])
(in-ns 'dude)
(defn dude [x])
$ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.439"} org.clojure/test.check {:mvn/version "RELEASE"}}}' -m cljs.main -t node -c dude
WARNING: dude is a single segment namespace at line 1 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 3 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 7 /private/tmp/repro/src/dude.cljs
#2019-01-1417:25rapskalianShootā¦so then portable code will have to use the qualified keyword everywhere at the moment?#2019-01-1417:26borkdudethe fully qualified one yes#2019-01-1417:26borkdudedid that in CLJS: https://github.com/clojure/clojurescript/commit/731be5e0916ad8d619c302bfc9b985c4d10daa8d#2019-01-1417:28rapskalianI may actually fall back to formally creating the namespace file for now, even if its mostly empty. Keyword fatigue is realā¦#2019-01-1417:28borkdudeitās not a huge pain. you could make a function that creates the keyword if you want to save characters#2019-01-1417:29borkdude@cjsauer we also have that in the app Iām working one, several almost empty ns-es for thisā¦#2019-01-1417:30rapskalianOnce CLJ-2123 is patched, I imagine it should be quick work to refactor#2019-01-1417:30borkdudecool. hopefully CLJS gets this too#2019-01-1418:06rapskalianWhat is the philosophy behind specāing attributes that are intended to be used as references to other entities? For example, I might have ::company/employees
which models a one-to-many relationship. How would I spec this attribute? My intuition is to use something like (s/coll-of ::employee)
, but the issue here is that I now have to define some minimum key-set that is ::employee
, which is a spec that I think is largely contextual. Further, there may be contexts where (s/coll-of (s/tuple #{::employee/uuid} ::employee/uuid))
are valid ::company/employees
, e.g. datomic idents.
Ultimately what I think Iām wondering is: can reference attributes only be specād within a given context, or at specific boundaries (e.g. fdef
)?#2019-01-1419:17rapskalianI think my confusion stems from the fact that a :db.valueType/ref
attribute can take on lots of different forms throughout an application, and so itās difficult to spec. For example, it could be a raw entity id, a {:db/id 000}
map, a [:db/id 000]
ident, some kind of custom [:employee/uuid #uuid "000"]
ident, or it could be a fully formed employee mapā¦all these can values live under the same ref attribute at one time or another.#2019-01-1420:02favilaI think only spec-ing the pull form (ie what you would get from a pull)#2019-01-1420:02favilaThose other variants are all part of the Tx map DSL#2019-01-1420:03favilaThis just shifts the problem around though#2019-01-1420:03favilaI think the TX spec would only spec the coll not the keys individually #2019-01-1420:06rapskalian>I think only spec-ing the pull form (ie what you would get from a pull)
Iāve been playing with a similar strategy. This way you can still make good use of generators without the concept of a database getting involved.#2019-01-1420:12rapskalian>I think the TX spec would only spec the coll not the keys individually
@favila by this do you mean something like (s/coll-of (s/keys))
?#2019-01-1420:16favilaColl-of whatever a tx is#2019-01-1420:20favilaColl of either Vecs with entity ref (= 2-tuple, long, or keyword) first item (Tx fns and raw add/retract) or tx maps; which are maps with entity ref keys and Tx map or col-of-txmap or entity ref or non-ref value or coll of non-ref value)#2019-01-1420:21favilaSo pretty open...#2019-01-1420:28rapskalianIn my tx function fdefs Iāve been specifying the :ret
much more tightly. Iām less interested in specāing the :ret
of any tx function, and more interested in specāing this specific tx function.#2019-01-1420:35rapskalianIāve been staring at the code more, and I think whatās troubling is that when I want to specify the form of some ref attribute in general, it canāt really be doneā¦the question of when is inescapable so to speak. Because if I specify the pull form for all of my ref attributes, now Iāve bound the keyword :company/employees
to one specific context: the fully realized one. I canāt use the name :company/employees
in other contexts in which it might make sense, because spec has already registered the āone true formā. Does this make sense, or am I complicating the issue? š¤#2019-01-1420:37rapskalianFor example, a tx function might return a collection that includes a map with the :company/employees
keyword in it, but it may not conform to the pull form. It may be a collection of idents instead, meaning it references pre-existing entities.#2019-01-1420:38rapskalianBecause Iāve already specād it to conform to the pull form, I canāt use that keyword again in a new contextā¦#2019-01-1420:42lilactownAssuming it's a map using s/keys, you could make everything optional#2019-01-1420:43rapskalianI had the same idea, and it works okay. I think the downside is that it forces you to resort to custom generators pretty much everywhereā¦#2019-01-1420:44lilactownBut this is the exact problem Rich stated in his last conj keynote Maybe Not#2019-01-1420:44lilactownThere's not a great solution for it in spec#2019-01-1420:46rapskalian>But this is the exact problem Rich stated in his last conj keynote Maybe Not
I think so as well. Pretty much anywhere that a map has to be specified, this problem will arrive. The concept of āentityā is still difficult to grasp in my opinion. āEntityā doesnāt seem to become concrete until some data arrives at a boundary/function.#2019-01-1421:16favila@cjsauer you are exactly right about the problem#2019-01-1421:17favilaI don't think this is the same problem Rich has acknowledged#2019-01-1421:17favilaspec has a strong opinion that the keyword of a map is (essentially) describing the type of its contents#2019-01-1421:18favilabut there are cases where that is not true and the keyword is really a structure rather than a type tag#2019-01-1421:18favilaone is DSLs: e.g. tx map dsl, or even the pull expression dsl#2019-01-1421:19favilain those cases the key of the map is a structural tag, not a type tag#2019-01-1421:19favilaand foreign system's data routinely make their map entries contextual to the type of thing they are in#2019-01-1421:20favilaboth these use cases I have found are bad fits for spec; the only workaround is predicates#2019-01-1421:20favilaor more keys and lots of key renaming#2019-01-1421:21favila"structural tag" is wrong#2019-01-1421:21favilawhat I mean is they are like structurally-expressed arguments to some function call implied by the dsl#2019-01-1421:22favila{:my/attr [:db/id :db/ident]}
is not expressing a :my/attr
value, it's telling you to do something to a :my/attr
value in some different context#2019-01-1421:23favilathey can't be interpreted apart#2019-01-1421:23favilaso it's the type of the entry itself that matters rather than the type of the value#2019-01-1421:24favilaso, maybe multi-spec on the key as the dispatch value is getting towards the right answer?#2019-01-1421:52rapskalian>I donāt think this is the same problem Rich has acknowledged
@favila Iāve just rewatched Maybe Not, and I think youāre right, itās not the same issue. It is exactly the type/structural conflation that youāve identified.#2019-01-1421:54favilaif anything, rich is moving even further away from keywords having any contextual meaning#2019-01-1421:54favilas/keys at least let you attach predicates to the whole#2019-01-1421:57rapskalianHe actually also mentions that :ret
in function specs is starting to smell, and itās in those :ret
specs that I encountered this issue. So that may be tellingā¦#2019-01-1421:58rapskalianIf I ease up on trying to ānail everything downā with specs, as Rich puts it, it may alleviate some of these limitations#2019-01-1421:59borkdudeWhat did he say about ret specs again?#2019-01-1422:01rapskalianHe touches on it at around this point in the video: https://youtu.be/YR5WdGrpoug?t=3174#2019-01-1422:09borkdudewhat heās saying there is that ret specs donāt tell you much, you almost always want fn specs. so he wants to refine fn specs (make them easier to use?)#2019-01-1422:30rapskalianTrueā¦I think the root of it is definitely this ācontextual keysā idea (keywords have different specs in different contexts). What might be cool is the ability to define a ādisjunctive schemaā:
(s/def ::widget (s/or* :A string? :B pos-int?))
and then use a function similar to Richās proposed s/select
function in order to mask the contextually relevant predicates, e.g. (s/select* ::widget [:A])
would mean āin this context, only condition :A
can match (anything other than a string would result in error)ā.#2019-01-1422:33rapskalianAlmost reminds me of tools.depsā alias flag clojure -A:a:b:c
#2019-01-1501:04favila#2019-01-1501:05favilaThis was something I wrote to tackle the specific case of having a more general spec (as a key in a map) that context (what kind of map it is in) would narrow#2019-01-1501:06favilae.g. :a could be 1 2 or 3, but when it's in a map with :map-type "foo" :a must be 1 or 2#2019-01-1501:08faviladealing with that case over and over with predicates was a hassle, so this macro (keys+) allowed adding an additional keyword->spec mapping for validity, conformance, and generator purposes#2019-01-1501:08favilabut the original "natural", "contextless" spec still had to validate#2019-01-1501:09favilaso it's not aimed at the DSL use case, this was to deal with data from a foreign system that had a kind of tagging/inheritance relationship. their fields and entites had a base definition that was overly broad, and could be narrowed without changing their structure#2019-01-1501:10favilathink xsd facets where the element names don't change#2019-01-1501:11favilathe purpose was to allow generic processors to understand everything, but particular producers/consumers to add additional constraints about what they would produce/consume#2019-01-1501:11favilae.g. a field is cardinality-many, but some processor guarantees they only ever put one value in that field#2019-01-1512:25misha@cjsauer you can s/or
or s/alt
lookup-refs spec with s/keys
. Or avoid speccing maps with lookup-ref values (`{:my/attr [:db/id :db/ident]}`). Or, if you can isolate lookup-ref key-values from actual data key-values, spec lookup-refs with key-agnostic spec, something like (s/map-of qualified-keyword? lookup-ref-spec)
.
I don't think having s/select
would solve spec overload (data or lookup) in this case, because: https://clojurians.slack.com/archives/C1B1BB2Q3/p1547474311308100#2019-01-1512:27mishaAnother option is to have s/or
+ (s/map-of keyword? not-a-look-up-value?)
where you expect already looked-up values.#2019-01-1516:43ShaneLesterSo I have [org.clojure/test.check "0.9.0"]
in my dependencies, but when Iām trying to use generators with spec Iām still getting a clojure.test.check.generators never required
errorā¦. Anyone have an idea of what I should try? Is there a different one for cljs potentially?#2019-01-1516:52borkdudehave you tried requiring it manually?#2019-01-1516:52borkdude@shanelester55 are you on CLJS or CLJ and which version?#2019-01-1516:53ShaneLesterIām on CLJS. Using shadow-cljs soā¦ not positive which version. Whichever version it is currently using, which I imagine is the latest.#2019-01-1516:54ShaneLesterIs requiring it manually just specifiying it at the top of the file where it is used?#2019-01-1516:54borkdudeyou should be able to know the CLJS version in order to deal with known issues. can you try *clojurescript-version*
in the REPL?#2019-01-1516:57borkdudethere were some changes around this recently, but they are only on master, not released yet, thatās why Iād like to know#2019-01-1516:57ShaneLesterSure thing. trying that nowā¦ for some reason giving me trouble#2019-01-1516:58borkdudeif you donāt have a REPL, just print it the console from your app#2019-01-1516:59borkdudeor type cljs.core._STAR_clojurescript_version_STAR_
in the browser console#2019-01-1517:00ShaneLesterAh that worked easily enough. "1.10.439"
#2019-01-1517:01borkdudeok, how do you use the generators. using clojure.spec.test.alpha?#2019-01-1517:02ShaneLesterCurrently using these#2019-01-1517:02borkdudeok, I imagine you want to call gen/sample to just see some generated data right?#2019-01-1517:03ShaneLestercorrect#2019-01-1517:03borkdudeyouāll need a require to [clojure.test.check.generators] in your ns as well#2019-01-1517:04ShaneLesterAhhh that did it.#2019-01-1517:04ShaneLesterThank you very much#2019-01-1517:05borkdudeactually just [clojure.test.check] should work too#2019-01-1517:06borkdudesince it requires all the other needed namespaces#2019-01-1517:06ShaneLesterMakes sense. That indeed works as well.#2019-01-1517:07ShaneLesterThanks again š Works well now#2019-01-1517:08borkdudeEnjoy š#2019-01-1517:29devthreposting my notes on spec-related content in the 'Maybe Not' talk (https://www.youtube.com/watch?v=YR5WdGrpoug) since the thread is buried.
1. separating out schema (shape) and selection (optionality). cool stuff!
- it's still not "just data". why the departure? everything in Clojure is just data, and for good reason. the entire core lib is built around manipulating data. it's the best part of clj.
- it's still place oriented, at least from what i gather so far
2. Better programmatic manipulation of specs. great! but again: why not make it data? then programmatic manipulation comes with.
Imagine if the schema for:
{:weather {:weatherbitio {:key "xoxb" :default {:zip "98104"}}}
:command {:prefix "!"}}
Was simply:
{:weather {:weatherbitio {:key string? :default {:zip string?}}}
:command {:prefix string?}}
There's elegance in mirroring the data structure you're specifying, kinda like how Datomic Pull queries mirror the data you get get back.#2019-01-1520:47schmeebecause this: https://clojure.org/about/spec#_decomplect_mapskeysvalues, https://clojure.org/about/spec#_sets_maps_are_about_membership_thats_it#2019-01-1520:47schmeethe answers to most questions about āwhy doesnāt spec do Xā are on that page#2019-01-1520:49schmeeyou might be interested in this though: https://github.com/HealthSamurai/matcho#2019-01-1520:53jstaabI came here to ask a very similar question. I do agree with the two points you linked above, but it does seem very strange to use static place orientation to achieve it. I'd like to programmatically generate specs based on my own domain modeling dsl, rather than repeat my entity shapes multiple times, use spec as my domain modeling language, or write complex macros to do both at once. I've heard the objection of "why is it so hard to generate composed specs programmatically", and I still haven't heard a good answer.#2019-01-1520:55schmeetheyāre working on a new version of spec which is supposedly more data-oriented, there are some details about the development here, but nothing is released publicly yet: http://insideclojure.org/2019/01/11/journal/#2019-01-1520:55jstaabI've heard of that, I have high hopes š#2019-01-1606:19ikitommi@U066S8QGH spec-tools has a data-layer on top of specs like you described:
(require '[spec-tools.data-spec :as ds])
(require '[clojure.spec.alpha :as s])
(def weather-spec
(ds/spec
{:spec
{:weather {:weatherbitio {:key string? :default {:zip string?}}}
:command {:prefix string?}}
:name ::weather}))
(s/valid?
weather-spec
{:weather {:weatherbitio {:key "xoxb" :default {:zip "98104"}}}
:command {:prefix "!"}})
; true
just functions & data.#2019-01-1606:22ikitommithe forms are mapped with a multimethod, so string?
gets a 'clojure.core/string?
form. doesnāt help with anonymous functions.#2019-01-1615:01devthlooks good! i'm planning to use it#2019-01-1520:58dustingetzbut string?
is code, not data#2019-01-1520:58dustingetzthe idea of predicates is core to spec, i dont see how spec could ever be jsut data#2019-01-1520:59devthdata for edges, predicates for leaf nodes#2019-01-1520:59Alex Miller (Clojure team)the symbol string?
is data#2019-01-1520:59Alex Miller (Clojure team)the form (s/coll-of string?)
is also data#2019-01-1520:59dustingetzthatās thin#2019-01-1520:59Alex Miller (Clojure team)itās not thin, this is the whole basis of lisp#2019-01-1521:00dustingetzhomoiconicity is about macros#2019-01-1521:01jstaabIf you call describe
it does return a list of symbols at runtime, so spec isn't just confined to macros. Admittedly it could be hard to do anything with more complex predicates#2019-01-1521:02Alex Miller (Clojure team)thatās just ā¦ not true#2019-01-1521:03dustingetzfor string? to have meaning weād have to syntax quote it for starters, and now weāre in the clojure reader, not the edn reader#2019-01-1521:03Alex Miller (Clojure team)and spec forms will be even more central in spec 2#2019-01-1521:04Alex Miller (Clojure team)string? has meaning in the context of a namespace#2019-01-1521:04dustingetzso now we hve (ns) forms and are very solidly in the clojure reader, not edn reader#2019-01-1521:05dustingetzthatās how i udnerstand it anyway, always looking to deepen š#2019-01-1521:05Alex Miller (Clojure team)if you have fully qualified symbols, then the edn reader is sufficient#2019-01-1521:05Alex Miller (Clojure team)for example, s/form always give you that#2019-01-1521:06Alex Miller (Clojure team)we are also moving towards a world where spec forms are a data form, but may not be the only data form#2019-01-1521:06Alex Miller (Clojure team)so some āmapā data form could also exist#2019-01-1521:07Alex Miller (Clojure team)but thatās not more or less data than the list form (it may be easier to perform some kinds of transformations on)#2019-01-1521:08dustingetzyoāre saying (s/coll-of ...)
is sugar for something deeper?#2019-01-1521:08Alex Miller (Clojure team)no#2019-01-1521:09Alex Miller (Clojure team)Iām saying that (s/coll-of string?)
is data. Some map syntax {:op coll-of :pred string?}
(not a real syntax) is also data. both are sufficient to describe a spec.#2019-01-1521:09favilaI don't think this matters. the power and problem is the predicates. predicates can only be data if you know what all of them mean (independently of an implementation--a well-known list of predicates)#2019-01-1521:10Alex Miller (Clojure team)yes, that is essential in the design of spec which from day 1 was predicate based#2019-01-1521:11favilaand that is what people mean when they say "specs are not data"#2019-01-1521:11dustingetzi accept that a qualified symbol is equivalent to a qualified keyword ā but not a closure or fn reference, only symbols#2019-01-1521:11favila> the idea of predicates is core to spec, i dont see how spec could ever be jsut data#2019-01-1521:11favilathat's I think what you mean @dustingetz?#2019-01-1521:11favilacertainly how I understood it#2019-01-1521:12favilain my own thinking about the "dataness" of spec#2019-01-1521:12dustingetzAs soon as someone puts #(and x y) as a spec pred, itās all over, no logner data#2019-01-1521:12Alex Miller (Clojure team)well, thatās false and we support that now#2019-01-1521:12dustingetzhow can you serialize that#2019-01-1521:12Alex Miller (Clojure team)(fn [&] ...)
#2019-01-1521:12dustingetzits a reference#2019-01-1521:13dustingetzit could be jvm interop there#2019-01-1904:30seancorfieldThank you!#2019-01-1903:24seancorfield(originally posted in #announcements by accident!)#2019-01-1903:28madstapI see that s/keys
is still part of the api in spec2. I'm a bit surprised given RH's talk on how he wants to separate that out into s/schema
and s/select
. Are these going to be added in addition to s/keys
then or is s/keys
going away in favor of them in the next iteration? If they'll all exist together, will it ever make sense to use s/keys
over s/schema
/`s/select`?#2019-01-1903:28seancorfield@madstap That's a bit unfair -- the refactor to spec2 started before Conj.#2019-01-1903:28seancorfieldThe shape/required stuff is further down the pike.#2019-01-1903:29seancorfieldThe work in spec2 is groundwork to prepare for that.#2019-01-1903:30madstapSure, my question is more about the direction spec will take. Is s/keys
"living on borrowed time" so to speak?#2019-01-1903:33seancorfieldYeah, I'd say s/keys
is living on borrowed time. Spec is still alpha so it's subject to API breakage.#2019-01-1903:34seancorfieldGiven the namespace changes, you can have both libraries loaded and both types of specs defined while you transition.#2019-01-1903:34seancorfieldThat's the important part of namespace changes.#2019-01-1903:37madstapYeah, just like RH talked about in his penultimate(?) keynote. Pretty cool to see how that approach works out in practice.#2019-01-1903:40seancorfieldThis is why next.jdbc
will be a new namespace at least and a new artifact more likely š#2019-01-1904:23Alex Miller (Clojure team)Unknown yet on s/keys. We havenāt started any of the work yet for the stuff rich talked about at conj#2019-01-1904:25Alex Miller (Clojure team)Look, people complain when we work in a private repo, then drop something at the end. Weāre trying to do something different here which means you get to see all the steps along the way, so things will be changing over a probably months#2019-01-1904:26Alex Miller (Clojure team)Some of it will be experimental #2019-01-1904:27madstapI'm not complaining, I hope it didn't come across that way#2019-01-1904:27seancorfieldI love that you're dropping stuff we can test against.#2019-01-1904:27Alex Miller (Clojure team)Just saying, donāt expect it all at once#2019-01-1904:27seancorfieldI don't much care which way you're going with the API -- we'll follow, regardless, even if we have to keep changing code š#2019-01-1904:29seancorfield(we're living on the bleeding edge for a reason -- it gives us an edge!)#2019-01-1904:29madstapAnd this iteration solves one of the big pain points which were programmable specs#2019-01-1904:34madstapOne observation about that though: The way spec op macros are implemented now breaks clojure.walk/macroexpand-all
, which will stack overflow if the form contains any spec macros.#2019-01-1904:38seancorfieldThat's good feedback for the development of spec2. Last time I tried our code against spec2, a lot of stuff broke. I'll be trying it again on Monday when I get back to work. Breakage is to be expected in prerelease work and if folks pitch in and try it, the core team get more feedback which is helpful!#2019-01-1908:54borkdudeI have this spec in spec1:
#?(:clj (s/def ::java-map
(s/with-gen #(instance? java.util.Map %)
(fn [] (gen/fmap #(java.util.HashMap. %)
(s/gen ::map))))))
On spec2 I get:
Caused by: java.lang.IllegalArgumentException: no conversion to symbol
at clojure.core$symbol.invokeStatic(core.clj:596)
at clojure.core$symbol.invoke(core.clj:589)
at clojure.spec_alpha2$explicate_1$fn__904.invoke(spec_alpha2.clj:314)
I have to look into this more, I just briefly tried it#2019-01-1914:19Alex Miller (Clojure team)The key thing is that specs have to start off as forms, not function objects. So here the anonymous function is getting evaluated too early and I suspect you need to quote the (fn ... ). That may be fixable though in spec since I turned with-gen into a macro. Iāll take a look when Iām at a computer.#2019-01-1920:08borkdudethanks#2019-01-2011:01borkdudeQuoting the fn didnāt help#2019-01-1910:03borkdudesame with:
#?(:clj (s/def ::char-sequence
(s/with-gen
#(instance? java.lang.CharSequence %)
(fn []
(gen/one-of (map #(gen/fmap %
(s/gen ::string))
[#(StringBuffer. %)
#(StringBuilder. %)
#(java.nio.CharBuffer/wrap %)
#(String. %)]))))))
#2019-01-1910:06borkdudeThis spec breaks:
(defn seqable-of
"every is not designed to deal with seqable?, this is a way around it"
[spec]
(s/with-gen (s/and seqable?
(s/or :empty empty?
:seq (s/and (s/conformer seq)
(s/every spec))))
#(s/gen (s/nilable (s/every spec :kind coll?)))))
(s/def ::seqable-of-map-entry (seqable-of ::map-entry))
with the message:
Caused by: java.lang.IllegalArgumentException: No method in multimethod 'create-spec' for dispatch value: speculative.specs/seqable-of
#2019-01-1910:20borkdudeI tried to fix it like this: https://github.com/borkdude/speculative/blob/spec-alpha2/src/speculative/specs.cljc#L86
But I get:
user=> (s/valid? ::ss/seqable-of-map-entry {:a 1 :b 2})
Execution error (IllegalArgumentException) at clojure.spec-alpha2/pred-impl (spec_alpha2.clj:132).
Iāll leave it at this for now.
Broken code is indicated with FIXMEās here: https://github.com/borkdude/speculative/blob/spec-alpha2/src/speculative/specs.cljc#2019-01-1918:36lambdamHello,
I have a case where having the keys of a map spec forced to be namespaced qualified, leads to a case like this:
(defn do-this [args]
;; ...
{:foo :bar
:resource {:key-a 1}})
(s/def ::foo keyword?)
(s/def ::key-a int?)
(s/def ::resource (s/keys req-un [::key-a]))
(s/fdef do-this
:ret (s/keys :req-un [::foo ::resource]))
(defn do-that [args]
;; ...
{:foo :bar
:resource {:key-b "plop"}})
(s/def ::key-b string?)
;; (s/def ::resource (s/keys req-un [::key-b])) <-- problem here: the spec is being redefined
(s/fdef do-this
:ret (s/keys :req-un [::foo ::resource]))
Here I would like to have to functions in the same namespace that return two different "partial views" of a resource along with other information in a map.
It seems that I can't due to the fact that two pieces of information are glued together: the unqualified key and the spec.
I feel like I miss kind of a scoped spec declaration or a function like this : (s/key :resource <spec>)
that I can embed in the declaration: (s/keys :req-un [(s/key :resource <spec>) ::foo])
.
But I feel also that I am maybe misusing the library.
Has someone bumped into this problem already?
Thanks#2019-01-1919:58misha@dam since you use :req-un
in all 3 s/keys
, you can pick another namespace for second ::resource
:
(s/def :baz/resource (s/keys req-un [::key-a]))
(s/def :quux/resource (s/keys req-un [::key-b]))
(s/fdef do-this :ret (s/keys :req-un [::foo :baz/resource]))
(s/fdef do-that :ret (s/keys :req-un [::foo :quux/resource]))
#2019-01-2008:24lambdam@misha thanks for this solution. I did that in a way for something a bit different and the problem I bumped into was that you cannot require the namespace containing the specs in another namespace and use it with a shortcut. Let's say the previous code was in a namespace a
.
(ns b
(:require [myapp.lib.a :as a]))
(s/valid? ::a/foo <whatever>)
(s/valid? :quz/resource <whatever>) <-- I have to remember that this comes from namespace myapp.lib.a
and doing this
(ns b)
(s/valid? :quz/resource <whatever>)
the dependency graph of spec definitions is not right and the compiler will complain (it happened to me and I spent time renaming a lot of specs)
Thus, even though I can do it, I tend to feel that it's a clumsy solution.#2019-01-2013:01mishatying all specs' ns to file ns - is often not a good idea#2019-01-2008:50lambdamAnother question that comes to my mind, how can I validate a map whose keys are string (typically data from an HTML form).
Doing this (s/def ::form-data (s/keys :req ["foo" "bar"]))
is half of the spec. I'd like to qualify the content of the keys.
Again I feel like missing a syntax like this: (s/def ::form-data (s/keys :req [(s/key "foo" string?) (s/key "bar" int?)]))
.#2019-01-2009:37lmergeni keep running into an issue with managing the complexity of my specs, and was wondering whether i'm "doing it all wrong". specifically, of quite a few specs, i need to have multiple variations of the same "schema", depending on the context.
to give an example, a user should not need to have an ID before it is inserted into the database, but it does need an ID when it is.
this is a simple example, but this pattern repeats. since ideally, i want my code to be able to declare whether it expects a a "stored user" or not, i currently compose these things as follows:
(s/def :user (s/keys ...))
(s/def :user/inserted (s/merge :user (s/keys ...))
on the surface this seemed like a good idea, but after a while of using this, i discovered downsides to this. most importantly: as soon as you have other specs that depend upon inserted users or not (e.g. a "company" vs "inserted company"), you repeat this pattern a lot, even when there is not necessarily a difference between a "company" and "inserted company".
as an alternative to this, i was thinking of making the state an explicit property i can multi-spec on. this makes the approach more data-driven. but then i wouldn't be able to directly say, "i expect an inserted user" in the code through specs anymore.
are there any alternatives to this ?
i hope my question is making a bit of sense š#2019-01-2009:38lmergenthis code is a good example: https://github.com/lagenorhynque/spec-examples/blob/master/specs/clj/spec_examples/geometry/specs.clj#L13#2019-01-2009:39lmergennow, code cannot just fdef
anymore saying it expects a ::shape/cube
or a ::shape/sphere
, it can only fdef on ::shape
#2019-01-2009:45lmergenthis is another discussion on the same topic => https://lispcast.com/reduce-complexity-with-variants/#2019-01-2016:52ericnormand@lmergen I think Rich Hickey has noticed that problem, too#2019-01-2016:52ericnormandhe talked about it at the conj and hinted at a big change to spec to make this easier#2019-01-2016:52lmergenyeah I saw that, sounds very promising!#2019-01-2016:52ericnormandbasically, you define a map spec separately from the selection#2019-01-2016:53ericnormandwhich lets you separate out the definition of the entity from the context#2019-01-2016:53lmergenyes so you declare what you need rather than what something is#2019-01-2016:54ericnormandawesome#2019-01-2016:54lmergenok thanks I kind of forgot about that#2019-01-2016:54ericnormandi have faced this same problem myself#2019-01-2016:54ericnormandproliferation of specs because, for instance, if you have all the CRUD operations, they each require slightly different data#2019-01-2016:54ericnormandthough you'd like to think of them as operating on the same entity#2019-01-2016:55borkdudeitās the same problems that type systems without extensible records have#2019-01-2016:56lmergenyes #2019-01-2016:57lmergenso now spec becomes a category system :)#2019-01-2016:57borkdudewhatās that#2019-01-2016:57lmergensorry I meant category as in category theory#2019-01-2016:58lmergenrather than describing types you describe traits#2019-01-2016:59lmergenI am fairly sure I have just mentioned a few forbidden words #2019-01-2016:59borkdudeā awkward silence ā#2019-01-2016:59borkdudeš#2019-01-2016:59lmergenhaha#2019-01-2017:03Alex Miller (Clojure team)I think itās most useful to think of it as oriented around value sets, as described by predicates#2019-01-2017:04lmergenIāll be watching your progress alex, just found the new spec repo#2019-01-2017:05Alex Miller (Clojure team)maps are composites of their attributes and the key is that semantics are derived from the attributes, not from the composite itself#2019-01-2017:05lmergenso they become self describing ?#2019-01-2017:07Alex Miller (Clojure team)not sure thatās the important bit#2019-01-2017:08lmergentrue #2019-01-2017:10Alex Miller (Clojure team)the new idea from Richās last keynote is that different functions can speak more precisely about which subset of the composite it cares about, and we can have better tools for describing what a function does with those inputs wrt to the output#2019-01-2201:21kennyIs there any way to get more info about why a spec generator is failing? This is the error message:
clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {:pred #object[clojure.spec.alpha$gensub$fn__1876 0x7230d43f "
Nothing in the stacktrace points to my code. I guess I could go through and individually test all my specs for which failed but in this case the spec is huge and that would be very time consuming.#2019-01-2201:22gfredericks@kenny not an answer to your direct question, but that particular failure is usually caused by an s/and
, where one of the non-initial predicates is not likely to happen by chance#2019-01-2201:22gfredericks(or all of them together, more accurately)#2019-01-2201:22kennyRight. There's a lot of those š#2019-01-2201:24gfredericksit's a long outstanding issue between spec and test.check; test.check's alphas have a feature in such-that
that would let spec report what spec is causing it#2019-01-2201:24gfredericksbut spec doesn't use that feature yet afaik#2019-01-2201:27seancorfieldIt doesn't help you now @kenny but when I'm writing specs, I pretty much always have a (comment ..)
form with calls to s/exercise
for each of my specs, and I make sure they generate as I'm writing them... for exactly this reason.#2019-01-2201:31kennyI have some decent guesses for the ones that fail but the feature @gfredericks mentioned would significantly improve this experience. Then you could get rid of all those comment blocks @seancorfieldš#2019-01-2201:32kennyAny idea if there's a jira issue for this?#2019-01-2201:35seancorfieldI have a lot of Rich Comment Forms in my code š#2019-01-2201:36seancorfieldI find they're a good way to keep notes about my thinking as I develop, as well as containing expressions that produce test data and show usage of functions -- and specs š#2019-01-2213:12lmergenwhat would be the recommend way to do a multi-spec dispatch for some nested map variable ?
e.g. let's say that i have this
{:entity/type :user
:entity/data
{:user/id 1234}}
and i want to multi-spec the :entity/data
based on the :entity/type
... this seems pretty tricky ?#2019-01-2213:13lmergenbecause this is not possible, although it's something that i would want:
(s/def :entity/data (s/multi-spec entity-data :entity/type))
(s/def :entity (s/keys :req [:entity/type :entity/data]))
#2019-01-2213:16lmergennow i could repeat :entity/type
as a property of :entity/data
again, but that seems redundant#2019-01-2213:43jeroenvandijkIf you really have to, you can use multispec with the dispatch on type. Repeat this for all the different types: (s/def :your/data any?)
(s/valid?
(s/coll-of (s/or :type (s/tuple #{:entity/type} #{:user})
:data (s/tuple #{:entity/data}
:your/data))
:kind map?)
{:entity/type :user
:entity/data :foo})
#2019-01-2213:45jeroenvandijkIf you control the design of the data, but i guess you dont, there is probably a better way more in line with clojure.spec#2019-01-2214:31lmergenš¤ not sure whether this is brilliant or evil#2019-01-2214:31lmergenit works though#2019-01-2214:34lmergenhow would you redesign the data ? lift the data inside :entity/data
into the parent map so that multispec works ?#2019-01-2214:34lmergenhow would you redesign the data ? lift the data inside :entity/data
into the parent map so that multispec works ?#2019-01-2221:39jeroenvandijkyeah sorry for giving the evil hack š Changing the data will make it much cleaner. And using multispec for example gives you very nice specific validation errors e.g. when using it with expound#2019-01-2222:54lmergenyes thanks a lot, your comment was helpful!#2019-01-2223:23jeroenvandijkbtw, not using a namespaced keyword, so :data
instead of :entity/data
could also be used as a fix instead of lifting the data inside. You would use s/keys
with :req-un
instead of :req
#2019-01-2214:55lmergenactually that makes a lot of sense #2019-01-2219:48mishaqualified keywords allow you to have a mix of "unrelated" keys in a flat map#2019-01-2220:26lmergenyes, it is obvious that I was doing this wrong #2019-01-2220:26lmergenmakes a lot of sense #2019-01-2312:02victorbHm, I'm a bit confused about the s/or
. Trying to get spec to accept that a key can be either a-map
or b-map
, both of them are fine, so far got this:
(s/def :test/a-map (s/keys :req-un [::name]))
(s/def :test/b-map (s/keys :req-un [::title]))
(s/def :test/key (s/or :test/a-map :test/b-map))
(s/def :test/map (s/keys :req-un [:test/key]))
(comment
(s/valid? :test/map {}) ;; false
(s/valid? :test/map {:key {:name "hello"}}) ;; false / should be true
(s/valid? :test/map {:key {:title "hello"}}) ;; true
)
#2019-01-2312:03jeroenvandijkyou need to label it with a keyword, e.g. (s/def :test/key (s/or :a :test/a-map :b :test/b-map))
#2019-01-2312:04victorb@jeroenvandijk ooh, so simple. Thanks a lot for the fast help! š#2019-01-2312:05victorbugh, should have been obvious if I just read the docs slower, missed the key+pred pairs. I'll blame it on too little coffee#2019-01-2312:08jeroenvandijkI had the same once, no worries#2019-01-2312:09jeroenvandijkIt doesn't blow up either so hard to know you did something wrong (without coffee š )#2019-01-2406:26dacopareI'm trying to write a spec for a map that satisfies two keys specs.
(s/def ::a string?)
(s/def ::b string?)
(s/def ::a-map (s/keys :req [::a]))
(s/def ::b-map (s/keys :req [::b]))
;; Satisfies both specs
(s/def ::a-and-b (s/and ::a-map ::b-map))
;; Extend an existing spec
(s/def ::another string?)
(s/def ::a-extended
(s/and ::a (s/keys :req [::another])))
But generation obviously fails. Is there a way to do this that produces a good generator?
Perhaps using the and
section in s/keys
?
(s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
#2019-01-2406:27seancorfields/merge
#2019-01-2406:27seancorfieldItās designed to merge multiple s/keys
specs.#2019-01-2406:28dacopareThank you very much @seancorfield, it seems to be the perfect tool for the job.#2019-01-2412:55victorbHow can I spec a map where the keywords can be anything (basically IDs in this case) but I do know what I want the values to be#2019-01-2413:01aisamuSomething involving #{}
, perhaps?#2019-01-2413:04borkdude@victorbjelkholm429 what about (s/map-of any? #{:value1 :value2})
#2019-01-2413:06victorbThanks @aisamu and @borkdude. map-of any?
seems to be what I was looking for.#2019-01-2413:08victorbEnded up with (s/map-of keyword? :my/other-spec)
š#2019-01-2413:15fmjreyBetter to change your spec before coding than changing it after coding!#2019-01-2414:38victorb@fmjrey funny you say that as I currently sit and trying to retrofit clojure.spec after getting the basic mvp features in place š#2019-01-2510:54borkdude@alexmiller left a few comments about the new SHA here https://github.com/borkdude/speculative/issues/124#issuecomment-457532789 and here https://github.com/borkdude/speculative/issues/124#issuecomment-457533593#2019-01-2513:55Ben HammondI've just started playing with
.
I can see that the :args
are getting verified, but I cannot see how to get the :ret
verified#2019-01-2513:56Ben Hammondthis is the code that I pinched from https://blog.taylorwood.io/2017/10/15/fspec.html
(defn digits
"Takes just an int and returns the set of its digit characters."
[just-an-int]
(into #{} (str just-an-int)))
=> #'dev/digits
(s/fdef digits
:args (s/cat :just-an-int int?)
:ret (s/coll-of char? :kind set? :min-count 1))
=> dev/digits
(spec.test/instrument `digits)
=> [dev/digits]
(digits 1)
=> #{\1}
(digits "1")
Execution error - invalid arguments to dev/digits at (form-init1603011188380229902.clj:1).
"1" - failed: int? at: [:just-an-int]
#2019-01-2513:57Ben Hammondso far so good; now I want to modify the :ret
to something impossible and retest#2019-01-2513:57Ben Hammond(s/fdef digits
:args (s/cat :just-an-int int?)
:ret (s/coll-of keyword? :kind set? :min-count 1))
=> dev/digits
(digits 1)
=> #{\1}
#2019-01-2513:58Ben HammondHmm I was expecting the return value to fail.
I can see the change in the doc string
(doc digits)
-------------------------
dev/digits
([just-an-int])
Takes just an int and returns the set of its digit characters.
Spec
args: (cat :just-an-int int?)
ret: (coll-of keyword? :min-count 1 :kind set?)
=> nil
#2019-01-2513:59Ben Hammondwhat do I need to do to get the :ret
spec verified?#2019-01-2513:59borkdude@ben.hammond ret is not checked during instrumentation#2019-01-2513:59borkdudeit is during generative testing#2019-01-2513:59borkdudethis is FAQ number 1 I guess#2019-01-2514:00Ben Hammondis that a feature?
or a bug?#2019-01-2514:00borkdudedesign decision#2019-01-2514:01Ben Hammondokay, so I have to manually call (spec/conform
that's not a biggie
Thanks#2019-01-2514:01borkdude@ben.hammond this library can help you with checking examples: https://github.com/borkdude/respeced/blob/master/README.md#check-call#2019-01-2514:01borkdudeThereās also orchestra which turns on ret checking in instrumentation#2019-01-2514:02Ben HammondWell I was just trying to pace out my basic understanding
don't need to get too fancy at this point#2019-01-2514:02Ben Hammondthanks#2019-01-2514:05vemvis there a good predicate for checking if x
can be used as a spec?
should be true for int?
, (spec/spec ...), #{1 2 3}
#2019-01-2514:09Ben Hammond(doc spec/def)
-------------------------
clojure.spec.alpha/def
([k spec-form])
Macro
Given a namespace-qualified keyword or resolvable symbol k, and a
spec, spec-name, predicate or regex-op makes an entry in the
registry mapping k to the spec. Use nil to remove an entry in
the registry for k.
=> nil
#2019-01-2514:09Ben Hammondany predicate should be fine#2019-01-2514:09Ben Hammond(which will include int?
and hashsets)#2019-01-2514:10Ben Hammondor the keyword identifier of an existing spec#2019-01-2514:10Ben Hammondor an inline spec declaration#2019-01-2514:12Ben Hammondby 'regex-op` I think they mean combinations of (spec/+
, (spec/cat
, ,(spec/alt` etc..?#2019-01-2514:13Ben Hammondhow do you tell if an arbritary function is a predicate? I'm not sure if you can distinguish function arity from the outside#2019-01-2514:13Ben Hammondother than by provoking ArityExceptions#2019-01-2514:15Ben HammondI guess you could get java.lang.reflect
to tell you
and there is probably a clojurey wrapping somewhere#2019-01-2514:15vemvnot sure if we're talking about the same thing#2019-01-2514:16vemv(defn spec? "whether x can can be used as a spec" [x] ...)#2019-01-2514:16vemv^ I'm seeking this#2019-01-2514:16borkdude@vemv you want a predicate that tells you if a thing fits inside (s/fdef x --> ? <--)
?#2019-01-2514:17borkdudespec already has a spec?
predicate, but that doesnāt do the same#2019-01-2514:17vemvyeah, not s/spec?
#2019-01-2514:18danielnealifn
?#2019-01-2514:18vemvseeking a filler for (s/def ::foo ->>> x <<<-)
#2019-01-2514:18borkdudeI guess fn?
and keyword?
and spec?
maybe?#2019-01-2514:18vemv(s/or :ifn ifn? :spec s/spec?)
?#2019-01-2514:19borkdudemay work.#2019-01-2514:20vemvyeah, guess so, the fun part of my question was seeking a built-in that did that#2019-01-2514:35Alex Miller (Clojure team)thereās some stuff in CLJ-2112 (specs for specs) patch#2019-01-2514:41vemvthanks!#2019-01-2514:35Alex Miller (Clojure team)keep in mind that all of these answers will be different in spec2 :)#2019-01-2615:59borkdudehash-map
canāt be instrumented and then called in CLJ but it can be in CLJS.
$ clj -A:test -m cljs.main -re node
ClojureScript 1.10.439
cljs.user=> (require '[speculative.core])
nil
cljs.user=> (require '[clojure.spec.test.alpha :as stest])
nil
cljs.user=> (stest/instrument `hash-map)
[cljs.core/hash-map]
cljs.user=> (apply hash-map [1])
repl:13
throw e__6573__auto__;
^
Error: Call to #'cljs.core/hash-map did not conform to spec.
$ clj -A:test
Clojure 1.10.0
user=> (require '[speculative.core])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (stest/instrument `hash-map)
[clojure.core/hash-map]
user=> (apply hash-map [1])
Execution error (StackOverflowError) at (REPL:1).
null
#2019-01-2617:29borkdudeI donāt understand why, because this has never been the case before with CLJ. With CLJS these are pretty common, because it has no direct linking.#2019-01-2618:46borkdudeThe cause is probably that spec-checking-fn
calls with-instrument-disabled
, which calls binding
which expands in a call to hash-map
, so thereās a loop.#2019-01-2619:04borkdudeA possible solution would be to replace in the binding
macro:
(push-thread-bindings (hash-map
with
(push-thread-bindings (clojure.lang.PersistentHashMap/create ~(var-ize bindings)))
i.e. inline the hash-map
call.
Probably very low priority, but if thereās interest, Iāll make the JIRA ticket.#2019-01-2621:07Alex Miller (Clojure team)Donāt want to depend on the class like that#2019-01-2621:08Alex Miller (Clojure team)If anything would be better to use RT/map or whatever #2019-01-2622:11borkdudeRT/map is tricky, because then I need into-array
which is not defined yet#2019-01-2622:12borkdudeWhen I depend on a local version of spec.alpha (not spec-alpha2), I get:
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:382).
clojure.spec.alpha$explain_out
when I call an instrumented function with invalid args. No clue what this is.#2019-01-2622:15borkdudeWhen I install it with mvn
it works#2019-01-2622:21borkdudeit seems like spec.alpha is AOT-compiled?#2019-01-2701:49Alex Miller (Clojure team)Yes#2019-01-2701:49Alex Miller (Clojure team)But core.specs.alpha is not#2019-01-2809:55jeroenvandijk@borkdude maybe something ugly as this would work: (let [hash-map0 (var-get #'hash-map)]
(alter-var-root #'clojure.core/hash-map (constantly (fn [& x]
(with-redefs [clojure.core/hash-map hash-map0]
;; validation here?
(apply clojure.core/hash-map x)
)))))
#2019-01-2809:58borkdude@jeroenvandijk if youāre interested, I have a potential fix for hash-map not being able to be instrumented here:
https://github.com/borkdude/speculative/issues/264#2019-01-2809:58borkdudeIāll probably give up on it for now. CLJS has plenty of similar issues like this (instrumenting apply, seq, etc.)#2019-01-2809:59borkdudeIām not sure what your solution accomplishes, but you could add it to the issue with some context#2019-01-2810:01jeroenvandijkFrom what I understand, hash-map
calls hash-map
itself when instrumented. So I thought of aliasing hash-map
to the old version of hash-map
during the validation itself. Just an idea, maybe it's shortsighted#2019-01-2810:03borkduderight. the spec-checking-fn replacing hash-map calls itself (because binding emits a call to hash-map), so it ends up in a loop.#2019-01-2810:19jeroenvandijkMaybe a complete clojure.core
fix would be to have (stest/instrument x)
to have x
to be aliased to it's previous value so it cannot interfere with the instrumentation itself when x
is used for the instrumentation.#2019-01-2810:20borkdudethat would be the more fundamental solution yes#2019-01-2810:21borkdudeit would also speed up core instrumentation probably#2019-01-2817:35buttergunsHi. I'm experimenting with Spec. I think I'm approaching it (incorrectly) from an OO background. Suppose I have a spec ::vehicle #{:car :truck :semi-truck :bicycle :motorbike}
. It is part of a map (s/keys :req [::vehicle ::color])
.
At some time later in processing, I have a function that requires 2-wheeled vehicles, so I might have ::two-wheeled-vehicle (s/and ::car has-two-wheels?)
.
My problem is I've already used the ::vehicle
key for the more "generic" specification of a vehicle. I would need to create a new map with new keys (`(s/keys :req [::two-wheeled-vehicle ::color])`), and copy all the values over, in order to validate the stricter specification. Am I doing it wrong?#2019-01-2915:44jkrasnayIām trying to use s/keys
with a vector of keys, such that I can use the same vector of keys elsewhere:
(def my-keys [:emp/id :emp/name])
(s/def ::my-spec (s/keys :req my-keys))
ā¦but s/keys
complains. Is there any way to do this?#2019-01-2915:51taylorthere are changes coming to spec to make this easier, but for now you gotta eval
#2019-01-2915:45mpenet> (eval `(s/def ::my-spec (s/keys :req ~my-keys)))#2019-01-2915:49jkrasnayAh, cool. Thanks.#2019-01-3008:41Josip GracinHi! clojure.spec.test.alpha/check silently ignores symbols which are not "checkable". This seems counter-intuitive to me because I'd expect it to fail when I call check on a symbol and the symbol does not have an associated spec (e.g. it is misspelled).#2019-01-3008:41Josip Gracinthe definition of check looks like this:#2019-01-3008:41Josip Gracin(defn check
([] (check (checkable-syms)))
([sym-or-syms] (check sym-or-syms nil))
([sym-or-syms opts]
(->> (collectionize sym-or-syms)
(filter (checkable-syms opts))
(pmap
#(check-1 (sym->check-map %) opts)))))#2019-01-3008:42Josip Gracinit seems to me that the (filter (checkable-syms opts)) might better go to the no-arg variant#2019-01-3008:42Josip Gracindoes this make sense?#2019-01-3013:10Alex Miller (Clojure team)File a jira#2019-01-3013:11Josip Gracinok, tnx!#2019-01-3015:15mpingHi, is there any good way to xform specās :problems
into something that could make sense for a human over a json api?#2019-01-3016:07bbrinckI suppose it depends on how technical the user is, but you could potentially use Expound here. #2019-01-3016:10bbrinckThe default options might be too noisy (they include the specs), but this can be changed easily. You can also customize the messages in some cases. Let me know if you want help with configuring expound or just need some sample code for your use case. #2019-01-3016:15bbrinckPhrase is another option#2019-01-3016:36mpingtks, I basically want to convert the specās :problems
into something more tractable for a human#2019-01-3016:37mpingit seems that iād have to change *explain-out*
right?#2019-01-3016:38bbrinckYep, thatās what Expound does :)#2019-01-3016:39bbrinckYou can just replace explain-out with the printer provided by Expound #2019-01-3016:39bbrinckhttps://github.com/bhb/expound#2019-01-3023:31mishais there a rationalization why aliased specs can't be overwritten in spec1?
(s/def :user/foo string?)
(s/def :user/bar :user/foo) ;;alias
(s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})}) ;;=> (["" ""])
(s/exercise :user/bar 1 {:user/foo #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])
(s/def :user/bar string?) ;;not alias
(s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])
Can I somehow decouple gen-override call site from the knowledge of spec (not) being an alias?#2019-01-3023:37mishaSticking generator on the spec at s/def
time works, but I'd like to avoid nailing generator to spec.#2019-01-3023:42mishaOverride for alias with custom generator works though:
(s/def :user/foo string?)
(s/def :user/bar (s/with-gen :user/foo #(s/gen #{"bar"})))
(s/exercise :user/bar 1)
=> (["bar" "bar"])
(s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})})
=> (["quux" "quux"])
#2019-01-3023:43Alex Miller (Clojure team)itās a bug#2019-01-3023:43Alex Miller (Clojure team)thereās a ticket for it#2019-01-3023:43mishadoh#2019-01-3023:43mishashould I wait for spec2? or spec 1 gets things fixed too?#2019-01-3023:44Alex Miller (Clojure team)hard to say right now#2019-01-3023:44Alex Miller (Clojure team)I think I actually fixed it already in spec 2, but I havenāt tested it explicitly#2019-01-3023:45Alex Miller (Clojure team)but no release of spec 2 yet#2019-01-3023:45mishado you have any guestimate for closest spec2 release?#2019-01-3023:46Alex Miller (Clojure team)2019#2019-01-3023:46Alex Miller (Clojure team);)#2019-01-3023:47mishaopieop#2019-01-3100:04seancorfieldFWIW, from spec-alpha2
, latest SHA: user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def :user/foo string?)
:user/foo
user=> (s/def :user/bar :user/foo) ;;alias
:user/bar
user=> (s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})}) ;;=> (["" ""])
(["" ""])
user=> (s/exercise :user/bar 1 {:user/foo #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=> (s/def :user/bar string?) ;;not alias
:user/bar
user=> (s/exercise :user/bar 1 {:user/foo #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])
(["" ""])
user=>
#2019-01-3100:05seancorfieldand your custom generator example: user=> (s/def :user/foo string?)
:user/foo
user=> (s/def :user/bar (s/with-gen :user/foo #(s/gen #{"bar"})))
:user/bar
user=> (s/exercise :user/bar 1)
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=> (s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})})
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=>
#2019-01-3101:27Alex Miller (Clojure team)I havenāt changed anything since we last talked#2019-01-3103:49seancorfieldThat was for @misha not you @alexmiller, since he asked how his code would behave on spec2 š#2019-01-3104:00Alex Miller (Clojure team)ah, well the thing still to do on aliases for you affects it :)#2019-01-3118:57buttergunsI'd like to substitute a generator for an "inner" spec while testing. i.e.
;; src file
(s/def ::zip pos-int?)
(s/def ::address (s/keys :req [::zip]))
(s/def ::person (s/keys :req [::address]))
;; test file
(gen/sample (s/gen ::person) 1)
=> {::address {::zip 1}}
;; pseudo code
(gen/sample (redefine-a-spec (s/gen ::person) ::zip #{10001 10002}))
=> {::address {::zip 10001}}
How can I do this?#2019-01-3118:58taylors/gen
takes an overrides map#2019-01-3118:58borkdude@mattmorten stest/check
also takes an overrides map in case you need that#2019-01-3118:58taylorsomething like (s/gen ::person {::zip new-spec})
I think?#2019-01-3119:00buttergunsThat was even simpler than I hoped for. Thanks!#2019-01-3120:45buttergunsI have a related question. I have a coll-of ::animal
, where ::animal
is a multi-spec. How do I provide overrides to force the ::animal that is generated?#2019-01-3120:45butterguns(s/def ::animal-type #{:dog :cat})
(s/def ::cat-attribute #{"very catty"})
(s/def ::dog-attribute #{"dog dog"})
(defmulti animal-type ::animal-type)
(defmethod animal-type :cat [_] (s/keys :req [::cat-attribute ::animal-type]))
(defmethod animal-type :dog [_] (s/keys :req [::dog-attribute ::animal-type]))
(s/def ::animal (s/multi-spec animal-type ::animal-type))
(s/def ::animals (s/coll-of ::animal))
(s/def ::petshop (s/keys :req [::animals]))
(gen/sample (s/gen ::petshop) 1)
=>(#:{:animals [#:{:cat-attribute "very catty", :animal-type :cat}
#:{:dog-attribute "dog dog", :animal-type :dog}]})
;; How do I override s/gen to get a list of :cat's only?
(gen/sample (s/gen ::petshop {??}) 1)
#2019-01-3120:47borkdudeNarrator: and this was harder than he hoped for.#2019-01-3120:49borkdudeso the spec for cat is (s/keys :req [::cat-attribute ::animal-type])
?#2019-01-3120:50buttergunsyep#2019-01-3120:50borkdudeI donāt have experience with spec and multimethods, so Iāll shut up now#2019-01-3120:51Alex Miller (Clojure team)name the spec returned by the defmethod and override it?#2019-01-3120:52Alex Miller (Clojure team)or youāre looking for a particular branch of the multimethod to be poked?#2019-01-3120:53buttergunsI just need cats!#2019-01-3120:53Alex Miller (Clojure team)override ::animal with the generator for the spec returned by (animal-type {::animal-type :cat}) ?#2019-01-3120:53borkdudeis multi-spec
relevant here?#2019-01-3120:54buttergunsGenius#2019-01-3120:55butterguns(gen/sample (s/gen ::petshop {::animal #(s/gen (animal-type {::animal-type :cat}))}) 1)
#2019-02-0109:50borkdudeI would like to have a better solution for this problem:
https://github.com/borkdude/speculative/issues/276#2019-02-0111:12borkdudeIs there something like s/undef
in spec? I guess not?#2019-02-0111:15eval2020IIRC thereās a ticket for that#2019-02-0111:17borkdudeWhat I actually want is to define a an fdef that doesnāt get instrumented when calling (stest/instrument)
, but this may be too niche.#2019-02-0111:19eval2020I had this ticket in mind: https://dev.clojure.org/jira/browse/CLJ-2060#2019-02-0111:20borkdudeoooh, nice š#2019-02-0111:20borkdudethis is already in spec#2019-02-0111:21borkdudebut this doesnāt work for fdef?#2019-02-0111:22borkdude(s/def clojure.core/hash-map nil)
works#2019-02-0111:23borkdudeseems to work also in CLJS#2019-02-0111:23eval2020would expect fdefās to be put in the registry#2019-02-0111:25borkdudecool, I might be able to use this, thanks @eval2020!#2019-02-0201:12dronerunning into issues with circular references between specs (::a references ::b in itās definition and vice versa). taking a step back to consider alternative ways to model the specs, but will spec2 support referring to undefined specs? As in:
(s/def ::a ::b)
(s/def ::b <spec>)
#2019-02-0203:34Alex Miller (Clojure team)Yes, and I have done some more work there to defer lookups more pervasively#2019-02-0202:38kennyI'd be nice to use s/assert
in production but if you s/assert
a spec that contains a fspec
, it will require test.check to check the value. Is is possible to disable this functionality in production but still use the s/assert
check?#2019-02-0203:29seancorfieldHow would it check the spec then @kenny?#2019-02-0203:30seancorfield(and you've got the same problem if you use s/valid?
or s/conform
, right?)#2019-02-0203:34kennyfn? is all I need in prod. #2019-02-0203:34kennyItād be nice to get Spec error messages without the overhead of generation.#2019-02-0203:38seancorfield(s/valid? fn? thing)
#2019-02-0203:39seancorfieldIf you want checks in production, put them in the code explicitly with s/valid?
and/or s/conform
.#2019-02-0203:40seancorfieldAsserts in production are a code smell, IMO, since they throw an Error
, not an Exception
so you're basically going to "kill" your request/process.#2019-02-0203:45seancorfieldIf you want "Spec error messages" then you don't really want s/assert
-- you want explicit code to check validity and call explain
or something.#2019-02-0415:12daniel-tcgplayerHey everyone, I'm experiencing something odd trying to implement spec into our project for the first time. I'm using fdef to define my function's spec, however it's not validating my arguments. I'm using clojure 1.10. Here's the code:
(defn inc-num [x]
(inc x))
(s/fdef inc-num
:args (s/cat :x number?)
:ret number?)
Calling this function (inc-num "a") throws a ClassCastException instead of the spec error. Any ideas?#2019-02-0415:14taylordid you s/instrument
the function too?#2019-02-0415:15taylorthat's what enables the :args
checking at runtime#2019-02-0415:16daniel-tcgplayerI did not! Let me go ahead and do this, maybe I got ahead of myself in the documentation š#2019-02-0415:16borkdudeyes, writing an fdef does not turn on instrumentation automatically. also note when you re-define the fdef, you have to call instrument again to make it effective#2019-02-0415:18daniel-tcgplayerI see. So for all functions that I'm relaying on fdef to spec, I must also instrument those functions#2019-02-0415:19borkdudeyes#2019-02-0415:19borkdudeyou can instrument all function at once by calling (stest/instrument)
#2019-02-0415:19daniel-tcgplayerPer namespace?#2019-02-0415:19borkdudeif youāre using component, you might want to hook it up to the start/stop cycle. no, globally#2019-02-0415:20tayloryep that'll instrument everything that's been loaded, across all namespaces. there's also unstrument
to do the opposite#2019-02-0415:21daniel-tcgplayerAwesome! Thanks everyone for the quick feedback, I'm movin' again. I'll go through the documentation more thoroughly#2019-02-0415:22taylor@U6TUZTAAF I wrote some more function spec examples here too: https://blog.taylorwood.io/2017/10/15/fspec.html#2019-02-0415:25borkdudeIāve never used fspec. does spec really check the arity of such a function when you call a higher order function whose function argument is specāed with it?#2019-02-0415:27taylor@U04V15CAJ fdef
uses fspec
internally, so there's not much difference AFAIK. when you spec a function that takes another function as an argument, spec invokes the passed function ~20 times ā a kind of mini-check ā to see if it conforms#2019-02-0415:27borkdudespec invokes then function when?#2019-02-0415:27taylorwhenever you call the higher-order function, if it's instrumented#2019-02-0415:28borkdudeit doesnāt just call it when itās called normally?#2019-02-0415:29taylornot sure I understand the question#2019-02-0415:29taylor(defn f [g] (g 1))
(s/fdef f :args (s/cat :g (s/fspec :args (s/tuple int?))))
(st/instrument `f)
(f #(doto % prn inc))
-1
0
-2
...
59
-69770
-1165
1
=> 1
#2019-02-0415:31taylorso whenever you call f
(instrumented), spec is going to do a lil mini-check of the function you pass to f
to see if it conforms to the fspec
, and if it does then of course f
will call g
again#2019-02-0415:31borkdudewhy is that sane behavior? I can see this being useful when doing generative testing#2019-02-0415:33borkdudewutā¦
$ clj
Clojure 1.10.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (s/fdef clojure.core/keep :args (s/cat :f ::keep-fn :coll seqable?))
clojure.core/keep
user=> (s/def ::keep-fn (s/fspec :args (s/cat :x any?) :ret any?))
:user/keep-fn
user=> (stest/instrument `keep)
[clojure.core/keep]
user=> (keep inc [1 2 3])
Execution error (FileNotFoundException) at user/eval144 (REPL:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
user=>
#2019-02-0415:34taylorI think you need test.check on your classpath#2019-02-0415:34borkdudeyeah, I know why this happens, Iām just surprised that it suddenly wants to do generative testing while it can just conform while running, like normal functions#2019-02-0415:35taylorthere's no way to check an fspec
without invoking the function though#2019-02-0415:36borkdudeoh, now I see, because they are not replaced by an instrumented versionā¦ right.#2019-02-0415:36borkdudebut still, itās a little weird this.#2019-02-0415:37borkdudeyou would have to be very wary of using side effects in fspecāed functions#2019-02-0415:38borkdudeI havenāt used fspecs in speculative yet and this may be a reason Iām not going to..#2019-02-0415:42taylorI think the problem is more about speccing higher-order functions in general, than it is purely about fspec
, b/c fdef
is just a shorthand for s/def
+ s/fspec
. another option is to just use fn?
or ifn?
when you're speccing HOFs, that way spec doesn't mini-check them#2019-02-0415:42borkdudeyes, thatās what weāre doing now#2019-02-0415:43borkdudein most cases itās sufficient, although it would maybe nice to be able to speak about function arity#2019-02-0415:47taylorone case where I can see value in fspec
ing args to HOFs is when you check
the HOF, spec will pass it a dummy function that expects the right :args
and returns values generated according to the :ret
spec#2019-02-0415:48borkdudethat would be useful, but I want this to be pluggable. so my fdef would not use fspecs, but I do want to plug them instead of ifn?
during generative testing.#2019-02-0415:49borkdudeyou can do that by overriding spec generators#2019-02-0415:50borkdudebut overriding ifn?
would be a bit too global, since the args and ret could use ifn?
with different properties#2019-02-0415:50borkdudeso then you would have to give the argument list a named spec#2019-02-0415:50borkdudewhich I what I do sometimes#2019-02-0514:55dronechiming in that the generator requirement for HOFs also does not seem sane to me. should be able to instrument without assumption that youāre using generative testing#2019-02-0514:55borkdudeIād say so too. @alexmiller maybe should be an option to turn this off on spec2?#2019-02-0515:02Alex Miller (Clojure team)itās a topic weāll revisit#2019-02-0420:16hmaurerIs there a way quick way, without using an external lib, to generate a random value from a spec while limiting the max depth when the value is a map?#2019-02-0420:17hmaurere.g. I have nested s/keys
specs and I would like to generate a random value for the spec but limit the depth of the data structure#2019-02-0420:17hmaurerI know there are some libs on github that do this, but I am wondering if there is a short way to do it out of the box with specs#2019-02-0420:18tayloris it a recursive spec? if so, there's a *recursion-limit*
binding you can use during generation. the other thing that comes to mind is passing in a small/fixed size
arg to the sample
function#2019-02-0420:21hmaurer@taylor yes itās recursive, although not directly. The spec is modelling an AST#2019-02-0420:23taylornot sure if this would have the effect you want, but maybe https://gist.github.com/taylorwood/232129ccd3cb809281fea591d46f1b8a#file-infix-spec-clj-L58#2019-02-0420:23hmaurer@taylor Iāll try that, ty! Also I am trying to generate random values for this spec:
(s/def :elogic/term (s/and (s/keys :req [:elogic.term/type])
(s/multi-spec elogic-term-type :elogic.term/type)))
but I am getting āCouldnāt satisfy such-that predicate after 100 tries.ā#2019-02-0420:23hmaurerany idea how I can make it work?#2019-02-0420:25tayloryeah that's b/c the generator for your s/and
is only based on the first inner s/keys
spec, and the generated values from that don't conform to the second multi-spec
#2019-02-0420:25hmaurer@taylor ah. Any way around this?#2019-02-0420:25tayloryou can wrap that s/and
with s/with-gen
to specify a custom generator, or override the generator elsewhere#2019-02-0420:26taylorsome of the spec functions take an overrides
map for this#2019-02-0421:01hmaureris there a way to define a spec on a string without seqāing it first?#2019-02-0421:01hmaurera regex spec#2019-02-0421:04Alex Miller (Clojure team)no#2019-02-0421:08hmaurer@alexmiller is it a generally bad idea to use specs to parse strings?#2019-02-0421:08Alex Miller (Clojure team)itās not recommended, but I literally canāt stop you :)#2019-02-0421:08Alex Miller (Clojure team)itās not recommended, but I literally canāt stop you :)#2019-02-0421:09hmaurerš what is the recommendation against it?#2019-02-0421:31souenzzoDo your own spec-inspired (and maybe backended by spec) library to specify string?!?!#2019-02-0421:33borkdudeI recently found this lib: https://github.com/miner/strgen
Itās cool, but maybe comes with limitations#2019-02-0421:33hmaurer@U2J4FRT2T not a bad idea š I would need to first learn more about how spec works though#2019-02-0421:33borkdudeThe nice thing about using spec for strings is that you get nice error messages and can generate examples.
For better error messages one could also use a parser combinator or parser generator.#2019-02-0421:35hmaurer@U04V15CAJ being able to generate examples is definitly useful#2019-02-0421:38Alex Miller (Clojure team)sorry, was on a call#2019-02-0421:38Alex Miller (Clojure team)the point is that spec regex were designed to spec sequential collections#2019-02-0421:39Alex Miller (Clojure team)strings are not sequential collections and the performance of the regex specs on them might be bad#2019-02-0421:39Alex Miller (Clojure team)and there are quite good options for parsing strings (namely, the regex built into the JDK and exposed by Clojure)#2019-02-0421:40Alex Miller (Clojure team)test.chuck has a function that generates strings from regexes https://github.com/gfredericks/test.chuck if you need that#2019-02-0422:32seancorfieldFWIW, we use test.chuck
s string regex generator quite a bit -- we're very happy with that, combined with regular string regex. I would not recommend trying to use spec's sequence regexes on a string.#2019-02-0422:53borkdude@U04V70XH6 good to hear. how does it compare to minerās lib? I see his name in the README of test.chuck. Was his lib merged into test.chuck?#2019-02-0422:53borkdude@U052852ES I think this thread is worth reading since you asked me about a related problem youāre solving with spec#2019-02-0423:12souenzzoWe can compile specs into regexp at macro time š#2019-02-0500:35seancorfield@U04V15CAJ No idea. First I've heard of Steve Miner's regex lib.#2019-02-0507:55thomas@U04V15CAJ I have seen it and I will have a look again, thank you.#2019-02-0508:27borkdudeI meant the test.chuck regex thing#2019-02-0509:25Vincent CantinWhere do clojurists usually place their fdef
?
- In the same namespace of their functions?
- In the same namespace with ā.specā prepended?
- In a separate, global /src/spec/
namespace tree?
- other?#2019-02-0511:02n2oI usually put it directly below the function definition, but i am not really happy with it to be honest.#2019-02-0511:16Vincent CantinWhy arenāt you fully satisfied with it?#2019-02-0512:04n2othe source code feels a bit messy. Simple clean functions, then a new definition below it just for specs... Just a personal impression. Some kind of choppy in my opinion#2019-02-0512:05n2oI tried to put them into a separate namespace but as you have to instrument them to really use the specs, the specs are like to be forgotten. So this does not seem to be a good solution#2019-02-0512:05n2oI then tried to put them below my source code, but in the same namespace (at the bottom of the file). I almost forgot all of them and was wondering, that I wrote specs for those functions when I scrolled down the file after some months. So this is also not a good solution#2019-02-0512:07n2oCurrently I put them directly together, so I don't forget them
(defn foo [x y] ,,,)
(s/fdef foo
:args (s/cat :x number? :y pos-int?))
#2019-02-0517:47seancorfieldIf I want the code to be usable without spec (i.e., on Clojure 1.8), I put fdef
s in a separate namespace (see clojure.java.jdbc
for example). Otherwise I put the fdef
immediately above the function it refers to.#2019-02-0609:10n2oOk, in my case I do not need to be downwards-compatible. But yes, then you don't have a different choice.#2019-02-0509:34borkdude@vincent.cantin The answer is: up to you. There is no ābest wayā regarding this. Personally, I co-locate it with the defn
.#2019-02-0513:53abdullahibraHello guys#2019-02-0513:53abdullahibrahow could i check the values of map only using clojure.spec, i know how can i do this for map keys names + values, but i'm in situation which i don't know what key names are but i know what types of values should be ?#2019-02-0513:53borkdudee.g. for checking string values: (s/map-of any? string?)
#2019-02-0513:53abdullahibrashould handle the values using s/coll-of#2019-02-0513:54abdullahibraah great#2019-02-0513:54abdullahibra@borkdude thanks#2019-02-0513:56abdullahibrais clojure.spec good alternative to type system in language like Haskell, ocaml ?#2019-02-0520:25hmaurermost definitely not#2019-02-0520:25hmaurerš#2019-02-0520:25hmaureritās very different and by no mean a substitute for static typing (as much as I like spec)#2019-02-0513:57borkdudeit has different applications, everythingās runtime, so you get no compile time guarantees, macros being the exception#2019-02-0513:58borkdudebut itās also more flexible than a type system. different trade offs.#2019-02-0514:06manutter51Also spec is still evolving, there are going to be changes, especially concerning how maps are specāed.#2019-02-0514:15victorbfor more details about the changes: https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-02-0514:22borkdudewould it make sense to co-develop the CLJS version of spec2 in the same repo and pull it outside CLJS itself, so CLJS never is behind on spec?#2019-02-0514:22borkdudelike test.check#2019-02-0514:26borkdudeam I right that you canāt use anonymous functions in specs anymore in spec2 so they would always have to be defined top-level? what about with-gen?#2019-02-0514:26borkdudethanks for the link @victorbjelkholm429, useful information#2019-02-0514:29Alex Miller (Clojure team)I do not think you should be tracking spec 2 changes in cljs yet - still in flux#2019-02-0514:30Alex Miller (Clojure team)you can use anonymous functions#2019-02-0514:30Alex Miller (Clojure team)what about with-gen?#2019-02-0514:31borkdudecan?#2019-02-0514:31Alex Miller (Clojure team)?#2019-02-0514:32borkdudeIām reading about āsymbolic specsā, it seems that you cannot use anonymous functions in there?#2019-02-0514:32Alex Miller (Clojure team)you can#2019-02-0514:32borkdudeSymbolic specs consist only of:
Spec forms (lists/seqs), with spec op in function position, composed of other symbolic specs
Qualified keywords (names that can be looked up in the registry)
Qualified symbols (predicate function references)
Sets of constant values (an enumeration)
#2019-02-0514:33Alex Miller (Clojure team)spec op can be fn
#2019-02-0514:33Alex Miller (Clojure team)so those are valid spec forms#2019-02-0514:33borkdudeah ok. I was confused, because of feedback you had earlier, I thought it wasnāt allowed anymore, but then I misunderstood#2019-02-0514:34Alex Miller (Clojure team)thereās a picture at http://insideclojure.org/2019/02/02/journal/ if that helps#2019-02-0519:54taylorI really like the music notes at the end of each post! There's some good lesser-known stuff on Zeppelin's In Through the Out Door too š are you a fan of The Meters?#2019-02-0519:54Alex Miller (Clojure team)yep and yep#2019-02-0519:56Alex Miller (Clojure team)been listening to the first Meters album a lot lately actually :)#2019-02-0514:34Alex Miller (Clojure team)it has a note about fn in the middle there#2019-02-0514:35Alex Miller (Clojure team)if you want to use anonymous functions at the top level with the api functions, you need to wrap them in s/spec#2019-02-0514:37borkdudealright. I donāt know how much in flux things are, but if you want me to give it a try again with speculative, let me know.#2019-02-0514:38Alex Miller (Clojure team)(s/valid? (s/spec #(> % 5)) 10)
#2019-02-0514:39Alex Miller (Clojure team)Iāve got at least one thing to fix still from Seanās feedback#2019-02-0514:39Alex Miller (Clojure team)and weāre working through best direction for some of the things you saw last time still#2019-02-0514:40Alex Miller (Clojure team)so probably not worth doing anything yet#2019-02-0523:05dronemaybe using spec in an unintended way, but I ran into a situation where there were two spec forms in an s/and
, the result of the first conformed the input and the second spec form in the s/and
was a predicate expecting the original input, not the modified/conformed form. it feels like validation should be independent of conforming#2019-02-0523:06droneand I donāt understand the purpose of conforming. is it intended to be like schemaās coercion? or is it to produce some intermediate form useful for spec-checking?#2019-02-0523:46Alex Miller (Clojure team)Re and, we have long considered having both āflowingā and nonflowing variants of and#2019-02-0523:46Alex Miller (Clojure team)And that might still happen in spec 2#2019-02-0523:47Alex Miller (Clojure team)The purpose of confirming is not coercion - itās to tell you which choices were made during validation for optional specs or components#2019-02-0523:48Alex Miller (Clojure team)That is, why was it valid?#2019-02-0523:58droneSounds good. The flowing was definitely surprising.#2019-02-0523:58droneThanks, got it#2019-02-0523:58droneA lot of design decisions #2019-02-0603:22flyboarderSo this is a spec error I ran into today and Im here thinking there must be a better way to get to the root of the problem#2019-02-0603:22flyboarder#2019-02-0603:34taylorwhatās on line 28 in bootstrap/feature.clj? is that your project?#2019-02-0603:36flyboarder(defn- feature-dispatch [[k v]] k)
#2019-02-0603:37tayloritās probably the destructuring of the first arg, maybe a keyword is being passed as the first arg#2019-02-0603:37flyboarder^ I was able to solve this the problem was
(spec/explain-data ::feature-gate feature)
when I needed
(spec/explain-data ::feature-gate [feature])
#2019-02-0603:37flyboarder@taylor ^#2019-02-0613:02borkdude@alexmiller I updated this comment just now: https://github.com/borkdude/speculative/issues/124#issuecomment-460944994#2019-02-0620:27borkdude@alexmiller do you plan to support this for spec-alpha2 as well? https://dev.clojure.org/jira/browse/CLJ-2060
I noticed it does not yet work in CLJS.#2019-02-0620:29Alex Miller (Clojure team)should work, havenāt looked at why yet, but wasnāt intentionally broken#2019-02-0620:29borkdudehttps://dev.clojure.org/jira/browse/CLJS-3049#2019-02-0621:19seancorfieldFYI, I just ran out entire test suite against the latest spec2 and there's just one failure (for s/valid?
on a large data structure for a very complex, nested spec) -- which was the unexplained failure I had before. The generator problems I mentioned before are fixed by the latest work.#2019-02-0621:28seancorfield@alexmiller What in spec (or spec2) causes this predicate to be used? (or (nil? %) (sequential? %))
#2019-02-0621:29seancorfield(it's not in our code so I assume it's what underlies s/cat
or something?)#2019-02-0621:48Alex Miller (Clojure team)I think thatās checked for all regex specs#2019-02-0621:48Alex Miller (Clojure team)yeah#2019-02-0621:48Alex Miller (Clojure team)I assume you see that fail?#2019-02-0621:51seancorfieldI've tracked it down a bit further... (def s [ ... sequence of hash maps ...])
(count s) ;;=> 6
(s/explain ::ba-decline (take 3 s)) ;;=> Success!
(s/explain ::ba-decline (drop 3 s)) ;;=> Success!
(s/explain (s/cat :one ::ba-decline :two ::ba-decline) s)
;; fails, claiming that the first element of s does not satify: (or (nil? %) (sequential? %)) in: [0] at: [:one]
#2019-02-0621:52seancorfield(this works with spec1)#2019-02-0621:53seancorfieldAm I missing something obvious about combining regex specs?#2019-02-0621:53seancorfieldThe ::ba-decline
spec is an s/cat
of three items.#2019-02-0621:54seancorfieldI'll see if I can create a minimal example that fails, based on this (unfortunately there are a lot of complex specs behind this).#2019-02-0622:03seancorfield@alexmiller (s/def ::x string?)
(s/def ::y int?)
(s/def ::z keyword?)
(s/def ::a (s/keys :req-un [::x]))
(s/def ::b (s/keys :req-un [::y]))
(s/def ::c (s/keys :req-un [::z]))
(s/def ::abc (s/cat :a ::a :b ::b :c ::c))
(def a {:x "x"})
(def b {:y 42})
(def c {:z :bar})
(s/explain ::abc [a b c])
(s/explain (s/cat :one ::abc :two ::abc) [a b c a b c])
works on spec1, fails on spec2#2019-02-0622:03seancorfieldon spec2 {:x "x"} - failed: (or (nil? %) (sequential? %)) in: [0] at: [:one]
#2019-02-0623:07Alex Miller (Clojure team)user=> (s/explain (s/cat :one ::abc :two ::abc) [[a b c] [a b c]])
Success!
certainly is suspicious :)#2019-02-0623:08seancorfieldIt's almost like regex specs don't unroll anymore! :rolling_on_the_floor_laughing:#2019-02-0623:14Alex Miller (Clojure team)I may have actually introduced that by introducing the delayed resolution of aliased specs#2019-02-0623:17Alex Miller (Clojure team)basically regex specs in the registry are shielded by the keyword (which is not a regex spec), so the code doesnāt believe they can be combined#2019-02-0623:18Alex Miller (Clojure team)so this may fight with the last changes - I canāt remember now if that was from forward references or from something about gens?#2019-02-0623:18seancorfieldThis was broken before you fixed either of those I believe. Certainly broken before the gensub fix.#2019-02-0623:19Alex Miller (Clojure team)well good to know then :)#2019-02-0623:19seancorfieldAlthough, with a quick clj -Sdeps ...
it's enough to test that repro case against any version of spec2 š#2019-02-0622:16Alex Miller (Clojure team)Iāll take a look, seems buggy to me#2019-02-0622:17Alex Miller (Clojure team)There is a lot gnarly code in that form/object transition. I could easily have broken something subtle#2019-02-0622:17Alex Miller (Clojure team)Thanks for the repro#2019-02-0622:18seancorfieldI was lucky that the first simple repro I tried still broke š#2019-02-0622:19Alex Miller (Clojure team)Hey, thatās a good sign#2019-02-0622:21seancorfieldAt this point, it really is a pretty minimal set of changes in our code to go from spec1 to spec2. It's frustrating that we can no longer use comp
and partial
to construct predicates and have to resort to #(..)
or (fn [x] ..)
but, fortunately, that didn't affect much code.#2019-02-0622:23seancorfieldWe have a few places we're having to use s/spec*
now so the "helper macros" mentioned in your latest journal will help clean that up when they drop in GitHub.#2019-02-0622:54Alex Miller (Clojure team)yeah, that has further solidified today and I think will be very useful#2019-02-0622:31borkdudeI still have at least 3 bugs with spec alpha 2 (see issue link above)#2019-02-0622:36seancorfieldInteresting. I never knew you could define a spec to nil
to make it go away...#2019-02-0622:37borkdudeItās a fairly recent addition.#2019-02-0622:52borkdudeMake that 2. The (s/def spec nil)
was a mistake on my part, I had to call unstrument
first (which failed because ::s/invalid
is not a valid any?
which I ran into)#2019-02-0622:52Alex Miller (Clojure team)oh good :)#2019-02-0622:52borkdudeundefining still works:
$ clj -A:test
Clojure 1.10.0
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (require '[clojure.spec-alpha2.test :as stest])
nil
user=> (defn foo [x] x)
#'user/foo
user=> (s/fdef foo :args (s/cat :x number?) :ret number?)
user/foo
user=> (stest/instrument `foo)
[user/foo]
user=> (foo "a")
Execution error - invalid arguments to user/foo at (test.clj:129).
"a" - failed: number? at: [:x]
user=> (stest/unstrument)
[user/foo]
user=> (s/def foo nil)
user/foo
user=> (stest/instrument `foo)
[]
#2019-02-0712:30borkdudeI thought I had to make upgrading existing spec libraries less painful is to have something like a reader conditional that can dispatch on the version of spec.
E.g.:
#?(:spec2 (:require [clojure.spec-alpha2] :as s) :default (:require [clojure.spec.alpha :as s]))
#?(:spec2 (s/def ā¦) :default (s/def ā¦))
This way a library can maintain compatibility with both versions of spec#2019-02-0716:23dominicm(try
(require '[clojure.spec-alpha2 :as s])
(catch Exception e
(require '[clojure.spec.alpha :as s])))
No? š#2019-02-0716:23dominicmdoesn't work for cljs though#2019-02-0716:23borkdudethatās the big deal, CLJS#2019-02-0716:24borkdudewe could use a goog-define for it in CLJS, but then still you cannot do anything on the namespace declaration level#2019-02-0716:25dominicmit's a shame that it doesn't work in cljs given the work that was done to make require
work there.#2019-02-0716:25borkdudethatās only for self-hosted AFAIK#2019-02-0716:25borkdudeand REPL usage#2019-02-0716:26borkdudeanyway, it would be nice to have a way have libraries support both versions. I bet it will be 2 branches and versions with ā-spec2ā suffixes for a while#2019-02-0716:27dominicmyeah, I think you're right#2019-02-0716:28dominicmyou could use a macro, and perform the require
voodoo at the clojure level perhaps š¤#2019-02-0716:28dominicmbut still not great#2019-02-0716:29borkdudeoh right, I think I saw someone āabuseā a data reader for that recently. Maybe it was @U050PJ2EU?#2019-02-0716:29dominicmit would work in shadow-cljs, which has the ability to specify custom :keywords which run#2019-02-0713:49borkdudewill spec-alpha2 be released at the same time as clojure 1.11 as one bundle?#2019-02-0713:54Alex Miller (Clojure team)unknown#2019-02-0713:59borkdudeI have yet to test the latest SHA. Will update you, probably in the weekend.#2019-02-0714:05Alex Miller (Clojure team)no worries#2019-02-0714:05Alex Miller (Clojure team)good catches#2019-02-0718:55joefromctcan anyone point out something i may have missedā¦ if i try to generate from a s/keys spec un-req
i get all blank mapsā¦ but if i use the same with req
i can generate data. Do you need a custom generator for anything un-req
?#2019-02-0718:56Alex Miller (Clojure team)do you mean req-un ?#2019-02-0719:03seancorfield@joefromct user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::a int?)
:user/a
user=> (s/def ::b string?)
:user/b
user=> (s/exercise (s/keys :req-un [::a ::b]))
([{:a -1, :b ""} {:a -1, :b ""}] [{:a 0, :b ""} {:a 0, :b ""}] [{:a -1, :b "75"} {:a -1, :b "75"}] [{:a 0, :b "zW7"} {:a 0, :b "zW7"}] [{:a -3, :b ""} {:a -3, :b ""}] [{:a -2, :b "Hlf"} {:a -2, :b "Hlf"}] [{:a 11, :b "M8r"} {:a 11, :b "M8r"}] [{:a -4, :b "K9Dn"} {:a -4, :b "K9Dn"}] [{:a 44, :b "o2rAl"} {:a 44, :b "o2rAl"}] [{:a -4, :b "SzHY1pTSN"} {:a -4, :b "SzHY1pTSN"}])
user=>
#2019-02-0719:03joefromctha, yup. that was it. very funny.#2019-02-0719:06joefromctattention to detail is important. thanks.
clojure
(s/def :my-test/name (s/and string? #(not= "" %)))
(s/def :my-test/developer (s/keys :req [:my-test/name ]))
(s/exercise :my-test/developer 2)
([#:my-test{:name "W"} #:my-test{:name "W"}] [#:my-test{:name "7"} #:my-test{:name "7"}])
;; d'oh `un-req` not a thing.
(s/def :my-test/developer (s/keys :un-req [:my-test/name ]))
(s/exercise :my-test/developer 2)
([{} {}] [{} {}])
#2019-02-0719:07seancorfieldI'm a bit surprised s/keys
doesn't complain that all its known options are missing user=> (s/exercise (s/keys))
([{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}])
user=>
#2019-02-0719:09Alex Miller (Clojure team)s/keys is a valid spec that will validate all registered keys in the map#2019-02-0719:09joefromctyeah i guess i did come across in the guide the mention of (s/keys) was valid and i guess the rule to fnās are āextra keys no-big-dealā#2019-02-0719:09Alex Miller (Clojure team)but of course it doesnāt know how to gen#2019-02-0719:10Alex Miller (Clojure team)just use this mnemonic ā¦ āI req-un the map should have a key like thisā#2019-02-0719:59jimbobā¦.nice#2019-02-0720:00jimbobwhats the idiomatic way to mandate presence of keys?
related: whats also the idiomatic way to mandate the lack of extraneous / undefined keys?#2019-02-0720:01jimbobi suppose for spec you can do something like (s/and (contains % #{mandated-keys})
#2019-02-0720:02borkdudemandate the presence of keys: req-un
not allow extra keys: this is not according to the philosophy of spec/Clojure I think#2019-02-0720:02jimbobinteresting. thanks#2019-02-0720:03taylorthere are some code snippets around for disallowing "extra" keys though @ben.borders even though it's counter to the design philosophy#2019-02-0720:03borkdudeyou can always hack around the design philosophy of Clojure š#2019-02-0720:04jimbobright, i figured.. seems like i should probably try to allign more with the philosophy#2019-02-0720:04jimbobiām sure there are valid reasons for that.#2019-02-0720:04jimbobwould just like to read about it i suppose#2019-02-0720:04Alex Miller (Clojure team)Rich did a whole talk about it#2019-02-0720:05Alex Miller (Clojure team)well, this covers a lot of other stuff, but there is a section in here: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md#2019-02-0720:06Alex Miller (Clojure team)grep for ācode for growthā#2019-02-0720:06jimbobthank you#2019-02-0720:06taylorI think one of the metaphors is "if you're expecting a truck to deliver your TV, you shouldn't care what else is on the truck" or something like that#2019-02-0720:06borkdudeThere have been some conversations about what spec is and isnāt suited for. E.g. it isnāt designed for coercion or āclosed worldā assumptions (not allowing extra keys), e.g. using it as API boundary protection#2019-02-0720:07Alex Miller (Clojure team)I think the greater point is actually that you shouldnāt design your systems that way, and if you donāt, spec will mesh well :)#2019-02-0720:08jimbobright.. we donāt now.. we allow our maps to have whatever values they have, and we grab the ones we care about#2019-02-0720:08jimbobwhich tends to be most or all of them.#2019-02-0720:08borkdudewell, sometimes I donāt want to have crap in my jsonb field, but thereās always select-keys, etc.#2019-02-0720:08jimbobright.#2019-02-0720:08Alex Miller (Clojure team)and you might be interested in the new stuff coming in spec 2 around that https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/MaybeNot.md#2019-02-0720:55seancorfield> I think the greater point is actually that you shouldnāt design your systems that way, and if you donāt, spec will mesh well :)
It's fine for a (REST) API to be passed a bunch of parameters it doesn't care about. It can just ignore them. Then spec works well for parameter validation.#2019-02-1013:14Jp SoaresI started to use REST API with compojure-api 2.0.0 that can coerce data with spec and generate swagger docs from it. I thought it would be the trend for specs, but reading this conversation maybe not.
It's better to use compojure-api with schema to follow the clojure philosophy in REST?#2019-02-1018:24seancorfieldI don't understand your question : what do you mean "clojure philosophy in REST"?#2019-02-0720:57seancorfield> well, sometimes I donāt want to have crap in my jsonb field, but thereās always select-keys, etc.
And you can pull the "known" keys out of a spec fairly easily (usually!) so that you can narrow your set of fields down to just what's in the spec (we do this around database insertion where we have a spec for the row representation of data).#2019-02-0721:00joefromcthow would a spec look for the output (https://github.com/clojure/clojure/blob/master/src/clj/clojure/xml.clj#L78) from clojure.xml\parse
look?
Below is what iām guessing at so far, but the ::content references ::element and vice versaā¦ not sure how to handle this. (Iām not worried about namespaces at the moment):
I think its tricky for me because itās a hierarchy.
(s/def ::tag keyword?)
(s/def ::attrs (s/map-of keyword? string?
:conform-keys true))
(s/def ::content-strings (s/coll-of string?))
(s/def ::content-maps (s/coll-of ::element?))
(s/def ::content
(s/or :content-map ::content-maps
:content-string ::content-strings ))
(s/def ::element (s/keys :req-un [::tag]
:opt-un [::content ::attrs]))
#2019-02-0723:45butterguns(gen/sample (s/gen string?) 10000)
#2019-02-0723:45butterguns...only gives me alphanumeric, for 10,000 iterations. Why is that?#2019-02-0723:46buttergunsNo symbols / unicode#2019-02-0723:47seancorfieldThat's probably all the string?
generator is coded to produce.#2019-02-0723:48seancorfieldtest.chuck
has a regex string generator that can produce a much wider range of character values, if you need to test that.#2019-02-0723:48buttergunsHmmm. I was trying to follow this blog post: https://lispcast.com/testing-stateful-and-concurrent-systems-using-test-check/#2019-02-0723:48seancorfieldFor example, here are some generated "email" strings, using that generator: (["č@Z.Gp" "č@Z.Gp"] ["ņ«@OG.lr" "ņ«@OG.lr"] ["\"ņ
Ø\"@z.DtS.K.Qpu" "\"ņ
Ø\"@z.DtS.K.Qpu"] ["ńŖń.ņ²Øņ²ņ ”ó¶@2N8.SuYn.FY.zBf" "ńŖń.ņ²Øņ²ņ ”ó¶@2N8.SuYn.FY.zBf"] ["ń±¢ó®¢ń¬§ņ¹ ō²±@3.QBB" "ń±¢ó®¢ń¬§ņ¹ ō²±@3.QBB"] ["šø£ńØ®¤ņ£ä±ó„¼.ņ¾.ņ®.󦚣©·ó½±šø.ģó
ń®¶ō¹šń¶.šŗ»ó£ńó
ņ”ņ¢@[3.61.526.313]" "šø£ńØ®¤ņ£ä±ó„¼.ņ¾.ņ®.󦚣©·ó½±šø.ģó
ń®¶ō¹šń¶.šŗ»ó£ńó
ņ”ņ¢@[3.61.526.313]"] ["\"ń¹š°šæŖµņ\"@[78.35.80.6]" "\"ń¹š°šæŖµņ\"@[78.35.80.6]"] ["\"ń¬ņØøń»\"@[00.5.0.8]" "\"ń¬ņØøń»\"@[00.5.0.8]"] ["\"ń±©\"@-Npu4W.unVx6R.DdNnk.4.A6tM.y.EJFHfw92N.ecYbisymo" "\"ń±©\"@-Npu4W.unVx6R.DdNnk.4.A6tM.y.EJFHfw92N.ecYbisymo"] ["\"ņµŖ¶ņ¤š¶ņš«ń¤¬ņ¼ńŗ·ń §ņŖ³\"@[0.881.6.929]" "\"ņµŖ¶ņ¤š¶ņš«ń¤¬ņ¼ńŗ·ń §ņŖ³\"@[0.881.6.929]"])
#2019-02-0723:49seancorfield(my editor does not render them all correctly š)#2019-02-0723:49butterguns... scrolling down to "Lets set up some Generators", he uses gen/string
instead of string?
and it produces the full range#2019-02-0723:50seancorfieldAh, there you go!#2019-02-0723:50buttergunsWould be nice not to have to override all my string?
specs with gen/string
however š#2019-02-0723:52seancorfieldI guess it depends on how important it is that you exercise your code with more exotic strings?#2019-02-0723:53buttergunsYeah, I guess so. It just surprised me. Testing against exotic strings strikes me as exactly within generative testing's wheelhouse#2019-02-0723:54seancorfieldWell... it is but this is a case where you need to opt into it.#2019-02-0723:57seancorfieldIt really depends on what parts of your data processing you need to stress test. If you only pass a string through your system and into the DB, you don't really care whether it's alphanumeric or all poop emojis.#2019-02-0723:58seancorfieldIf you have specific string processing that does some sort of parsing, it might well be useful to stress test it with really exotic strings.#2019-02-0723:58seancorfieldNot everything is important to test to the same level.#2019-02-0723:59seancorfieldIt might be more useful, for example, to use a string generator that only ever produces an empty string, a short readable string, or a giant long string.#2019-02-0800:01buttergunsThat's a good point. I think this perhaps exposes a weakness in my understanding of generative testing. My brain says "what will this really test that can't be better accomplished with targeted unit testing"#2019-02-0800:01buttergunswacky strings is often my go-to answer#2019-02-0800:02buttergunsBut this is a larger question, somewhat off topic#2019-02-0800:04seancorfieldMy experience so far leads me to think that the power of generative testing is more in the combinations of data it uses to exercise code rather than the specific values of an individual item.#2019-02-0800:08buttergunsCan you elaborate? To that point: I feel like that is what makes gen testing so difficult. E.g. I may have a "Command" that has two parameters. But only a very narrow combination of those two parameters actually make sense. Otherwise the "Command" is just gibberish.#2019-02-0800:09buttergunsAnd when I start using (gen/elements ["very" "small" "number" "of" "valid" "inputs"])
it sorta doesn't feel like generative testing anymore#2019-02-0800:15seancorfieldIf you have interdependent parameters, that's certainly a bit trickier to test. You'd need to write a spec for the possible combinations that were valid, and make sure it's generatable.#2019-02-0800:17seancorfieldAn example from one of our apps. We have a spec for the set/sequence of valid interactions a user (an admin in this case) can have with a sequence of data. Given that (complex!) spec, we can generate arbitrary but valid combinations of operations the user could run. We use those generated combinations of operations to drive an HtmlUnit test to verify that a) they are actually possible and don't produce errors and b) that the end state for the transformed sequence of data is still valid (using its own set of specs).#2019-02-0800:18seancorfieldAutomated UAT of a web app, via generative testing š#2019-02-0800:23buttergunsAh, ok this makes a lot of sense! I think I'm getting hung up on the idea that I have to check the result of applying these operations, in literal sense. Like, "after a generated operation, I should inspect the operation and foo should == bar". It sounds like you're saying "after a generated operation, make sure everything conforms to spec, plus no errors thrown, and then just move on"#2019-02-0800:36seancorfieldIt's really going to depend on your specific system under test. In another part of our system, we have certain pairs of functions that are inverses of each other, so for those we'll do full-on property-based testing to ensure that, for arbitrary "valid input", calling (= (inverse-fn (some-fn some-input)) some-input)
.#2019-02-0800:38seancorfieldAnd then in other situations, you might want to specifically test handling of bad data so you might write a spec for some of the things that your input cannot be, and then run tests to ensure that for arbitrary "bad input" you get some sort of appropriate failure response (i.e., that given bad input you don't get a success response).#2019-02-0800:38seancorfield(that might help you detect combinations of bad input that you unexpectedly allow through your app -- a gap in your validation)#2019-02-0801:27ericnormandjohn hughes (creator of quickcheck and now property-based testing consultant) always talks about how prop-based testing is exploratory#2019-02-0801:27ericnormandyou model the system you are checking, and often the tests donāt pass#2019-02-0801:28ericnormandyou investigate and find that your test was missing something#2019-02-0801:28ericnormandso you modify it and run it again#2019-02-0801:28ericnormandit fails again, it was missing something else#2019-02-0801:29ericnormandmy point is that property based testing always requires work to model the system#2019-02-0801:29ericnormandthere is no āone wayā to test#2019-02-0801:30ericnormandi think property based testing is a good name. youāre looking for algebraic properties to test for#2019-02-0801:31ericnormand@seancorfieldās example of inverses is a good property#2019-02-0801:31ericnormandanother is idempotence#2019-02-0801:31seancorfieldYup, you often start with a property you believe should hold and sometimes you discover it doesn't -- so either the property is wrong or your system doesn't preserve when it should š#2019-02-0801:32ericnormandcommutativity, associativity, identity, zero#2019-02-0801:32ericnormandall good properties to start with#2019-02-0801:32ericnormandthen you can get creative#2019-02-0801:32ericnormandas in āitās commutative under these conditionsā#2019-02-0801:33ericnormandor itās commutative under this comparison operator#2019-02-0801:33ericnormand(instead of =)#2019-02-0801:35ericnormandas for generators, i have made use of three#2019-02-0801:35seancorfieldThis output value should always exceed this input value. The output should always be an ordered sequence (regardless of its actual contents). The output value(s) should always be between these bounds (possibly based on input values). Etc.#2019-02-0801:35ericnormandalways valid data, junk data, and almost valid data#2019-02-0801:35ericnormandthe almost valid is the hardest to model#2019-02-0801:36ericnormandfor me at least#2019-02-0801:37ericnormanditās almost as if you need to start with valid data and break it#2019-02-0801:37ericnormandin a random way#2019-02-0801:37ericnormandthose are to test error conditions as sean was saying#2019-02-0801:38seancorfieldMutation testing.#2019-02-0801:38ericnormandah!#2019-02-0801:38ericnormanddoes such a thing exist in clojure?#2019-02-0801:39seancorfieldIn theory, you could build such a system (to mutate code). Easier to mutate input data and see if your system breaks in unexpected ways š#2019-02-0801:39ericnormandyes#2019-02-0801:40ericnormandif d doesnāt pass spec, f should throw an exception#2019-02-0801:40ericnormanditās hard to test that thoroughly#2019-02-0801:41ericnormandto feel confident youāve found all of the ways you could do the data wrong #2019-02-0801:41ericnormandit would be interesting to make a generator that could mutate any other value randomly#2019-02-0801:42ericnormandadd or subtract 1 from numbers#2019-02-0801:42ericnormandadd a char to a string#2019-02-0801:42ericnormanddrop an element from a list#2019-02-0801:42ericnormandetc#2019-02-0815:10borkdude@alexmiller one down. https://github.com/borkdude/speculative/issues/124#issuecomment-461833318#2019-02-0815:11borkdudeItās probably an issue with not fully qualifying something in a macro#2019-02-0815:14borkdudemight be this one: https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/spec_alpha2.clj#L931#2019-02-0815:28Alex Miller (Clojure team)I fixed it, then broke it again :) Iāve reverted my last changes#2019-02-0815:38borkdudeit works now, except for some error messages that look strange (see issue). I think weāve now seen the first time that all speculative specs can be instrumented. š#2019-02-0815:41borkdudeI have to port respeced to spec-alpha2 before I can run the test suite, Iāll do that sometime soon#2019-02-0815:46Alex Miller (Clojure team)Iām not getting whatās weird about the errors#2019-02-0815:47borkdudeok.
(atom 1 {:validator 1})
Execution error - invalid arguments to clojure.core/atom at (test.clj:129).
{:validator 1} - failed: keyword? at: [:options :clojure.spec-alpha2.impl/k]
The failed keyword should be something like :atom/validator
#2019-02-0815:47borkdudeI think?#2019-02-0815:48Alex Miller (Clojure team)spec 1 version of the first one is the same#2019-02-0815:48Alex Miller (Clojure team)user=> (atom 1 {:validator 1})
Execution error - invalid arguments to clojure.core/atom at (REPL:1).
{:validator 1} - failed: keyword? at: [:options :clojure.spec.alpha/k]
#2019-02-0815:48Alex Miller (Clojure team)this may be a pre-existing weakness of keys*#2019-02-0815:48borkdudeinteresting, Iāll try#2019-02-0815:48Alex Miller (Clojure team)whatās weird about the dissoc one?#2019-02-0815:49borkdudeOn spec 1 I get:
user=> (dissoc 1)
Execution error - invalid arguments to clojure.core/dissoc at (REPL:1).
1 - failed: map? at: [:map :clojure.spec.alpha/pred] spec: :speculative.specs/map
1 - failed: nil? at: [:map :clojure.spec.alpha/nil]
On spec 2 I get:
user=> (dissoc 1)
Execution error - invalid arguments to clojure.core/dissoc at (test.clj:129).
1 - failed: map? at: [:map :clojure.spec-alpha2/pred]
1 - failed: nil? at: [:map :clojure.spec-alpha2/nil]
#2019-02-0815:50borkdudeso itās clear which of my own specs the argument is violating#2019-02-0815:50Alex Miller (Clojure team)so this bit: spec: :speculative.specs/map
#2019-02-0815:50borkdudeI guess so.#2019-02-0815:51borkdudeMaybe it would also be good for expound to test this behavior, since it relies on things like this (@bbrinck)#2019-02-0815:52Alex Miller (Clojure team)yeah, Iām sure there are some subtle things like this that have broken, particularly in regex world#2019-02-0815:52borkdudeI donāt have specific demands for this, but expound probably has. Nowās the chance to get it fixed š#2019-02-0815:54borkdudeAre spec objects that are defined in the registry backward compatible with spec1? I donāt mean how you are defining them, but the actual result of defining a spec?#2019-02-0815:54Alex Miller (Clojure team)no#2019-02-0815:54bbrinck@borkdude Good idea! I will start a spec2 branch soon for expound to see what works and what needs changes.#2019-02-0815:54Alex Miller (Clojure team)for the moment they are the same protocol, but that protocol is in a different ns so not compatible#2019-02-0815:55Alex Miller (Clojure team)migration/release is another whole thing - weāve talked about it at length but not going to worry about it for now till weāre much closer#2019-02-0815:55borkdude@alexmiller if that were true, I think there would be a more smooth upgrade path. spec2 would just register it in both the old and new registry and old consumers would still work#2019-02-0815:55borkdudeah ok, makes sense.#2019-02-0815:56bbrinckIām wondering if expound may need to depend on spec1 and spec2 at the same time. Even if expound has different namespaces that use spec1 and spec2, presumably a lib could upgrade to spec2 but still use a lib that uses spec1. #2019-02-0815:57Alex Miller (Clojure team)that should be fine (presuming you have a version of clojure aware of both registries, etc)#2019-02-0815:57Alex Miller (Clojure team)the clojure integration aspects will require a clojure release - lots of open questions about exactly what that will cover still tbd#2019-02-0815:58bbrinckYeah#2019-02-0815:59borkdudeIāve thought about supporting two versions of the lib, one with a -spec2 suffix in the version. Could work.#2019-02-0816:00bbrinckThat makes sense, as you think about migration/release it would be useful for lib authors to come up with some practices as well, informed by your decisions. I suspect many libs will be in the situation where they want to be spec2 compat but must assume that some deps use spec1. #2019-02-0816:00borkdudebut then again, thereās only one or two places right now where I needed to change something? the finicky difference is the namespace declarations#2019-02-0816:01borkdudeanyway, now it probably not the time
> but not going to worry about it for now till weāre much closer
š#2019-02-0816:01bbrinckIf all spec-libs require both spec1 and spec2, then at least the incremental cost of using any spec-lib wonāt be that much :)#2019-02-0816:04borkdudeAnd then we have CLJS in the mix as well, so if you write CLJC there is that to consider too#2019-02-0818:59gklijsIs there a cljs way to statisfy inst?
I used it in a cljc spec and ran into trouble reading the java Instant, then just switched to number, but there might be a nicer way?#2019-02-0819:01gklijsIs this it? https://cljs.github.io/api/syntax/inst-literal#2019-02-0819:14lilactownare you asking how to make a value that is an inst
? AFAIK it's just a (js/Date.)
#2019-02-0819:32gklijsI should explain in more detail. I have a list of images in de back-end (Clojure) with just an id of the image and the time it was uploaded for now. I want to spec and use this list in the front-end (Clojurescript). I already have some (de)serialize code based on the spec to not repeat the keys everywhere. So I think I could get away with transforming the Clojure Inst to a string in such a way it's easy for Clojurescript to make it into a js/Date. Which gives me my answer by using a DateTimeFormatter.ISO_INSTANT
to format the Instant to string, it should be possible to use Instant.parse(x)
to get back the Instant while at the same time in Clojurescript I can use #inst x
to get the js/Date.#2019-02-0819:36lilactownif I understand correctly, what you are trying to figure out is how to transfer the instant over the wire?#2019-02-0819:36lilactownfrom the back-end to the front-end?#2019-02-0819:37lilactownFWIW, you can't call #inst x
where x
is a var or some other binding; #inst
is a literal, so you'd have to write the date literally #inst "1980-01-01"
#2019-02-0819:38lilactownwhat I would suggest is using something like EDN or Transit to communicate between your back end and front-end, so that they can transfer the inst
literally without having to format to a string and then re-parse the string#2019-02-0819:57gklijsI already use EDN, but that goes wrong, because there is no Instant object in cljs. You where right about the literal, I need (js/Date. x)
to create it.#2019-02-0819:57lilactownare you reading the EDN in?#2019-02-0819:57gklijsyes#2019-02-0819:58lilactownyou might just need to provide the correct arguments to the reader#2019-02-0819:59lilactownhttps://github.com/Lokeh/punk/blob/e6bb3b719571e79a6540597db50163fc4b9d8a4f/ui/src/punk/ui/core.cljs#L464#2019-02-0820:03gklijsThanks, might work better. I actually use reader/read-string
now..#2019-02-0819:40Frank HenardHey everyone... question about using s/merge
with s/multi-spec
: https://stackoverflow.com/q/54599149/59439
I put it on SO for easier findability#2019-02-0822:12borkdude@alexmiller I now ported my testing library to spec-alpha2, this works.
Now Iām running the tests of speculative and I ran into a bug which I can reproduce as follows:
user=> (require '[clojure.spec-alpha2.gen :as gen])
nil
user=> (gen/sample (s/gen any?))
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval189$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.core$any_QMARK_
#2019-02-0822:13borkdudeI can probably workaround this by using the test.check generator directly, but just fyi.#2019-02-0822:36borkdudeThis does work: (gen/sample (gen/gen-for-pred string?))
#2019-02-0822:40borkdudeThis too:
user=> (gen/sample (s/gen (s/spec string?)))
("" "I" "" "4Cv" "" "Cs" "" "864w" "4" "w8TIh7")
#2019-02-0822:59seancorfielduser=> (gen/sample (s/gen (s/spec any?)))
([] nil [] () {[] ()} () nil nil {} ([{\M :nK21+I.s*-J_ce9P+.-B!ll?._0?!07zMs*.k4N-y_A.?T?b.V*!mC/c-m-?k7, \q .z?} [:?57*-_?*x0:wg!!*v66+:+!Nwv:?*:Vti3*78I7:2_EVDZ8Dij:1:xC0 -1.375]]))
user=>
#2019-02-0823:00seancorfieldBecause any?
isn't a symbol, so you need s/spec
now.#2019-02-0823:00seancorfieldThis seems expected to me, given the spec1 -> spec2 changes.#2019-02-0823:00seancorfield^ @borkdude#2019-02-0823:18borkdudeIām not 100% clear on this. E.g. it says in https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha
> Symbolic specs consist only of:
ā¦
> Qualified symbols (predicate function references)
If Iām reading that, I would say that
(gen/sample (s/gen `any?))
should work since itās a qualified symbol referencing a predicate function?#2019-02-0823:20borkdudeIt might as well be an accidental breaking change. Iāll wonder what Alex thinks. Iām off to bed now.#2019-02-0823:41seancorfieldGiven that (s/valid? any? :foo)
etc all reject any?
(and string?
etc), I think it's pretty clear it's deliberate @borkdude#2019-02-0823:42seancorfieldany?
isn't a spec, it's a predicate. You can define a spec in terms of a predicate. So (s/def ::foo any?)
is valid and then (s/exercise ::foo)
will be valid but (s/exercise any?)
is not.#2019-02-0823:43seancorfieldAnd then (s/spec any?)
produces a spec from a predicate. So (s/exercise (s/spec any?))
is valid.#2019-02-0823:44seancorfieldThe line between specs and predicates is very blurred in spec1.#2019-02-0900:43Alex Miller (Clojure team)Sean is correct. s/gen takes a spec object. A predicate symbol or function is not a spec object. Invoking s/spec with a predicate will expand into a qualified symbolic spec and pass it through spec*, yielding a spec object#2019-02-0900:43Alex Miller (Clojure team)So (s/spec any?) or you can do the work of the spec expander and do (s/spec* `any?)#2019-02-0902:07buttergunsQuick question: any way to supply a seed to gen/sample
?#2019-02-0902:09buttergunsI want to do some performance testing / benchmarking. Hence getting a predictable sample is critical#2019-02-0902:28seancorfield@mattmorten It sounds like you really need to provide your own overridden generators that produce the same sequence over and over again?#2019-02-0902:36buttergunsHmmm. That would involve a lot of duplication. In my test ns, I have 20+ lines of generator code to produce a complex data-structure in my app. I'm very happy with it. I'd rather say "give me seed 1 of this generator" than go back and write duplicate generators for each of my (many, very nested) attributes that produce a constant value#2019-02-0905:11flyboarderWhy doesnāt spec/assert let me set my own message, seems like an optional 3rd arg would be really convenient#2019-02-0907:33borkdude@seancorfield @alexmiller alright, thanks. Now that this is clear Iāll change those #2019-02-0910:00borkdudeFound a new bug:
speculative.specs=> (s/def ::or-spec (s/with-gen (s/or :vector vector) #(s/gen (s/spec vector?))))
:speculative.specs/or-spec
speculative.specs=> (s/valid? ::or-spec [1 2 3])
false
#2019-02-0910:00borkdudewithout s/with-gen
it works, with it doesnāt#2019-02-1105:07Alex Miller (Clojure team)fixed in latest#2019-02-0910:03borkdudeIt also works when not using s/or
+ s/with-gen
so it seems s/or
specific#2019-02-0910:24borkdudeI bumped into another one, but this might be related, so Iāll wait for this one to be resolved#2019-02-0913:42Alex Miller (Clojure team)You have vector in the spec, not vector?#2019-02-0913:42Alex Miller (Clojure team)Not sure thatās the cause but seems like a thing to fix first#2019-02-0915:37borkdudeStill doesnāt work with vector?
.#2019-02-1011:24borkdudeThis works in spec1 and 2: (gen/sample (s/gen (s/every number? :kind vector?)))
This works in spec1 but doesnāt work in spec2. Should it? (gen/sample (s/gen (s/every number? :kind coll?)))
#2019-02-1014:14Alex Miller (Clojure team)I think so? Not sure why it wouldnāt #2019-02-1016:22borkdudeWell it doesnāt yet. I posted it in the issue#2019-02-1105:05Alex Miller (Clojure team)I did find and fix a bug in the gen from a :kind, but this particular example should and does fail in both versions. coll?
can generate any kind of collection, including maps, which will not pass with something like:#2019-02-1105:06Alex Miller (Clojure team)user=> (gen/sample (s/gen (s/every number? :kind coll?)))
Error printing return value (IllegalArgumentException) at clojure.lang.RT/seqFrom (RT.java:553).
Don't know how to create ISeq from: java.lang.Double
#2019-02-1105:06Alex Miller (Clojure team)thatās with spec 1, but I see same on spec 2 now. You might sometimes get lucky and get enough non-maps for this to work occasionally.#2019-02-1011:39borkdudeApart from this issue and the or-spec issue, the speculative tests pass with spec-alpha2!#2019-02-1012:23borkdudeThe specs are now backward compatible with spec1. So if spec2 and spec1 would have the same namespace names, it could just be a drop-in replacement.#2019-02-1012:26borkdudeFirst succesful build of the spec-alpha2 branch š https://circleci.com/gh/borkdude/speculative/tree/spec-alpha2#2019-02-1012:54borkdudeIt just occurred to me that libs can maintain compatibility with spec1 and spec2 by doing this:
(:require [clojure.spec.alpha :as s1] [clojure.spec-alpha2 :as s2])
and then define specs for each version of spec:
(s1/def :foo number?) (s2/def :foo number?)
#2019-02-1013:20borkdude@jpsoares106 FWIW yada is not moving to spec for this reason. Schema will remain to be used to API boundary validation/coercion.#2019-02-1018:27seancorfield@borkdude what is "this reason" in your comment above? We use spec extensively for "API boundary validation / coercion" and it works great. #2019-02-1018:37borkdude@seancorfield Malcolm talked about it here: https://clojurians-log.clojureverse.org/yada/2019-01-18/1547833475.076000#2019-02-1018:38seancorfieldThat doesn't answer the question -- WHY?#2019-02-1018:40borkdudeMalcolm referred to this post on SO where it says that clojure.spec is not indented for coercion:
https://stackoverflow.com/a/49056441/6264
Iām not sure what I said anymore, lost track.#2019-02-1018:40seancorfield(And, yes, I read a lot of the follow up discussion there -- none of it seems substantive. It's all opinion)#2019-02-1018:41borkdudeThis opinion comes from the clojure.core team. But if it works for you, nobody will stop you I guess š#2019-02-1018:42seancorfieldI think it's misrepresenting what Alex said. #2019-02-1018:43seancorfieldAnd it specifically omits Alex's justification, which was that it forces all clients of the spec to use the coercion. Which in this case is EXACTLY what you want here. #2019-02-1018:44borkdudeWhat do you mean by āitā, the SO article?#2019-02-1018:44seancorfieldDave's answer there yes #2019-02-1018:49borkdudeI havenāt tried using spec in this way. What I want at the API level is not having junk to come into my system which makes it all the way into the database in some jsonb field. Also I want coercion at the API level (e.g. query params are always strings and some have to be numbers, array, etc.).
All I know is that yada does this for me using Schema and I havenāt tried to make this work with spec. The main author of yada doesnāt want to use spec for this anymore. Thatās all I know š.#2019-02-1018:50borkdudeI wish there was a clear and more elaborate article on this subject so I had more clarity on this myself.#2019-02-1018:53seancorfieldWhen I'm not just on my phone I'll write more about this. We were an early adopter of spec and use it very heavily in production for a lot of different things. I've probably spent more time discussing this aspect with Alex than anyone else outside of Cognitect š #2019-02-1018:54borkdudePlease do!#2019-02-1018:57borkdude> We do this for parameters in our REST API, for long, double, Boolean, date, etc ā we have two specs for each: one that is a spec for the target type in the domain model (which in these cases is just defined as the appropriate built-in predicate), and one that is a spec for the API level (which accepts either the target type or a string that can be coerced to the target type). Then we use the appropriate spec at the appropriate ālevelā in our application.
How do you deal with āextraā data coming into your API? How do you deal with preventing junk going into your persistence layer?#2019-02-1018:59lilactownselect-keys? seems like thatās something spec wonāt save you from#2019-02-1018:59borkdudeyeah, a nested select keys actually which is what Schema and other tools are good at#2019-02-1019:01borkdudee.g. EQL#2019-02-1019:06lilactowncurious to know how sean handles that too. Weāve been using spec for our services, but just relying on whatever reititās integration with spec-tools does#2019-02-1019:16ikitommispec-tools allows one to drop all extra keys while doing coercion.#2019-02-1019:16ikitommiitās automatically on in reitit & compojure-api#2019-02-1019:20seancorfieldRegarding that quote from me: we have different specs for different layers. We have a spec for the persistence layer and we have code around the persistence layer that derives the set of keys (i.e., the set of columns) from the specs and uses that to narrow the keys to just those the database will accept. I.e., we are open for extensions, as far as the maps are concerned, until we have to close the set for storing it in a system that isn't open š#2019-02-1019:24seancorfieldAt the API boundary, we have specs that are specific to the REST parameters -- strings as inputs (of course) and validation that does the minimal coercion in order to validate those arguments.#2019-02-1019:25borkdudeIt would be useful to have a function that worked generically on a spec that selected just the data that the spec describes, without changing the spec. Is that public?#2019-02-1019:27seancorfieldGiven that a REST API requires coercion, you have a limited number of choices here. You must do some coercion somewhere. You do it before, during, or after the validation step. You can't validate without some coercion. There isn't much point in doing the coercion twice -- and you don't want different code inside validation and outside it. You have the choice of doing all that coercion upfront and then validating the result, or you can do it in place via spec as part of the validation. Anyone criticizing combining that in spec should also be criticizing Schema for the same thing, IMO.#2019-02-1019:31seancorfieldThe objections to doing coercion in a spec are based on a number of things. Alex has repeatedly pointed out that if you do coercion in a spec, you are forcing that coercion on all clients of that spec -- which is a concern if you're building reusable specs and I agree that you shouldn't do coercion in a general, reusable spec.#2019-02-1019:33borkdudeTo be clear, whatās the problem of forcing coercion on a user of the spec?#2019-02-1019:33ikitommiAlex talks about using conform
to do coercion.#2019-02-1019:33ikitommiitās always on.#2019-02-1019:35seancorfieldRight. So you only want to use it in a context where you need that coercion in order to do the validation.#2019-02-1019:35ikitommidid a gist how spec-tools (and reitit + c-api) solves this: https://gist.github.com/ikitommi/68f662a399a90e8a70308ffcd4b3e752#2019-02-1019:35ikitommiwould like to see the solution baked into spec itself, hereās the most relevant issue: https://dev.clojure.org/jira/browse/CLJ-2251#2019-02-1019:40seancorfieldI'm with Alex that spec shouldn't be used for JSON transformation/parsing š Our REST API mostly has simple string input that gets coerced as part of the validation that those strings are valid numbers, dates, booleans -- but where we have JSON input, we use a JSON library to do that aspect of parsing/coercion.#2019-02-1019:42ikitommisure, the json strings get parsed with Jsonista/Cheshire and then the values are coerced. I would argue that ādropping keys that are not part of the specā happens in the coercion part. Or you do it manually, which is IMO not a good idea.#2019-02-1019:43seancorfieldWe disagree on that, but that's fine.#2019-02-1019:43ikitommihopefully spec will add support for coercion. the current ways of doing it (conform or form walking) are kinda hacks and need to go.#2019-02-1019:45seancorfieldI doubt it, given how much and how often Alex et al have argued against coercion in spec š#2019-02-1019:46borkdudeClearly thereās a tension between some users of spec and the authors of spec on this subject (coercion/stripping). I believe a good article/guide on http://clojure.org about this topic and the recommended way to go about it would be cool.#2019-02-1019:53ikitommiI guess the upcoming select
could be used to strip out data not defined in it?#2019-02-1019:54seancorfieldNo, it's for selecting which keys to check in a given context.#2019-02-1019:56seancorfieldSo that it can decomplect the overall shape of the data spec from the conformance needed in specific situations.#2019-02-1020:56ikitommi@borkdude you asked about a function just to strip out just keys, just remembered that there is a existing transformer doing only that. So, this works (but requires spec-tools):
(require '[spec-tools.core :as st])
(s/def ::zip int?)
(s/def ::country keyword?)
(s/def ::address (s/keys :req-un [::zip ::country]))
(s/def ::name string?)
(s/def ::user (s/keys :req-un [::name ::address]))
(st/coerce
::user
{:name "liisa"
:TOO "MUCH"
:address {:zip 33800
:INFOR "MATION"
:country "INVALID"}}
st/strip-extra-keys-transformer)
; {:address {:zip 33800, :country "INVALID"}, :name "liisa"}
#2019-02-1020:56borkdudecool š#2019-02-1020:57ikitommiand you can compose json-transformer
with strip-extra-keys-transformer
so itās applied in single pass.#2019-02-1101:19joshkhhow might i go about creating a generator for BigDecimal values? is this a job for fmap?
(defn bigdec? [n] (instance? BigDecimal n))
(s/def :bank/balance bigdec?)
(s/gen :bank/balance)
ExceptionInfo Unable to construct gen at: [] for: :bank/balance clojure.core/ex-info (core.clj:4739)
#2019-02-1109:18misha@joshkh
;; Clojure 1.10.0
(s/exercise decimal?)
=>
([0.5M 0.5M]
[1.0M 1.0M]
[2.0M 2.0M]
...
#2019-02-1109:18misha(defn decimal?
"Returns true if n is a BigDecimal"
{:added "1.0"
:static true}
[n] (instance? BigDecimal n))
#2019-02-1114:48borkdudemaybe stupid question, but whatās the benefit of s/defop
over a macro?
https://github.com/borkdude/speculative/issues/124#issuecomment-462352972#2019-02-1114:52mpenetit's basically a parameterized spec, not 2 different specs#2019-02-1114:54Alex Miller (Clojure team)the benefit is that it forms back to the op#2019-02-1114:54mpenetI didn't check the source but reading the comments about it on the dev notes that what it felt like#2019-02-1114:55Alex Miller (Clojure team)that is, you have added a new symbol to the spec op language#2019-02-1114:55Alex Miller (Clojure team)and yes, itās a parameterized fixed spec op#2019-02-1114:55Alex Miller (Clojure team)not a combinator of arbitrary other spec ops#2019-02-1114:56Alex Miller (Clojure team)itās not intended to cover every possible case, just address one common need#2019-02-1114:56borkdudewhatās the benefit of āforms back to the opā?#2019-02-1114:57Alex Miller (Clojure team)as in the example at https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha, you have created something with your own semantics#2019-02-1114:57mpenets/form return value is your op#2019-02-1114:57Alex Miller (Clojure team)user=> (s/form ::zip)
(user/bounded-string 5 9)
#2019-02-1114:57Alex Miller (Clojure team)so you retain your semantics#2019-02-1114:57borkdudeah.
I was just trying this out. I imagined you would get a specific error message for bounded-string:
user=> (s/explain (bounded-string 0 2) "foooo")
"foooo" - failed: (<= 0 (count %) 2)
#2019-02-1114:58Alex Miller (Clojure team)youāre getting the more specific error messages from the definition of bounded-string#2019-02-1114:58mpenetyou can also map the op to something useful in your context for other means I guess#2019-02-1114:59mpenetuser/json-object#2019-02-1114:59mpenetetc#2019-02-1114:59Alex Miller (Clojure team)many of the spec ops have internal constraints that will generate explain problems#2019-02-1114:59Alex Miller (Clojure team)so this is the same as other spec ops#2019-02-1114:59Alex Miller (Clojure team)like regexes check whether the input is nil? or sequential?#2019-02-1114:59Alex Miller (Clojure team)keys checks for map?#2019-02-1114:59Alex Miller (Clojure team)etc#2019-02-1115:00Alex Miller (Clojure team)it may be that some further adjustments should be made in the explain generator (just deferring to the definition spec right now)#2019-02-1115:02Alex Miller (Clojure team)I didnāt highlight it, but inst-in
, int-in
, and double-in
are all derived specs and I reimplemented all of those in terms of defop#2019-02-1115:02Alex Miller (Clojure team)they are effectively all parameterized compound specs#2019-02-1115:02borkdudeThis gives more or less the same result when it comes to error messages:
user=> (defmacro bounded-string2 [min max] `(s/and string? #(<= ~min (count %) ~max)))
#'user/bounded-string2
user=> (s/explain (bounded-string2 0 2) "foooo")
"foooo" - failed: (<= 0 (count %) 2)
but s/form
will give the expanded spec form, so thatās indeed different#2019-02-1115:04Alex Miller (Clojure team)the other big thing is that with defop, we actually def a macro#2019-02-1115:04Alex Miller (Clojure team)well I guess youāll get the same effect if youāre using defmacro here#2019-02-1115:05Alex Miller (Clojure team)(vs just implementing the lower-level create-spec
)#2019-02-1115:08borkdudehttps://github.com/clojure/spec-alpha2/blob/bfba0e37f1b72b2723eda9a7887a96a08e92698d/src/main/clojure/clojure/spec_alpha2.clj#L757 š#2019-02-1115:14borkdude@mpenet
> itās basically a parameterized spec, not 2 different specs
Not sure what you mean by this. If you call (bounded-string 1 10)
and (bounded-string 1 20)
you will get two different spec objects#2019-02-1115:15mpenetyeah but the form share quite a bit, I guess the facts it's 2 distinct spec objects is an impl detail#2019-02-1115:16mpenetas I said I don't know more than what I read on the blog post + intuition of what's the final intent#2019-02-1115:19mpenetfor lib authors it's nicer to read (s/sorted-coll-of x comparator)
than (s/and (s/coll-of x) #(..))
#2019-02-1115:20borkdudewhen would a lib author read this? not trying to argue for the sake of arguing, just want to get it clear for myself š#2019-02-1115:21borkdudeI havenāt used s/form
and friends myself yet, so I havenāt needed this feature much (probably out of ignorance)#2019-02-1115:21mpenetex: spec-tools when trying to understand specs to build json schemas#2019-02-1115:22mpenethere we have specs that are used to validate json params also, one of which is some kind of derivative of coll-of, it makes dealing with it much nicer too#2019-02-1115:22borkdudebut compared to a macro you donāt see a benefit there yet? I mean when reading and writing the literal code?#2019-02-1115:23mpenetnot sure what you mean#2019-02-1115:23mpenetmore readable s/form is a plus for me, ability to build your own s/coll-of like ops is good too#2019-02-1115:24borkdudeI mean, when you define a macro you get to write same code:
user=> (defmacro bounded-string2 [min max] `(s/and string? #(<= ~min (count %) ~max)))
#'user/bounded-string2
user=> (bounded-string2 1 100)
#object[clojure.spec_alpha2.impl$and_spec_impl$reify__1407 0x64b3b1ce "
#2019-02-1115:24borkdudewhen are you using s/form
?#2019-02-1115:24borkdudemaybe this is used by explain logic?#2019-02-1115:26mpenetanytime you want to understand what's behind a spec#2019-02-1115:26mpenetprogramatically#2019-02-1115:26borkdude> ability to build your own s/coll-of like ops is good too
this is limited with s/defop
since you cannot pass in any spec argument. e.g. (my-coll-of (s/nilable ::my-spec))
wonāt work?#2019-02-1115:26mpenetno idea#2019-02-1115:26mpenetas I said I read only the blog post#2019-02-1115:26mpenetand what's here#2019-02-1115:27borkdudeI was referring to this link that I posted earlier: https://github.com/borkdude/speculative/issues/124#issuecomment-462261842#2019-02-1115:27borkdudethatās what left me wondering about defmacro
vs s/defop
#2019-02-1115:29borkdudemaybe not a huge limitation since you can always def the āanonymousā spec first#2019-02-1115:44Alex Miller (Clojure team)int-in
, inst-in
, double-in
are all good examples where this is useful too#2019-02-1115:44Alex Miller (Clojure team)they are all compound parameterized specs#2019-02-1115:45borkdude> where this is useful
What exactly are you referring to?#2019-02-1115:46borkdudethe result of s/form
?#2019-02-1115:47Alex Miller (Clojure team)I just mean general cases where s/defop is better#2019-02-1115:47Alex Miller (Clojure team)symbolic specs form a language, defined by the ops#2019-02-1115:48Alex Miller (Clojure team)using a macro that expands to a compound spec is fine - youāre using an initial step to produce something in the language#2019-02-1115:49Alex Miller (Clojure team)using defop lets you create new ops in the language for the special case where the op can be defined in terms of other spec ops (and is not parameterized by another spec)#2019-02-1115:50Alex Miller (Clojure team)and if you need that, you can drop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instance#2019-02-1115:50Alex Miller (Clojure team)(but a lot more code is required)#2019-02-1115:51borkdudeis nilable
an op? then why can (s/nilable ::foo)
not be passed as an argument to defop, but ::foo
can?
sorry, bit confused about āsymbolic specsā#2019-02-1115:51Alex Miller (Clojure team)nilable is an op#2019-02-1115:51Alex Miller (Clojure team)as I said above, defop is not designed to be parameterized by other symbolic specs#2019-02-1115:52borkdudeso passing ::foo
just accidentally works?#2019-02-1115:52Alex Miller (Clojure team)whatās the example?#2019-02-1115:52borkdude(seqable-of ::foo)
vs. (seqable-of (s/nilable ::foo))
#2019-02-1115:54Alex Miller (Clojure team)it really goes to the evaluation model. spec names (fq keywords) are a little special in that they eval to themselves and also they are the only thing other than a spec object explicitly handled in the spec api functions.#2019-02-1115:56Alex Miller (Clojure team)in this case, ::foo evaluates to a valid symbolic spec (where another spec form evaluates to a spec object, which is not a symbolic spec)#2019-02-1115:57Alex Miller (Clojure team)so it will work, but youāve created a narrow constraint on how it can be used#2019-02-1115:57borkduderight, so itās something that happens to work, but not really the common use case for defop#2019-02-1115:58Alex Miller (Clojure team)yeah, I hadnāt really thought about that#2019-02-1115:58Alex Miller (Clojure team)youāre also not going to get proper explain path-ing with it as a spec created by defop is considered to be a single op#2019-02-1115:59Alex Miller (Clojure team)so if the sub-spec fails, you wonāt get the parent spec in the path#2019-02-1116:01borkdudeok. to conclude: defop is not designed to support spec arguments. If you want that, either write a macro and accept less helpful error messages and s/form output, or ādrop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instanceā which requires more code#2019-02-1116:03Alex Miller (Clojure team)yes#2019-02-1116:03Alex Miller (Clojure team)although I think in the macro case, the errors are almost exactly the same#2019-02-1116:03Alex Miller (Clojure team)so I would maybe quibble with that part#2019-02-1116:08borkdudenot going to publish this anywhere, so I think for now itās clear š#2019-02-1116:15borkdudeI notice that regular pre-defined predicates are also not supported in defop
:
user=> (s/defop first-pred [pred] (s/and (pred (first %))))
#'user/first-pred
user=> (s/valid? (first-pred number?) [1 "f"])
Maybe a bad example#2019-02-1116:30Alex Miller (Clojure team)again, evaluation#2019-02-1116:31Alex Miller (Clojure team)the definition in defop is not going to evaluated - it should be a valid symbolic spec but where the parameterized values are substituted (defop is literally going through and replacing the args with their values)#2019-02-1116:47borkdudeMade a typo. This works:
user=> (s/defop first-pred [pred] (s/and #(pred (first %))))
user=> (s/valid? (first-pred number?) [1 "f"])
true
#2019-02-1116:53seancorfieldFor specs parameterized by other specs, you can do something like (defn domain-keywords
"Given a spec, return a new spec that can conform a keyword or string to
a keyword that is constrained by the given spec."
[spec]
(s/spec* `(s/with-gen (s/and ::keyword ~spec)
(fn [] (g/fmap name (s/gen ~spec))))))
(that and bounded-string
above come from the World Singles Networks' codebase)#2019-02-1116:54borkdude@seancorfield what benefit does that have over writing domain-keywords
as a macro?#2019-02-1116:56Alex Miller (Clojure team)well, itās a function so you can apply it, so usual benefits of function over macro#2019-02-1116:57Alex Miller (Clojure team)(with the caveat that the spec arg needs to be a form, not an object, here)#2019-02-1116:57seancorfieldIt was a function in the spec1 world (without (s/spec* ..)
and the quote/unquote, so we made it a function in the spec2 world. Minimal change (and it still composes and applies etc).#2019-02-1116:58borkdude@seancorfield I had a similar thing with seqable-of
. Function in spec1, but when I turned it into a function in spec2 using s/spec*
, I could not provide specs like (s/nilable ::foo)
because I got an error, so then I made it a macro.#2019-02-1116:58seancorfieldI plan on writing up a (probably long) blog post on our conversion from spec1 to spec2 when Alex tells me spec2 is stable enough for that to be widely valuable š#2019-02-1116:58Alex Miller (Clojure team)you canāt use this with in s/def though (like (s/def ::x (domain-keywords ...))
)#2019-02-1116:59seancorfieldRight. And we use s/defop
for those sorts of things.#2019-02-1116:59Alex Miller (Clojure team)but you could with the functional entry point s/register
which takes an object (which is what domain-keywords
returns)#2019-02-1117:00Alex Miller (Clojure team)@seancorfield btw, I spent some time working on the regex thing and I need to undo the changes I made to support forward references in regexes#2019-02-1117:00Alex Miller (Clojure team)which will fix the nesting issue, but re-break forward references#2019-02-1117:00seancorfield@borkdude Mostly, we've found changing our defn
spec builders over to s/defop
has been all we've needed in the most common cases. A few have needed s/spec*
instead.#2019-02-1117:00borkdude> Right. And we use s/defop
for those sorts of things.
Sorry, referring to what?#2019-02-1117:01seancorfield@alexmiller That's fine -- the forward reference issue only affected one spec in our entire code base so I can just move it below the sub-specs š#2019-02-1117:01Alex Miller (Clojure team)Forward refs in regex is solvable via different means but I need to talk to Rich before I commit to a path on that and heās out today#2019-02-1117:01seancorfield@borkdude "those sorts of things" = "use this with in s/def"#2019-02-1117:02borkdudeaāight#2019-02-1117:04borkdudeI canāt write a very long blogpost about transitioning to spec2. All I had to do is report bugs, which were all fixed, wrap a bunch of predicates in s/spec
, refactor one predicate to #(not (sequential %))
instead of (complement sequential?)
and turn a private function into a macro.#2019-02-1117:05borkdudeI think I might write a tweet about it instead.#2019-02-1117:06Alex Miller (Clojure team):)#2019-02-1117:06Alex Miller (Clojure team)@seancorfield reverted the fwd reference fix, which should fix the nesting issue (but break that fwd reference)#2019-02-1117:08borkdudedid forward referencing ever work? didnāt know#2019-02-1117:10Alex Miller (Clojure team)yes, in many cases#2019-02-1117:11Alex Miller (Clojure team)in general, specs delay lookup of named specs until use#2019-02-1117:11Alex Miller (Clojure team)I fixed several places where that wasnāt being done#2019-02-1117:13Alex Miller (Clojure team)but changes in how regexes are implemented mean that we effectively broke it for them#2019-02-1117:14Alex Miller (Clojure team)regex impls used to not be specs but would get spec-ized when needed. in spec 2, regexes actually are spec instances (thanks to metadata protocol implementation!) which simplifies the code in many places, but removed the point where this delay naturally happened before#2019-02-1117:15Alex Miller (Clojure team)fixing it is ā¦ tedious#2019-02-1117:15borkdude(s/declare ::foo)
š#2019-02-1117:15Alex Miller (Clojure team)yeah, no thanks :)#2019-02-1118:08seancorfield@alexmiller Good to know. I'll run a full test suite with the latest spec2 shortly.#2019-02-1119:47seancorfield@alexmiller Confirming that our full test suite runs on the latest spec2, with that one forward reference regex spec moved after the specs that it refers to.#2019-02-1119:49Alex Miller (Clojure team)š#2019-02-1119:49borkdudespeculative still works as well#2019-02-1213:56guyIf youāre going to be using spec in a project is it better to use spec-2 now instead?#2019-02-1214:07Alex Miller (Clojure team)no#2019-02-1214:07Alex Miller (Clojure team)we have not yet done any releases of spec 2#2019-02-1221:30guyThanks!#2019-02-1217:05seancorfield(if you're using CLI / deps.edn, you can at least start testing against spec2 to see what code changes you might need to make at some future point @guy)#2019-02-1221:31guythanks!#2019-02-1221:02mishahttps://clojurians.slack.com/archives/C03RZMDSH/p1550005322074900#2019-02-1221:25aisamuWe usually create aliases (`alias`) and then use ::
#2019-02-1221:30misha1) assuming alias :datomic.client.protocol
= pro
: ::pro/response
is fine, but ::pro.response/body
just does not work, (as keyword ns is just a string with no "structure" inside, afaik). So you'd need to make an alias for :datomic.client.protocol.response
as well. And if you don't have datomic.client.protocol.response and datomic.client.protocol namespaces as files, thats 2 lines for create-ns
, and 2 for alias
, all just to save 5 words :(
2) afaik, clojurescript does not have alias
function (or I did not find it yesterday)#2019-02-1221:35mishayeah, https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/core.cljc#L42
(ns cljs.core
(:refer-clojure :exclude [ ...
#
#2019-02-1306:07caleb.macdonaldblackIs there a way to create an anonymous spec? So instead of defining it, I can just create one and pass it into spec functions?#2019-02-1306:13seancorfield@caleb.macdonaldblack I suspect the answer is "yes" but it's different between spec1 and spec2...#2019-02-1306:14caleb.macdonaldblackspec2 as in the next version of spec?#2019-02-1306:14seancorfieldIn spec1, you can mostly use a predicate interchangeably with a spec. In spec2, you can construct spec objects on the fly.#2019-02-1306:14seancorfieldYes, clojure.spec-alpha2
#2019-02-1306:14caleb.macdonaldblackWell I'm now excited for spec2#2019-02-1306:15seancorfieldHahaha... well, there are no releases yet, but if you're using deps.edn
, you can test against it.#2019-02-1306:17caleb.macdonaldblackI'm using leiningen. I may as well wait until an official release. I'm glad to see spec moving in that direction#2019-02-1306:19seancorfieldHave you seen Rich's talk from Conj? (Maybe Not)#2019-02-1306:19seancorfieldHe talks about future direction for spec...#2019-02-1306:20caleb.macdonaldblack"Maybe Not"?#2019-02-1306:20caleb.macdonaldblackif so I haven't seen it. I'll have a look though#2019-02-1306:22seancorfieldhttps://youtu.be/YR5WdGrpoug#2019-02-1306:23seancorfieldSpecifically he talks about how s/keys
complects the shape of data and the actual checks that are needed in any given context.#2019-02-1306:27caleb.macdonaldblackinteresting#2019-02-1306:51Alex Miller (Clojure team)in both spec 1 and 2 you can create a spec object and pass it to any of the spec api functions#2019-02-1306:51Alex Miller (Clojure team)with the caveat that s/keys relies on having registered key specs to rely upon#2019-02-1306:52Alex Miller (Clojure team)there may be more support in spec 2 for map selection specs with anonymous key specs#2019-02-1306:53Alex Miller (Clojure team)I am working in that area right now#2019-02-1307:17caleb.macdonaldblackAh okay i didnt know that#2019-02-1314:25rascioHi here! I'm playing with clojure specs, I'm struggling in trying to use generators for a spec with regex:
(s/def ::entity (s/and string? #(re-matches #"DEST:\d+")))
Am I wrong in defining something? Or the regex can't be generated by the default lib?#2019-02-1314:27borkdude@manuelrascioni youāre missing a %
:
user=> (s/def ::entity (s/and string? #(re-matches #"DEST:\d+" %)))
:user/entity
user=> (s/valid? ::entity "DEST:1")
true
#2019-02-1314:30borkdudeYour spec will generate, but the likelihood that it generates strings that will satisfy the predicate is extremely small. You probably want to provide a generator with the spec, using s/with-gen
#2019-02-1314:39rascio@borkdude ah...thank you! It seems I still have to train my eye to check for this kind of mistakes...I will check the docs for the with-gen
thank you for the hint!#2019-02-1314:44borkdude@manuelrascioni E.g.:
(s/def ::entity (s/with-gen (s/and string? #(re-matches #"DEST:\d+" %)) #(gen/fmap (fn [i] (str "DEST:" i)) (s/gen nat-int?))))
(gen/sample (s/gen ::entity))
("DEST:1" "DEST:0" "DEST:1" "DEST:2" "DEST:2" "DEST:7" "DEST:16" "DEST:41" "DEST:1" "DEST:4")
#2019-02-1314:47rasciogreat! thank you, it is very helpful!#2019-02-1314:50rasciojust to check if I understood well, the fmap
is used to "customize" the value generated by a generator, and return a generator, right?#2019-02-1314:54borkdudefmap returns a new generator which transforms the values generated by the mapped-over generator using a function#2019-02-1315:02buttergunshttps://github.com/miner/strgen#2019-02-1315:22Alex Miller (Clojure team)a newer better version of that is in test.chuck#2019-02-1315:23Alex Miller (Clojure team)https://github.com/gfredericks/test.chuck#string-from-regex#2019-02-1315:26borkdudeI recently ran orchestra which tests ret-specs with speculative on a body of code. The only ret spec which wasnāt correct was for an fdef for which I had not used generative testing.#2019-02-1315:28borkdudeit was e.g. re-find
, re-matches
, etc. for which I had not considered that it could also return nils, as in
(re-find #"(a)?(b)" "b")
["b" nil "b"]
I wonder how I could have written a generator for this.#2019-02-1315:29borkdudeI would have to generate regexes and strings that would sometimes match, sometimes not#2019-02-1316:13borkdudeThis fun experiment is able to generate regexes that seem to not terminate when executedā¦
(defn test-re-find []
(let [regex-gen (gen/fmap (fn [parts]
(let [s (str/join parts)]
(re-pattern (str/join parts))))
(s/gen (s/* (s/cat :part
(s/or :string string?
:group (s/with-gen string?
#(gen/fmap (fn [s]
(str "(" s ")"))
(s/gen string?))))
:maybe (s/? (s/with-gen string?
#(gen/fmap (fn [s]
(str s "?"))
(s/gen string?))))))))
matcher-gen (gen/fmap (fn [[r s]]
(re-matcher r s))
(gen/tuple regex-gen (s/gen string?)))]
(map re-find (gen/sample matcher-gen 100))))
#2019-02-1316:13borkdudeuser=> (gen/sample regex-gen)
(#"" #"" #"" #"()15?" #"(2)" #"(3)" #"()9?F?JWwff1" #"xE7(re9)(79W)26E?()jKqXKk5?(DY1Xa)()(4m)2qNS3?" #"gW7GAJ9p22Z4o4eWJ?" #"(Gi8r)RQ22uBC?(jj0PolFmd)h7?Taz()(7)GwE0")
#2019-02-1319:38borkdudeIām trying this now:
(s/def ::regex.char #{"a" "b"})
(s/def ::regex.group (s/with-gen string?
#(gen/fmap (fn [s]
(str "(" s ")"))
(s/gen ::regex.pattern))))
(s/def ::regex.maybe (s/? (s/with-gen string?
#(gen/fmap (fn [s]
(str s "?"))
(s/gen ::regex.pattern)))))
(s/def ::regex.pattern (s/* (s/or :char ::regex.char
:group ::regex.group
:maybe ::regex.maybe)))
This gives me a stackoverflowā¦
(binding [s/*recursion-limit* 1]
(gen/sample (s/gen ::regex.pattern)))
#2019-02-1320:13borkdudeHow do I get generators to play nice with conformers?
(s/def ::my-spec (s/and int? (s/conformer str)))
(gen/sample (s/gen ::my-spec)) ;; => (0 -1 0 0 0 0 6 42 -1 -3) <- want strings here
#2019-02-1320:25seancorfieldAs written, your spec accepts (only) numbers -- and it is generating numbers that your spec accepts. That's the correct behavior.#2019-02-1320:27seancorfieldConforming numbers to strings as part of a spec feels very wrong to me (and you know what an advocate I am for certain types of coercion in specs! š)#2019-02-1320:28borkdudethis was just an example, not something Iām doing for real#2019-02-1320:29borkdudethe thing I was doing for real was the regex.pattern above, where I want to generate strings, but describe those strings in terms of spec#2019-02-1320:30seancorfieldGenerators must produce values that are acceptable to your spec.#2019-02-1320:31seancorfield(and we've had repeated cautions from @alexmiller not to use spec regex for string parsing/generation stuff š )#2019-02-1320:33borkdudefor parsing yes, because of performance, there are better tools, but for generation, I currently donāt know a better tool š#2019-02-1320:35seancorfieldWhy not use test.chuck
s regex string generator?#2019-02-1320:35seancorfield(or did I miss your rationale for not using that?)#2019-02-1320:35borkdudeI want to generate regexes, not strings that are matched by a given regex#2019-02-1320:38borkdudeonce I have that, I can use test.chuck to generate strings from the generated regexes. and then I can use stest/check to test re-find, etc.#2019-02-1320:52Alex Miller (Clojure team)why not make a regex for regexes?#2019-02-1322:38aisamuCan't tell if serious#2019-02-1323:08Alex Miller (Clojure team)I'm not sure either#2019-02-1320:52Alex Miller (Clojure team)then use test.chuck on it#2019-02-1320:58borkduderegex language cannot be expressed with a regex, I think you need a CFG tool like spec#2019-02-1321:45borkdudethis kinda works:
(s/def ::regex.pattern
(s/* (s/cat :pattern
(s/alt :char #{\a \b}
:group (s/cat :open-paren #{\(}
:inner-pattern ::regex.pattern
:closing-paren #{\)}))
:maybe (s/? #{\?}))))
(s/valid? ::regex.pattern (seq "(ab)"))
(s/valid? ::regex.pattern (seq "ab(ab)?"))
(map str/join (binding [s/*recursion-limit* 2]
(gen/sample (s/gen ::regex.pattern))))
(defn test-re-find []
(let [regex-gen (gen/fmap (fn [r]
(re-pattern (str/join r)))
(s/gen ::regex.pattern))
matcher-gen (gen/fmap (fn [[r strs]]
(re-matcher r (str/join strs)))
(gen/tuple regex-gen (s/gen (s/* #{"a" "b"}))))]
(let [matchers (gen/sample matcher-gen)]
(map re-find matchers))))
(test-re-find)
At least Iām now finding return values that I didnāt account for in an early version of the spec, e.g.:
["ba" "a" "a" "" "" nil nil "" "" ""]
#2019-02-1321:46borkdudeafk now#2019-02-1410:01borkdudeSeems to work now. Iām generating regexes using a simplified regex spec (that exists solely for generating) and Iām generating strings that match it using test.check. Iām using these combinations to test re-find, etc. Thanks for the suggestions all š#2019-02-1410:01borkdudehttps://twitter.com/borkdude/status/1095985994590027777#2019-02-1410:02borkdudeThe only problem left is making it work on CLJS and self-hosted CLJS š https://github.com/gfredericks/test.chuck/blob/master/src/com/gfredericks/test/chuck/generators.cljc#L244#2019-02-1410:04borkdudeI think Iāll just replace the generator on CLJS with a simpler one#2019-02-1411:33gfredericks@borkdude there's a PR for that š#2019-02-1411:33gfredericksI'm not sure if it works though#2019-02-1411:34borkdudePR seems out of date. Even if itās incomplete I think itās better than nothing? @wilkerlucio#2019-02-1411:36gfredericksyou're suggesting I just merge it?
I'm not sure what you mean by "out of date"#2019-02-1411:37borkdudethat the PR has merge conflicts#2019-02-1411:41wilkerlucioI wouldn't merge it, the range of things it works is quite narrow at this point, I'm using something simpler to generate strings, if we want to get that working in cljs (which would be awesome) that code needs better testing and impl, not good to merge as is IMO.#2019-02-1411:42wilkerluciowhat I'm using for cljs these days is a much simpler string generator that knows just some basic patterns (numbers, letters, alphanum):#2019-02-1411:42wilkerlucio(ns string-gen
(:require [com.wsscode.test.chuck.charsets :as charsets]
[clojure.test.check.generators :as gen]))
(defn charset->gen [charset]
(let [size (charsets/size charset)]
(if (zero? size)
(throw (ex-info "Cannot generate characters from empty class!"
{:type ::ungeneratable}))
(gen/fmap (partial charsets/nth charset)
(gen/choose 0 (dec size))))))
(def type->charset
{"D" (charsets/range "0" "9")
"W" (charsets/union (charsets/range "0" "9") (charsets/range "a" "z") (charsets/range "A" "Z"))
"A" (charsets/range "A" "Z")})
(defn parse-int [x]
(js/parseInt x))
(defn token->gen [token]
(cond
(string? token)
(gen/return token)
(keyword? token)
(if-let [[_ t n] (re-find #"([DAW])(\d+)" (name token))]
(gen/fmap #(apply str %)
(gen/vector (charset->gen (type->charset t)) (parse-int n)))
(throw (ex-info "Invalid keyword token" {:token token})))
:else
(throw (ex-info "Invalid token" {:token token}))))
(defn string-gen [tokens]
(->> tokens
(mapv token->gen)
(apply gen/tuple)
(gen/fmap #(apply str %))))
#2019-02-1411:47borkdudeIāll consider tweaking this. Thanks#2019-02-1411:35borkdudeI just realized that core.match could play nice with clojure spec conform results (in general, not related to this regex discussion)?
https://stackoverflow.com/a/54687183/6264#2019-02-1411:36borkdude(probably realized a little late)#2019-02-1413:02joshkhi'm probably missing something obvious -- how can i define a spec so that its value conforms to one of a collection of specs? for example, a person can have a pet that's either a mammal or a fish, neither of which share a common attribute.
(s/def :animal.class/mammal (s/keys :req [:utters/count]))
(s/def :animal.class/fish (s/keys :req [:fins/count]))
(s/def :person/pet (s/or :animal.class/mammal :animal.class/fish))
(s/explain :person/pet {:utters/count 4})
val: #:utters{:count 4} fails spec: :animal.class/fish at: [:animal.class/mammal] predicate: (contains? % :fins/count)
=> nil
#2019-02-1413:03joshkhwhoops - that was meant for #beginner#2019-02-1413:04borkdude(s/or :animal.class/mammal :animal.class/fish)
=> (s/or :mammal :animal.class/mammal :fish :animal.class/fish)
#2019-02-1413:04borkdudeyou need to tag the alternatives with a key#2019-02-1413:04joshkhah!#2019-02-1413:04joshkhi saw that in cat/alt, didn't realise it applied to /or. thanks as usual, borkdude.#2019-02-1413:40andy.fingerhutborkdude, are you basically looking for bugs in the JVM's and/or JavaScript's regex library using spec and generative testing?#2019-02-1413:42andy.fingerhutI wouldn't be surprised if you find bugs in those, and/or test.chuck's generate-a-string-matching-a-regex-from-a-regex code, or all of the above. Just so you realize that the time scale for fixing the JVM and/or JavaScript's regex library might be a bit long š Those libraries must be challenging to maintain.#2019-02-1413:45borkdudeThe situation is like this: someone used Orchestra with speculative specs. Orchestra checks ret specs at runtime (in contrast to spec). It turned out the ret specs of regex functions were incomplete and I could have found this if I had used generative testing (which I have for almost all specāed functions except these). Now I have almost fixed this generative testing, but I ran into the CLJS limitation of test.chuck as a last issue.#2019-02-1413:46borkdudeI can bypass this by generating simpler strings for CLJS. It will still find the case I forgot to spec.#2019-02-1413:48borkdudeIām not using it for anything more serious than this, just wanted to see how for I could go with it.#2019-02-1414:16mpenetany planned changes (or options) about fn as argument triggering gen, something like wrapping their arguments with asserts instead so that they fail at invoke time ?#2019-02-1414:17mpenetI think there was a jira issue about this#2019-02-1414:21Alex Miller (Clojure team)will revisit, haven't yet#2019-02-1417:02djtangoIs there a way to provide a custom generator for a spec distant to the spec's definition/local to the test context? The docs for instrument
say that the :replace
only relates to fn-specs
and :gen
option is only for stubbed vars#2019-02-1417:03borkdude@djtango do you want to test with stest/check or instrument?#2019-02-1417:07djtangoI think right now want to manually generate some inputs using gen/sample
#2019-02-1417:07djtangothough could also do stest/check
- was it stest/check
that lets you also patch in your own generators?#2019-02-1417:07djtangothat seems familiar#2019-02-1417:07borkdudeyes,
(stest/check `my-function {:gen {::my-spec (fn [] (gen/ā¦))}})
#2019-02-1417:10djtangoawesome thanks#2019-02-1417:10djtangobut with instrument
no luck?#2019-02-1417:10borkdude> :gen overrides are used only for :stub generation.#2019-02-1417:11borkdudeIām not sure what :replace
does, I have never used it š#2019-02-1417:12borkdude:replace replaces a fn with a fn that checks args conformance, then
invokes the fn you provide, enabling arbitrary stubbing and mocking.
#2019-02-1417:12borkdudeš¤#2019-02-1509:11borkdude@wilkerlucio FWIW:
$ clj -R:test -m cljs.main -re node
Downloading: com/wsscode/test-chuck-string-from-regex-cljs/com.wsscode.test-chuck-string-from-regex-cljs/0.3.0/com.wsscode.test-chuck-string-from-regex-cljs-0.3.0.pom from
Downloading: com/wsscode/test-chuck-string-from-regex-cljs/com.wsscode.test-chuck-string-from-regex-cljs/0.3.0/com.wsscode.test-chuck-string-from-regex-cljs-0.3.0.jar from
ClojureScript 1.10.516
cljs.user=> (require '[com.wsscode.test.chuck.core :as sfr])
Unexpected error macroexpanding instaparse.core/defparser at (regexes.cljs:18:1).
Error parsing grammar specification:
Parse error at line 1, column 1:
./resources/com/gfredericks/test/chuck/regex-cljs.bnf
^
Expected one of:
<
Īµ
eps
EPSILON
epsilon
Epsilon
(*
#"[^, \r\t\n<>(){}\[\]+*?:=|'"#&!;./]+(?x) #Non-terminal"
#2019-02-1609:47borkdude(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.gen.alpha :as gen])
(s/def ::foo (s/keys* :req-un [::a ::b]))
(s/def ::a number?)
(s/def ::b number?)
(s/valid? ::foo [:a 1 :b 2]) ;; true
(gen/generate (s/gen ::foo {::a (fn [] (gen/return 1))})) ;;=> (:a -15164 :b 24.0)
any reason I canāt override a key generator in a keys*
spec?#2019-02-1609:59Alex Miller (Clojure team)no, probably a bug#2019-02-1611:58borkdude@alexmiller JIRA here: https://dev.clojure.org/jira/browse/CLJ-2483
I could take a look myself if youāre still accepting patches for spec1.#2019-02-1612:51borkdudeI think itās a more general problem:
(gen/generate
(s/gen (s/with-gen ::foo
(fn []
(s/gen (s/cat :key-a #{:a} :val-a ::a :key-b #{:b} :val-b ::b))))
{::a (fn [] (gen/return 1))})) ;;=> (:a -1 :b "OXi2")
Overriding the generator on a spec thatās made with with-gen
has no effect apparently?#2019-02-1614:49Alex Miller (Clojure team)Oh, thatās true based on the impl for sure#2019-02-1620:10don.dwoskeI finally watched the Maybe Not talk and have some comments. Agree totally based on our history of modeling a biomedical research domain that schema/select
is a great idea. A context is needed in order to know the required/optionality. For us information gets added to entities over time, what is an optional field in one stage in the pipeline, may be required in a future stage.
However, I think something additional is missing. Our predicates also depend on the context. The allowable values for a key depend on the context, so we want to attach narrowing predicates at that 'select' time, not just the required/optionality part.
Hopefully a simple example: if we define a Car
entity, with a make
, the make
is validated against a CV list that has lots of makes in it. However, if we are working within a particular dealership context - a dealership which only sells Jeeps and Land Rovers, we want to add a new predicate at that select time which not only requires the make
be present, but that it also be one of those two values, and not just any kind of make. Not only is the shape gaining requirements in some contexts, the allowable values are as well.
The predicates added at select time are always and
- they reduce the set of allowable values and cannot make it more general - thus, there is no breaking change to the original s/def
contract for the key. Rich was close to this idea during his talk and applied it to shapes, but not to predicates. Hopefully, I've described all this well enough, and I'm not missing something obvious, but my Google/Slack-fu didn't turn up anything relevant on this. Thanks.#2019-02-1712:30wilkerlucio@U4NQYBCU8 I trully believe in this idea of information getting adding over selections over time, maybe you will be interesting to check a library that I maintain that pushes this idea futher, and provides a system where by defining inputs and outputs as maps (with schema/select
like definitions), the system can transition from some input to some output, given there is some available path in the system. this is the lib: https://github.com/wilkerlucio/pathom and if you like you can see I talk I gave about it (also in this conj :)) https://www.youtube.com/watch?v=yyVKf2U8YVg#2019-02-1922:37johanatanThereās no documentation on this project yet but this is the intended usage of: https://github.com/johanatan/validaze#2019-02-1922:38johanatanYou can get an idea how to use it by reading the tests (including spec generators) and source code.#2019-02-1922:39johanatanThe strong point of this lib is very user-friendly validation failure error messages.#2019-02-1621:06Alex Miller (Clojure team)you can s/and additional predicates any time you like#2019-02-1822:44don.dwoskeBut that changes things in a global context... an earlier stage context doesn't want to 'see' the additional s/and for that def.#2019-02-1823:00Alex Miller (Clojure team)Iām saying you can s/and additions predicates into a new spec, either at the point of use or registered#2019-02-1823:16don.dwoskeBut isn't the point of schema/select being able to reuse the same schema in different select contexts, so that registering new specs is not necessary. e.g. I want to use the same schema in two contexts 1) makes allow only land rovers and jeeps and 2) makes allow fords and chryslers ... what I'm doing there is wanting to add predicates at the select stage. Am I missing something here?#2019-02-1823:20don.dwoskeIt's not always a linear accretion and narrowing of information - there are forks where one part of the system has different rules for validating cars than another part (e.g. one dealership has different rules than another) but the schema for a car is always the same.#2019-02-1918:51buttergunsWould something like this work in spec2?
(s/def ::model string?)
(s/def ::car (s/keys [::model]))
;; Would "select" the ::model key AND apply additional rules
(s/select ::car [(s/and ::model #(is-made-by-jeep? %))])
(s/select ::car [(s/and ::model #(is-made-by-ford? %))])
#2019-02-1918:52butterguns(pseudo code of course)#2019-02-1919:01Alex Miller (Clojure team)not currently#2019-02-1919:04buttergunsI'm not sure I understand when you say "you can s/and additional predicates any time you like". I want to add additional predicates at "selection-time", like above#2019-02-1919:05Alex Miller (Clojure team)I get what you're asking#2019-02-1919:06Alex Miller (Clojure team)I don't have an answer for you right now#2019-02-1919:08buttergunsNo problem! Was just spit-balling answers for @U4NQYBCU8#2019-02-2018:44don.dwoskeThanks. Just want to be understood and acknowledged, no answers needed... I think you get it. @UFQAPAUU8 example is approx. the thing we want.
However, another thing to make clear is that selecting for required/optionality is orthogonal to adding predicates to keys on the schema within a context. A key may still be optional in a spec context - but if the key/value happens to be there, I want to add predicates to that key's spec. Some psuedo-stuff :
(s/def ::id int?)
(s/def ::make string?)
(s/def ::model string?)
(s/def ::color string?)
(s/def ::car (s/schema [[::id ::make ::model ::color]]))
(favorite-car car) needs ::make, ::model
(favorite-car) checks (::make is 'jeep') and (::model is 'wrangler')
(favorite-car) checks ::color is 'red' or 'black'
For favorite-car
, make and model are required, color is still optional. Predicates in the context are added to both required and optional fields to narrow the specification.#2019-02-1721:23hmaurerSpeaking of schema/select
, whatās the current state of this? Is alpha code already available?#2019-02-1721:52Alex Miller (Clojure team)Spec 2 is at https://github.com/clojure/spec-alpha2#2019-02-1721:52Alex Miller (Clojure team)But nothing about select there yet#2019-02-1807:46jaihindhreddyI want to spec a fn target
, like this:
(s/fdef target
:args (s/+ ::a)
:ret ::a
:fn #(= (f (:ret %) i)
(apply g (map (fn [x] (f x i))
(:args %)))))
#2019-02-1807:47jaihindhreddyWhere f
and g
are functions I've written, and i
is any integer i.e, for any integer i
, this property must hold. Can I express this with spec, or should I drop down to test.check
?#2019-02-1813:36misha> Iāve been working on the new select functionality for spec 2 this week
ā„#2019-02-1814:18gfredericks@jaihindh.reddy I think if i
is not an arg to target
then you can't easily do that with spec; you'd have to maybe write another function target*
that also takes i
, and you'd only use it for testing, which is a bit weird#2019-02-1912:23sonnytocan anyone answer this question about spec? https://groups.google.com/forum/#!topic/clojure/IPY9YukiLI0#2019-02-1912:56Alex Miller (Clojure team)You need to force eval somehow so macro around it, use eval, etc. this is an area where there are more and different answers in spec 2#2019-02-1912:57sonnytothanks. i used def-impl directly . is this a problem?#2019-02-1913:53Alex Miller (Clojure team)it's fine (although won't work in spec 2)#2019-02-1913:54Alex Miller (Clojure team)there is a new function in spec 2 called register
that is similar#2019-02-1916:07borkdudeIs metadata and docstrings on specs considered for spec2? E.g. Iād like to conditionally instrument them based on metadata#2019-02-1916:16Alex Miller (Clojure team)yes#2019-02-1918:29borkdudebananadance#2019-02-1918:33Alex Miller (Clojure team)it was in scope for spec 1 we just never got to it :)#2019-02-2001:02jsa-aerialI have question that more experienced people here will likely think obvious. Say I have a map with some keys, two of which need to have the same value. For example m, {::id v1 ::nm v2 ...}, and I want to enforce (= (m ::id) (m ::nm)). I haven't seen this sort of example, but presumably it is covered by spec?? Thanks in advance for any insight!#2019-02-2001:30jsa-aerialHmmmm, this doesn't give errors, but it doesn't work either:#2019-02-2001:31jsa-aerial#2019-02-2001:40jsa-aerialHmmmm, looks like id-eq-nm?
needs to use (m :sid)
and (m :snm)
#2019-02-2015:42kvltHey all. I'm writing specs for a few controller functions. They take in request maps and hand off those maps to have work done.
The issue I'm experiencing is that a collection inside of that (`body-params`) can vary by quite a bit depending on which endpoint is called and I don't like the idea of writing a really loose spec that can work with all of the say, user endpoints. As such:
(s/def ::body-params (s/keys :opt [::username ::email ::....]))
and would prefer to have specs setup for each call:
(s/def ::body-params (s/keys :req [::email] :opt [:...]))
.
Without putting each endpoint into a new file. How do I go about handling this?#2019-02-2017:24johanatanIsnāt it possible to have more than one ns form per file?#2019-02-2015:47borkdudeI think this is a problem which spec2 solves#2019-02-2015:48borkdudeyou can watch the latest Rich Hickey talk on youtube if you want to know more#2019-02-2015:48borkdudenot that this is solving your problem right nowā¦ sorry š#2019-02-2015:52kvltYeah, I saw the talk. I was just kinda hoping that there was something I could do naow#2019-02-2016:07buttergunsCould a multi-spec be used for this?#2019-02-2016:14butterguns(s/def ::email string?)
(s/def ::username number?)
(s/def ::request-type #{:change-email :show-username})
(s/def ::change-email-request (s/keys :req [::email] :opt [::username]))
(s/def ::show-username-request (s/keys :req [::username] :opt [::email]))
(defmulti request-type ::request-type)
(defmethod request-type :change-email [_] ::change-email-request)
(defmethod request-type :show-username [_] ::show-username-request)
(s/def ::body-params (s/multi-spec request-type ::request-type))
(s/valid? ::body-params {::request-type :change-email ::email "boop"})
=> true
(s/valid? ::body-params {::request-type :change-email ::username 1234})
=> false
(s/valid? ::body-params {::request-type :show-username ::email "boop"})
=> false
(s/valid? ::body-params {::request-type :show-username ::username 1234})
=> true
#2019-02-2016:15buttergunsYou'll have to do a step first that assigns the request-type
key to the map, depending on the controller that was callled#2019-02-2020:45guyMight be the wrong place to ask,
But with https://github.com/jeaye/orchestra
are you still supposed to use,
https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check
To check the instrumented fdefās?#2019-02-2021:01borkdude@guy as far as I know orchestra only changes instrumentation (it includes ret + fn checks). you still define specs with normal spec, so everything should still work. orchestra uses different namespaces, so it doesnāt patch/replace spec itself (I think)#2019-02-2021:02borkdude@jeaye might be able to confirm or correct this#2019-02-2021:07guyThanks!#2019-02-2021:07guyAlso iām a little confused about how to use :fn
#2019-02-2021:07guyas part of an fdef
#2019-02-2021:08borkdudehereās an example: https://github.com/borkdude/speculative/blob/master/src/speculative/core.cljc#L465#2019-02-2021:08guyAm i supposed to try and link the :args
and :ret
using :fn
?
As in when a certain input is something, then the return should be something, using fn
to validate that?#2019-02-2021:09jeayeYep, you still use spec for everything. Just use orchestra to enable instrumentation and consider using defn-spec
to clean up your fns.#2019-02-2021:12guyok thanks š#2019-02-2021:09borkdude@guy with fn you can do additional checking on arg + ret and dependencies between them#2019-02-2021:11jeaye:fn
is optional, so you'll likely know when you need it.#2019-02-2021:11borkdude@guy e.g. in the group-by spec I linked I did an additional check that group-by should not produce more elements than the input collection had#2019-02-2021:13guyš¤#2019-02-2021:13guyok#2019-02-2021:15guyI guess it might be an issue with my understanding.
Lets say you have a function that takes some args and returns true or false.
Would you ever want to use :fn
to sort of specify that when the args are nil, you return false#2019-02-2021:15guyIām not sure if thats a bad example, but when i check an instrumented function, i want the args and return to make sense#2019-02-2021:16guyor maybe iām thinking about it the wrong way :shrug:#2019-02-2021:16borkdudeI think in the case of simple predicates fn specs might not be the most helpful. The clojure spec guide might provide you with more helpful ones.#2019-02-2021:17guyIāll read the guide again and check that speculative lib too. Thanks š#2019-02-2022:45droneā¤ļø defn-spec
#2019-02-2022:47droneweāve rolled our own for defrecord, but would like to see standard spec-enhanced forms be a thing. but I have a feeling theyāre most useful for those of us using spec as a poor manās type system, which is probably something Hickey wants to discourage#2019-02-2023:02Alex Miller (Clojure team)can you explain what you're talking about?#2019-02-2023:03Alex Miller (Clojure team)what is "spec-enhanced forms"?#2019-02-2100:00dronejust like defn-spec adds inlined specs for function return values and parameters, defrecord-spec adds inlined specs for fields, along with adding specs to the built-in record construction functions#2019-02-2100:00droneeh, s/inlined/inplace#2019-02-2100:01Alex Miller (Clojure team)Ah. Rich is considering a spec-enhanced defn.#2019-02-2100:12dronedefn and defrecord are the obvious ones to gain spec-enhanced forms. specs for protocol implementations could also be nice (in defrecord/deftype or extend-type). default specs for protocols (in defprotocol)? protocol related stuff I havenāt thought enough about. but have noticed it feels like a āspec gapā#2019-02-2100:52Alex Miller (Clojure team)Protocols canāt be specāed#2019-02-2101:02flyboarder@mrevelle if you keep your protocol an implementation detail and dont expose it as your public api itās not a problem#2019-02-2101:03flyboarderfor example (my-func)
could be a specād function and (-my-func)
could be a protocol function#2019-02-2101:04flyboarderthen (my-func)
calls (-my-func)
internally#2019-02-2101:04flyboarderThis is what we did with hoplon#2019-02-2101:42Alex Miller (Clojure team)These days we consider that a best practice in the core team#2019-02-2103:35droneprotocols could be specād#2019-02-2103:40drone@flyboarder yeah, I suppose. not a fan of the redundancy and lack of documented expectations that specs may provide. it can also be useful to allow others to implement your protocols (see loom and ubergraph)#2019-02-2103:58dorabWhat am I doing wrong here?#2019-02-2103:58dorabuser> (require '[clojure.spec.alpha :as s])
nil
user> (require '[clojure.spec.test.alpha :as stest])
nil
user> (require '[clojure.spec.gen.alpha :as sgen])
nil
user> (s/def ::one-arg-fn (s/fspec :args (s/cat :x any?)))
:user/one-arg-fn
user> (defn map-vals
"Map the function f over all the values of the associative collection coll."
[f coll]
(reduce-kv (fn [m k v]
(assoc m k (f v)))
(empty coll)
coll))
#'user/map-vals
user> (s/fdef map-vals :args (s/cat :f ::one-arg-fn :coll associative?))
user/map-vals
user> (def xmap {:a " foo" :b "bar "})
#'user/xmap
user> (stest/instrument)
[user/map-vals]
user> (map-vals clojure.string/trim xmap)
Execution error - invalid arguments to user/map-vals at (REPL:65).
(nil) - failed: (apply fn) at: [:f] spec: :user/one-arg-fn
user>
#2019-02-2104:10Alex Miller (Clojure team)when you use an instrumented fspec, it will actually generate values according to the fspec args spec and invoke the function you pass with them#2019-02-2104:10Alex Miller (Clojure team)here you declared that the fspec function takes an any? arg#2019-02-2104:10Alex Miller (Clojure team)the generator generated nil and invoked clojure.string/trim with it#2019-02-2104:10Alex Miller (Clojure team)which throws#2019-02-2104:12dorabAh. Thanks. So, I should use a s/with-gen
in the spec definition of ::one-arg-fn
?#2019-02-2104:12Alex Miller (Clojure team)so it's effectively telling you that when passing clojure.string/trim to map-vals, you have passed a function that will not take an any?#2019-02-2104:12Alex Miller (Clojure team)you can#2019-02-2104:12Alex Miller (Clojure team)or instead of fspec, many people find it easier to just use ifn?
#2019-02-2104:13Alex Miller (Clojure team)this generative test on instrument of fspec is a feature that many people find surprising (or wrong)#2019-02-2104:13Alex Miller (Clojure team)and something we're going to revisit in spec 2#2019-02-2104:14dorabOK. Thanks for the explanation. What would be the recommended way in spec1 to write a spec for a single arity function?#2019-02-2104:17dorabOr, is the recommendation to just use ifn?
as you mention above?#2019-02-2104:28Alex Miller (Clojure team)I would just do that#2019-02-2104:28Alex Miller (Clojure team)with current state, I don't find that I gain anything but headaches with fspecs#2019-02-2104:32dorabThanks.#2019-02-2105:57gklijsWhat would be the best way to prepare for spec 2. I now sometimes use the :opt is it useful to rewrite those to :req and make the values also allow nil?#2019-02-2106:27seancorfield@gklijs s/keys
will stay as-is, as far as I know, so you will be able to move to s/schema
and s/select
piecemeal over time as you need it.#2019-02-2106:28seancorfieldI have our codebase running against spec2 with minimal changes (by depending on the latest git SHA of clojure/spec-alpha2
-- since we use CLI/`deps.edn`) but it's still very much a moving target with no formal release (and still some bugs being worked out).#2019-02-2106:30seancorfieldThe main thing you may trip over is if you have used a predicate in a context that is really expecting a spec -- spec1 allows that but spec2 does not. Also, some constructs are now stricter about what constitutes a "spec".#2019-02-2106:31seancorfieldYour best preparation at this point is to read https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-02-2106:31Alex Miller (Clojure team)the fate of s/keys is not yet determined#2019-02-2106:31seancorfieldOooh... so it may go away before spec2 is "released"?#2019-02-2106:32Alex Miller (Clojure team)maybe, but really don't know yet#2019-02-2106:32seancorfieldThat would make the migration from spec1 to spec2 a lot more work!#2019-02-2106:32Alex Miller (Clojure team)at the moment I'd say it's more likely to still be there#2019-02-2106:33Alex Miller (Clojure team)the idea behind schema has shifted a lot since Rich's talk, not sure if it will survive in the form described then#2019-02-2106:33Alex Miller (Clojure team)he's deep in the hammock :)#2019-02-2106:33seancorfieldInteresting... I look forward to seeing how this evolves then...#2019-02-2106:34seancorfieldAnd, yeah, I feel that hammock... I'm still deep in thought about next.jdbc
š#2019-02-2106:34Alex Miller (Clojure team)@gklijs I would not rewrite opts to nilable reqs - I think that's the wrong direction#2019-02-2106:34Alex Miller (Clojure team)really, unless you're a "riding the bleeding edge" guy like Sean, I would just wait#2019-02-2106:35Alex Miller (Clojure team)select will have both required and optional selections#2019-02-2106:36Alex Miller (Clojure team)although the semantics around optional stuff is a little different#2019-02-2106:39gklijsOk, I'll leave it for now then. I now use it as part of (de)serialisation, with optional added defaults on a subset of what can be specced, also using the spec's to generate the cljs functions to edit the data. I probably need to write some sort of migration at a point, but that should be trivial. Might use another namespace to spec the existing data in spec2 at some point.#2019-02-2115:26guyIf you have two specs but you want to have a shared id between the two, is that possible in a generator? But also the idās are generated as well. š#2019-02-2115:29borkdudeexample please?#2019-02-2115:31mishayou'll need custom generator, which probably will have to generate examples for both specs at the same time. Another funky option is to override default id gen to generate predefined small set of ids, and then filter out examples generated by "default" gens which happen to share an id#2019-02-2115:31mishayou'll need custom generator, which probably will have to generate examples for both specs at the same time. Another funky option is to override default id gen to generate predefined small set of ids, and then filter out examples generated by "default" gens which happen to share an id#2019-02-2115:32guythanks! that makes sense#2019-02-2115:34mishafirst option would look like:
1) gen set of ids,
2) gen set of obj1 with default gen
3) gen set of obj2 with default gen
4) override ids in pairs of obj1 obj2 with ids from 1)#2019-02-2115:35mishathis will allow you to leverage default spec gens w/o any modification, and just build higher level one#2019-02-2115:37mishawriting a gen for entire family of interdependent objs is hard -_-#2019-02-2201:35jsa-aerialSo, what is the status of CLJ-2320? In my case I can limp along with s/merge, but this is (according to the issue itself) a 'major' priority but does not appear to have had any activity since last June. OTOH, it doesn't have any votes either. I don't have an account so can't log in to vote anyway...#2019-02-2201:37jsa-aerialAlso, why would what is effectively a 'logic system' use short circuiting semantics for an 'and' operator???#2019-02-2203:22seancorfield@jsa-aerial Anyone can create a JIRA account, I believe...#2019-02-2203:23seancorfieldI think s/and
really has to be short-circuiting since some predicates may blow up if they don't have a guard predicate ahead of them...?#2019-02-2216:07jsa-aerial@U04V70XH6 Sounds like a likely explanation, but in this context that 'should not happen'. Put another way, that should be up the user's predicates. If a 'guard' operator is wanted - call it 'guard' or some such.#2019-02-2208:09Christian JohansenIs there any way to associate a docstring with a spec?#2019-02-2208:10Josip Gracinnot in spec1, AFAIK#2019-02-2208:10Christian Johansendo you know if it's planned for 2?#2019-02-2208:12Josip GracinI should probably let Alex answer that, but I believe it will probably be in spec2#2019-02-2208:15Josip Gracinmy belief is based on previous discussions of this on slack and mailing lists#2019-02-2208:26Christian Johansenok, cool. I haven't followed either of those closely, so crossing my fingers š#2019-02-2208:27mishahttps://clojurians.slack.com/archives/C1B1BB2Q3/p1550592458214700
https://clojurians.slack.com/archives/C1B1BB2Q3/p1550601206215400#2019-02-2209:06Christian Johansenah, cool, thanks!#2019-02-2213:22slipset@christian767 I think @bbrinck has done some trickery in expound wrt docstrings on specs....#2019-02-2213:23slipsetNo, that was custom error messages. #2019-02-2213:25Christian Johansenok š#2019-02-2221:38kennyIs there a good way to debug slow generators when running a Spec check? #2019-02-2310:42misha@kenny https://www.youtube.com/watch?v=FihU5JxmnBg#2019-02-2310:42mishaĀÆ\(ć)/ĀÆ#2019-02-2316:23adc17Is there a way of following progress on the s/schema
/`s/select` functionality that Rich mentioned in his talk? I can't see an issue tracking this on Jira (I also can't see spec in the clojure-contrib
libraries)?#2019-02-2316:33misha@alexchalk17 https://github.com/clojure/spec-alpha2 and http://insideclojure.org/#2019-02-2316:48Alex Miller (Clojure team)clojure-contrib is very old and deprecated#2019-02-2421:04eoliphantHi, trying to muddle through a modeling scenario in the context of spec. In one of my domains I work with grants. so a grant has a unique :grant/number
thatās a string of some format āGRT-999ā, etc., string? + regex and Iām good. But i also have to deal with the fact that various āissuersā of grantās have their own formats. Issuer A, uses āGRANT-999", B uses āG_999ā,etc. So, Iām grappling with somehow providing ācontextā to the spec. Or are these better modeled as different things entirely :issuer-a/grant-number
, :issuer-b/grant-number
?#2019-02-2421:09borkdudeI can imagine that a grant has a :grant/issuer
and based on that value you can verify if the :grant/number
is valid, possibly using s/and
or maybe using a multi-spec?#2019-02-2507:31misha#2019-02-2507:34misha@eoliphant ^^^ this is a bit simplified, as you probably will not have issuers as keyword, but it can be expanded with mapping of issuer to number format.#2019-02-2507:36mishayou'd also need to customize generators a bit, if you need them.#2019-02-2510:49Audriuswhat if to use Spec with regular (non namespaced) keywords? should it work OK?#2019-02-2510:49Audriusis there a too t generate spec from data structure?#2019-02-2510:51borkdude@masta I believe spec-tools has such a tool#2019-02-2511:49misha@masta read through this chapter https://clojure.org/guides/spec#_entity_maps#2019-02-2511:49mishathere is https://github.com/stathissideris/spec-provider#2019-02-2514:08eoliphantThanks @misha, @borkdude. One other bit of fun is that at the end of the day, the number of issuers/formats could be large and might end up in a db#2019-02-2514:35rickmoynihanIs it possible to turn clojure/core.specs
off during macroexpansion? So you can see the failing expansion?#2019-02-2514:37Alex Miller (Clojure team)https://clojure.org/guides/faq#skip_macros#2019-02-2514:37rickmoynihanš#2019-02-2514:37rickmoynihanthanks#2019-02-2514:39rickmoynihanIām assuming thereās a reason this couldnāt also be a set!
able var? Not that restarting my REPL is that difficult! š#2019-02-2514:40Alex Miller (Clojure team)well it's in a system property as stuff is potentially checked while you're loading before you have a chance to set it#2019-02-2514:41Alex Miller (Clojure team)it could probably also be a dynvar (although that would need to be checked on every macroexpansion, so probably some perf impacts)#2019-02-2514:46Audrius(spec/fdef data->graph
:args ::data
:ret ::graph)
why this does not work? I mean it will take for ever to start...#2019-02-2514:47Alex Miller (Clojure team)"not work" == ?#2019-02-2514:51Audriusprocess starts for ever#2019-02-2514:54Alex Miller (Clojure team)is data->graph a macro or a function?#2019-02-2514:55Alex Miller (Clojure team)if a function, it shouldn't do anything until you instrument#2019-02-2514:55Audriusit is a function...#2019-02-2515:09Audriusstill running š¤ maybe it is too complex for Spec to handle...#2019-02-2515:10Audrius(spec/def ::graph (clojure.spec.alpha/def
:importer.datamodel/graph
(clojure.spec.alpha/coll-of
(clojure.spec.alpha/or
:collection
(clojure.spec.alpha/coll-of
(clojure.spec.alpha/keys
:req-un
[:importer.datamodel/label]
:opt-un
[:importer.datamodel/center
:importer.datamodel/edge_id
:importer.datamodel/global_id
:importer.datamodel/name
:importer.datamodel/source
:importer.datamodel/target
:importer.datamodel/typ]))
:simple
clojure.core/keyword?))))
(spec/def ::data (clojure.spec.alpha/def
:importer.datamodel/data
(clojure.spec.alpha/coll-of
(clojure.spec.alpha/or
:collection
(clojure.spec.alpha/coll-of
(clojure.spec.alpha/keys
:req
[:importer.datamodel/global-id]
:opt
[:importer.datamodel/center
:importer.datamodel/part-of
:importer.datamodel/type]
:opt-un
[:importer.datamodel/attributes
:importer.datamodel/datasource
:importer.datamodel/features
:importer.datamodel/ioi-slice
:importer.datamodel/name
:importer.datamodel/url]))
:simple
clojure.core/keyword?))))
(spec/fdef data->graph
:args ::data
:ret ::graph)
#2019-02-2515:10AudriusI have this...#2019-02-2515:10Audriusand runs for ever to start...#2019-02-2515:23guyHave you tried (exercise ::data 3)
to see if outputs anything?#2019-02-2515:29Audriusalso runs for ever...#2019-02-2515:31AudriusFound it...#2019-02-2515:31Audrius(spec/def ::data (clojure.spec.alpha/def
is recursive or something š#2019-02-2519:52jrwdunhamHi here! I'm hoping someone can help me out with what is probably a basic question about clojure spec#2019-02-2519:52jrwdunhamI have this function:#2019-02-2519:52jrwdunham(defn starts-with-any?
"Return truthy if s starts with any character in chrs; otherwise nil."
[s chrs]
(seq (filter (fn [c] (string/starts-with? s (str c))) chrs)))
#2019-02-2519:52jrwdunhamand this spec:#2019-02-2519:52jrwdunham(s/fdef starts-with-any?
:args (s/cat :s string? :chrs string?)
:ret (s/nilable (s/coll-of char?))
:fn (s/or :nil-case #(-> % nil?)
:non-nil-case
(s/and #(= (-> % :ret set first) (-> % :args :s first))
#(= 1 (-> % :ret set count))
(fn [x] (subset? (-> x :ret set) (-> x :args :chrs set))))))
#2019-02-2519:53jrwdunhamif I run (stest/check
starts-with-any?)` on this I get a failure#2019-02-2519:53Alex Miller (Clojure team)#(-> % nil?)
== nil?
btw#2019-02-2519:53jrwdunhamyeah, thanks, I'll change that#2019-02-2519:54borkdude> I get a failure.
needs more detail.#2019-02-2519:54jrwdunham{:clojure.spec.alpha/problems ({:path [:fn :nil-case], :pred (clojure.core/fn [%] (clojure.core/-> % clojure.core/nil?)), :val {:args {:s "", :chrs ""}, :ret nil}, :via [], :in []} {:path [:fn :non-nil-case], :pred (clojure.core/fn [%] (clojure.core/= 1 (clojure.core/-> % :ret clojure.core/set clojure.core/count))), :val {:args {:s "", :chrs ""}, :ret nil}, :via [], :in []}), :clojure.spec.alpha/spec #object[clojure.spec.alpha$or_spec_impl$reify__2046 0x3d686af1 "
#2019-02-2519:54jrwdunhamit's failing on a nil
ret, but shouldn't it pass on the first branch of s/or
?#2019-02-2519:55borkdudeitās failing on the path [:fn :nil-case]
, so that would be the first branch#2019-02-2519:55jrwdunhamoh right, and it needs :ret
#2019-02-2519:55jrwdunhamd'oh#2019-02-2519:56Alex Miller (Clojure team)yeah#2019-02-2519:56jrwdunhamhaha, thanks. sorry for the typo question#2019-02-2519:57Alex Miller (Clojure team)duckie#2019-02-2522:53robertfwIs there a recommended/succinct way to reuse a string regex spec in keyword form? That is, I have a spec that uses a regex to define a string format (in this case, it's an ID format). In some places, that value is used as a keyword in a map. I could break out the specifics, but was wondering if there is an easy way to do something in the spirit of (s/def ::id-as-keyword (as-keyword ::id-spec))
#2019-02-2612:12Audriushow to make spec for list of maps? '({}{}{})
?#2019-02-2612:12mpenet(s/coll-of map?)
#2019-02-2612:13mpenetbut most of the time you want something more precise than map?#2019-02-2612:13Audriusbut how to make that the maps also conforms to a spec?#2019-02-2612:15mpenetreplace map? with a namespaced keyword that points to a spec#2019-02-2612:15mpenetor a spec directly#2019-02-2615:53jsa-aerialAny reason why (s/explain-data a-spec a-value)
does exactly what is expected (and is correct) and given the exact same input (s/valid? a-spec a-value)
fails with a class cast exception? This is on 1.9#2019-02-2615:54borkdudedo you have the spec and example data? did you test in clj 1.10?#2019-02-2616:08favila@jsa-aerial https://dev.clojure.org/jira/browse/CLJ-2372 maybe?#2019-02-2616:45jsa-aerial@favila doesn't look to be that (might be peripherally related). I think the issue is related to a misuse of s/merge
in my case. I have a set of predicates def'd to specs. None of these are map/key related (like with s/keys
) so none involve a map. Then I use s/merge
(because s/and
short circuits...) on these. Now, s/merge
is happy to do this. s/explain-data
is happy to do 'the right thing' and check all predicates of the merged spec. But s/valid?
has a bug as well, but instead of 'doing the right thing' it just blows up. I put 'right thing' in quotes because I suppose s/merge
should not allow this in the first place. All of this is a shame - s/and
should not be short circuiting in the first place. This is a logic system after all. If you are concerned about guarding flow through to other predicates then your predicates are what's broken. I suppose you could have a guard
operator for that sort of case, but IMO, that would still be bogus. ĀÆ\(ć)/ĀÆ#2019-02-2616:46favilawhy would you want s/and not to short-circuit?#2019-02-2616:48jsa-aerialBecause you are saying that the spec should satisfy all predicates. Like in propositional logic. Not programming languages#2019-02-2616:49jsa-aerialThe reason this is really useful is because you would like to catch all the errors in one go - not get one report to user. fix that one, get the next, report to user, etc. That is really annoying and just wrong#2019-02-2618:51favilabut s/and accepts specs and conformers#2019-02-2618:51favilanot just predicates#2019-02-2618:51favilawell, nm just conforming is a problem#2019-02-2618:52favilaso you would have to drop conformers, or introduce some trickery about evaluation#2019-02-2618:52favilaeven if all were evaluated order would still have to matter#2019-02-2618:55seancorfield@jsa-aerial s/and
flows data through all the specs so it must be short-circuiting -- it has an inherent order and you can't rely on being able to apply subsequent specs if an earlier spec fails. (and I feel we've had this conversation before)#2019-02-2619:13jsa-aerialIt is what it is - I think if you wanted that 'guarded flow` capability that another name would have been far better. I don't really care TBH, I just think it was a poor choice.#2019-02-2619:04mishaYeah, each next spec in s/and
receives conformed value from prior one. So if one step is invalid ā next would just blow up with random noise instead of useful explain data.#2019-02-2619:09mishaotherwise how would you s/conform
to s/and
spec? conform only last step? none? Most of the s/and specs I saw ā check fields first, and then some relation between them next (e.g. this id and that id are the same.), which is basically: 1st step is conforming, last is not#2019-02-2619:09favilaif s/and were restricted to predicates, and predicates were written defensively (which they should be IMO), then I think what @jsa-aerial proposes would be fine; order could be used for making generators, but all could be checked in parallel#2019-02-2619:10mishahow do you restrict to predicates only?#2019-02-2619:10favilaI mean if conformers were not legal#2019-02-2619:11favilabascially, you would have to drop conformers as a feature of s/and#2019-02-2619:11mishawell, this excludes s/keys opieop#2019-02-2619:11mishaor anything containing s/or
#2019-02-2619:12favilaI don't follow?#2019-02-2619:13favilaI'm just talking about s/and. Maybe for clarity I should say this would not be s/and anymore. s/nonconforming-and or s/parallel-and#2019-02-2619:13favilas/all maybe#2019-02-2619:14mishaI'd call it s/every
or something. But "flowing" behavior was surprising to me back then, I agree#2019-02-2619:15mishabut, how would such s/every be more useful than clojure.core/and
?#2019-02-2619:15favilaevery predicate would be tested unconditionally#2019-02-2619:15favilaI should say, could be#2019-02-2619:16borkdudeis this for form validation or something?#2019-02-2619:16favilaI'm guessing#2019-02-2619:16mishaok, clojure.core/every-pred
#2019-02-2619:16favilasomething user-facing#2019-02-2619:18jsa-aerialNot necessarily, could be anything. You just want to be able to report all findable problems at once - not piecemeal#2019-02-2619:16favilabut I haven't found spec very useful for things like that#2019-02-2619:18jsa-aerialActually it works amazingly good when coupled with phrase
#2019-02-2619:16mishawhat extra would it provide?#2019-02-2619:16favilareport all errors instead of just the first one#2019-02-2619:17borkdudemaybe build your tool on top of spec, but not use s/and to collect all the errors. just call s/explain for each field for example#2019-02-2619:17mishayou can't exercise it, can you? because it involves conforming#2019-02-2619:17favilaconforming and excercising seem orthogonal?#2019-02-2619:18mishaexercise gives you pairs of [generated conformed], I think?#2019-02-2619:19favilathe conformed of s/every would always be identity#2019-02-2619:19favilaor whatever, s/every-pred#2019-02-2619:19mishaI think you'll be able to implement such s/every
on top of spec2#2019-02-2619:19favilano conforming#2019-02-2619:20favilaI'm more curious what the explain-data would look like#2019-02-2619:20mishaok, what if one of the items in s/every would be spec with s/and inside? it'll propagate conformed value, I think#2019-02-2619:21seancorfieldIt's worth pointing out that Spec2 draws a harder line between specs and predicates -- and s/and
accepts specs, not arbitrary predicates. See https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-02-2619:21favilathe point is, this spec would simply not do that#2019-02-2619:21mishawell yeah, explain - "conforms"#2019-02-2619:23seancorfield(! 591)-> clj
Clojure 1.10.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::foo (s/and int? (partial > 10)))
:user/foo
user=> (s/valid? ::foo 10)
false
user=> (s/valid? ::foo 1)
true
user=> ^D
Tue Feb 26 11:22:41
(sean)-(jobs:0)-(~/clojure)
(! 592)-> clj -A:spec2
Clojure 1.10.0
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::foo (s/and int? (partial > 10)))
:user/foo
user=> (s/valid? ::foo 10)
Execution error (IllegalArgumentException) at clojure.spec-alpha2/spec* (spec_alpha2.clj:197).
No method in multimethod 'create-spec' for dispatch value: clojure.core/partial
user=>
#2019-02-2619:23misha@seancorfield I don't see "not arbitrary predicates" there. It says "Qualified symbols (predicate function references)" are ok#2019-02-2619:23seancorfield@misha See the code I just posted above.#2019-02-2619:24mishawhat if you defn
it first, instead of partial?#2019-02-2619:25favilaprobably it needs an fspec#2019-02-2619:25favila(just guessing)#2019-03-2621:45Alex Miller (Clojure team)hoping to do some work on this in 1.11#2019-03-2621:46Alex Miller (Clojure team)I feel your pain :)#2019-03-2711:09zcljDo anyone know of a library that can produce specs given a json schema?#2019-03-2712:05ikitommi@zclj havenāt heard, but would have use for that too. there is an issue in spec-tools, but no work going on: https://github.com/metosin/spec-tools/issues/154. Happy to help if someone starts doing that.#2019-03-2712:31zclj@ikitommi thanks for the info. I will do some experiments on my specific case and if it is generic enough I would be happy to contribute to the issue in spec-tools#2019-03-2713:05ikitommiIs there a way to force a fdef
to validate always the inputs? Schema had the :always-validate
metadata for this.#2019-03-2713:08ikitommie.g.
(require '[schema.core :as s])
(s/defn ^:always-validate interceptor-x
[opts :- {:string? s/Bool}]
...)
(interceptor-x {})
; Syntax error (ExceptionInfo) compiling at (test.cljc:150:1).
; Input to interceptor-x does not match schema:
;
; [(named {:string? missing-required-key} opts)]
#2019-03-2713:07ikitommiSomething like stest/instrument
but for non-test use.#2019-03-2713:11rickmoynihanNot that I know of.
You can use a :pre
condition on a function that calls s/valid?
, though youāll not get an s/explain-*
style error if you do.#2019-03-2713:12rickmoynihanIf you actually want that though, why canāt you just call
(st/instrument #{`the.fdef.i.want.to.always/validate})
#2019-03-2713:14rickmoynihanIād probably just call s/valid?
and raise an s/explain-data
if the check failed explicitly inside the function though if I wanted that ā or however you want to report the error.#2019-03-2713:14ikitommiThat would work, but that would not contribute to the function documentation.#2019-03-2713:15rickmoynihanIt would if it was both an fdef and that š#2019-03-2713:16ikitommicalling clojure.spec.test.alpha
from non-test code also doesnāt sound like a good idea.#2019-03-2713:20ikitommiIāll write an issue out of that.#2019-03-2713:32ikitommihttps://dev.clojure.org/jira/browse/CLJ-2498#2019-03-2714:26rickmoynihanregarding configuring components on startup you might want to look at what integrant does:
https://github.com/weavejester/integrant/#specs#2019-03-2714:46ikitommiThatās nice, thanks! Doing about the same with reitit: https://github.com/metosin/reitit/blob/master/modules/reitit-http/src/reitit/http/coercion.cljc#L12#2019-03-2714:48ikitommiI would still like a support functional way of doing that: have a function that takes options and returns the component (that has a spec). And the function args should be validated too.#2019-03-2714:21drone@ikitommi Iām not sure it addresses your exact issue, but orchestraās instrument
enables checking of :ret
and :fn
#2019-03-2714:24dronethis has been better for how we use spec; where we enable instrumentation during development to try and get type-checking-like feedback while writing code#2019-03-2714:29Alex Miller (Clojure team)@ikitommi it is intentional that this functionality doesn't exist for functions as the goal is to have no overhead for function specs at runtime#2019-03-2714:29Alex Miller (Clojure team)but I'll leave it open for now as maybe it would be useful to have an opt-in feature for that#2019-03-2714:31ikitommiopt-in would be great.#2019-03-2714:34ikitommi@mrevelle orchestra is great, but it also has the dynamic *instrument-enabled*
which can disable the instrumentation.#2019-03-2723:42jeayeWe just use something like the following in our code:
(when-not-prod
(let [instrumented (sort (stest/instrument))] ; [orchestra.spec.test :as stest]
(timbre/trace :instrumenting instrumented)))
#2019-03-2819:31borkdude@ikitommi If you want to always validate, also in prod, why not use s/valid?
for this?#2019-03-2819:32borkdude(s/valid? (:args (s/get-spec `my-fn)) args)
something like that#2019-03-2819:34borkdudeor if you donāt like using s/get-spec
you can write the args spec separately and use that in fdef
, youāll still get the docstring#2019-03-2819:39dronewe use s/valid?
in asserts for things that should never happen and both s/valid?
and s/conform
in control flow (`if`, condp
, match
, etc). not advocating to do that, but sharing a related need we felt and what weāre trying out to address it#2019-03-2819:45borkdudeuser=> (s/fdef foo :args (s/cat :i int?))
user/foo
user=> (def args-spec (:args (s/get-spec `foo)))
#'user/args-spec
user=> (defn foo [i] (if (s/valid? args-spec [i]) i (throw (ex-info "n00" (s/explain-data args-spec [i])))))
#'user/foo
user=> (foo 1)
1
user=> (foo "1")
Execution error - invalid arguments to user/foo at (REPL:1).
"1" - failed: int? at: [:i]
#2019-03-2819:48borkdude^ @ikitommi#2019-03-2819:48borkdudes/assert
also works, but we have assertions turned off in prod#2019-03-2819:48borkdudeso s/valid?
always works#2019-03-2819:49borkdudeand I bet you can wrap this in a macro like (def-checking-fn name & body)
#2019-03-2820:26ikitommi@borkdude thanks! looks just right. In my case, it's a framework calling the specced functions at runtime, so I can use apply
& args to make a generic functional proxy.#2019-03-2820:32ikitommi... or just extract the function spec if that exist and validate by the framework. indeed.#2019-03-2820:29ikitommithat said, tempted to add an s/defn
with identical syntax as Schema somewhere. There are several ones (all bit different) out there already. The Lisp Curse?#2019-03-2820:39droneI think the only public one thatās maintained is orchestraās defn-spec
#2019-03-2821:05Alex Miller (Clojure team)fwiw, I think weāre probably going to add something with spec 2 (or maybe Clojure 1.11)#2019-03-2821:18metametadataI like https://github.com/Provisdom/defn-spec syntax as it's friendly out-of-the-box to Cursive IDE (specs are defined inside defn metadata). but maybe orchestra's macro is also supported by Cursive#2019-03-2821:49buttergunsThe other day, I asked a question about speccing Java object fields
https://clojurians.slack.com/archives/C1B1BB2Q3/p1553612761304000
I thought I'd check back in, and say that I got intimate with s/conformer
! My solution is to use a conformer to convert the Java object into a clojure map of {<fieldname-keyword> <field-value>}
, and then just use s/keys
(s/def ::device (s/and (s/conformer (conform-java-fields ::summary #(.getSummary %)
::report #(.getReport %)))
(s/keys :req [::summary ::report])))
I learnt something! š#2019-03-2821:58Alex Miller (Clojure team)note that this spec won't work backwards with unform though#2019-03-2822:00Alex Miller (Clojure team)in general, this kind of thing is frowned upon as it bakes transformation into the spec and doesn't give registry consumers the option of whether to do the transformation. I would probably recommend to instead actually convert the objects to Clojure data, then use simple data specs on it.#2019-03-2822:01buttergunsSo, doing the transformation in code, instead of within a spec?#2019-03-2822:02Alex Miller (Clojure team)yeah#2019-03-2822:02Alex Miller (Clojure team)conformer was really imagined primarily as a tool for spec op creators, not for api users#2019-03-2822:03buttergunsOK, that makes sense. Thanks for the advice. I'm making progress#2019-03-2821:58borkdudere: https://clojurians.slack.com/archives/C1B1BB2Q3/p1553807158018600
it would be nice if spec offered a bit more integration between fdef args and defn args somehow#2019-03-2822:02Alex Miller (Clojure team)well the big thing likely here is to integrate the requires/provides semantics via select, and also to introduce a way to talk about returns in terms of args more descriptively. the :ret/:fn as it stands now is likely to change#2019-03-2901:20kenny@borkdude @mrevelle I've run into problems using s/valid?
in production. If my specs use fspec
s, it will throw in prod because no test.check. We work around this by never passing a function to a function that uses s/valid?
but seems a bit dangerous. Would be nice to be able to disable fspec
generative checking in certain production.#2019-03-2908:18ikitommiIs there a way to extract the fdef
specs, given one has a function at hand? e.g. need to get the symbol out of the function to be able to look up for itās spec. Functions are quite opaque in clojure (or Iām just doing it wrong)#2019-03-2908:21ikitommisomething like this, but for real (and without eval
):
(defn kikka [])
(-> kikka
str
(str/replace #"@.*" "")
(str/replace #"_" "-")
(str/split #"\$")
(->> (apply symbol)))
; user/kikka
#2019-03-2912:05Alex Miller (Clojure team)You can demunge
the class name#2019-03-2912:05Alex Miller (Clojure team)But wonāt always work#2019-03-2912:20Alex Miller (Clojure team)In general, itās better if you can start from a symbol than start from a function object#2019-03-2913:14ikitommiThanks. In my case, I just donāt have the symbols available, just functions. Knowing the arity of the functions at runtime would be useful too to be able to fail fast in case of invalid arity of the (framework) functions (middleware, interceptors etc). Any change of getting the function symbol & arity queryable for runtime?#2019-03-2913:47Alex Miller (Clojure team)function objects are just function objects - the fact that a particular var is referring to them is in some ways incidental.#2019-03-2913:48Alex Miller (Clojure team)same for arity, although that one in particular is one that affects code evolution over time (it's common for arities to expand but still be backwards compatible)#2019-03-2913:48Alex Miller (Clojure team)I know these seem like obvious things to have on a function, but I think the implications are probably a lot more far reaching and subtle than you expect#2019-03-2913:50Alex Miller (Clojure team)there are a lot of good reasons to fight this kind of introspection and instead try to drive from symbols or vars#2019-03-2913:50Alex Miller (Clojure team)so, in short, no, not eagerly looking to make these changes#2019-03-2914:04drone@kenny we donāt use fdef
because of this. And I agree, disabling the test generator req would help. I think it was mentioned as possibly coming in spec2#2019-03-2914:35stathissiderisa bit of a philosophical question: I know spec uses qualified keywords, but do people normally go for the :com.my-company.invoice/total
style of keywords or the :invoice/total
style?#2019-03-2914:48Alex Miller (Clojure team)if you are writing a public library, then I think you endeavor to start your qualifier with something you control (by trademark, domain name, etc).#2019-03-2914:48Alex Miller (Clojure team)if you are writing a private app, then you should make qualifiers "sufficiently unique"#2019-03-2914:49Alex Miller (Clojure team)whether that means including company, department, product, whatever is relevant#2019-03-2915:22stathissideris@alexmiller thanks, thatās a balanced view which I think makes sense#2019-03-2918:08dominicmI'm starting to regret using particularly unique keywords that include the company name and sub project as a prefix for an application.
I'm considering adding aliases to human written edn because of it.#2019-03-2919:10carocadhey guys, does somebody knows what is the url of the api docs for spec-alpha2 ? The link seems to be broken in https://github.com/clojure/spec-alpha2 š#2019-03-2919:12Alex Miller (Clojure team)I'm not sure I've ever built it#2019-03-2919:12Alex Miller (Clojure team)the doc builder works off of a release and we haven't released a version of it yet#2019-03-2919:15Alex Miller (Clojure team)in other words, the link is right, the target is absent :)#2019-03-2919:15Alex Miller (Clojure team)I can try to build it but I'm probably not going to have time today#2019-03-2919:21Alex Miller (Clojure team)Posted a writeup on the (still in work) schema and select in spec 2: https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-03-2919:29carocadthanks a lot š#2019-03-2920:23seancorfieldI know what I'll be doing this weekend (while my wife's in China)... š#2019-03-3114:09eggsyntaxLooking good, thanks Alex! Couple of thoughts/questions on that document:
- on first encounter, it seems confusing (and complecting š ) to treat schemas sometimes as though they're specs (`s/def`, same registry) and sometimes not (can't be used for valid?
etc). Why not either a) let them act as specs (maybe a schema acts like (s/select my-schema)
, ie all optional, when treated as a spec?) or b) make them fully separate (separate registry, can't use s/def
to register them)?
- What do you mean by 'SPI'? 'Serial-parallel interface'? š¤
- In the "get-movie-times" example, is there a way to say that address is optional, but if present it must include zip? I know I've encountered use cases like that in practice.#2019-03-3120:27seancorfieldYes, omit address from the vector but still provide the map for the sub-select user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::name string?)
:user/name
user=> (s/def ::street string?)
:user/street
user=> (s/def ::city string?)
:user/city
user=> (s/def ::zip string?)
:user/zip
user=> (s/def ::addr (s/schema [::street ::city ::zip]))
:user/addr
user=> (s/def ::user (s/schema [::name ::addr]))
:user/user
user=> (s/def ::data (s/select ::user [::name {::addr [::zip]}]))
:user/data
user=> (s/valid? ::data {::name "Sean"}) ; ::addr is optional
true
user=> (s/valid? ::data {::name "Sean" ::addr {}}) : but when provided it must contain ::zip
false
user=> (s/valid? ::data {::name "Sean" ::addr {::zip "94546"}})
true
user=>
#2019-04-0103:35Alex Miller (Clojure team)@U077BEWNQ - schemas as specs would be pretty useless since map specs are open (any map would validate) so we decided to make them different. There are some real subtle questions in the idea of wanting to have both for different purposes though and I haven't had a chance to talk through it with Rich yet. might change.
- SPI = service provider interface, for those building new kinds of specs/schemas, vs API which is what consumers use. probably should have explained that more clearly.#2019-03-3017:28didibusHow would you spec the following: [:keyword1 '(:one :or) '(:more :lists) :keyword2 '(:one :or) '(:more :lists)]
where each pair of :keywordN '(:one :or) '(:more :lists)
is optional?#2019-03-3017:29didibusI tried:
(s/cat :keyword1-pair (s/?
(s/cat :keyword1 #{:keyword1}
:keyword1-lists (s/+ list?)))
:keyword2-pair (s/?
(s/cat :keyword2 #{:keyword2}
:keyword2-lists (s/+ list?))))
But it doesn't work.#2019-03-3017:33didibusNevermind, it does work. I had a typo.#2019-03-3017:34didibusI have another question though. It seems that conform evaluates the forms?#2019-03-3017:35didibusBasically, my above spec is for a macro. And I expect the lists to be code forms. Then I call conform to get it conformed, but it seems to evaluate the forms.#2019-03-3017:36didibus(defmacro foo [& bar]
(s/conform ::my-spec bar))
(foo [:keyword1 (+ 1 1)])
Hum... Unless its evaluated on the print.#2019-03-3017:38didibusAh yes, nevermind again#2019-03-3021:34seancorfield@alexmiller I'm trying the latest Spec2 and running into a new failure (s/defop length-limited-string
"Given n return a spec for a string that is less then n characters
long."
[n]
(s/and string? #(>= n (count %))))
;=> #'ws.billing.specs/length-limited-string
(s/def :wsbilling/braintree-id (s/nilable (length-limited-string 64)))
;=> :wsbilling/braintree-id
(s/def :wsbilling/risk-id :wsbilling/braintree-id)
;=> :wsbilling/risk-id
(s/get-spec :wsbilling/braintree-id)
;=> #object[clojure.spec_alpha2.impl$nilable_impl$reify__12249 0x689396e2 "
This worked fine on the last version I tried, and that last s/get-spec
call returned a spec, as expected, not nil.#2019-03-3021:34seancorfieldIf I change :wsbilling/risk-id
to (s/def :wsbilling/risk-id (s/spec :wsbilling/braintree-id))
then it works.#2019-03-3021:39seancorfieldUnfortunately we have a lot of specs that are defined as aliases of other specs. I happened to spot this one because we have a test that verifies every key in a related spec has a spec defined for it...#2019-03-3021:40seancorfieldAh, it looks like that spec exercises and conforms just fine, it's only get-spec
that "fails" by returning nil
.#2019-03-3021:52seancorfieldOK, it looks like we used to call s/spec
in these cases with Spec1, but that stopped working in Spec2 with the API split into symbolic specs and spec values, so I changed it to s/get-spec
and checked it returned non-`nil`. But what I probably should have done was used s/form
instead which seems to do the right thing in Spec2. It still seems surprising that s/get-spec
returns nil
for the case above tho'...#2019-03-3022:54Alex Miller (Clojure team)I know why youāre seeing that#2019-03-3023:02seancorfieldIs it a bug or should I just not be doing it that way?#2019-03-3023:02seancorfield(I have almost everything passing again on Spec2 -- still trying to track down one failure in one of our newer apps)#2019-03-3023:07seancorfieldOK, everything passes. The only change I had to make was that get-spec
issue (in three places).#2019-03-3023:07seancorfieldI'll take a break for a bit and then start looking at schema
/`select`...#2019-03-3101:50Alex Miller (Clojure team) Bug#2019-03-3103:55seancorfieldNeed a JIRA for it @alexmiller?#2019-03-3103:55Alex Miller (Clojure team)Nah#2019-03-3107:36danielcomptonhttps://danielcompton.net/2019/03/31/announcing-defn-spec#2019-03-3109:06ikitommi@danielcompton awesome! It seems to use schema directly, have you though about copying just the needed parts? Takes 1sec to load just the schema.core
to the repl and most of it is not needed here, I guess?#2019-03-3113:01y.khmelevskii@danielcompton as I can see defn-spec is slightly similar to https://github.com/gnl/ghostwheel, right?#2019-03-3116:45danielcompton@ikitommi yeah I've pretty much copied most of the impl from schema into defn-spec, there was just a little bit left in the macros namespace and I wanted to get a release out over the weekend#2019-03-3116:45danielcomptonBut that's on the list to do, shouldn't be much more work there#2019-04-0107:34tangrammerhi spec people!!
Iām trying to temporarily redefine the spec global registry in the same way as with-redefs
does ā¦
My intention is to reduce possible valid values, eg: instead of only checking that value is a string, validate that is exactly foo
value
so far, this is the code that iāve reachedā¦
(s/def ::simple string?)
(s/def ::composed
(s/keys :req [::simple]))
(with-redefs [s/registry (constantly (assoc (s/registry) ::simple (#'s/res #{"foo"})))]
[(s/valid? ::composed {::simple "other"})
(s/valid? ::composed {::simple "foo"})]
) ;; [false true]
(s/valid? ::composed {::simple "other"}) ;; true
š¤ what do you think about using with-redefs to redefine specs?#2019-04-0112:09Alex Miller (Clojure team)Why?#2019-04-0112:29tangrammerthe final idea is to apply authorisation#2019-04-0112:32tangrammerbasically if certain path values of the speced data are āallowedā, the full data could be viewed by a user or not#2019-04-0112:33tangrammeriās it more clear now?#2019-04-0112:38Alex Miller (Clojure team)Sounds like a nightmare for such a critical role#2019-04-0112:48tangrammercould you expand a bit why you think it could be a nightmare?#2019-04-0113:04Alex Miller (Clojure team)why would you base something as critical as auth on something as fragile as with-redefs?#2019-04-0113:07tangrammersorry i didnāt realise about āThese temporary changes will be visible in all threadsā š¬ ā¦ I was trying for something thread isolated ā¦ but with-redefs was my quick first attempt#2019-04-0115:20djtangoif you want thread-local rebinding - use binding
#2019-04-0115:21djtangothough so far the only things I've found with-redef
useful for is orthogonal instrumentation of functions (e.g. timing functions) or stubbing in testing#2019-04-0115:22tangrammer(def ^:dynamic my-fun string?)
(s/def ::simple #'my-fun)
(s/def ::composed (s/keys :req [::simple]))
(s/def ::extra-composed (s/keys :req [::composed]))
(assert (s/valid? ::extra-composed {::composed {::simple "one"}}))
(assert (= (binding [my-fun #{"foo"} ]
[(s/valid? ::extra-composed {::composed {::simple "other"}})
(s/valid? ::extra-composed {::composed {::simple "foo"}})])
[false true]))
(assert (s/valid? ::extra-composed {::composed {::simple "other"}}))
#2019-04-0115:22djtango(though since spec I've not really redef'd fns for testing)#2019-04-0115:22tangrammerexample using binding
^^ š#2019-04-0115:23djtangoya - still hard to see how / when you might apply that but at least it's thread-safe now#2019-04-0115:23tangrammer:+1:#2019-04-0115:26tangrammer@U064X3EF3 maybe less nightmare now with binding
?#2019-04-0115:34Alex Miller (Clojure team)Less but the idea of changing specs still seems like an idea likely to cause problems later #2019-04-0115:46tangrammerI'll take into account, thanks both for your help!#2019-04-0116:06djtangoIf it's a top level spec you can do:
(let [the-spec (if p? ::a ::b)]
(s/valid? the-spec data))
If it's a nested spec, you could try using :req-un
to have (s/keys :req-un [:very-specific/a])
and (s/keys :req-un [:general/a])
alternatively, you could try experimenting (s/or ...)
If you're using namespaced keys on your data, it's hard to understand why the spec for a namespaced entity would change#2019-04-0121:33tangrammermore than change, the spec should be more specific. so instead of having a predicate string?
would be #{"value1" "value2" "value3"}
#2019-04-0121:34tangrammerbeing these values the returned of a sql execution#2019-04-0112:42tangrammerthe thing (at this level) is getting spec validation messages#2019-04-0113:05djtangogiven that specs are typically named with just data can't you dispatch at real-time the spec you might want to validate against#2019-04-0113:05djtango(with no other knowledge of what you're trying to achieve)#2019-04-0116:04mathpunkI've got a question about spec design. The situation: I have an entity in mind, which I call a run
(for test run). It has two variants: it might be a test run from CI, or it might be a test run from my local machine.#2019-04-0116:06mathpunkSome data is present in runs of any variant, like results
and a revision
(the SHA of the codebase during the run). Only CI runs have a pipeline
. Both variants have logs
but only the local variant has api
logs under that key.#2019-04-0116:08mathpunkMy design problem is, I'm trying to use mostly namespaced keywords, so that I can nest specs. For instance,
(s/def ::revision (fn [s] (boolean (re-find #"[a-f0-9]+" s))))
(s/def ::meta-v1
(s/keys :req [::date ::job ::pipeline ::revision]
:opt [::suites]))
#2019-04-0116:08mathpunkBut then I realized, ooooh when I start modeling runs of the local variant, project.ci.runs/revision
is overly specific#2019-04-0116:09mathpunkDo you have any thoughts on what I should keep in mind while spec'ing these variants out? Thanks!#2019-04-0116:11mathpunkI guess I should add, I'm trying to be careful about not over-spec'ing things --- this could be an example of, Don't spec this part#2019-04-0116:18mathpunk(The value-add that I see of spec'ing this data is, I've been doing these runs for months, and iterating on how they're shaped, so I'm interested in validating that I understand what 'type' of run I'm looking at)#2019-04-0116:19Alex Miller (Clojure team)a good guiding principle is to just try to spec the truth - what are all the values an attribute can actually take on?#2019-04-0116:20Alex Miller (Clojure team)if it's not constrained to the regex, then that's not the truth, it's just one variant#2019-04-0116:20Alex Miller (Clojure team)it's always possible to spec generally, then s/and an additional constraint for a particular context#2019-04-0116:23mathpunkš¤ So maybe,
:project.run/revision
:
#2019-04-0116:23mathpunkoh wait, you said 's/and', not 'and'... I'll have to think about that a minute#2019-04-0117:14kennyHow do you guys use clojure.test.check.generators/let
with Spec? It doesn't appear to be exposed in clojure.spec.gen.alpha
.#2019-04-0201:14taylorI just add the dependency explicitly to the test check namespace #2019-04-0201:15kennyWe don't allow test.check as a runtime dependency.#2019-04-0201:15taylorInstead of using the aliased stuff from spec#2019-04-0201:15taylorYou can add it as a dev only dependency #2019-04-0201:16kennyRight but then you can't define your generators next to your specs.#2019-04-0201:16taylorAlso, youāre depending on it already if youāre actually using any of its functionality#2019-04-0201:17kennyWe don't use the functionality in production but want to define the generators next to our specs.#2019-04-0201:17taylorThe stuff in spec are merely aliases to the test check definitions#2019-04-0201:17kennyTrue but it does it lazily -- it only requires test.check if you actually invoke a function in the gen namespace.#2019-04-0201:18taylorI supposed you could use the same strategy#2019-04-0201:19kennyYes but that begs the question of why it wasn't included to begin with. It may be because it is a macro.#2019-04-0210:24gfredericksyes, a macro has to run at compile time#2019-04-0210:25gfredericksso the normal trick that allows you to not have a runtime dependency on test.check doesn't work#2019-04-0217:14kennySo the solution is to copy and paste the let macro elsewhere?#2019-04-0218:58gfredericksYou can do that, you could relax your requirement to not have test.check in prod, you could wrap in a simple proxying macro that expands to an exception throwing generator when tc isn't on the classpath#2019-04-0219:00kennyThe problem with including test.check on the cp in prod is certain code paths check input structures via s/valid?
. If, by accident, a function is passed in a map of things, it would get generatively checked. That would have a significant impact on performance.#2019-04-0219:03gfredericksThey skip that based on the namespaces being requirable or not?#2019-04-0219:03kennyNot sure what you mean.#2019-04-0219:04kennyIf test.check isn't available on the cp and a fn is passed to the s/valid?
call, an exception will get thrown.#2019-04-0219:05kennyIf it is, no exception is thrown and the performance will slow down silently.#2019-04-0219:06gfredericksAh, I see, I didn't notice the "by accident" part
In any case, the other two approaches would work though#2019-04-0219:07kennyThe latter seems like a good approach. Seems like a worthy addition to the Spec gen namespace as well.#2019-04-0301:11gfredericksI can see that#2019-04-0217:07zalkyHi all, is there an predicate for whether something is a valid spec? Something like specize
(not public api), but returns true or false. I guess I could do:
(defn spec?
[spec]
(try
(do (s/conform spec nil) true)
(catch Exception e
false)))
But maybe I missed something easy?#2019-04-0217:11buttergunshttps://clojuredocs.org/clojure.spec.alpha/spec_q#2019-04-0217:13zalky@mattmorten, thanks for the response. The issue with that is it returns false things that are valid specs, like functions and keywords that resolve to spec objects.#2019-04-0217:14buttergunsAh, sorry, misunderstood. No I don't think I know of anything like you ask#2019-04-0218:13robertfwAre there any suggestions for tracking down "Couldn't satisfy such-that predicate" errors? The exception it throws doesn't appear to give any clues as to which predicate is the problem. I'm wondering if there is a faster option that my current approach of narrowing down my recent changes#2019-04-0218:16Alex Miller (Clojure team)unfortunately, not really atm. I will tell you that most of the time it's from an s/and#2019-04-0218:16Alex Miller (Clojure team)where the initial gen is too wide for the subsequent filter(s)#2019-04-0218:17robertfwyeah, I narrowed it down pretty quickly and it was indeed an and
. just one of those minor annoyances.#2019-04-0218:22Alex Miller (Clojure team)there is some new stuff in the latest test.check but we haven't yet looked at how to make it better to expose the info out through spec#2019-04-0218:22Alex Miller (Clojure team)certainly, it's annoying, and I have spent a lot of time doing the same#2019-04-0223:20didibusQuestion, How do you s/keys a Java Map? Is that even possible? It fails because it says a Java Map is not a map?#2019-04-0223:26robertfwMy first guess would be to write your own predicate function to validate it yourself, I wouldn't expect s/keys
to support it#2019-04-0301:49Alex Miller (Clojure team)spec is based on Clojure maps and is not intended to support Java colls#2019-04-0303:19didibusYa, I think I was confused, I thought all Java collections were Clojure's, but its the other way around only. And since Java Lists are seqable, anyways. I got around it.#2019-04-0303:19didibusthanks#2019-04-0310:22djtango@alexmiller I seem to have some vague memory that you guys were maybe looking at alternatives to generatively testing functions that are argument inputs or map entries, but can't seem to find anything to follow on this subject?#2019-04-0310:22djtangoam I remembering correct? This was prompted by some of the discussion @kenny and @gfredericks were talking about#2019-04-0315:57Alex Miller (Clojure team)not sure what you're talking a bout#2019-04-0315:58Alex Miller (Clojure team)can you give me some more hints? :)#2019-04-0316:17kennyI believe he may be referencing the ability to disable generative fspec
testing in production.#2019-04-0317:27djtango^ this#2019-04-0317:28djtangobut am I imagining that there would have been an alternative way to check if the specs were still correct?#2019-04-0317:38Alex Miller (Clojure team)oh, yeah, my current thought on this is to instead of generatively testing the function to wrap the function with something that checks arg specs of the inputs#2019-04-0409:09djtangoAh nice - is any of the ideas/progress on this happening anywhere public?#2019-04-0411:05Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2#2019-04-0411:06Alex Miller (Clojure team)And I write a weekly journal at http://insideclojure.org#2019-04-0413:45djtangoha - ok is there a jira on this specific bit?#2019-04-0413:47Alex Miller (Clojure team)there is an old ticket for it#2019-04-0413:51Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2217#2019-04-0413:58djtango:+1: awesome thanks!#2019-04-0413:59Alex Miller (Clojure team)there may be others#2019-04-0319:27mpenetThat would be a huge improvement#2019-04-0321:04hadilsQ: I want to define a spec some thing like:
(s/def ::firstName string?)
(s/def ::lastName string?)
(s/def ::email utils/email-address?)
(s/def ::type #{:receive-only :personal :business})
(s/def ::businessName string?)
(s/def ::receive-only-user (s/and (s/keys :req-un [::firstName ::lastName ::email ::type]) (::type equals :receive-only))
#2019-04-0321:05hadilsIs this possible?#2019-04-0321:28hadilsNvm, I answered my own questionā¦#2019-04-0411:32orestisIs there a way for spec to express āif this key is present, this other key must also be presentā?#2019-04-0411:33orestisActually - if this key has this value, this other key must also be present#2019-04-0411:34orestisI suspect itās something you can do with multi-spec?#2019-04-0411:58orestisGot something working multi-spec, but it seems that I have to name the intermediates ā like, if I need a timestamp to be present if something is of type A, I have to name a spec āsomething-with-timeā, whereas the normal case is āsomething-plainā. Or perhaps naming isnāt necessary, and my multi-method can just return an (s/keys [])
?#2019-04-0413:31Alex Miller (Clojure team)(s/keys)
is valid#2019-04-0413:31Alex Miller (Clojure team)and will still validate all registered keys#2019-04-0413:32Alex Miller (Clojure team)sounds like an excellent use case for schema/select in spec2#2019-04-0413:43orestisOh thatās right ā I guess what the actual requirement is, is that for some specific subtype of foo, you have to provide ābarā. If you provide ābarā for a different subtype, well, itās still valid but going to be ignored anyway.#2019-04-0413:46orestisLooking forward to seeing how schema/select looks like.#2019-04-0413:51Alex Miller (Clojure team)it looks like https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-04-0413:52Alex Miller (Clojure team)you can try it now if you like#2019-04-0413:58djtangoThis feels like a bit of an abuse of s/keys
but is this what you mean @orestis (s/def ::m (s/keys :req [::a (or ::a (and ::foo ::bar))] :opt [::foo ::bar]))
#2019-04-0413:59Alex Miller (Clojure team)note that or
here is not xor
#2019-04-0414:00djtangoindeed - got caught out by that once#2019-04-0414:00djtangobut really was just loose reasoning more than anything#2019-04-0414:02orestisI thought that there were incompatibilities between current spec and spec2 ā thatās why I shied away so far.#2019-04-0414:03orestis(As in, Iād rather migrate my specs and introduce the new stuff to the team when itās released)#2019-04-0414:11orestisBTW today was the first time I shared a spec between frontend and backend. So nice! Thanks a lot to everyone involved š#2019-04-0416:35Alex Miller (Clojure team)correct - spec and spec2 are not compatible, but you could try it if you like (I wouldn't use spec2 for anything yet)#2019-04-0417:10seancorfield@orestis We have a branch of our 80K+ line codebase at work running on Spec2 but it's still changing a lot and we've tripped over numerous bugs that have led us to huddle with @alexmiller several times in order to figure out workarounds (and sometimes just waited for the bug to get fixed).#2019-04-0417:10seancorfieldI agree with Alex that it isn't production ready yet but it's definitely interesting to try out.#2019-04-0417:11seancorfieldOur biggest issue was that we had sort of mixed predicates and specs in some places -- that Spec1 allows but Spec2 draws a harder line in the sand around.#2019-04-0417:12orestisIrrelevant to spec2: Someone mentioned that in 1.10 destructuring is done via a spec, but it seems like destructure
is same as before. Is there a spec for what could go into a destructuring binding?#2019-04-0417:12orestisThanks for chipping in @seancorfield ā Iāll put it down on my list to try out, but the backlog is very big š#2019-04-0417:13seancorfieldI know that feeling š#2019-04-0417:13Alex Miller (Clojure team)there is a spec for destructuring in core.specs.alpha, used by many other specs#2019-04-0417:14Alex Miller (Clojure team)it's not used for destructuring#2019-04-0417:14seancorfieldWe've been on the bleeding edge for a long time at World Singles Networks. We regularly take alpha/beta builds to production, and we were early adopters of spec (and very heavy users of it in production). But even for us, Spec2 isn't quite ready yet š#2019-04-0417:14Alex Miller (Clojure team)https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L4-L44#2019-04-0417:14orestisCool!#2019-04-0417:15Alex Miller (Clojure team)btw, Sean, current spec2 should fix that issue you reported (as a side effect of other changes)#2019-04-0417:15Alex Miller (Clojure team)schema/select have been changing a bit this week, but probably not in ways that break anything you've done#2019-04-0417:15seancorfield(oh @alexmiller meant to mention that we've had 1.10.1-beta1 in production for all processes since 3/26 -- no problems!)#2019-04-0417:15Alex Miller (Clojure team)cool#2019-04-0417:16Alex Miller (Clojure team)I've been keeping https://github.com/clojure/spec-alpha2/wiki/Schema-and-select basically up to date though#2019-04-0417:16Alex Miller (Clojure team)will write up some more in weekly journal#2019-04-0417:16seancorfieldThanks, Alex. I may go back to our branch tomorrow and update it and see where we are now. I was working on an enhancement over the last few days that was just screaming out for schema
/`select` BTW.#2019-04-0417:18seancorfieldBusiness wanted an extra field added to a database table (for which we have a spec) and that cascaded up through several layers -- and several specs -- to two of our REST APIs (which also have specs). Dealing with optionality across those tiers is something that schema
/`select` will really help us with, I think.#2019-04-0417:18orestisOn the destructuring front then ā Iād like to write a macro that given a destructuring form as in ::binding-form
, and some kind of other specification (a GraphQL input object, specifically), ensures that the symbols used in the destructuring are present and and in the correct place in the other specification. This is a partly an aid to catch typos, partly a cool thing to show off. Would that ::binding-form
spec be of any use? Perhaps conform
could give something back?#2019-04-0417:19orestisOr should I ājustā hand-roll a validator?#2019-04-0417:27seancorfield@orestis My understanding is that binding form spec already exists and you can use it directly (we do this in one place in our code -- which led to an interesting workaround when we tried Spec2 since the core specs continue to use Spec1 and we needed to bridge between them!).#2019-04-0417:28orestisYeah, TIL: (s/valid? ::cs/binding-form '{:keys [a b c :d]}) ;;=> true
#2019-04-0417:28seancorfield;; ugly workaround to pull spec1 internals into spec2 world:
(eval '(s/def :clojure.core.specs.alpha/bindings
(s1/get-spec :clojure.core.specs.alpha/bindings)))
(s/fdef with-connection
:args (s/cat :bindings :clojure.core.specs.alpha/bindings
:body (s/* any?))
:ret any?)
#2019-04-0417:28orestisNever expected that a keyword would be a valid destructuring key.#2019-04-0417:29seancorfield(I'm shocked that the above workaround even works but...)#2019-04-0417:34Alex Miller (Clojure team)we added keyword support a few years ago, specifically so that you could use ::a, ::x/a, etc and get autoresolve functionality#2019-04-0417:34Alex Miller (Clojure team)that's somewhat been superseded now by :x/keys support but still there for backwards compatibility#2019-04-0417:37orestisGotcha.#2019-04-0417:39orestisSeems like spec isnāt the tool Iām looking for the job I want to do. I would like something that given something like {foo :bar}
and {:keys [a :b]}
would give me the keys that will be destructured on the map. Perhaps I could do some hackery by actually calling destructure
and looking at the generated bindings ā but that seems a recipe for trouble.#2019-04-0417:42orestisSorry for the stream of thought about this. I am wondering if this is actually viable or possible, and if I could get some value out of the core specs (apart from seeing that there are a ton of possible use cases that I never knew existed š )#2019-04-0417:56Alex Miller (Clojure team)oh, yeah, I don't know of anything that will do this job#2019-04-0417:57Alex Miller (Clojure team)unless you're will to eval, in which case you could probably abuse the destructure
function#2019-04-0418:00Alex Miller (Clojure team)user=> (->> '[{:keys [a :b]} {foo :bar}] destructure (apply hash-map) keys)
(a map__2953 b)
#2019-04-0418:01Alex Miller (Clojure team)you will see some gensym cruft (although the cruft is real! those are intermediates that are getting bound)#2019-04-0418:01orestisThis will probably lose the structure for nested restructuring though. Since it will linearize all the bindings. #2019-04-0418:02Alex Miller (Clojure team)yes, it resolves to a single let binding#2019-04-0418:02Alex Miller (Clojure team)but that is the reality of what's happening. going in reverse seems challenging.#2019-04-0418:03orestisI thing walking the tree and having a cond based on the core specs to see which kind of restructuring is there and recurring accordingly would make more sense. I will have to maintain the ācurrentā position against an external tree structure anyway, to answer questions such as āis this key valid for this sub treeā#2019-04-0418:05Alex Miller (Clojure team)there's a lot of destructuring options so lots to do there#2019-04-0418:06orestisHopefully thereās a 80/20 reward (as is usually the case with this kind of linting).#2019-04-0418:08orestisOr there are dozens of academic paper arguing about whether this is possible š#2019-04-0418:08Alex Miller (Clojure team)doesn't seem impossible :)#2019-04-0418:08Alex Miller (Clojure team)just tedious#2019-04-0418:20orestis(defn get-destructured-keys [binding]
(->> (destructure [binding {}])
(filter list?)
(filter #(= (first %) 'clojure.core/get))
(map #(nth % 2))))
(get-destructured-keys '{:keys [foo bar]})
(get-destructured-keys '{a :foo
b :bar})
#2019-04-0418:20orestis70% of the value š#2019-04-0418:25orestis(let [(gql-args {foo :foo} :something) {:foo "bar"}]
foo)
=> this complains that let doesnāt conform to spec. Bummer. I hoped that my macro would run first.#2019-04-0418:26Alex Miller (Clojure team)nope :)#2019-04-0418:26Alex Miller (Clojure team)due to delayed eval, outer is in control first#2019-04-0418:28orestisPerhaps I could do a tagged literal?#2019-04-0418:36Alex Miller (Clojure team)tagged literals are read as objects at read time, can't see that's going to help you in expansion#2019-04-0418:36Alex Miller (Clojure team)you could make your own let variant#2019-04-0418:38orestisWas hoping that I could replace the tagged literal with a valid let binding form at read time, then the let macro wouldnāt know the difference (if I understand the compilation phases correctly)#2019-04-0418:43orestisToo much for a Thursday evening ā gotta run! Thanks for your time.#2019-04-0418:57Alex Miller (Clojure team)might actually work to some limited degree (I imagine some editors and context would run into issues with this)#2019-04-0418:59Alex Miller (Clojure team)user=> (set! *data-readers* {'foo (fn [x] ['a x])})
{foo #object[user$eval2967$fn__2968 0x5b0902b4 "
#2019-04-0419:00orestisOh, thatās how you do a dynamic data reader? I was looking at https://clojure.org/reference/reader#tagged_literals and it suggested binding
which didnāt work in my case.#2019-04-0419:02Alex Miller (Clojure team)it's a dynvar and it's set by the RT so you can set! it for thread-local scope#2019-04-0419:03Alex Miller (Clojure team)or you can set it up in the data_readers.clj file#2019-04-0419:10orestiswhat about the binding suggestion in the docs? Would that work?#2019-04-0419:12Alex Miller (Clojure team)it should, but seems inconvenient for this#2019-04-0419:13Alex Miller (Clojure team)user=> (binding [*data-readers* {'foo (fn [x] ['a x])}]
(let #foo 5 a))
5
#2019-04-0419:15orestisHm, I was doing something wrong. Oh well, thanks for clarifying. Doing it as a tagged literal seems weird but I donāt want to write a separate let macro (perhaps in practice it wonāt be an issue). Thanks again :)#2019-04-0419:18bronsawhat, that should not work#2019-04-0419:18bronsa@alexmiller was that from the same repl you set!
earlier? I think it was only working because of that#2019-04-0420:37Alex Miller (Clojure team)ah, it was#2019-04-0602:08gfredericksset!
in that context, used as a past tense verb, is pronounced "set-bung"#2019-04-0612:54ikitommiIs there a way to construct the selection in s/select
programmatically? e.g. have the selection somewhere as data (in db, edn, etc) and make a select out of it at runtime.#2019-04-0612:55ikitommisame for s/schema
ā¦#2019-04-0612:59ikitommialso, could the selection syntax allow something special that effects the whole keyset, not present an individual key. Could be just a function or a special Protocol.
(s/def ::passwords-must-equal #(= (:password1 %) (:password2 %)))
(s/select ::user [::user ::password1 ::password2 ::passwords-must-equal])
;; or
(s/select ::user [::user ::password1 ::password2 #(= (:password1 %) (:password2 %))])
#2019-04-0613:07ikitommiā¦ if that special keyset-validator would be implemented with a protocol, it would be great to have a function which takes both the spec
and the value
as arguments, so one could easily write something like ::closed-keyset
: the function could extract the list of defined keys from the spec
argument and use that to verify that the value
doesnāt have any extra keys. And if the validation could return the ::s/problems
instead of just ::s/invalid
, it would be easy to write those, e.g. no need to write separate explain too.#2019-04-0613:40metametadata+1, would be nice to have something like that. because currently relying on forms, macros or low-level *-impl
helpers is quite cumbersome (e.g. see my attempt at implementing merge-able closed keys spec with user-friendly explain here: https://groups.google.com/d/msg/clojure/duY3ojPwPYo/Wgvk9PsaCAAJ)#2019-04-0614:36ikitommiI too have almost mergeable closed specs with Spec1, but next hurdle is to get deep-mergeable specs...#2019-04-0617:40Alex Miller (Clojure team)You have two routes to programmable specs in spec 2#2019-04-0617:40Alex Miller (Clojure team)You can make forms, and call spec*#2019-04-0617:41Alex Miller (Clojure team)Or you can make your own syntax that wraps something with defop#2019-04-0617:41Alex Miller (Clojure team)Or actually third option is making your own spec instance from scratch but thatās a much heavier lift#2019-04-0617:42Alex Miller (Clojure team)We are evaluating support for preds in specs - tbd#2019-04-0617:43Alex Miller (Clojure team)But we are working on another kind of checking for closed specs#2019-04-0617:43Alex Miller (Clojure team)Sorry, gotta run#2019-04-0703:57didibus> But we are working on another kind of checking for closed specs
That's really good news. I've had to implement closed keys spec in spec 1 to deal with the following three common case of defects:
1. Keys are mistyped, wrong qualified ns or name, like thinking it is id instead of identifier. Or just plain typo.
2. Keys are misplaced. Like putting the address on the user when it actually goes in the nested profile map. Address on profile being optional, open specs don't catch this bug.
3. Combinations are mistaken. Like say a map which has a query and insert key, this is supposed to be interpreted by some engine, but it is now ambiguous if the operation is supposed to be to query or to insert.#2019-04-0704:01didibusI don't favour the use of closed specs, because they can cause unecessary breakage. And I'm not saying closed specs is the best solution to these above issues. But my team has faced these often, leading to real production bugs. And I haven't found a better solution to them yet.#2019-04-0704:04didibusThe only better idea we had was to seperate the spec of the producer from the one of the consumer. Specifically, code that looks to an input map to read from doesn't need to validate the input as being closed. As all the above bugs are on the producing code. That way, consumers are not broken by producer accretion that are backward compatible.#2019-04-0704:04didibusBut producers would validate their output maps using Closed spec. To ensure they didn't make any of the above mistakes.#2019-04-0704:05didibusWe never implemented it, because with spec1, it would be too tedious to keep seperate producer/consumer specs.#2019-04-0705:37orestisYeah, catching typos is a great use of closed specs. I was attempting going at it from a usage point of view, warning if you are trying to a lookup a key that isnāt guaranteed to be there. #2019-04-0705:37orestisA spec-aware linter would be great.#2019-04-0706:56ikitommi@orestis maybe you could work with @borkdude on clj-kondo for that? Would be awesome.#2019-04-0706:57orestisYeah, I was actually looking into this before. A tricky thing is that you probably need runtime info to get to the spec registry #2019-04-0706:58orestisYou could perhaps just collect s/keys calls statically for a good first approximation though. #2019-04-0707:00ikitommi@didibus there is https://github.com/bhauman/spell-spec for spec1 just for that. Sadly, it also macros so can't program/compose with those.#2019-04-0707:17ikitommi@alexmiller thanks for the info. My use case is this:
reitit components can declare partial specs for the route data they are mounted to, e.g. a parameter-coercion-interceptor says that the route data can have :parameters
key, with a map with keys :query
, :path
etc. for a given route. All the partial specs of the components mounted to a route will be deep-merged for the final effective spec. e.g. interceptor requiring data of form {:parameters {:path ...}}
and another with {:parameters {:multipart ...}}
should be merged to require effectively {:parameters {:path ā¦, :multipart ā¦}}
. And there should be an way to validate that merged spec as āclosedā to fail-fast on typos.
I have just that working using data-specs
+ spell-spec
+ spec-tools
coercion and will demo that soon, but I would want the users of the library not to require anything else than the core.spec to define the component specs.
With spec2, I would like to support the selection
/ schema
syntax as data so that one could say that āmy component requires [{:parameters [{:query ::parameter-map}]}]
and the other [{:parameters [{:multipart ::parameter-map}]}]
and I could just merge those + spec to close them (with a small helper) into [{:parameters [{:query ::parameter-map} {:multipart ::parameter-map} ::closed-spec]} ::closed-spec]
kinda thing, that could be used to check the strict configuration.#2019-04-0707:20ikitommialso, noticed that the nested non-qualified keys donāt work yet, is that a bug or a feature? [{:a int?}]
seems to work, but [{:a [{:b int?}]}]
doesnāt.#2019-04-0711:11Alex Miller (Clojure team)It works if you wrap the inner in s/schema (or register the inner with a name)#2019-04-0707:27borkdude@orestis > You could perhaps just collect s/keys calls statically for a good first approximation though.
Thatās a nice idea. Iāll consider that for clj-kondo.#2019-04-0707:29ikitommiwith top-level merging & spec1#2019-04-0707:32borkdudesomething like this? https://github.com/borkdude/clj-kondo/issues/56#2019-04-0707:58didibusThat would be neat yes#2019-04-0707:59didibusOne thing I was thinking also is to make a custom validate function which can be called with an option to be closed or open.#2019-04-0708:00didibusThat way I could use the same spec, but have the producer code validate it assuming no added keys can be present#2019-04-0711:12Alex Miller (Clojure team)Thatās exactly the direction weāre going #2019-04-0711:12Alex Miller (Clojure team)Itās in how you check, not in the spec#2019-04-0711:38orestisThat makes a ton of sense!#2019-04-0715:26gfredericksliterally friday at work somebody was debugging some code that walks & transforms a plumatic/schema to add openness everywhere, with exactly the same motivation#2019-04-0719:00quollIām pushing up against leaving for my daughterās swimming training, so I figure I should ask in hereā¦
Iām using an ns
form with metadata, in the same way that clojure.core does. i.e.
(ns ^{:doc "some docs" :author "me"} my-ns)
#2019-04-0719:00quollHowever, this fails on Clojure 1.10, due to āExtra inputā#2019-04-0719:01gfredericksworks fine for me if I paste that in a repl#2019-04-0719:01quollsure, I can just go back to a standard docstring and drop the :author metadata, but given that clojure.core uses this I thought Iād question it#2019-04-0719:01quollyup, same here#2019-04-0719:01quollbut leiningen hates to build it š#2019-04-0719:02quolllet me put together something minimal#2019-04-0719:02gfredericksyeah, I just tried it in leiningen and it worked fine#2019-04-0719:03gfrederickshalf of the time when I'm debugging something, trying to get a minimal reproduction also exposes the real problem#2019-04-0719:09quollOK, itās looking like I had a typo elsewhere that got through the compiler before spec was introduced š³#2019-04-1114:48vemvIs there something like (describe-spec :my/spec)
that will print what :my/spec
is about? Maybe recursively.
Loosely analog to macroexpand
/`macroexpand-all`#2019-04-1115:02vemv(for now I use Emacs niceties but I'm interested in editor-agnostic tooling)#2019-04-1115:19Alex Miller (Clojure team)doc
works on registered specs#2019-04-1115:20Alex Miller (Clojure team)it is, however, not deep/recursive#2019-04-1115:20Alex Miller (Clojure team)(doc :my/spec)
#2019-04-1116:38fedregHi all, can someone help me with specāing a higher-order fn? One of the arguments to a fdef
is a higher order fn that Iāve defined as: (s/def ::foo-fn (s/fspec :args any? :ret ::foo))
This works except running something like test.check
on the fn that takes foo-fn
as an arg is super slow. ā¦If I just pass ::foo
in instead of ::foo-fn
everything generates as quickly as usual so Iām guessing Iām doing something wrongā¦ š . Any thoughts? thx!!
ā¦ohā¦ and ::foo has no args but not sure how to express that.. :args nil
doesnāt work#2019-04-1117:06Alex Miller (Clojure team):args (s/cat)
is prob best#2019-04-1117:07Alex Miller (Clojure team)fspecs are instrumented by generating args and invoking the passed function a bunch of times#2019-04-1117:07Alex Miller (Clojure team)this is sometimes surprising and/or bad for people#2019-04-1117:07Alex Miller (Clojure team)an alternate option to using an s/fspec
here is to just spec it as ifn?
#2019-04-1117:10fedreg@alexmiller Will try those out! Thanks!!#2019-04-1117:18fedregā¦Using :args (s/cat)
instead of any?
definitely sped things up.. Canāt use ifn?
for my specific use case but still generating / running through my system about 5 results per second which is pretty fast. ā¦just spoiled by how fast test.check
usually runs data!#2019-04-1117:34Alex Miller (Clojure team)any?
can generate very large nested colls#2019-04-1117:35Alex Miller (Clojure team)(s/cat)
can only generate one thing, ()
#2019-04-1117:43fedregyes.. I knew that any
was slowing things down but couldnāt figure out how to do no argsā¦ Didnāt think to try (s/cat)
so thanks for that!#2019-04-1217:06fedregCan anyone point me to an example of passing custom generators into the options of spec.test.alpha/check
? :gen map from spec names to generator overrides
ā¦I need to limit the range of some generated ints and canāt seem to get it. thx!#2019-04-1506:23Jakub HolĆ½ (HolyJak)Given a spec, how can I generate random value(s)? Does spec.gen/generate take a spec or is there another fn? Thank you!#2019-04-1507:46jaihindhreddyclojure.spec.alpha/gen
takes a spec and returns a generator for it.
clojure.spec.gen.alpha/sample
takes a generator and gives you a few examples. Takes an optional number of examples to generate
clojure.spec.alpha/exercise
takes a spec and gives you a seq of pairs, each containing generated example value, and the same value conformed.
Check out their docstrings.
Also, it's highly recommended to read the Spec guide in its entirety (https://clojure.org/guides/spec)
It clears a lot of these things up (including potential gotchas like you need to include test.check
for some of the functionality to work)#2019-04-1515:43jaihindhreddy@holyjak ^#2019-04-1614:22borkdudecan macro spec checking (for core macros) be disabled from the REPL instead of with a Java property?#2019-04-1614:32Alex Miller (Clojure team)not currently. such a thing could be done by just setting the clojure.lang.RT/CHECK_SPECS flag, but that's not public currently#2019-04-1616:40jumarIs there any reason why the clojure.spec.skip-macros
isn't re-loaded dynamically in Compiler every time it's needed? Performance?#2019-04-1616:43Alex Miller (Clojure team)Didnāt seem necessary#2019-04-1614:33Alex Miller (Clojure team)you could remove those specs from the registry#2019-04-1614:33Alex Miller (Clojure team)s/def with nil will remove#2019-04-1614:34borkdudeI tried (clojure.spec.alpha/def clojure.core/for nil)
already, but that didnāt do it#2019-04-1614:34borkdudeoh it does work, sorry#2019-04-1614:35borkdudeI was testing if for
supported beginning with a let
but it doesnāt š
(for [:let [x 1] y (range x)] y)
#2019-04-1614:36borkdudeand I wanted to make sure I wasnāt getting an error from the spec. thanks#2019-04-1621:18hiredmanI dunno if nit picking on spec-alpha2 is useful feedback yet, but https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/spec_alpha2.clj#L51-L54 is checking if an object implements the interface behind the Schema protocol instead of checking for satisfying the Schema protocol (the namespace imports the interface behind the Spec and the Schema protocols, but is careful to use the Spec protocol everywhere and not the interface)#2019-04-1621:47Alex Miller (Clojure team)Yeah, thatās actually intentional for perf, same in spec 1#2019-04-1621:47Alex Miller (Clojure team)Well same for spec?#2019-04-1621:48Alex Miller (Clojure team)May change#2019-04-1621:52hiredmanah, I see, I missed the use of the interface there. and it looks like spec? will return true if you implement the interface, or if you have metadata with the conform* function(satisfy the protocol via metadata), but will return false if you extend the protocol to some type, which seems idiosyncratic#2019-04-1621:55hiredmanI guess I'll stay tuned#2019-04-1622:05Alex Miller (Clojure team)Yeah#2019-04-1622:05Alex Miller (Clojure team)In any case, Iām aware :)#2019-04-1623:05bronsahttps://dev.clojure.org/jira/browse/CLJ-1814 homerdisappear#2019-04-1916:58lilactownI have a multi-spec for different query types to a service. I also want to dispatch on the query type to actually query the service#2019-04-1916:58lilactownwhat's the best way to consolidate these two things?#2019-04-1917:00lilactowncause atm I have:
(defmulti query-type :type)
(defmethod query-type :view/id [_]
(s/keys :req-un [:content.query/id]))
(defmethod query-type :fragment/id [_]
(s/keys :req-un [:content.query/id]))
and then for implementation:
(defmulti query (fn [conn q] (:type q)))
(defmethod query :view/id
[{:keys [url]} {:keys [id]}]
{:url (str url "/view/" id)
:method :get
:as :json})
(defmethod query :fragment/id
[{:keys [url]} {:keys [id]}]
{:url (str url "/view/" id)
:method :get
:as :json})
and then I need to create a wrapper function so I can fdef the implementation, right?#2019-04-1918:12mishaafaik spec does not work for multimethods, but you can declare implementation function(s), fdef it/them, and make your multimethod call it/them#2019-04-1921:39lilactownit looks like multi-specs s/keys check all available keys, even when not specified?#2019-04-1921:39lilactownactually not sure if this is multi-specs or s/keys doing this#2019-04-1921:40Alex Miller (Clojure team)s/keys does this#2019-04-1921:41lilactownhrm#2019-04-1921:41seancorfieldIf you have a qualified key that matches a spec, it will be checked, whether it's specified in s/keys
or not.#2019-04-1921:41Alex Miller (Clojure team)this is documented in the s/keys docstring#2019-04-1921:41lilactowngotcha - it's just not what I want right now š#2019-04-1921:56dominicm@alexmiller just read your inside clojure. The api seems very global for closed specs. Have you given much thought to how it might be scoped?#2019-04-1921:58Alex Miller (Clojure team)Scope in what dimension?#2019-04-1921:58Alex Miller (Clojure team)It is very global, like instrumentation #2019-04-1921:59dominicmMaybe with a dynamic binding, so thread local.#2019-04-1922:00dominicmThinking of a web server with many concurrent requests where you want the closed spec on the boundary, but internally within processing the open spec may be used (as other sources are merged).#2019-04-1922:01borkdudeInteresting choice to turn off and on āclosednessā for specs. Would it make sense to combine this with select somehow?
Like (s/def ::closed-spec (s/select ::open-spec ...))
?#2019-04-1922:02Alex Miller (Clojure team)It should work with select too, I just havenāt done that yet. Thereās just a lot of copied code there at the moment#2019-04-1922:03borkdudeno, I mean, turn an open spec into a closed one using select, not by a global flag#2019-04-1922:03borkdude(then select would have to support that)#2019-04-1922:03Alex Miller (Clojure team)then, no :)#2019-04-1922:04Alex Miller (Clojure team)Closeness is intentionally not part of the spec language and wonāt be#2019-04-1922:04Alex Miller (Clojure team)Itās part of the api#2019-04-1922:04Alex Miller (Clojure team)@dominicm still thinking about this aspect#2019-04-1922:05borkdudebut when you would like to use a spec in different ways, you would have to toggle this flag all the time?#2019-04-1922:05Alex Miller (Clojure team)Thatās the aspect weāre think about still#2019-04-1922:06borkdudeI guess you can just make a copy of the spec, like (s/def ::foo ::bar)
, but if you close bar, will foo also be closed?#2019-04-1922:07Alex Miller (Clojure team)Well thatās not a copy, itās an alias#2019-04-1922:07borkdudeyeah#2019-04-1922:07dominicm@borkdude feels like working around the intentional limitation to make closed&open copies.#2019-04-1922:07Alex Miller (Clojure team)The answer is tricky, and may change#2019-04-1922:09dominicm@alexmiller btw, why the global on/off switch? My reaction was that I'm going to see someone turning it on globally unconditionally in some project and removing the upside of open specs.#2019-04-1922:10Alex Miller (Clojure team)Do you just mean the no-arity version of close/open?#2019-04-1922:23dominicmI do, yeah#2019-04-1922:11borkdudeis s/valid? part of the spec language? why not make it an option to those functions?#2019-04-1922:11Alex Miller (Clojure team)s/valid? is api, not spec language#2019-04-1922:12borkdudeyeah ok. I think having it as an option is more flexible than setting it globally#2019-04-1922:12Alex Miller (Clojure team)Still might do that. Breaks all existing APIs and specs though#2019-04-1922:12borkdudeextra arity doesnāt break?#2019-04-1922:13Alex Miller (Clojure team)It will break the protocol#2019-04-1922:14Alex Miller (Clojure team)Weāve also been thinking about a fast s/valid? path. Right now thatās tunneled through the conform* protocol, which does a lot of useless work if just validating. So that could also be a flag, or a separate protocol method#2019-04-1922:14borkdude(s/valid2? ā¦)
š#2019-04-1922:15Alex Miller (Clojure team)Those could all be flags internally on conform*#2019-04-1922:15Alex Miller (Clojure team)We are already in a new namespace, no need to number#2019-04-1922:16Alex Miller (Clojure team)But ideally could minimize breakage for some of those spec libs#2019-04-1922:17Alex Miller (Clojure team)Anyhow, what Iām interested in hearing now is how you would use it#2019-04-1922:17Alex Miller (Clojure team)We can work out the best answers from that#2019-04-1922:17borkdudea fast s/valid? path would certainly help things like core specs which quickly become noticable in performance#2019-04-1922:18Alex Miller (Clojure team)Itās not going to be like 10x or anything#2019-04-1922:18Alex Miller (Clojure team)It really matters when you are constructing conformed outputs#2019-04-1922:18borkdudeI think the closedness aspect is something where people choose different tools or use workarounds for spec right now at API level things, like maybe Reitit and yada (which now uses Schema), but Iāll let those maintainers chime in#2019-04-1922:22Alex Miller (Clojure team)Like how often are you likely to use the same spec in both open and closed check modes on the same spec? And where different, is it per usage or per thread or per what?#2019-04-1922:27dominicm@alexmiller I can imagine closed for the incoming boundary, open internally when merging additional internal data, and potentially closed again when sending it on to another consumer (e.g. A service which accepts too much or a database)#2019-04-1922:29borkdudethatās exactly how I would use it too and probably in combination with select#2019-04-1922:30dominicmI'm not the maintainer of yada, but I'm influential there. For incoming web requests, I'm more interested in:
- not checking extra keys
- not seeing extra keys#2019-04-1922:31dominicmI don't want to throw if there's extra keys, probably, just silently drop them.#2019-04-1922:33dominicmSorta, but recursive :)#2019-04-1922:57seancorfieldMy first reaction is: ugh, no! Mutable global state on spec behavior just feels horribly wrong. I could have a spec defined in one place and just looking at that and the uses of it no longer tells me whether extra keys are permitted or not, because any random code, anywhere in the system can toggle that setting :face_vomiting: :exploding_head: š #2019-04-1922:58seancorfieldIt seems like the absolute worst of all worlds, to be honest.#2019-04-2000:27droneso, we use records and have a macro that generates a spec which includes checking fields as well as asserting the object is an instance of the backing record class (the latter part can also be used in isolation as a fast check using instance?
). because of this, I didnāt realize keys
generates open specs#2019-04-2000:30droneah, partial nm. forgot about :req
and :req-un
, which we use behind the scenes#2019-04-2000:30dronewhatās the use case for a spec that describes keys but permits maps that have none of those keys?#2019-04-2000:32droneand agreed with above that mutating closedness seems like a bad idea#2019-04-2000:55seancorfield@mrevelle One observation about records is that they are open too -- you can assoc
in any additional keys you want and it remains an instance that record type.#2019-04-2000:56droneyeah, and that makes sense. but my expectation is that a record (or keys spec) provides a base set of keys that should be present#2019-04-2000:56seancorfieldMy view of :opt
in s/keys
is that it mostly exists for generative testing (and partly for documentation).#2019-04-2000:57drone(s/def ::f string?)
(s/def ::l string?)
(s/def ::s (s/schema [::f ::l]))
(s/valid? ::s {::x 10}) ;; "extra" keys are ok
;;=> true
#2019-04-2000:57dronethat seems wrong. but maybe Iām missing something#2019-04-2000:58seancorfieldselect
is how you specify required-ness in spec2.#2019-04-2000:58droneso the default, when no select
is provided, is to not include anything?#2019-04-2001:04seancorfieldhttps://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-04-2001:05seancorfield"Schemas do not define "required" or "optional" - it's up to selections to declare this in a context."#2019-04-2001:06seancorfielduser=> (s/valid? (s/select ::s [::f ::l]) {::x 10})
false
user=>
#2019-04-2001:07dronethanks, sorry for the easily researched question. and ughā¦#2019-04-2001:07seancorfielduser=> (s/valid? (s/select ::s [::f ::l]) {::x 10 ::f "F" ::l "L"})
true
user=>
#2019-04-2001:11seancorfieldThe schema
/`select` stuff is going to be very helpful for us -- it will simplify our API specs quite a bit. Instead of needing to have different specs for otherwise very similar API calls, we can have a schema
for the "family" of parameters that can be passed in and then use select
to specify what a particular API requires.#2019-04-2001:12seancorfieldI'm still thinking about how it can help with the specs around our database, but I think it can help with the required-for-insert vs required-for-update scenarios.#2019-04-2001:17droneyeah, sub-spec selection seems like a good idea. I think itās just that the undefined default behavior is to not include any of the keys/fields#2019-04-2001:18dronespec2-scratch.core> (s/valid? ::s {::x 10 ::f 10})
false
#2019-04-2003:58seancorfield@mrevelle This is not valid because ::f
's value is not string?
per your spec for ::f
. That makes perfect sense to me.#2019-04-2003:59seancorfields/keys
works that way today.#2019-04-2004:01droneOh, I donāt disagree. Was pointing out that a unselected spec field becomes active when it matches a key even though itās not selected.#2019-04-2004:01seancorfieldYes, which makes sense in the context of the way Spec has always worked.#2019-04-2004:02seancorfieldIf you added a spec for ::x
as string?
then (s/valid? ::s {::x 10})
would fail -- because 10
is not string?
.#2019-04-2004:03seancorfieldThat's how s/keys
works today. That's how s/schema
and s/select
work in Spec2: if you provide additional keys, they're still checked if there's a spec for them.#2019-04-2004:05seancorfielduser=> (s/def ::y string?)
:user/y
user=> (s/valid? (s/select ::s []) {::x 10})
true
user=> (s/valid? (s/select ::s []) {::x 10 ::y 11})
false
user=> (s/valid? (s/select ::s [::f]) {::x 10 ::y 11 ::f "s"})
false
user=> (s/valid? (s/select ::s [::f]) {::x 10 ::y "a" ::f "s"})
true
#2019-04-2004:06droneagain, I understand and am not disagreeing with you. But you pointed out that the docs for spec2 say use of a schema without select
is undefined and Iām providing examples of how saying something is undefined doesnāt mean there isnāt a default#2019-04-2004:07droneand for my use case, itās a poor default#2019-04-2004:07seancorfieldI didn't say it was undefined (as in "undefined behavior"). I quoted the part that says schema
says nothing about required-ness -- that's what select
is for -- and that decoupling is exactly what Rich was talking about in Maybe Not.#2019-04-2004:09seancorfieldWhat I do think is odd, right now, is that you can select
keys that are not in the .#2019-04-2004:10dronefrom the docs:
> Schemas do not define ārequiredā or āoptionalā#2019-04-2004:11droneand that is weird#2019-04-2004:11seancorfieldI disagree. It's exactly what Rich talked about.#2019-04-2004:11seancorfieldThis, on the other hand, seems wrong: user=> (s/def ::s (s/schema [::f ::l]))
:user/s
user=> (s/valid? (s/select ::s [::f ::x]) {::f "s"})
false
user=> (s/valid? (s/select ::s [::f ::x]) {::f "s" ::x 10})
true
#2019-04-2004:12seancorfield(there's no spec for ::x
but the s/select
call makes it required, even tho' it isn't part of ::s
)#2019-04-2004:12droneI guess my point is, based on Richās talk and the current spec2 docs, (s/valid? ::s x)
shouldnāt be permitted since the schema is being used without a select
#2019-04-2004:13seancorfieldIn the first cut of schema
/`select` that landed in the repo, schemas were not specs, so that was not allowed.#2019-04-2004:13seancorfieldI'm not sure why the change was made to turn schemas into specs.#2019-04-2004:15droneah, I see#2019-04-2004:16seancorfieldThis is the commit that changed that https://github.com/clojure/spec-alpha2/commit/1aa141dcdadca88c25afa73e486b3707eaed9d99#2019-04-2004:17droneIām mostly just bummed that spec seems to be going away from being a more-flexible structural typing substitute and into something that Iām not sure fits at all with how I use Clojure#2019-04-2004:18dronethanks for the link#2019-04-2004:22seancorfieldI'm not sure I follow you... select
is much more powerful than s/keys
-- and now you can define the overall shape of your data once, and then validate against the relevant parts of it as needed, without having to write multiple s/keys
specs that all overlap.#2019-04-2004:24seancorfieldReading over the wiki page again in more detail, I wonder if the drive for that change (the commit above) came from wanting to treat schemas like specs because of data generation?#2019-04-2004:26seancorfieldThere's an example of generating against a schema, which produces random selections of keys from the set in the schema. Which would all be "valid" in any context that accepted that schema without placing any required-ness constraints on the contents. Seems like a slim driver to me tho'...#2019-04-2004:32dronethatās true, itās not that something canāt be done. but itās unclear how compatible it is with other use cases. donāt want to end up working against the language#2019-04-2004:33droneand I think youāre probably right about generation as motivation#2019-04-2017:40seancorfieldAlex responded to my question on that commit: "The big thing that tipped it is the unqualified key support which does enforce the specs of unqualified keys (in addition to checking the values of any qualified keys against the registry)." -- because the unqualified key schemas include the spec and don't exist outside the schema (so the schema is a useful spec mostly for unqualified keys which would not otherwise be checkable).#2019-04-2001:18dronebut I didnāt select ::f
from ::s
ā¦#2019-04-2001:24ikitommi@alexmiller About the closed specs. I agree on @dominicm on the open/closed on the borders. We want to drop out extra keys at the borders (recursively) and also coerce the values. Inside they can be open. Spec-tools (and by so, reitit) does both stripping extra keys & coercion already, but using the s/form
parsing, which feels wrong. There is CLJ-2251 I wrote about the different ways to walk the Specs from inside, not outside. Also the fast path of just validating.#2019-04-2001:25dronei.e., based on what Rich presented and my lurking on here, Iād expect select
to be used to choose a subset of keys when the full record isnāt expected or needed. having it default to none seems silly when itās much easier to write (s/select ::s [])
when none are required than enumerate all the fields in a spec when you want all to be required. I also think itās less surprising, but thatās subjective#2019-04-2007:25seancorfield> when itās much easier to write (s/select ::s [])
when none are required than enumerate all the fields in a spec when you want all to be required
FYI (s/select ::s [*])
is the "all fields required" case#2019-04-2015:47dronecool, thatās a bit better#2019-04-2001:34ikitommiI would put the closed-checking into select
: Schemas would be open by default, but one can define select default to open or closed maps, which can be explicitly defined to different (sub)selects too.#2019-04-2001:35ikitommi(s/select ::user [::id ::name ::address {::address [::street s/closed]} s/closed])
#2019-04-2001:37ikitommi(s/select ::user [::id ::name ::address {::address [::street]}] [s/closed]) ;;3rd arg will be merged into all submaps
#2019-04-2001:46wilkerluciomaybe moving to the select get back on the closed problem again, it will always be closed, I agree with @borkdude in this one, maybe the point to make this decision is s/valid?
, this way you keep the possibility to stay open, but can do closed checks when it makes sense#2019-04-2001:53ikitommiwith s/valid?
is would be all or nothing. In s/select
, one could close it just partially. Not sure if there is a real life case for that thou.#2019-04-2405:48didibusIt seems you register which spec are to be validated as closed and open. So the s/valid could be partially closed as well, if you broke it up into named specs.#2019-04-2003:24eggsyntaxIndependently wrote a response to the open/closed approach which pretty much echoes the same things other folks are saying above ā global-stateful-ness bothers me, what about threads, etc. The need for closedness seems to me like itās usually context-specific and scope-limited, so I was expecting a limited-scope approach. My first inclination would be to be able to do something like
(s/valid? (s/closed-variant ::bar) my-bar)
but I understand youāre trying to make sure it doesnāt become part of the spec language.
If itās about limiting it to the API, what about a separate s/closedly-valid?
(or strictly-valid?
or strictly-correct?
or whatever)? And maybe an equivalent for s/explain
. Or I like what I think @wilkerlucio is suggesting, making it available as an extra-arity arg to s/valid?
(off by default so it doesnāt break existing code).#2019-04-2003:56seancorfield@eggsyntax My first thought was s/valid-closed?
but then you need a variant of each of the explain*
functions and conform
and probably several others... which gets ugly fast...#2019-04-2020:42eggsyntaxWhat about maybe a separate s/closed?
function that just verifies whether the data structure contains only the keys in the spec? It doesnāt seem like a big burden to do
(and (s/closed? ::spec x) (s/valid? ::spec x))
#2019-04-2007:29metametadataIf closed/open status becomes an option to some s/valid*
fn then I don't see how the closed spec can be specified to validate args in defn-spec
or something like that. So "closedness" should be attached to the spec instance.
And I guess I'd also intuitively expect (s/close-specs ::s)
to return a new instance of the spec. Even though I don't yet have cases where I need to open or close the already created spec, I'd expect it to return new values instead of "mutating" existing ones. Otherwise, there's a risk to have all the old problems with mutable data like threading, reasoning about code, etc.#2019-04-2017:30adamShould I be using spec for server side form validation or https://funcool.github.io/struct/latest/ or something else?#2019-04-2017:41seancorfield@somedude314 We use Spec for that. We use it for API parameter validation, server-side form validation, and several other places in our production code.#2019-04-2017:42adam@seancorfield do you use any helper libraries built on top the spec or just vanilla spec?#2019-04-2017:44seancorfieldJust vanilla spec. We wrote some code on top of explain-data
to turn Spec failures into domain-specific error messages but that's it.#2019-04-2017:45seancorfieldAnd we mostly do it like this: (let [params (s/conform ::my-spec (:params req))]
(if (s/invalid? params)
(report-error (s/explain-data ::myspec (:params req)))
(process-form-data params)))
(pseudo-code)#2019-04-2017:47adamCool thanks. I will give it a try. This is the first time I am trying to validate stuff in Clojure so still unsure what is practical and what is not.#2019-04-2022:47ikitommiDid a demo in Clojure/North about closed specs, with spec.alpha, spec-tools, spell-spec and expound, using the new reitit error formatter: https://vimeo.com/331602826. The code calls spec-tools.spell/closed-spec
function on a spec and gets a closed (spell-)spec back.#2019-04-2320:45robertfwI have a question about how people handle writing specs for an API endpoint. I'm speccing out ring request maps, and have several endpoints in the same namespace. a ring spec typically involves providing specs for :status
, :params
, sometimes :body
. However my spec names end up being quite wordy; for example, the expected body for a successful response of a given endpoint ends up being :my.ns.endpoint-name.response.success/body
. Over the course of defining several endpoints, with potentially multiple possible responses, I end up with quite a lot of typing. I've been considering either moving each endpoint spec into its own namespace, so that I can use ::
shorthand, but I'm not wild about the proliferation of files that will create. My other thought is to create some macro or function to help generate these specs. Any thoughts?#2019-04-2321:35seancorfield@robertfrederickwarner You can create a namespace alias without needing an actual namespace file.#2019-04-2321:53robertfwthis looks to work nicely š thanks#2019-04-2405:23didibusWow didn't know that#2019-04-2321:36seancorfield(alias 'm (create-ns 'ws.domain.member))
and then use ::m/whatever
#2019-04-2321:37robertfwah, I did not know about alias
. I had considered trying using (ns ...)
but rejected it as it seemed to be pretty unidiomatic#2019-04-2321:39seancorfield@alexmiller has mentioned the possibility of improving alias usage but I don't think any concrete proposal has been made yet.#2019-04-2321:39robertfwIt would be nice if it were possible to define a map spec assigning specs to given keys; I know that seems to be going against the grain of spec but it would cut down some lines of code needed to spec out json endpoints#2019-04-2321:40robertfwe.g... we get data coming in as :some_value
whereas in our system it's :some-value
, so there's another spec to handle the conversion#2019-04-2321:40robertfwI've been digging around to see if anyone has written helpers for that but no luck#2019-04-2321:43Alex Miller (Clojure team)spec2 has support for this for unqualified keys in s/schema#2019-04-2321:43robertfwprayers: answered#2019-04-2321:44Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-04-2321:45robertfwIs there a rough idea of when spec2 might be "ready"?#2019-04-2321:46seancorfieldDefine "ready" š We have a branch of our codebase at work that runs on Spec2.#2019-04-2321:48robertfwgood question - I guess, active development finished and moving into a phase of broader community use, similar to where spec.alpha is now#2019-04-2321:52Alex Miller (Clojure team)I would say it's likely on order of months given the set of work we have talked about#2019-04-2321:53seancorfieldCould you give us an overview of what additional work on Spec2 is still in the pipeline?#2019-04-2321:53Alex Miller (Clojure team)hard to say whether that's 2 months or 6 but certainly I would love have it before the conj#2019-04-2321:54robertfwthat's what i suspected. thanks š#2019-04-2321:58Alex Miller (Clojure team)we have a list of 10 areas of work for spec2#2019-04-2322:00Alex Miller (Clojure team)some of those we'd like to work through before "release" and re-integration, some could be add-ons later#2019-04-2322:01seancorfieldI'd like to cut our project at work over to Spec2 but I don't know how much rework to expect (I mean, I'm used to some level of rework given how often we rely on Clojure alphas š )#2019-04-2322:03Alex Miller (Clojure team)you mean rework between now and "the end" ?#2019-04-2322:05seancorfieldBetween now and non-alpha status.#2019-04-2322:05Alex Miller (Clojure team)well, I wouldn't recommend that yet but I'm not expecting that the existing apis are going to change much#2019-04-2322:05seancorfieldAlthough we're heavily reliant on Spec1 Alpha in production so I don't know why I'm all that worried...#2019-04-2322:11eggsyntax@alexmiller don't know if you already saw it, but there are a few more ideas above on s/closed
.#2019-04-2518:33pyrHi#2019-04-2518:33pyrnow that usage of spec is pervasive and given the fact that defrecord produces handy functions#2019-04-2518:35pyrwould it make sense to add another one which would create a predicate#2019-04-2518:35pyrto test for protocol satisfaction?#2019-04-2518:36pyri.e (defrecord MyRecord) ;; => implies ->MyRecord, map->MyRecord, and MyRecord?#2019-04-2518:38borkdudeyou mean like (instance? Foo (->Foo))
?#2019-04-2518:38dronewhere MyRecord?
is a predicate returning true if its argument is an instance of the backing MyRecord
class and/or it has the required record fields with values that match some field specs?#2019-04-2518:39pyr@mrevelle: yes#2019-04-2518:40pyrthe former#2019-04-2518:40pyrmeh, now that I say it out loud, it makes more sense for protocols#2019-04-2518:40pyrnevermind#2019-04-2518:43droneI think thereās value in specāing fields of records (just as there is in specāing plain-old maps). as for protocols there are some technical reasons they arenāt supported. the workaround that was thrown around in the past is having protocol method implementations call a specād function#2019-04-2518:45pyrok#2019-04-2518:48dronere: records. it sounds like most people donāt use them (?) and stick with maps and occasionally add some :tag
or :type
entry to achieve the ability to determine the ākindā of map#2019-04-2518:51dronenot sure if records were too JVM-centric or if they feel too restrictive to the free-spirited majority opposed to types#2019-04-2518:52borkdudewerenāt they a little bit better for performance?#2019-04-2518:53borkdudealso in Clojure Applied it is recommended for ādomainā objects, although I rarely see them used#2019-04-2518:53pyrI like the fact that you can directly attach protocols#2019-04-2518:54droneyeah, you get fast single dispatch (like a Java class, since theyāre implemented as Java classes)#2019-04-2518:54Alex Miller (Clojure team)fwiw, I would probably de-emphasize records in a 2nd ed of Clojure Applied#2019-04-2518:56pyrextend-via-metadata will help with that#2019-04-2518:57droneI use them all over in machine learning, data analysis, and program analysis work#2019-04-2518:57pyr@alexmiller so fewer records but protocols still emphasized, correct?#2019-04-2518:58pyrspec and qualified keywords make maps more attractive indeed#2019-04-2518:58pyrbreaking out of the protocol + record-implementation will be hard#2019-04-2518:58pyrespecially in our component heavy codebase#2019-04-2518:58droneseems like thereās plenty of boiler plate that packaging things as records could manage for you#2019-04-2519:00dronedefrecord-spec
where each field has an associated spec. specād record creation functions, reference the field specs elsewhere when performing a subset select with select
#2019-04-2519:01dronethe only place I see them being awkward is when youāre building up the fields for a record. which I believe is an example Rich referenced in Maybe Not or another recent-ish talk#2019-04-2519:02dronebut, itās not that bad to just use maps with a sub-select spec until youāve gathered all the required fields and then throw it into a record#2019-04-2519:04dronefeels like serialization is motivating much of this. e.g., you may have some fields that are handy but can be computed from other fields in the map. so when you serialize maps for storage in a database, you drop those derived fields. but then results from the database are āincompleteā and processing is required to generate the derived fields before you could construct the record#2019-04-2519:08dronedatomic killed records, is what Iām saying (with tongue in cheek)#2019-04-2710:55kommenplaying around with spec-alpha2, I was wondering if there is a way to make s/conform
of an s/select
only return keys of a map which are actually specified in the select.#2019-04-2710:56kommen(s/def :foo/id string?)
(s/def :foo/text1 string?)
(s/def :foo/text2 string?)
(s/def ::foo (s/schema [:foo/id :foo/text1 :foo/text2]))
(s/def ::foo-only-text1 (s/select ::foo [:foo/id :foo/text1]))
(s/conform ::foo-only-text1 {:foo/id "bar" :foo/text1 "t1" :foo/text2 "t2"})
;; => #:foo{:id "bar", :text1 "t1", :text2 "t2"}
#2019-04-2716:15Alex Miller (Clojure team)No, by design#2019-04-2716:16Alex Miller (Clojure team)Itās not working this way at the moment but select over a closed schema should throw invalid on that case #2019-04-2716:16Alex Miller (Clojure team)The close stuff is going to get completely overhauled though so current api will change #2019-04-2914:02dominicmI was curious about this, could you share what the current thinking is? Even with the disclaimer of it being heavy WIP?#2019-04-2914:05Alex Miller (Clojure team)rather than the stateful api (which was intended to be similar to instrument), will be more op-specific, and actually more generic than just "closing", will open up some api space for other kinds of checking variants#2019-04-2914:05Alex Miller (Clojure team)so, new options on valid?, conform, assert, etc - anything with checking#2019-04-2914:11dominicmSounds really interesting. Looking forward to seeing how much leverage this provides.#2019-04-3008:20ikitommichecking variants like runtime transformations / coercion? or just validating? anyway, generic sounds great. :+1:#2019-04-3011:41Alex Miller (Clojure team)Not transformations#2019-05-0115:03metametadata> so, new options on valid?, conform, assert, etc - anything with checking
it looks like it will create friction for adopting closed maps in defn-spec
and other libs which don't know about new options in the methods and rely on the fact that every spec itself describes everything needed for validation#2019-05-0115:05Alex Miller (Clojure team)that's something defn-spec will have to reckon with. function specs themselves are going to go through significant changes and making an integrated defn+function spec syntax is in scope.#2019-05-0115:14metametadatathanks.
fwiw, this is the typical way I add specs to protocol methods using defn-spec
at the moment (`sp` here contains custom spec helpers):
(:require [common.logic.specs :as specs]
[spec-plus.core :as sp]
[clojure.spec.alpha :as s]
[defn-spec.core :as ds]))
(defprotocol Protocol
(-fetch-places [_ postal]))
(ds/defn-spec fetch-places
{::s/args (sp/pos any? ::specs/postal)
::s/ret (sp/seq-of ::specs/place)}
[this postal]
(-fetch-places this postal))
#2019-05-0115:16dominicmI guess there's a open question of whether the function level is the right place for deciding closedness. Maybe the other flags will clarify this.#2019-05-0115:19metametadatayes, it's interesting to see how function specs will play with closed maps.
and speaking of that, this is how I use closed maps in functions currently:
(ds/defn-spec build-and-push
{::s/args (sp/pos ::component
(sp/speced-keys :req-un [::path ::image-id ::tag]))} ; sp/speced-keys implements a closed s/keys spec
[component {:keys [path image-id tag]}]
...
#2019-05-0115:25dominicmI'm suggesting that maybe it's up to the caller sometimes whether to apply certain constraints.#2019-05-0115:32Alex Miller (Clojure team)these flags will probably be available on calls like instrument or check so you could supply those constraints that way#2019-05-0115:37metametadatawould be cool if it allowed to spec a function in such a way that some args are closed and some are not, e.g. (foo <closed-map> <integer> <open-map>)
#2019-05-0115:41dominicmThat sounds like an important filter.#2019-05-0116:58Alex Miller (Clojure team)the planned api will allow that#2019-05-0116:59Alex Miller (Clojure team)well, on a per-spec basis that is. if you want to use the same spec in both modes in a single context, not sure about that (or whether that's even a real use case)#2019-05-0117:28dominicmHmm, merging an old-user with new-user where the new user is coming over the wire and needs to be locked down, but old-user is from the db?#2019-04-2913:21johnhttps://nakedsecurity.sophos.com/2019/04/29/nist-tool-boosts-chances-of-finding-dangerous-software-flaws/ Apparently this just reduces the number of combinations one might test by choosing more unique combos per iteration. Seems like maybe a dimensionality reduction scheme over random parameters. Thought it might be related to spec gen scenarios.#2019-04-3008:25ikitommiBending spec1 to support things that we need for validating configuration, and that hopefully are available with spec2 itself: closing (data-)specs recursively. No macros, just data.#2019-04-3008:26ikitommiuses spell-spec
and expound
for most of the work.#2019-04-3010:26vemv(spec/def a int?)
(spec/valid? `a 2) ;; -> true
Sometimes I make this typo (`a` vs. ::a
) but it happens to work
Is it intentional?#2019-04-3011:54Alex Miller (Clojure team)Itās a side effect of the impl#2019-04-3011:55Alex Miller (Clojure team)Symbols are used as valid names for function specs (s/fdef)#2019-04-3012:29vemvgot it, thanks!#2019-05-0116:28vemv(spec/def :foo/bar int?)
(spec/def :baz/quux string?)
(spec/keys :req [...])
Is it possible to define a keys
spec such that the :foo/bar
key is required, but the :baz/quux
spec is validated for that specific key?
i.e. I want to override a spec without renaming the key
(rationale: :foo/bar
is a Datomic attribute. I cannot or don't want to rename it. And :foo/bar
has a existing spec, useful/correct 99% of the times)#2019-05-0116:40valeraukonot that i know of, no#2019-05-0116:42vemvWorking around that with a fn
for now š#2019-05-0120:29eskemojoe007I have a hash-map
where the keys and values are used almost as a linked list {"a" "b", "b" "c", "c" "a"}
where a,b,c could be other strings. This is used in a map where I have a spec, but I don't know how to write a spec that says, let the keys and values be any ol' string.#2019-05-0120:33drone(s/map-of string? string?)
#2019-05-0120:38eskemojoe007Well that was easy...#2019-05-0120:39eskemojoe007One more question for now. I have a unique ID for many items. They are used throughout the speced map. When using generators, it doesn't understand the relationships between the IDs. Any way to enforce that?#2019-05-0121:50favilahttps://github.com/reifyhealth/specmonstah is in this space#2019-05-0120:41seancorfield@david.folkner You can either (s/and (s/map-of ...) uniqueness-predicate)
or write your own generator. The former is "easy" if it's able to actually satisfy the criteria. The latter is harder but more likely to work.#2019-05-0120:42eskemojoe007Perfect. I'll take a look at making a generator, that will likely fit my application a bit better. I already have the functions to build up the data for my actual application, so a custom generator can leverage those.#2019-05-0121:13Alex Miller (Clojure team)for the latter, see gen/fmap
and/or gen/bind
#2019-05-0121:14Alex Miller (Clojure team)http://blog.cognitect.com/blog/2016/8/10/clojure-spec-screencast-customizing-generators might be a good example#2019-05-0301:36Nolanjust trying out alpha2
for the first time and finding (s/or ...)
to be StackOverflowError
ing on everything i try. seems like im missing something pretty big but havent been able to piece it togetherādo the regex macros need to be used with s/spec*
?#2019-05-0301:43Nolanah, seems like its nrepl relatedā¦#2019-05-0301:47Nolanseems to have resolved itself, what a baffling experience indeed.#2019-05-0302:20Nolanis there a concise way of expressing something similar to s/or
where data that conforms to multiple predicates returns all of the matches instead of just the first? something similar to:
(s/def ::example (s/... :e even? :s #(< % 42))
(s/conform ::example 2) ;; => ([:e 2] [:s 2])
#2019-05-0302:29Alex Miller (Clojure team)no, not really#2019-05-0302:31Nolangood to know. thank you alex!#2019-05-0312:54jumarI'm not sure how to approach this problem...
I have multiple implementations of an "auth provider" (e.g. db, ldap, etc.) and they are represented as a namespaced map like this:
#:auth-provider{:active? true
:type "ldap"
:config {:port 636
:host ""
:bind-dn-format "uid={username},cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org"
:search-base "dc=demo1,dc=freeipa,dc=org"
:connection-timeout 10000
:response-timeout 10000
:ssl? true}}
The problem here is the :config
key. It's widely different across different implementations and I'd like to have different specs for that.
My problem is that AFAIK you can have only one spec for such a key - in my case that's fairly generic:
(s/def :auth-provider/config (s/map-of keyword? any?))
(s/def ::auth-provider-spec (s/keys :req [:auth-provider/type
:auth-provider/active?]
:opt [:auth-provider/id
:auth-provider/default-role
:auth-provider/config
:auth-provider/priority-order
:auth-provider/role-mapping]))
Now I want to "override" spec for the :auth-provider/config
key for each implemantation - currently I use this hacky approach:
(s/def ::ldap-config (s/keys :req-un [::host ::port ::connection-timeout ::response-timeout]
:opt-un [::bind-dn-format ::search-base]))
(s/def ::ldap-provider-spec (s/and
::auth-specs/auth-provider-spec
;; this is less descriptive than using `s/keys` directly
;; but we cannot use `s/keys` because `:auth-provider/config` default
;; spec is already registered in auth-specs namespace
;; and you cannot have two different specs for the same namespaced key (i.e. `:auth-provider/config`)
#(s/valid? ::ldap-config (:auth-provider/config %))))
But that's not only awkward it's also problematic from the "error reporting" point of view - in case something inside the :config
key is invalid I get pretty useless error message that the spec failed but I don't know why:
... Reason: Invalid LDAP configuration: -- Spec failed ---------
... Relevant specs ------- :auth.providers.ldap-provider/ldap-provider-spec: (clojure.spec.alpha/merge :auth.specs/auth-provider-spec (clojure.core/fn [%] (clojure.spec.alpha/valid? :auth.providers.ldap-provider/ldap-config (:auth-provider/config %)))) ------------------------- Detected 1 error
#2019-05-0312:56jumarI tried merge
and multi-spec
(might not know how it should be used properly) but failed basically for the same reason - cannot define different specs for the key :auth-provider/config
#2019-05-0313:03manutter51Wonder if you could do something with the namespaces on the key? Like, instead of :auth-provider/config
for all the variants, have :auth-provider-ldap/config
, :auth-provider-db/config
, etc.#2019-05-0313:06jumarThat's not an option I'm afraid.#2019-05-0313:04Alex Miller (Clojure team)Youāre in the ballpark of s/multi-spec - have you tried that?#2019-05-0313:07jumarAs commented in the reply, I got stuck basically for the same reason (not able to define different specs for the same namespaced-key) - I might be doing it wrong, though.#2019-05-0313:10jumarroughly something like this:
(defmulti auth-specs/provider-type :auth-provider/type)
(defmethod auth-specs/provider-type "ldap" [_]
(s/keys :req [:auth-provider/config]))
#2019-05-0313:11jumarNow I don't know how to define different versions of :auth-provider/config
...#2019-05-0313:20Alex Miller (Clojure team)one way would be to do this one level up - the map containing :auth-provider/config#2019-05-0313:21Alex Miller (Clojure team)or I find it's always helpful to come back to the truth of the matter - what values can a key take on? here :auth-provider/config has multiple sets of things#2019-05-0313:21Alex Miller (Clojure team)so spec it as s/or of different s/keys specs#2019-05-0313:21Alex Miller (Clojure team)(some of which might reuse the same attributes)#2019-05-0313:26jumaridea with s/or
is interestig - I'll try that. Thanks!#2019-05-0318:33denikit looks like unform
is not aware of nonconforming
or am I doing sth wrong?
(s/def ::bar
(s/or :baz integer?))
(s/def ::foo
(s/nonconforming ; <========
(s/keys :req-un [::bar])))
(s/def ::tez
(s/tuple any? ::foo))
(s/conform ::tez [123 {:bar 4}])
; [123 {:bar 4}]
(s/unform ::tez [123 {:bar 4}])
; Error: nth not supported on this type function Number() { [native code] }
; at Function.
#2019-05-0318:38deniksmaller example
(s/def ::bar
(s/or :baz integer?))
(s/def ::foo
(s/nonconforming
(s/keys :req-un [::bar])))
(s/conform ::foo {:bar 4})
(s/unform ::foo {:bar 4})
#2019-05-0322:21Alex Miller (Clojure team)those both roundtrip fine for me w/o error#2019-05-0322:22Alex Miller (Clojure team)are you on cljs? or clj?#2019-05-0617:29denikcljs#2019-05-0617:42denikjust tested and can't reproduce in CLJ.#2019-05-0619:53yuhanIs it good practice to place all specs together in a dedicated my-project.specs
namespace?#2019-05-0620:05Jakub HolĆ½ (HolyJak)I do not know but on my project I created a separate .domain
ns with specs so that I could share them with multiple namespaces and it is nice not to have it all through the core ns. So it works for me š#2019-05-0620:33yuhanThanks! I was reading a few posts and it seems there isn't really a one "correct" way of doing things#2019-05-0620:34yuhansharing specs between namespaces is the reason I'm thinking of doing it as well, and it probably makes sense for s/fdef
specs to remain next to their respective functions#2019-05-0718:26buttergunsI personally put specs above the code, in the same namespace. It makes sense that logically, if I want to see the "Event" spec, it would be in the event.clj namespace for example#2019-05-0718:28buttergunsI also have a common.clj
namespace, which contains shared functions. It also contains common specs, like ::timestamp
, which is used in many places#2019-05-1015:52rickmoynihanIād say it can make sense for libraries at a certain scaleā¦ though even there you may just want to put them where you need them.
For an application youāll inevitably end up wanting to spread things out.#2019-05-0619:58yuhanI also avoid using namespace alias ::keyword syntax and most of the specs have short 3-8 character prefixes that don't actually correspond to any of my project namespaces, wondering if that's good practice too#2019-05-0620:06Jakub HolĆ½ (HolyJak)They don't need to correspond to actual namespaces, the main thing is I guess readability.#2019-05-0819:55favilawhat's the best way to spec that some relationship holds between the keys and values of a map?#2019-05-0819:56favilain my case I have a map whose keys describe some range, and the vals are vecs of items which should be in the range described by the key#2019-05-0819:57kennys/and
?#2019-05-0819:58favila(s/and (s/map-of ::x ::y) kv-rel-holds?)
#2019-05-0819:58favilanothing better?#2019-05-0820:00Alex Miller (Clojure team)You can spec it as a coll of entry tuples#2019-05-0820:00Alex Miller (Clojure team)Then you get both k and v#2019-05-0820:02favila(s/coll-of (s/and (s/tuple ::x ::y) kv-rel-holds?) :into {} :kind map?)
#2019-05-0820:02favilalike that?#2019-05-0820:02favilagenerator for that seems easier to make#2019-05-0911:07benthis is probably a really simple question, but what are the practical differences between using :pre
and :post
vs spec/fdef
for function validation?
As far as I can tell, fdef
gives you a bit more control, but otherwise :pre
/`:post` are just there because theyāre more ālightweightā/easier to read?#2019-05-0913:31eskemojoe007I'm a noob with this stuff, but I think :pre
and :post
run at runtime. fdef
doesn't run at runtime, only during testing.#2019-05-0913:53benthis rings true with the docs now I read them back, actually#2019-05-0913:53benthanks!#2019-05-0916:46jumarfn specs defined via fdef
don't run by default - they run in whatever environment you want if you toggle those via stest/instrument
(https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument)#2019-05-0916:47jumarAnd yes, it's not usually recommended for production use, only for testing/development#2019-05-1007:08thumbnail:pre
, and :post
can be toggled by rebinding *assert*
as well though#2019-05-1017:41eskemojoe007I'm trying to make a custom generator for the deck of cards example in the spec
guide. I have a spec that looks like (s/def ::player (s/keys :req [::name ::score ::hand ::scored]))
. Each key already has the proper tested generator, but for ::player
It shouldn't allow duplicate cards in the hand and scored category for example. I need to make a custom generator for ::player
that checks that forces them to be unique.#2019-05-1017:42eskemojoe007I guess I didn't ask a question. Question1: Is it possible to make a custom generator for just part of the ::player
?#2019-05-1017:45seancorfieldI'm not sure I'm reading your problem statement correctly: are you just trying to ensure ::hand
generates unique cards, and ::scored
generates unique cards, or are you also trying to ensure there's no duplication between ::hand
and ::scored
?#2019-05-1017:45seancorfieldIf it's the former, you can just write custom generators for those specs. If it's the latter, you pretty much have to write a generator for the whole ::player
spec.#2019-05-1017:47eskemojoe007Its the later. Given a typical deck of 52 cards. Scored is cards that have scored, then are out of play (Making go-fish).#2019-05-1017:48seancorfieldThen, yeah, you have no choice really -- you need a generator for ::player
#2019-05-1017:48eskemojoe007So I have to make a custom generator for the whole ::player
and I can't rely on the generator even for the other keys such as ::score
?#2019-05-1017:49seancorfieldYour ::player
generator will run the generators for the individual elements if it wants default behavior.#2019-05-1017:50eskemojoe007What does that syntax look like?#2019-05-1017:51seancorfield(s/gen ::my-spec)
gets you a generator for a spec#2019-05-1017:52eskemojoe007Ahh...perfect!#2019-05-1017:52eskemojoe007That makes sense, just didn't realize thats what it was doing.#2019-05-1017:52eskemojoe007Thank you.#2019-05-1017:54seancorfieldAnd (gen/hash-map ...)
to generate the ::player
hash map from those generators.#2019-05-1017:54seancorfield(assuming gen
is clojure.spec.gen.alpha
)#2019-05-1017:58eskemojoe007So a very redundant generator would look something like this: (s/def ::player (s/with-gen
(s/keys :req [::name ::score ::hand ::scored])
#(gen/hash-map ::name (s/gen ::name)
::score (s/gen ::score)
::hand (s/gen ::hand)
::scored (s/gen ::scored))))
#2019-05-1017:59seancorfieldYup, that would get you started.#2019-05-1017:59eskemojoe007Perfect. I'll post my solution just for reference, but I appreciate the help.#2019-05-1018:00seancorfieldI'd probably have a spec for a deck of cards with a custom generator to ensure they're all distinct (it could just shuffle
an ordered generation of all the cards), then ::scored
would be the first random N of those and ::hand
would be the next random M of them.#2019-05-1018:04eskemojoe007The ::scored
spec generator is weird, it takes exactly 4 of the same rank card to score. So I was going to run that first, then ommit any results from the ::hand
based on that.#2019-05-1018:05eskemojoe007But taht seems like a terrible idea.#2019-05-1018:05eskemojoe007I'm gonna make a full deck spec.#2019-05-1317:55eskemojoe007Finally got back to this. I came up with something that looks like
(gen/sample (gen/bind (s/gen ::scored)
#(gen/hash-map ::name (s/gen ::name)
::score (gen/return (/ (count %) 4))
::hand (gen-hand %)
::scored (gen/return %))))
(defn remove-by-rank
[cards ranks]
(vec (reduce (fn [new-cards rank]
(remove #(rank-match? % rank) new-cards))
cards
ranks)))
(defn gen-hand
[scored-cards]
(gen/bind (s/gen ::hand)
#(gen/return (remove-by-rank % (should-score? scored-cards)))))
#2019-05-1317:56eskemojoe007So it generates the scored cards first, uses gen/bind
to get those, and bases the the other generation based on that (I didn't include all the functions).#2019-05-1116:48denikI'm using s/conform
to give a richer shape (e.g. places -> names / sequential -> associative) to data flowing through my program. However, after I conformed a value, it trickles through my program and is updated. The crux is that I don't have a spec for the conformed value, which would be useful. Since conform is doing the reshaping work, it must be possible to generate that spec. Currently, I'm hacking this behavior with s/unform
and then s/conform
#2019-05-1119:11Alex Miller (Clojure team)that's why you shouldn't use conform to transform your data#2019-05-1119:12Alex Miller (Clojure team)conform is designed to tell you how it parsed, not to serve as a general purpose transformation engine#2019-05-1202:35denikIs anyone aware of a library that is designed to transform data based on a spec or similar? cc @alexmiller#2019-05-1202:40Alex Miller (Clojure team)https://github.com/wilkerlucio/spec-coerce#2019-05-1203:17seancorfieldResist the temptation tho' @denik -- spec is not intended to be used to drive transformations š#2019-05-1204:00ikitommithere is also https://cljdoc.org/d/metosin/spec-tools/0.9.2/doc/spec-coercion#2019-05-1215:46denik@alexmiller @seancorfield @ikitommi to be clear, I'm not looking to coerce data, rather I want to to give it a richer shape (e.g. places -> names / sequential -> associative). I have a concise place-based syntax like [:user/add [:post 5 :authors] {:id 10 :user/name "Eve"}]
that I want to turn into
{:op :user/add
:path [:post 5 :authors]
: entity {:id 10 :user/name "Eve"}}
This would be easy in vanilla clojure but there are some gotchas, for example path
is optional. With other edge cases writing that in vanilla clojure would get ugly fast. Spec's s/?
+ conform (or similar) seem like a perfect fit. So if it's not conform
, I'm wondering if there is a straightforward approach to parse and unparse while leveraging specs?#2019-05-1215:49stathissideris@denik s/conform
would work for this, but Iāve seen people advising against using it#2019-05-1217:38Alex Miller (Clojure team)cond->
is great for optional stuff#2019-05-1217:38Alex Miller (Clojure team)Iād just use core stuff#2019-05-1218:53denik@alexmiller not for parsing#2019-05-1219:31Alex Miller (Clojure team)there are libs for data structure parsing like https://github.com/cgrand/seqexp that do the "parsing" of data#2019-05-1315:33denikthanks! unfortunately clj only#2019-05-1220:18ikitommiPuzzled. I would have said that s/conform
would have been a perfect tool (without any add-onns) for parsing just that example. What are the downsides of using spec for parsing?#2019-05-1221:59Alex Miller (Clojure team)conform is fine for parsing#2019-05-1222:00Alex Miller (Clojure team)Request above is for additional transformation and enhancement of the result#2019-05-1315:31denikconforming/parsing + generation of a spec for the parsed result (comformed shape) so that it can be checked in other function invocations as well as fspecs.
I would find this very useful:
1. sparse data comes in from a request, e.g. [:user/add [:post 5 :authors] {:id 10 :user/name "Eve"}]
2. conform
data to use names instead of indexes/places=>
{:op :user/add
:path [:post 5 :authors]
:entity {:id 10 :user/name "Eve"}}
3. use a derived spec of the conformed value from (2) to check validity of the value as it is updated throughout the program#2019-05-1312:02benI have a bunch of specs that are quite repetitive to write out manually. Is there an easy way to define spec (e.g. w/in the current ns) with a function?#2019-05-1312:10benEssentially I have a map that looks something like:
{:event :some-kw
:metadata {:a 1 :b 2}}
where the spec of :metadata
depends on the value of :event
, which I want to check with spec#2019-05-1312:13benSo I think I could do something like:
(s/def :event1/event ...)
(s/def :event2/event ...)
;; and so on
(s/def :event1/metadata ...)
(s/def :event2/metadata ...)
;; and so on
(s/def :event1/message (s/keys :req-un [:event1/event :event1/metadata]))
(s/def :event2/message (s/keys :req-un [:event2/event :event2/metadata]))
;; etc
(s/def ::message (s/or :event1/message :event2/message ...))
But this seems extremely inelegant. Feels like Iām missing something obvious but Iām not sure what#2019-05-1312:26codonnellSounds like a good use case for a multispec.#2019-05-1312:36Alex Miller (Clojure team)Yes, also repetitive code can be made less repetitive with a macro#2019-05-1312:59ben> One common occurrence in Clojure is to use maps as tagged entities and a special field that indicates the ātypeā of the map where type indicates a potentially open set of types, often with shared attributes across the types.
yes it does š thank you, @codonnell#2019-05-1322:59yogthosdoes anybody know if there's a workaround for this issue https://dev.clojure.org/jira/browse/CLJ-2482#2019-05-1323:06yogthosI guess dropping down to clj 1.9.0 works#2019-05-1323:08Alex Miller (Clojure team)The linked issue in the comments has some patches people have been using.#2019-05-1323:08Alex Miller (Clojure team)Patches to Clojure that is#2019-05-1323:09Alex Miller (Clojure team)Iām not sure which of the many approaches on there is really the best#2019-05-1323:09Alex Miller (Clojure team)But we will definitely take a look at it in 1.11#2019-05-1323:17yogthosthanks, and 1.9 seems to work so I'll just stick with that for the time being#2019-05-1412:57rickmoynihanAny ideas on how to spec this?
Essentially I have a heterogenous map; if a key in that map is a vector of length N then Iād like to spec that the value of that key in the map is a sequence of tuples also of length N.
If a key in the map is not a vector, than its key can be any?
#2019-05-1413:02Alex Miller (Clojure team)spec it as a collection of kv tuples#2019-05-1420:03Alex WhittWhen defining your fspec's :fn
, what's the intended way to interact with args that are s/or
specs? For example:
{:fn #(-> % :args :arg-name)
for such an arg yields a vector that starts with the keyword for the s/or
variant. Here's how I could approach it:
(s/def ::example (s/or :map map?
:num int?))
(defn myfn [int-arg example-arg])
(s/fdef myfn
:args (s/cat :int-arg int?
:example-arg ::example)
:fn #(-> % :args :example-arg
(as-> [path value]
(or (not= path :map)
(contains? value (-> % :args :int-arg))))))
(myfn 1 {2 :a 3 :b}) ;; => fails
Here, :fn
ensures that the first arg is a key in the second arg if the second arg is a map. Is this how I should be writing these :fn
s, or is there a better way?#2019-05-1421:21Alex Miller (Clojure team)if it's just an inter-arg dependency, you can s/& that predicate onto the args spec#2019-05-1421:21Alex Miller (Clojure team)rather than using :fn, which has access to both args and ret#2019-05-1421:22Alex Miller (Clojure team)alternately, you could encode the args with an s/alt for your two alternatives#2019-05-1421:26Alex Miller (Clojure team)(s/alt :not-map (s/cat :int-arg int? :example-arg #(not (map? %)))
:map (s/& (s/cat :int-arg int? :example-arg map?) #(contains? %2 %1)))
#2019-05-1421:26Alex Miller (Clojure team)something like that for the args if I understood all that right#2019-05-1421:27Alex Miller (Clojure team)and as an aside, when something is hard to spec like this with options, it's often a good sign that your function is doing two things and having 2 functions might be better#2019-05-1514:02Alex WhittAhh that looks better! And thank you for the advice, I'll think about whether I can split this up or not.#2019-05-1521:13lellisHi all! There is some lib that i can create an ER model based in my spec definition?#2019-05-1607:47jaihindhreddyTake a look at Hodur. Tries to DRY the essence of the domain model. You define one central model from which you generate the models for each thing you use (search engine, database, message queues etc.)
https://github.com/luchiniatwork/hodur-engine#2019-05-1607:47jaihindhreddyI'm sure you can extend hodur to make it generate an ER model too. Hodur's possibly too heavy conceptually for your specific use case.#2019-05-1613:38lellisTks!#2019-05-1620:47robertfwI'm trying to create an fdef :args
spec for a multi-arity function in the general form of
(defn my-func ([x] (my-func x nil)) ([x y] (do-thing x y)))
I have a handful more args than just x & y, and don't want to repeat myself, so my first thought was to do something like
(def base-args [:x ::my-x-spec]) (s/fdef my-func :args (s/alt :without-y (apply s/cat base-args) :with-y (apply s/cat (conj base-args :y ::my-y-spec)))
- but alas, s/cat is a macro so I can't do that. I've had another peruse of the spec docs but nothing jumped out at me. any suggestions?#2019-05-1620:49robertfwCan I make an (s/cat base-args)
and then append onto it somehow?#2019-05-1620:53robertfwI understand that I can just provide a full list of [:x ::my-x :y ::my-y]
, but I'd like to be able to generate examples of each arity#2019-05-1620:58Alex Miller (Clojure team)Just build a single s/cat with nested optional additional args with s/?#2019-05-1620:59Alex Miller (Clojure team)Or you could build each arity as an s/cat that combined the prior with an additional arg#2019-05-1621:27robertfwThanks, I'll look at those options#2019-05-1710:07vemvIs there a performance difference between running essentially the same spec checking via instrumentation, vs. via :pre
?#2019-05-1719:28andy.fingerhutFor a general spec, I have no idea. For a very specific very simple spec of checking whether some clojure.set functions are given arguments return true for the predicate set?
or not, there is a pretty big performance difference I measured, with measurements given starting in about the second screenful of the README for this project: https://github.com/jafingerhut/funjible#2019-05-1818:17vemvnice one!
But my question was focused on instrumentation vs :pre.
Guess I can try it myself though#2019-05-1711:10kszaboAm I right to assume that until spec2
gets finished the way to go to emulate s/schema
behavior is to only define (s/keys :opt [:schema/values])
kind of specs and use fdefs
with use-case specific :req
specs?#2019-05-1711:11kszabothat way those can be later transformed in s/select
ās#2019-05-1712:23Alex Miller (Clojure team)Yeah#2019-05-1805:03dottedmagI'm trying to write a dissector of a complex protocol for Wireshark. I have started by writing it manually in C, but it seems to be a very tedious task. I'm going to consume this protocol from Clojure, so I was thinking about specifying it using spec, and then generating C code from it. Has anyone seen anything similar?#2019-05-1805:04dottedmag(C can be replaced with Python or Lua ā does not matter much in this context)#2019-05-1805:04seancorfieldI bet if you ask that during Pacific daylight hours, my colleague @hiredman would have a fair bit to say. He's a big fan of generating stuff from spec and similar data-based systems.#2019-05-1805:49dottedmagIs there a way to have "switch-by-value" without describing all variants in one gigantic (s/alt)
? I'm dealing with a protocol that has a command ID byte that describes format of what is following, and then another subcommand ID in some commands etc. I don't want to have (s/alt :0x00 ::0x00-command :0x01 ::0x01-command ...)
ā maybe I can register them separately?#2019-05-1805:49dottedmagSomething like a multi-spec for sequences.#2019-05-1818:15y.khmelevskiiHi! Does anybody know when spec2 for clojurescript will be available? Also, just curious, was spec2 release date announced?#2019-05-1818:45Alex Miller (Clojure team)Still months to go probably#2019-05-1818:45Alex Miller (Clojure team)No date#2019-05-1818:54y.khmelevskiigot it, thank you#2019-05-1819:26Alex Miller (Clojure team)Although at some point maybe weāll decide to lock it and just make additive changes after that so who knows #2019-05-1820:02y.khmelevskiiIt would be great to deploy cljs.spec-alpha2
in the same state as clojure.spec-alpha2
now. During development not stable clojure.spec-alpha2 is ok but I need cljs.spec-alpha2 as well for sharing new specs between frontend and backend#2019-05-1820:19Alex Miller (Clojure team)afaik no one has actually started working on a port of the changes in spec-alpha2 to cljs yet. The implementation changes are fairly large (and still under way) so I'm not sure it makes sense to start that work until we feel the internals have started to stabilize, and I wouldn't say that yet.#2019-05-1820:21y.khmelevskiiunderstand, thanks for info! I will look forward to it#2019-05-1920:31Jakub HolĆ½ (HolyJak)how do you do generative testing with spec? I now use (defspec xyz
100
(prop/for-all [v (s/gen ::my-spec), res (myfn v)]
(do (s/assert ::my-result res)
(my-check-some-property-on v res))))
but shouldn't s/exercise-fn be usable for this? Though, contrary to s/gen it doesn't allow me to specify overrides and thus test more special / infrequent cases?#2019-05-2005:05Jakub HolĆ½ (HolyJak)I see I should have used https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check instead. Still, how to best integrate with clojure.test? Something like
(deftest xyz
(if-let [f (:failure (st/check ::my-spec)]
(throw f)
(is true)))
?#2019-05-2010:37conanI use this namespace:
clojure
(ns ic.test-util
(:require [clojure.spec.test.alpha :as stest]
[clojure.test :refer :all]
[expound.alpha :as expound]))
(defn check
"Passes sym to stest/check with a :max-size option of 3 (i.e. generated sequences will have no more than 3 elements,
returning true if the test passes or the explained error if not"
[sym]
(let [check-result (stest/check sym {:clojure.spec.test.check/opts {:max-size 3}})
result (-> check-result
first ;; stest/check accepts a variable number of syms, this does not
:clojure.spec.test.check/ret
:result)]
(when-not (true? result)
(expound/explain-results check-result))
result))
#2019-05-2010:37conanthen my tests look like this:
(ns ic.date-test
(:require
[clojure.test :refer :all]
[ic.date :as date]
[ic.test-util :as tu]))
(deftest inst->local-date-time-test
(is (true? (tu/check `date/inst->local-date-time))))
#2019-05-2010:38conanthis will either pass, or give an expound-formatted error showing where the :args
, :ret
and :fn
specs went wrong for the generative tests run for my function (in this case, inst->local-date-time
)#2019-05-2016:33Jakub HolĆ½ (HolyJak)thanks a lot!#2019-05-2011:53borkdude@holyjak Iām using this lib for it: https://github.com/borkdude/respeced#2019-05-2011:53borkdudeso in the context of this question: https://github.com/borkdude/respeced#successful#2019-05-2011:54borkdudethat function also checks if the sequence of results is not empty#2019-05-2016:23boyanbDoes anybody have experience they would be willing to share in regard of human readable messages at API boundaries via spec? We've looked at expound(which doesn't fit) and phrase(which could do the job, but I am personally not convinced by the predicate focused approach). It feels that a simpler solution focused around explain-data and a message registry(similar to the one found in expound) would produce a better result. Has anybody implemented/is currently using spec for this purpose?#2019-05-2016:27jeroenvandijk@boyanb Can you share how you feel expound is not a fit?#2019-05-2016:30boyanbIt's not really designed with the idea of message formatting for "users". Expound could very easily be enhanced or parts/ideas of it lifted into a library that could fit it. AFAIK, while looking at the code, there were several places where we needed paramtrization/additional control that is currently not available by the public facade of the library,#2019-05-2016:30boyanb(had to do with custom printers and expected outputs)#2019-05-2016:32boyanbI could probably dig in a little further and find the concrete examples. Are you using it with user facing messages in any way?#2019-05-2016:32jeroenvandijkI've used expound like this (defn validate-data [spec data]
(if (s/valid? spec data)
:ok
(if-let [explain-data (s/explain-data spec data)]
(let [expound-state
(try
(expound/expound spec data)
:ok
(catch Exception e
(println "Expound had difficulties using " explain-data)
:error))]
(throw (ex-info (if (= expound-state :ok)
"Spec error (see stdout)"
"Spec error (see explain data)") {:explain explain-data})))
(throw (ex-info "Specs are in a weird state, as we can't explain why data is invalid" {})))))
#2019-05-2016:32jeroenvandijkBut I agree it is not perfect#2019-05-2016:34jeroenvandijkI've actually used this in a clojurescript environment (using with-out-str). Worked pretty will when you do a validation on every key change#2019-05-2016:34boyanbYes#2019-05-2016:34kenny@boyanb We had a similar problem. We wanted to spec our API using Clojure Spec but we didn't want to expose Spec's error messages to our users (most people are not used to seeing error messages in that format). This immediately meant Expound was out of the question (too similar to Spec). I looked at Phrase but it seemed like you'd need to duplicate some code to get error messages. I didn't really like that. The overall approach Phrase took made sense -- take the output of explain-data, match it, and convert it to error messages. The matching part seemed like a perfect fit for core.match. So I took that direction and wrote https://github.com/Provisdom/user-explain. I didn't have time to write docs for it š IMO the library is much more general than Phrase due to all the features of core.match. It's also pretty simple -- only 75 LOC š#2019-05-2021:06ericstewartthank you for sharing this! Going to take a look as I have been on the same path as you and you are further ahead it seems.#2019-05-2021:07kennyOf course! LMK if it works out for you.#2019-05-2016:34boyanbour implementation was with-out-str exactly.#2019-05-2016:35boyanb(when with expound). In the end, we still didn't have the formatting we wanted. Main point is, underlying users are really not familiar with anything clojure and shouldn't care about implemenmtation and in the end we needed fine grained control to explain-data to be able to format path within spec + spec message as we needed.#2019-05-2016:36boyanb@kenny - what you describe matches exactly our needs. I'll take a look.#2019-05-2016:39kennyThe tests may be helpful for documentation. I'd recommend just reading the 75 line implementation though. There's a lot of areas I want to improve. As you start diving into the data produced by explain-data, you realize that there's a ton of "core" predicates that need to be handled. A common case is #(contains? % kw)
which is used with s/keys
to validate keywords exist on the map. I'd like to provide a way get automatic nice error messages for all the "core" predicates.#2019-05-2016:40boyanbOooh, that's great. @kenny. Looking at the tests it's almost exactly what we are looking for. I'll play with it and let you know if we decide to ship.#2019-05-2016:40boyanbYes, the tests are where I started ;o)#2019-05-2016:49kennyThere's a couple weird things about the implementation that I really need to write down before I forget:
1. Since defexplainer
does not have a name associated with it, the only way to uniquely identify "explainers" is via the matching map. If you change the matching map (i.e. add or remove keys), the old matcher will still be def
'ed. You'll need to run clear!
to reset everything. This is really annoying and I don't have a great solution atm. The most obvious solution would be to name every defexplainer
.
2. Order of defexplainer
s does matter. If you def
the most general explainer first, it will always get matched first. Ensure more specific matches are def'ed before general ones. Technically this could be fixed by sorting based on some sort of heuristic but I didn't have the time to work out what that should be.#2019-05-2016:50boyanbThank you#2019-05-2016:51kennySure. LMK if it ends up working out or if you guys take another approach.#2019-05-2016:51boyanbWill do#2019-05-2114:25Jakub HolĆ½ (HolyJak)does (spec.test/instrument)
(w/o args) instrument 1. all functions (ie vars) in all namespaces, mine, libs, and clojure? or 2. just the current one? thanks!#2019-05-2114:25Jakub HolĆ½ (HolyJak)does (spec.test/instrument)
(w/o args) instrument 1. all functions (ie vars) in all namespaces, mine, libs, and clojure? or 2. just the current one? thanks!#2019-05-2114:33kszabohttps://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument#2019-05-2114:34kszaboall vars#2019-05-2116:13Jakub HolĆ½ (HolyJak)yeah I read that. So option 1 right?#2019-05-2116:17kszabohttps://github.com/clojure/spec.alpha/blob/0269b2cfefe4df7b68710decc94c623db4a6f630/src/main/clojure/clojure/spec/test/alpha.clj#L248
everything returned by instrumentable-syms
#2019-05-2116:18Jakub HolĆ½ (HolyJak)OK I guess I can run it to get the answer ,thx#2019-05-2116:28seancorfieldIt will instrument all the s/fdef
s that have been loaded.#2019-05-2116:29seancorfieldAs an example, clojure.java.jdbc
has all its s/fdef
s in a separate namespace so the library's functions would only be instrumented by (s/instrument)
if you have required the clojure.java.jdbc.specs
namespace.#2019-05-2211:38Jakub HolĆ½ (HolyJak)thanks a lot, Sean!#2019-05-2118:19ejelomeThis might be answered, but I watched Matthias Felleisen's talk about types: "Types are like the Weather, Type Systems are like Weathermen".
And I remembered clojure.spec and noticed that it somehow reflects his ideas (putting contracts instead of making a type system for an untyped language, e.g. typed.clojure [which afaik is now discouraged/discontinued] and that type inference is near to impossible to implement in a untyped language like Clojure).
So the question is ... is clojure.spec the answer to Felleisen's talk?#2019-05-2118:28Alex Miller (Clojure team)no, although spec was influenced by Racket's contracts, along with other stuff#2019-05-2118:29Alex Miller (Clojure team)spec had been in development for about 4 months at the time of that talk#2019-05-2118:33seancorfield"typed.clojure [which afaik is now discouraged/discontinued]" -- I thought Ambrose was still actively working on this...?#2019-05-2118:35Alex Miller (Clojure team)he just defended his dissertation on it#2019-05-2118:35Alex Miller (Clojure team)so it has been, but not sure where it's going from here. would be a good question for him#2019-05-2118:38Alex Miller (Clojure team)iirc, while I was away at Clojure/west when that talk was given, Rich wrote all the regex stuff in spec#2019-05-2118:54ejelomethanks @alexmiller, although it didn't came directly from racket's contracts, I'm still glad that they're going almost on the same direction (no type inference, or implementing a type system [just to say we also can do type checking]).#2019-05-2118:59Alex Miller (Clojure team)well, it's not type checking, and the difference is important#2019-05-2118:59Alex Miller (Clojure team)type checking is about proving things early#2019-05-2119:00Alex Miller (Clojure team)spec is about verifying predicative constraints dynamically#2019-05-2211:51Jakub HolĆ½ (HolyJak)@alexmiller To improve docs of the lazy-loaded gen/* functions, since including the whole docstring is not feasible according to https://clojure.atlassian.net/browse/CLJ-2018, what about at least changing the docstring to contain the URL of the function's online documentation, such as https://clojure.github.io/test.check/clojure.test.check.generators.html#var-elements for elements
?#2019-05-2212:13Alex Miller (Clojure team)Sure#2019-05-2212:14Jakub HolĆ½ (HolyJak)Should I make a jira issue and send a patch?#2019-05-2212:20Alex Miller (Clojure team)Go for it#2019-05-2212:23Alex Miller (Clojure team)I assume you are aware we have migrated jira to new system...#2019-05-2217:10Jakub HolĆ½ (HolyJak)@alexmiller I wanted to try my patch by including local clone of spec.alpha in my project but starting clj
in my project then fails with ...
Caused by: java.lang.Exception: #object[clojure.spec.alpha$and_spec_impl$reify__1047 0xf9b5552 "
Any idea what I do wrong?
I have cloned to /Users/me/tmp/spec.alpha
, git hash 5228bb7. In my project's deps.edn
: {:deps {org.clojure/clojure {:mvn/version "1.10.1-beta2"}
org.clojure/spec.alpha {:local/root "/Users/me/tmp/spec.alpha"}
...}
(This is before I did any changes to the code. java -version
1.8.0_192, OSX)
But mvn package
in the spec.alpha
project runs just fine.#2019-05-2217:13Jakub HolĆ½ (HolyJak)Same problem when running clj in the spec.alpha project: š clj
Clojure 1.10.0
user=> (load-file "src/main/clojure/clojure/spec/alpha.clj")
Syntax error macroexpanding clojure.core/defn at (alpha.clj:78:1).
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x1698fc68 "
#2019-05-2217:22Jakub HolĆ½ (HolyJak)(Added https://clojure.atlassian.net/browse/CLJ-2512 for the docstring change)#2019-05-2212:14Jakub HolĆ½ (HolyJak)Question: I often do something like the following, to restrict the domain of the generated values to ensure interesting conflicts: (s/fdef filter-adult-users
:args (s/cat :youngsters (s/coll-of ::uid :kind set?), :users ::users) #_... )
(deftest filter-adult-users-spec
(let [[user-ids] (sg/sample (sg/set (s/gen ::uid)
{:min-elements 1
:max-elements 20})
1)]
(is (true? (check `filter-adult-users
{:gen {::uid (constantly (sg/elements user-ids))}})))))
I.e. I have a let
where I use sample
to generate a random, small set of data then used in generator overrides in the test itself. Do you do that too? Is there a better way?
update Sometimes I need to use the customized random data from multiple generators, which prevents I. believe the usage of simple generator derivation such as gen/let and gen/rmap#2019-05-2214:08mishahave a look at test.check/fmap
, test.check/bind
and test.check/let
:
https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#combinators
https://github.com/clojure/test.check/blob/master/doc/intro.md#bind
@holyjak
https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L1570#2019-05-2214:41Jakub HolĆ½ (HolyJak)thank you! but what if eg 2 different generators need the value? any tips?#2019-05-2214:56tangrammeryep, thatās a very interesting need ā¦#2019-05-2215:09Jakub HolĆ½ (HolyJak)example: having a function taking a map with known values and a list of "things", the fn throws if any " thing" has an unknown value. If I want to test its other functionality I must ensure that things only use the values in the map get I prefer not to hardcode the map.#2019-05-2215:46mishayou can test.check/let
the generator, which generates [map things] tuples, and then apply function you want to test to the generated tuples#2019-05-2216:27Jakub HolĆ½ (HolyJak)good idea! But I guess I would need to invoke it manually instead of using spec.test/check#2019-05-2310:05mishadefining wrapper which applies your function to the tuple, and specing/checking wrapper instead might be an option for you#2019-05-2310:08tangrammercould anyone write/share a gist detailed example š ? š#2019-05-2311:11Jakub HolĆ½ (HolyJak)BTW I tried to replace my (let [uids ..] (check .. {:gen {::uid (const. (sg/elements uids)))}}))
with (check ..
{:gen {::uid (constantly
(gen/bind
(sg/set (s/gen ::kd/sid) :num-elements 1)
sg/elements))}})
but it does not seem to really work. If I replace it with (constantly (sg/return (s/gen ::uid)))
then I get many (desired) "collisions" in the tests but with āļø I get none. So my hypothesis is that the set of values is re-created every time that a new value fpr ::uid is generated, instead of creating it once and reusing it every time ::uid is requested.#2019-05-2406:51Jakub HolĆ½ (HolyJak)@U051KJGTX a gist of what exactly? As mentioned above, bind did not work for me, and showing a let wrapping check with some :gen overrides is perhaps not all that interesting?#2019-05-2412:52Jakub HolĆ½ (HolyJak)@U051KJGTX here is an example of what misha proposed, a wrapper fn taking a tuple of related inputs: (defn apply-profile-compute-totals-wrapper
"Smart wrapper around apply-profile-compute-totals that 'unpacks' the subscr-summaries+userid->profile tuple
we generate so that both have the same subscribers before invoking the wrapped fn
"
[subscr-summaries+userid->profile]
(let [[subscr-summaries userid->profile] subscr-summaries+userid->profile]
(apply-profile-compute-totals
{:db/userid->profile userid->profile}
subscr-summaries)))
(s/def ::subscr-summaries+userid->profile
(s/with-gen
(s/cat :subscr-summaries ::kd/subscr-summaries, :userid->profile :db/userid->profile)
;; GENERATOR: Ensure that the generated userid->profile have a profile for every subscriber in
;; subscr-summaries (b/c those without profile would have been filtered out before)
(constantly
(gen/let
[subscr-summaries (s/gen ::kd/subscr-summaries)
profiles (sg/vector (s/gen :db/profile) (count subscr-summaries))]
[subscr-summaries
(zipmap
(keys subscr-summaries)
profiles)]))))
(s/fdef apply-profile-compute-totals-wrapper
:args (s/cat :1 ::subscr-summaries+userid->profile)
;:args (s/cat
; :org (s/keys :req [:db/userid->profile])
; :subscr-summaries ::kd/subscr-summaries)
:ret ::kd/subscrs+profile-usages)
(st/check `apply-profile-compute-totals-wrapper)
#2019-05-2413:12tangrammerthanks a lot for sharing ā¦ Iāll give a try next week!#2019-05-2216:03mishahere is more explanation about fmap and bind: https://youtu.be/F4VZPxLZUdA?t=655#2019-05-2306:41yuhanHow do I instrument/unstrument functions on a per-namespace basis?#2019-05-2306:50Jakub HolĆ½ (HolyJak)Not sure but perhaps https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/enumerate-namespace could help?#2019-05-2306:54Jakub HolĆ½ (HolyJak)(st/instrument (st/enumerate-namespace 'my.core))
Enum. enumerates all vars but instrument seems to be happy with that and presumabely skips those that are not spec-instrumentable.#2019-05-2307:03yuhanthat's just what I was looking for, thanks!#2019-05-2307:07yuhan(stest/instrument (map symbol (vals (ns-interns *ns*))))
#2019-05-2312:36Nolanam i missing something regarding s/union
and s/select
? iām trying to do something like this:
(require '[clojure.spec-alpha2 :as s])
(s/def ::n number?)
(s/def ::o odd?)
(s/def ::schema1 (s/schema [::n]))
(s/def ::union1 (s/union ::schema1 [::o]))
(s/select ::union1 [*]) ;; => IllegalArgumentException
(s/select (s/union ::schema1 [::o]) [*]) ;; => IllegalArgumentException
#2019-05-2312:55Alex Miller (Clojure team)probably broken atm. top-level union is probably going to go away anyways#2019-05-2312:56Alex Miller (Clojure team)I'll take a look when I next get a chance#2019-05-2313:05Nolanok, perfect. appreciate the info!#2019-05-2319:36ikitommicould the s/def
fail-fast on totally bogus spec-forms? what would be a spec for the spec-form
arg in s/def
?#2019-05-2319:36ikitommi(s/def ::a 1)
;; :user/a
(s/valid? ::a 1)
;; Syntax error (ClassCastException)
;; java.lang.Long cannot be cast to clojure.lang.IFn
#2019-05-2319:39Alex Miller (Clojure team)it could check some things better, for sure#2019-05-2319:55Alex Miller (Clojure team)what's valid is changing between spec 1 and 2#2019-05-2319:58Alex Miller (Clojure team)in spec 2 the valid symbolic forms are keywords, sets, symbols, and lists/sequences which are spec forms where a known spec op is in op position (ops are an extension point via multimethod)#2019-05-2319:59Alex Miller (Clojure team)we may add another datafied/map form, still tbd#2019-05-2320:00Alex Miller (Clojure team)notably, function objects are not allowed (different than spec 1)#2019-05-2320:00Diegohey all, does anyone know why canāt I do something like this with spec?
(s/def ::profile (s/keys :req-un {::id int?
::name string?
::photos (s/* (s/keys :req-un {::type int?}))}))
#2019-05-2320:02Alex Miller (Clojure team)you mean, inline a spec in s/keys?#2019-05-2320:02Alex Miller (Clojure team)by design, spec is trying to enable a shared global registry of attribute specs#2019-05-2320:03Alex Miller (Clojure team)the attribute spec is considered to be more important than containers of those attributes#2019-05-2320:03Alex Miller (Clojure team)as such, we require you to strongly associate a spec with an attribute name#2019-05-2320:04Alex Miller (Clojure team)spec 2 will have more support for inlining attribute spec defs for unqualified keys (common with json interop)#2019-05-2320:09DiegoI see. Being new to spec I find that not being able to inline specs makes it harder for me to reason about them but I suppose that might change once I get more familiar with it, and one I learn about the best practices on creating specs (any suggested links?).#2019-05-2320:13Diegobtw, thanks for the response @alexmiller#2019-05-2320:18Alex Miller (Clojure team)have you read the guide? https://clojure.org/guides/spec ?#2019-05-2320:18Alex Miller (Clojure team)or the rationale? https://clojure.org/about/spec#2019-05-2321:06Nolancurious if there is a significant expected runtime difference between computing s/select
s dynamically and def
ing them prior. e.g. if the following s/valid
calls were going to be called in a loop:
(require '[clojure.spec-alpha2 :as s])
(s/def ::schema1 (s/schema [...]))
(s/def ::select1 (s/select ::schema1 [...]))
(def select2 (s/select ::schema1 [...]))
(s/valid? ::select1 {...})
(s/valid? select2 {...})
(s/valid? (s/select ::schema1 [...]) {...})
#2019-05-2321:16Alex Miller (Clojure team)you should expect that we have spent no time yet on perf aspects of select :)#2019-05-2321:19Alex Miller (Clojure team)in general though, the first two are reusing the same spec object, which means work can be done once at spec object creation time (optimization work is basically shifting as much work as possible into construction time here) whereas the last one would pay that cost every time#2019-05-2321:19Nolanexactlyāwas about to say that i suppose iād always def it, if it werenāt truly dynamic. was just tangentially curious about the perceived cost of the dynamism there#2019-05-2321:20Alex Miller (Clojure team)so in general, I would tend toward either using an object saved in var or from the registry (those are probably approx the same)#2019-05-2321:20Alex Miller (Clojure team)but also note that the tradeoff, as usual, is in not picking up changes to specs in the ... there#2019-05-2321:21Alex Miller (Clojure team)so you might make the opposite choice at the repl, if you're in development#2019-05-2321:22Alex Miller (Clojure team)generally, I am just working in a file, that has all the specs I'm working on in it, and I just reload the file, which registers and recompiles all of the specs, and I don't care#2019-05-2321:22Alex Miller (Clojure team)it is useful to have that mental model though#2019-05-2321:25Nolanright, right. got it. thats a premium tip. spec2 is awesome, have been having a blast getting some experience. it alleviates essentially all of my old s/keys
woes š„³#2019-05-2321:34Alex Miller (Clojure team)great!#2019-05-2406:54Jakub HolĆ½ (HolyJak)Does anybody have any idea why might I be getting the error
> Syntax error macroexpanding clojure.core/defn at (alpha.clj:78:1).
> #object[clojure.spec.alpha$and_spec_impl$reify__2183 0x1698fc68 "clojure.spec.al/cdn-cgi/l/email-protection"] is not a fn, expected predicate fn
when trying to start a REPL in the spec.alpha
project and loading the code? I did this: $ git clone # sha 5228bb75fa10
$ cd spec.alpha
$ clj
Clojure 1.10.0
user=> user=> (load-file "src/main/clojure/clojure/spec/alpha.clj")
Syntax error macroexpanding clojure.core/defn at (alpha.clj:78:1).
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x1698fc68 "
#2019-05-2407:06Alex Miller (Clojure team)spec is aot compiled, might be a conflict since you are forcing a load of the clj, which recompiles the protocol (yet all the old instances are from the prior protocol). should probably use load instead?#2019-05-2407:46Jakub HolĆ½ (HolyJak)Ah, OK, thank you, that helped!
I am used to load-file
and do not really know how it works, I guess I should learn š#2019-05-2417:47seancorfield@U0522TWDA I'm curious how you got into the habit of using load-file
in the first place? It seems like require
would be the "obvious" thing to learn first...#2019-05-2417:49Alex Miller (Clojure team)yeah, that seems weird#2019-05-2417:51Jakub HolĆ½ (HolyJak)Perhaps a bad habit from Node.js where require is idempotent and does not actually reload the code from the disk.
Also, I see that Cursive's "load file in repl" used to reload the code actually calls load-file
, at least when running REPL via main (instead of via lein)#2019-05-2417:52Jakub HolĆ½ (HolyJak)Ah, no, Clojure require
does not reload the code either
> Loads libs, skipping any that are already loaded.
So what are you saying? That the 1st time I should use require but after I change the code, to get the changes in, then I should use what?#2019-05-2417:53Jakub HolĆ½ (HolyJak)(I guess I will get the answer after I finish Eric's REPL course)#2019-05-2417:53seancorfield(require ... :reload)
or (require ... :reload-all)
#2019-05-2417:54Jakub HolĆ½ (HolyJak)Also, require requires that the code is on the classpath while load-file does not care about that. But I guess that is not a problem if I only use lein repl
in lein project and clj
in deps.edn projects as those have the correct paths autom.#2019-05-2417:54seancorfieldI think editors are likely to use load-file
since they have a filesystem path, rather than a namespace -- but I view load-file
as a tooling/system-level hook. Always interesting to hear how other folks developed their Clojure habits...#2019-05-2407:06Alex Miller (Clojure team)or require?#2019-05-2614:41y.khmelevskiiHi everyone! Can you please explain me why this spec doesnāt work correctly:
(s/def ::name string?)
(s/def ::src string?)
(s/def ::width pos-int?)
(s/def ::height pos-int?)
(s/def ::size
(s/schema {:src ::src
:width ::width
:height ::height}))
(s/def ::file
(s/select
(s/schema {:name ::name
:size ::size})
[:name :size {:size [*]}]))
(s/valid? ::file {:name "test"
:size {}})
;; => true
;; but should be false
#2019-05-2614:47y.khmelevskiibut when I use fully-qualified keywords, s/valid?
works correctly#2019-05-2620:30seancorfield@y.khmelevskii I tried the following and it also produces true
user=> (s/def ::file (s/select (s/schema {:name ::name :size ::size}) [:name :size {:size [:src :width :height]}]))
:user/file
user=> (s/valid? ::file {:name "x" :size {}})
true
I'm not sure what restrictions are in place for unqualified key usage. It is all alpha, after all.#2019-05-2623:54benzapHas clojure.spec 2.0 landed already, or is it still in the development stage?#2019-05-2623:59seancorfieldVery much alpha at the moment @benzap#2019-05-2623:59seancorfieldThere hasn't even been an "alpha" release yet -- it's just on GitHub.#2019-05-2700:05benzapOkay, thanks for the update!#2019-05-2700:09seancorfieldAt work we have a branch of our code running on spec-alpha2
just to keep an eye on changes. We haven't started using any of the schema
/`select` stuff yet. Alex has repeatedly told me it's not ready to run in production yet š Even for us, who run alpha stuff in production all the time š#2019-05-2700:12benzaphaha, well it sounds like it's on it's way then, that's good to hear š#2019-05-2700:13seancorfieldYeah, still some weird quirks to be ironed out but it wasn't too painful to get our code running on the new version straight from GitHub (since we're an all-`deps.edn` setup at work)#2019-05-2808:25zcljany advice in how to create a spec for a "string-date" where the string should be a valid RFC3339 format?#2019-05-2810:24Mikko HarjuHi! Is there a mechanism to get exhaustive errors from a spec or how would one approach validating maps with required keys and keys that depend on the content of another one? For instance, given a spec like (s/and (s/keys :req-un [::start-date ::end-date ::foo]) end-date-after-start-date?)
Iād like to be able to have the end-date-after-start-date?
error to be shown also when the key :foo
is also missing from the map.#2019-05-2810:25Mikko HarjuBy default s/and does short circuiting on the first failing spec#2019-05-2810:43Mikko HarjuOne option would be to split the spec and use s/merge
, are there any other valid options?#2019-05-2811:49Jakub HolĆ½ (HolyJak)Any idea why, given this spec (s/def ::account (s/with-gen delay? (constantly (sg/return (delay nil)))))
does (sg/sample (s/gen ::account))
fail with
> Unable to construct gen at: [] for: :myapp/account
? Doesn't the with-gen
add the generator?#2019-05-2901:49Chris ReyesIām interested in using spec (for the first time) for a side project Iām working on. Where should I put the spec definitions? (Or is that a controversial question?)#2019-05-2901:50Chris ReyesI found this https://stackoverflow.com/questions/37942495/where-to-put-specs-for-clojure-spec
But Iām not sure they came to a consensus in any of the answers/comments. (Or maybe Iām not sure how to interpret it because Iām still pretty new to spec)#2019-05-2901:53seancorfield@chrisreyes Most people tend to put data specs in their own namespace, possibly with a few utilities for processing that data, but put function specs next to (above) the functions to which they apply.#2019-05-2901:54seancorfieldIf you want specs to be "optional" for some functions, it makes sense to put them in a separate namespace, so that users can decide whether to load them or not.#2019-05-2901:55Chris ReyesOkay, thanks!#2019-05-2901:56seancorfieldThere is no "right way" or "wrong way" -- whatever is most convenient/makes the most sense.#2019-05-2901:57seancorfieldWhen I added function specs to next.jdbc
recently, I put them all in a separate namespace, so users could choose whether to use them or not https://github.com/seancorfield/next-jdbc/blob/master/src/next/jdbc/specs.clj#2019-05-2901:58seancorfieldThat ns actually contains fdef
s for two other namespaces within the next.jdbc
project, just because that was the most natural/convenient way to set things up.#2019-05-2901:58seancorfieldBut having fdef
s separate isn't as common as having s/def
s for data separate.#2019-05-2901:59seancorfieldPart of the issue is that specs can serve a lot of different purposes. They can be used for testing in several ways. They can be used to support development (`st/instrument` for example). They can be used in production for data validation (and other things).#2019-05-2909:23hlolliWhat's & rest
in spec. I want to spec [:a :B :C :D] and make sure that first is :a and the rest can by of any amount of any type? (s/cat :need-a ::need-a (&rest?))#2019-05-2909:31djtango(s/cat :need-a ::need-a :rest (s/* any?))
#2019-05-2909:32djtangothe nested s/*
(and other regex ops) is flattened out by default so that doesn't mean [:a [:b ...]]
#2019-05-2909:34hlolliahh I see š thanks dj! Still after 2 years of spec, these basics are still troubling me#2019-05-2909:34djtangothat one definitely requires "grokking"#2019-05-2909:35hlolliyes I would have expected a nested sequence for that#2019-05-2909:36djtangoI guess with a lisp hat on you would expect it to nest, but if you were trying to imagine expressing regex via s-expressions it would look like that too so :shrug:#2019-05-2909:37djtangos/tuple
behaves how you might expect iirc#2019-05-2909:47djtangoDo people do generative testing over side-effectful code and/or integration tests?
I feel it is nice to be able to put an fdef
spec over an API endpoint on a webserver (e.g. required params and possible responses) but am unconvinced setting up the check as it starts becoming a bit like a super verbose example test. I suppose instrument with :stub
option could work but I'm trying to find a nice way of tying fdefs
into integration tests...#2019-05-2909:57Jakub HolĆ½ (HolyJak)Speaking about integration / system tests, these talks https://lispcast.com/testing-stateful-and-concurrent-systems-using-test-check/, https://youtu.be/zjbcayvTcKQ and Datomic Simulat could be of interest.
Regarding side-effects, I guess it depends on their kind. Some you certainly want to avoid when testing. (x with datmoic, you can use an "in-memory" local copy of the DB and never "commit" / transact into the original DB yet have everything there for verification.#2019-05-2910:01djtangothanks for sharing#2019-05-2910:01djtangowill take a look!#2019-05-2910:03djtangoalas we're not using Datomic at work though#2019-05-3010:44hlolliSo I have this fdef
(s/fdef define-fx
[env]
:args (s/cat :fx-name ::fx-name
(s/keys* :req [::fx-name]
:req-un [(or ::orc-string ::orc-filepath ::orc-internal-filepath)]
:opt [::fx-form ::ctl-instr ::num-outs
::init-hook ::release-hook
::release-time ::config]))
:ret :instrument-controller fn?)
for this macro
(defmacro define-fx [fx-name & {:keys [orc-string fx-form ctl-instr num-outs release-time
init-hook release-hook instance-config] :as env}] )
but there seems to be no instrumentation if I forget to provide required args, or type them incorrectly? Any obvious error here?#2019-05-3016:59dronethereās no key for the second key-pred pair in s/cat
#2019-05-3017:13droneand [env]
seems wrong should be removed, unless thereās some form of fdef
Iām unfamiliar with#2019-06-0416:25hlolliyup that's right, works now, thanks!#2019-05-3119:40mpdairyso i have an idea for instrumentation with spec. It would be nice if the arguments to a function with a failing fdef were saved somewhere so they could be inspected in the repl, since a lot of times the spec error message is not very useful, and sometimes is so large it nearly crashes emacs.#2019-05-3119:41mpdairyspeaking of that, is there a way to truncate the length of spec error messages?#2019-05-3119:42ghadispec error messages should be really short as of the last changes#2019-05-3119:42ghadiwhat version clojure + spec are you using?#2019-05-3119:46mpdairyorg.clojure/spec.alpha "0.2.176"
and clojure 1.10.0#2019-05-3119:48ghadiare you sure you're not mistaking the ex-data for the exception message?#2019-05-3119:48ghadi(that's the latest release)#2019-05-3119:49mpdairyyeah, it's the explain data. it's when it's showing the "actual" datatype, which can be huge#2019-05-3119:50ghadithat's a tooling thing... CIDER should have a toggle for it#2019-05-3119:50mpdairyoh ok#2019-05-3119:50ghadi(It used to be that the ex-data was also serialized into the message string, which is admittedly awful, but that is no longer the case)#2019-05-3119:51mpdairyoh yeah i remember those times#2019-06-0410:38Jakub HolĆ½ (HolyJak)Hello, how do I spec protocol methods? Just (s/fdef protocol-method :args (s/cat :this any? ...))
?#2019-06-0410:56metametadataYou'll have to wrap the protocol method into a spec-able function. E.g. using defn-spec
:
(defprotocol Protocol
(-foo [_ x]))
(ds/defn-spec foo
{::s/args (sp/pos any? ::specs/x)
::s/ret ::specs/y}
[this x]
(-foo this x))
#2019-06-0412:32djtangowhat is (sp/pos ...)
?#2019-06-0414:57metametadata@U0HJD63RN ah, it's a local helper to reduce verbosity:
#?(:clj
(defmacro -cat
[& body]
(if (-cljs-env? &env)
`(cljs.spec.alpha/cat
#2019-06-0415:06djtangoah nice - I had a feeling that was how it worked, just got my hopes up it might be something in spec-2 š#2019-06-0417:56colinkahnIs there a way to control the recursion depth for spec generators?#2019-06-0417:57colinkahnMy use case is a recursive tree-like spec that Iām generating using (gen/generate (s/gen ::my-spec))
#2019-06-0417:58Alex Miller (Clojure team)there are some dynamic var knobs in the spec.alpha namespace#2019-06-0417:59Alex Miller (Clojure team)https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/*recursion-limit*#2019-06-0417:59Alex Miller (Clojure team)also, if you have any collection specs, I usually use :gen-max 3
to limit nested collections from going out of control#2019-06-0418:00Alex Miller (Clojure team)even given those, I think there are some known issues where the recursion is not well controlled#2019-06-0418:03colinkahnAwesome, :gen-max 3
is working for me, thanks!#2019-06-0421:35hlolliso for a string? bool in a macro, I get
{:path [:define-fx-params :orc-string],
:pred clojure.core/string?,
:val
(str
(slurp
(io/resource
"panaeolus/csound/fx/udo/shred.udo"))
"\ngkTransPose init 1\ngkTransRand init 0.1"),
:via [:panaeolus.csound.macros/orc-string],
:in [:orc-string]}
So it's obviously a string, but it's a macro, so I understand it sees a list at this point, any good tip to make this string check?#2019-06-0421:38hlolli(s/def ::orc-string #(string? (eval %)))
works actually#2019-06-0423:05Alex Miller (Clojure team)Thatās not a good pattern#2019-06-0423:06Alex Miller (Clojure team)By evalāing here youāre basically ruining the lazy evaluation of macros and could even cause issues#2019-06-0423:07Alex Miller (Clojure team)In general macro specs are often tricky to write unless youāre trying to enforce positional constraints#2019-06-0423:07Alex Miller (Clojure team)For something like this I would probably not spec it at all#2019-06-0423:08Alex Miller (Clojure team)But would spec that arg as any?
if I was#2019-06-0500:00seancorfield@hlolli Remember that macros take code as input and produce code as output -- they don't see the runtime type of that code.#2019-06-0500:05hlollihmm, my macros are the few things in my app that I actually want to spec, because of their interface nature to my app. Yeh, I can ofc spec a presence/absence of an argument. Or spec a function that the macro calls... also a solution.#2019-06-0501:00Alex Miller (Clojure team)the latter is a good idea#2019-06-0501:01Alex Miller (Clojure team)keep in mind that macro specs are checked at compile time. so they can only check things you know at compile time.#2019-06-0501:02Alex Miller (Clojure team)they can't check things about the values, because the values don't exist yet#2019-06-0501:02Alex Miller (Clojure team)so macro specs are great for checking syntax (in core we use them for ns, defn, destructuring, etc) - things that are macros with their own syntax defined by the macro#2019-06-0501:03Alex Miller (Clojure team)function specs are checked (when instrumented) at runtime when you have values in hand#2019-06-0501:03hlolliyup, so this makes total sense to me. I should keep it a habit to do as little work as possible in a macro, I only need the def on a symbol, and I can forward the rest to a function*#2019-06-0501:04Alex Miller (Clojure team)in general, having pairs like that is one common thing people do#2019-06-0501:04Alex Miller (Clojure team)with the caveat that the macro expands to a function call, so if you want that to be part of your public api, available outside the ns, then the function needs to be public too#2019-06-0501:05Alex Miller (Clojure team)sometimes that feels dirty#2019-06-0501:05Alex Miller (Clojure team)spec itself has this all over (s/and is a macro that expands to s/and-spec-impl etc)#2019-06-0501:06Alex Miller (Clojure team)we have significantly changed that in spec 2... but that's a longer story#2019-06-0501:07hlolliyeh I see, it's dirty in a forgiveable way. Other plus is that when I'm working in the repl, I only need to change the function once (given that I have a `(def ~symbol function) pattern), instead of re-evaling all def instances#2019-06-0501:09Alex Miller (Clojure team)yep#2019-06-0516:58dangercoderHi! Never looked at how Spec works when it comes to data that depend on other data.
Is it possible to use clojure.spec to validate and generate relational data? Let's say I have a map like this:
{:min-amount 100.00M
:max-amount 10000000.00M
:amount 133.00M}
:amount must be within the bounds of min-amount and max-amount.#2019-06-0516:59Alex Miller (Clojure team)you can do this with custom generators#2019-06-0517:00Alex Miller (Clojure team)gen the min and the max and then use gen/bind to create a generator that produces values in the range and packages them all together#2019-06-0517:00kszaboa tool in this area: https://github.com/reifyhealth/specmonstah#2019-06-0517:01dangercoderYeah I am listening to a podcast about Specmonstah right now#2019-06-0517:01kszabonot for the usecase you mentioned, but for relational data generation#2019-06-0517:01dangercoderThanks @alexmiller I will look more into how custom generators work! š#2019-06-0522:08plinshey everyone, I want to spec a map with 2 values (`:key1` :key2
), they are both ints so ill use integer?
..
but the second key must be bigger than the first, whats the best way to achieve that?#2019-06-0523:34seancorfield@plins wrap the s/keys
in s/and
and add a predicate#2019-06-0523:38seancorfield@plins if you need more concrete guidance than that, LMK and I'll paste an example#2019-06-0600:07plinsthank you very much but I've managed to do it š#2019-06-0615:45sveri#2019-06-0615:50Alex Miller (Clojure team)you're just getting the spammed ex-info data there, not a crafted message#2019-06-0615:50Alex Miller (Clojure team)which clojure version and where are you seeing this?#2019-06-0615:51Alex Miller (Clojure team)seeing it in repl (if so, which repl) or from lein command or other command?#2019-06-0709:31sveriGetting back to this problem. I did some digging and it turns out, if I run this function as part of a web request it gets wrapped into a exception clojure.lang.ExceptionInfo
and printed out like seen above vs calling the function from the repl and seeing the more readable output.
What would be the idiomatic way to use spec in this environment?
I could catch the exception, and display some better error? Is it possible to extract a more readable error from the exception?
Two things that bug me here.
1. ExceptionInfo seems like a very generic exception and I would have to look into it to check if its a exception regarding spec.
2. I would have to do this for every function that accepts / returns a web request while I would like to have it printed readable everytime, without having to do manual exception handling, if possible.#2019-06-0615:57sveriI see this in the cursive REPL when running a function of mine that I specced.
There is also a lengthy stacktrace available and what fails is clojure.spec.test.alpha$spec_checking_fn$conform_BANG___3024.invoke
Clojure version is 1.10.1 and spec is spec.alpha 0.2.176#2019-06-0616:06Alex Miller (Clojure team)what kind of repl in Cursive?#2019-06-0616:07Alex Miller (Clojure team)I'm most likely going to suggest you ask in #cursive as its really dependent on who is controlling the printing, which here is either cursive or nrepl#2019-06-0616:08Alex Miller (Clojure team)that is, if you used lein or clj in this same scenario, I think you would get a much better output#2019-06-0616:58vemvI post here a variation of something that I posted in #clojurescript this morning, but is essentially a different problem (it affects JVM clojure)
I have a ns l
from library L which requires clojure.core.specs.alpha
, and uses one of its specs via spec/def
.
And I have a ns a
from app A that requires l
.
When running A, I get java.lang.Exception: Unable to resolve spec: :clojure.core.specs.alpha/params+body
. This doesn't happen when running L.
I use the Reloaded workflow with vanilla requires and project organisation. All deps are the latest#2019-06-0617:19vemv@alexmiller :
The problem appears to boil down to the fact that just that specific spec is absent from the registry#2019-06-0617:19vemv#2019-06-0618:10Alex Miller (Clojure team)there were some core spec renames in the 1.10-era core.specs.alpha - are you seeing mismatches maybe?#2019-06-0618:11Alex Miller (Clojure team)yeah params+body was a rename in 1.10-era from 1.9-era#2019-06-0618:12Alex Miller (Clojure team)changes happened in core.specs.alpha 0.2.44#2019-06-0618:14Alex Miller (Clojure team)so would be good to know the version of that being used by L and A, and also whether aot is happening on L#2019-06-0618:15vemvhi! I've been doing archeology for the last hour and I still can't make sense of things
even when using 1.10.1 and the latest clojure.core.specs.alpha
dep (stable, or snapshot, or none at all, since it's bundled), (require '[clojure.core.specs.alpha] :reload-all)
won't get me the params+body spec in this particular project
maybe Leiningen is doing sth evil#2019-06-0618:19vemvno aot in either side. tried lein clean
, no luck either#2019-06-0618:23Alex Miller (Clojure team)Clojure 1.10.0
user=> (require '[clojure.core.specs.alpha :as sa])
nil
user=> ::sa/params+body
:clojure.core.specs.alpha/params+body
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/get-spec ::sa/params+body)
{:clojure.spec.alpha/op :clojure.spec.alpha/pcat, :ps [:clojure.core.specs.alpha/param-list {:clojure.spec.alpha/op :clojure.spec.alpha/alt, :ps ({:clojure.s...
#2019-06-0618:23Alex Miller (Clojure team)just using clj here#2019-06-0618:29vemvyes, in the original library ("L") it just works
in a larger app ("A") using L it doesn't
Did whatever I could to bisect, no luck
I'll try creating another app, since A is quite large tbh, increasing the chances for weird interactions#2019-06-0618:41Alex Miller (Clojure team)does the registry contain any of the core.specs.alpha stuff?#2019-06-0618:41Alex Miller (Clojure team)(keys (s/registry))#2019-06-0618:48vemvmost of them yes. see #clojure , found the thing#2019-06-0619:17sveri@alexmiller its the same in the clj repl. But I have a lot of libraries loaded, not sure of something messes around with the output.#2019-06-0622:48plinsdid some googling but not found the anwser, is it possible to share specs between clj and cljs files? do I need to write .cljx files?#2019-06-0622:57Alex Miller (Clojure team)you can write specs in .cljc files and share them#2019-06-0623:40plinsany recommended documentation on how to setup clc files and share them? something has to be done in project.clj?#2019-06-0700:10seancorfield@U3QUAHZJ6 It's mostly automatic. The compilers look for both .cljc
files and their own suffix (`.clj` or .cljs
).#2019-06-0622:58Alex Miller (Clojure team).cljx is old and shouldn't be used anymore#2019-06-0707:08sveri@alexmiller I just checked my error in a minimal example and indeed the output is very different:
Execution error - invalid arguments to off/add-nutriments at (core.clj:9).
"" - failed: #{"kJ" "kcal" "kCal" "g" "mg"} at: [:product :nutriments :clojure.spec.alpha/pred :sugars_unit] spec: :off/unit
"" - failed: #{"kJ" "kcal" "kCal" "g" "mg"} at: [:product :nutriments :clojure.spec.alpha/pred :fat_unit] spec: :off/unit
"" - failed: #{"kJ" "kcal" "kCal" "g" "mg"} at: [:product :nutriments :clojure.spec.alpha/pred :salt_unit] spec: :off/unit
{:salt "3.5", :sugars_unit "", :energy-kcal "201.81514210652017", :energy_unit "kcal", :energy_value 202, :proteins_value "17.6", :proteins_unit "", :carbohydrates_unit "", :saturated-fat_value "8.2", :fat 12, :energy 845, :salt_value "3.5", :saturated-fat_unit "", :fat_100g 12, :sugars_value 6, :sodium_100g "1.37795275590551", :carbohydrates_value 6, :proteins_100g "17.6", :fat_unit "", :saturated-fat "8.2", :sugars_100g 6, :sodium "1.37795275590551", :sugars 6, :carbohydrates 6, :energy_100g 845, :salt_100g "3.5", :saturated-fat_100g "8.2", :salt_unit "", :carbohydrates_100g 6, :proteins "17.6", :fat_value 12} - failed: nil? at: [:product :nutriments :clojure.spec.alpha/nil] spec: :off/nutriments
I do use the same cursive repl so it must be some library that tinkers with the spec output settings. Thanks for the hint.#2019-06-0707:09sveriAnd yea, this output definitely is much more helpful.#2019-06-0712:11Alex Miller (Clojure team)In 1.10.1 you should also get a full path location in the first line too#2019-06-0811:11mishais there a better way to make s/conform
label values conformed to s/coll-of
other than wrap s/coll-of
in s/or
?
(s/conform (s/coll-of any?) [])
=> []
(s/conform (s/or :vector (s/coll-of any?)) [])
=> [:vector []]
#2019-06-0811:13mishaneed it for a recursive spec, to uniformly label all tree nodes, to dispatch on conform-labels in a case
#2019-06-0812:48Alex Miller (Clojure team)Canāt say I have any better idea other than to do this walk after the conform instead#2019-06-0813:33misha(re-)labeling conformed tree is doing (almost) double work. and implies having double knowledge of "how nodes look like" in the code: in spec and inline during the walk.
Or, worse, dispatching with call to s/valid? or s/conform again, which will conform entire subtree from current node down.
the (s/or)
wrappers are hacky, but not that bad for irregular trees, all things considered:
(s/def :user/leaf int?)
(s/def :user/foo (s/coll-of (s/or :user/leaf :user/leaf :user/bar :user/bar :user/foo :user/foo)))
(s/def :user/bar (s/coll-of (s/or :user/leaf :user/leaf)))
(s/def :user/tree
(s/or
:user/leaf :user/leaf
:user/foo :user/foo))
(s/conform :user/tree [[1] 2 [[3]]])
;;=> [:user/foo [[:user/bar [[:user/leaf 1]]]
[:user/leaf 2]
[:user/foo [[:user/bar [[:user/leaf 3]]]]]]]
#2019-06-0814:48Alex Miller (Clojure team)my complaint is basically that you're turning conform into a meat grinder to get a specific data structure you want, and that's not what it's designed for - it's designed to tell you a) is this valid? and b) why?#2019-06-0815:15potetmI thought it was also deigned to be ādestructuring on steroidsā#2019-06-0815:15potetm(I forget where I heard that.)#2019-06-0815:23Alex Miller (Clojure team)it has some capability in that regard when conforming regex for syntax structures in macros, but even then it's not destructuring, more making structures amenable to destructuring#2019-06-0906:38mishaI completely agree with your complaint. But it works! kappa#2019-06-0906:39misha(went with loop+case and weaker jit errors instead of spec for such tree this time, though)#2019-06-0907:08mishafurther destructuring of a form conformed to regex-spec and to or-spec is different: map vs "twople". In case of a tree, you'll walk 1 node at a time, which makes
(let [[tag form] conformed]
(case tag ...))
preferable to
(let [{:keys [tag1 tag2 tag3 ...]} conformed]
(cond ;; I guess?
tag1 ...
tag2 ...))
#2019-06-0907:09mishabut latter brings the same edge cases as (contains? #{...} x)
vs. just (#{...} x)
: nil, false#2019-06-1002:38andy.fingerhutThere is an example spec using s/double-in
on the spec guide page: (s/def ::dubs (s/double-in :min -100.0 :max 100.0 :NaN? false :infinite? false))
I have tried changing the false
occurrences to true
, and leaving out the :NaN?
and :infinite?
keys, but I always see (s/valid? ::dubs ##Inf)
evaluate to false
, as well as (s/valid? ::dubs ##-Inf)
and (s/valid? ::dubs ##NaN)
. Is that expected?#2019-06-1002:44andy.fingerhutI guess it makes sense for ##-Inf
to be invalid if a :min
value is specified that is a double value other than ##-Inf
, and similarly for :max
values and ##Inf
.#2019-06-1002:44andy.fingerhutBut it seems like specifying either :min
or :max
changes the validity of ##NaN
from true to false.#2019-06-1002:44andy.fingerhutno matter what the value associated with the :NaN?
key happens to be#2019-06-1002:49Alex Miller (Clojure team)might be the ordering of the checks#2019-06-1002:50Alex Miller (Clojure team)if you (s/form :;dubs) you can see what it expands to#2019-06-1003:02Alex Miller (Clojure team)maybe buggy at a glance as it's an s/and and it starts flowing the boolean rather than the double#2019-06-1018:37andy.fingerhutI doubt that is a hot ticket item for anyone, but created a JIRA ticket as a reminder: https://clojure.atlassian.net/browse/CLJ-2516#2019-06-1213:23carkhmulti-spec dispatching on the first value in a sequence and returning the syntax for the rest of that sequence, possible at all ?#2019-06-1213:24carkhall examples of multi-specs i could find were about maps#2019-06-1213:26carkhwhat i'm trying to do : (s/def ::node (s/cat :tag keyword? (multispec dispatching on the tag here)))
#2019-06-1213:30Alex Miller (Clojure team)it is not only maps, can definitely be made to work with other things (I've got some stuff doing that)#2019-06-1213:32Alex Miller (Clojure team)you can make your multimethod dispatch on first
#2019-06-1213:32Alex Miller (Clojure team)and then you have to be a little careful with the retag function#2019-06-1213:32Alex Miller (Clojure team)I actually have an example of this in CLJ-2112 (specs on specs) I think#2019-06-1213:33Alex Miller (Clojure team)https://clojure.atlassian.net/secure/attachment/10625/spec-forms.patch#2019-06-1213:33Alex Miller (Clojure team)you'll see spec-form
is a defmulti on first
#2019-06-1213:33Alex Miller (Clojure team)and the ::spec
spec uses (s/multi-spec spec-form (fn [val tag] val))
#2019-06-1213:34Alex Miller (Clojure team)and then the defmethods cover the different cases with different starting symbols#2019-06-1213:38carkhthanks looking at your example#2019-06-1213:42carkhok got it working on the whole "sequence", i'm guessing this can't be done the way i want, dispatching on the first element in order to spec the rest of the sequence#2019-06-1213:42carkhnot a huge deal, just a little usability issue i guess#2019-06-1213:44Alex Miller (Clojure team)isn't the example above what you're describing?#2019-06-1213:44Alex Miller (Clojure team)I'm using it to spec a form sequence based on the operator type at the beginning, so seems like the same thing?#2019-06-1213:52carkh(s/cat :tag keyword? :rest (s/multi-spec node-type :rest))
it looks like it's looking for a sequence in the rest#2019-06-1213:53carkhthe mmultispec is returning something like (s/cat ....) a flat thing#2019-06-1213:53carkhis multispec like s/spec in this regard ?#2019-06-1213:54carkh(i have a hard time following your example)#2019-06-1213:55Alex Miller (Clojure team)you don't want an s/cat here at all - the multi-spec is the sequence spec#2019-06-1213:55carkhbut i need something to put around my stuff returned from the multimethod =)#2019-06-1213:55carkhbut yes i understand#2019-06-1213:56Alex Miller (Clojure team)I don't understand what that meant#2019-06-1213:56Alex Miller (Clojure team)the multimethod returns the spec for the sequence#2019-06-1213:56carkhright#2019-06-1213:56carkhin the typical example for a typed map, you spec for the type keyword outside the multimethod#2019-06-1213:57carkhthen only spec for what's relevant to the multispec user in the multimethod#2019-06-1213:57carkhbut i think what i'm asking is impossible#2019-06-1213:58Alex Miller (Clojure team)afaict what you're asking for is the example I gave. I don't get why it's not what you want.#2019-06-1213:59Alex Miller (Clojure team)are you looking for something different in the conformed value shape?#2019-06-1214:00carkhin your example, i want the :f #{'clojure.core/fn}
out of the multimethod#2019-06-1214:00carkhjust like for the typical typed map examples#2019-06-1214:01carkhactually i'm wrong on that#2019-06-1214:01carkhevent/type is indeed present in the multi-method in the guide#2019-06-1214:02carkhok then sorry for the noise ><#2019-06-1214:03Alex Miller (Clojure team)you can post-process the conformed value to do whatever you want of course#2019-06-1214:04carkhi was just weary that all the users of the multimethod will have to spec for the "tag"#2019-06-1214:04carkhnot a huge deal since all those users are me =)#2019-06-1214:04carkhthanks for your time !#2019-06-1214:16Alex Miller (Clojure team)yeah, you can just spec it as any?
if you like#2019-06-1214:16Alex Miller (Clojure team)it's already been chosen by the time you get into the multimethod and the value will flow through#2019-06-1214:17Alex Miller (Clojure team)I spec'ed it in that code just for clarity#2019-06-1217:04colinkahnWhen multi-specing where the specs are s/keys
and the retag is a keyword, is it best practice to include the retag key as :req
in the defmethods keys spec?#2019-06-1217:23Alex Miller (Clojure team)not sure what you mean by "include the retag key as :req
"#2019-06-1217:23Alex Miller (Clojure team)oh, you mean as a :req in the s/keys#2019-06-1217:24Alex Miller (Clojure team)well it is probably required to get to that point in the first place#2019-06-1217:25Alex Miller (Clojure team)and either way it's going to end up in the conformed value#2019-06-1217:25Alex Miller (Clojure team)I'm not sure that it matters either way, depends on whether you see the retag key as part of the data or as frame around the data, but that's probably dependent on your code#2019-06-1217:51colinkahnOk cool, yeah I was doing it quite consistently in :req
and ran into something where I was adding a :gen
to the keys spec in the defmethod and found it a bit unergonomic to have to include the retag in the generator as well, which got me feeling like maybe it was better not to include it at all. I have been also specing the retag key mostly as something rather generic like keyword?
too.#2019-06-1311:12vemvHey there,
I wrote down this design https://github.com/reducecombine/fridge/issues/9 50% for fun, 50% b/c it might actually be necessary at work.
It doesn't seem too extravagant (in fact I view it as very aligned with RH-isms: immutable/never-broken specs, growth by accretion, no renames etc), but still, there is a gap between theory and practice š i.e. my design might actually have flaws in a real system.
It would be super appreciated if someone with relevant experience could comment in the issue.#2019-06-1311:12vemvHey there,
I wrote down this design https://github.com/reducecombine/fridge/issues/9 50% for fun, 50% b/c it might actually be necessary at work.
It doesn't seem too extravagant (in fact I view it as very aligned with RH-isms: immutable/never-broken specs, growth by accretion, no renames etc), but still, there is a gap between theory and practice š i.e. my design might actually have flaws in a real system.
It would be super appreciated if someone with relevant experience could comment in the issue.#2019-06-1311:45Alex Miller (Clojure team)Seems fine. I guess my only real question is whether apps and models really need that level of absolute separation that comes from putting the app name in the attribute qualifier#2019-06-1311:47Alex Miller (Clojure team)Seems like you will end up basically copying the same attribute definition everywhere (and require unnecessary data transformation at each step). The whole idea of the global registry is to get reuse#2019-06-1311:48Alex Miller (Clojure team)There is more coming on datafied specs in spec 2 btw#2019-06-1312:15vemv> Seems like you will end up basically copying the same attribute definition everywhere
Not sure I can follow.
Let's say I have N repos, 1 per microservice, plus 1 special repo, the 'immutable spec registry'
Over there, I (spec/def :messaging.blue-app.v1/age integer?)
And microservices can consume that spec (and quite concisely, via require ... :as ...
). There's no copying in that
> and require unnecessary data transformation at each step
What did you mean?#2019-06-1313:22Alex Miller (Clojure team)I assume these services pass data to each other. It seems improbable that you won't have multiple services handling the same semantic attribute (like "age"). in your scheme, that attribute will show up in the messages for all the services - :messaging.blue-app.v1/age
, :messaging.red-app.v1/age
etc#2019-06-1313:23Alex Miller (Clojure team)if a service ever takes a map with one variant and returns a map with the other variant, you have rebuild the map and change the attribute name#2019-06-1313:24Alex Miller (Clojure team)whereas if you had a :messaging.common.v1/age
, it could flow between services and be shared#2019-06-1313:24Alex Miller (Clojure team)which was the whole point of the global registry#2019-06-1317:47seancorfieldThis is why we tend to use more generic domain-level prefixes for our specs, rather than specific/namespace-qualified names.#2019-06-1318:24vemvUnderstood!
I'll give it a think, however I find some value in having this design:
Each microservice can retain a specific meaning and type for age
. That way, services can evolve independently, without coordination.
The opposite might resemble a distributed monolith, where making a change/decision needs asking everyone else (other codebases / other team members) first
(btw, messaging is via Kafka so the map-in-map-out philosophy doesn't apply that much)#2019-06-1321:23rapskalian@U04V70XH6 could you give an example of what you mean by ādomain-levelā? Would you tend to favor :user/email
over :myapp.auth.user/email
?#2019-06-1321:49seancorfieldFor stuff that stays "inside the company", the qualifier doesn't need to be any more specific than the broadest level of uniqueness needed. So, yeah, if you want to standardize on an email representation for all your users across the whole company, :user/email
is fine. It shouldn't be part of any public APIs/libraries you create -- but using commonly known domain names inside your company helps reuse across multiple systems.#2019-06-1321:52seancorfieldA specific example is :wsbilling
for us -- for a lot of things that are related to our World Singles Billing services rather than being tied to a single app, these specs are reused by all apps that support and interact with those services. We've used :wsbilling
there rather than just :billing
because there are other generic billing services we interact with, so this just hints that it's about our billing services rather than anything else we interact with.#2019-06-1323:01rapskalianVery helpful, thanks#2019-06-1406:36jaihindhreddy@U45T93RA6 "making a change/decision needs asking everyone else (other codebases / other team members) first", as RH says in Spec-ulation, the way we use the word change tends to complect growth and breakage, which are completely different. As long as you're only doing growth, these is no need to ask (or even tell) anyone.#2019-06-1406:45seancorfieldThat's why it's so important to me that next.jdbc
will never make breaking changes -- only accretive/fixative changes, since it moved into beta. I learned a lot of lessons from maintaining clojure.java.jdbc
for the last eight years! š#2019-06-1407:36vemv@ jaihindh, how do you tell if a change is breaking or not? as hinted in the document, One man's non-breaking is another's breaking
. My approach is pessimistic in that it assumes all changes are potentially breaking.
At the same time, it encourages services to keep sending/accepting messages at old versions, so you don't actually break 3rd parties.
This philosophy is partly justified by the fact that atm we don't have a tool that automatedly analyses spec compatibility. Might be unfeasible given that specs can be backed by arbitrary code#2019-06-1411:22jaihindhreddyI don't have the answer (don't even have an answer) for how to partition "change" into "growth" and "breakage". And no matter how hard we try, at some point people will rely on things that are broken, making one man's fixation another man's breakage, etc. But if we keep changing our names by incorporating versions into them and ask people to assume they are potentially breaking, we still lack a stable set of names that can act as a stable basis to reason about our data across time. That, is the real systems problem right?
Sorry for the blabbering & noise š
#2019-06-1416:06seancorfieldYou don't change the names. Just introduce new names. The old names still exist and have the old behavior. Folks can continue to rely on those old names "forever". #2019-06-1413:30dmarjenburghIs there a convenient way to separate specs definitions and their generators? I want to define my specs with my main code, but only use generators in test code. I include test.check
in my classpath only with the test
alias. I know spec loads generators lazily for this purpose, but not sure how to use s/with-gen
when the generator argument uses stuff depending on test.check#2019-06-1413:42dmarjenburghOkay, I think I got it. I was calling gen/fmap
when my namespace of custom generators was loaded.#2019-06-1516:47misha(->> [0 1 2 3]
(partition-all 2)
(into {}))
ClassCastException [trace missing]
(->> [[0 1] [2 3]]
(into {}))
=> {0 1, 2 3}
š#2019-06-1516:48misha(->> [0 1 2 3]
(partition-all 2)
(map vec)
(into {}))
=> {0 1, 2 3}
#2019-06-1519:20andy.fingerhutone of the places in Clojure where vectors vs. non-vectors makes a difference.#2019-06-1519:34y.khmelevskiihey, can you please help me to understand why lein throw this error
If there are a lot of uncached dependencies this might take a while ...
clojure.lang.ExceptionInfo: Could not find artifact org.clojure:spec-alpha2:jar:0.2.177-SNAPSHOT in central ()
{:lib org.clojure/spec-alpha2, :coord {:mvn/version "0.2.177-SNAPSHOT", :deps/manifest :mvn}}
#2019-06-1519:37y.khmelevskiiI use leiningen together with deps.edn#2019-06-1519:40bronsasnapshot releases are not in maven central, they're in sonatype#2019-06-1519:40bronsatry adding
in your repos#2019-06-1519:41y.khmelevskiiwhich way? via :mvn/repos
?#2019-06-1519:41bronsayeah#2019-06-1519:43y.khmelevskii:mvn/repos {"clojars" {:url ""}
"central" {:url ""}}
it doesnāt work#2019-06-1519:58y.khmelevskiiand my dependency
org.clojure/spec-alpha2 {:mvn/version "0.2.177-SNAPSHOT"}
#2019-06-1520:00bronsanot sure if lein picks up repos from deps.edn#2019-06-1520:00bronsaif you're sure that snapshot exists, maybe trying adding that repo in .lein/profiles.clj
and retry#2019-06-1520:00bronsadon't have any other suggestions, sorry#2019-06-1609:52hlollihow is it possible to prevent a generator from doing a retry without re-calling a generator function? Here's an ugly phone number generator
(defn phone-number-generator []
(gen/fmap (fn [[cc phone-num]]
(str "+" cc " (0) " phone-num))
(gen/tuple (gen/return (rand-nth ["1" "37" "39" "44" "49"]))
(gen/return (-> (* Integer/MAX_VALUE (Math/random))
int
str)))))
printing the validation from the spec
"VALID?" "+1 (0) 265957916" false
it gets print 100 times exactly the same, the generator is only called 1x or 2x
that all ends with
Execution error (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
#2019-06-1611:20gfredericks@hlolli generally you design the generator so that it has no need to retry#2019-06-1611:20gfredericksI can't see the spec so I don't know what that would be#2019-06-1611:21hlolliI see, it's this here
(defn-spec phone-number-valid? boolean?
[phone-number string?]
(let [instance (PhoneNumberUtil/getInstance)
phone-number-parsed (.parse instance phone-number "")]
(.isValidNumber instance phone-number-parsed)))
(s/def :user/phone-number ; format
(s/with-gen phone-number-valid?
generators/phone-number-generator))
#2019-06-1611:23gfredericksokay well there's some stuff buried in the jvm method
but for example, is the last portion supposed to be a particular number of digits?#2019-06-1611:25hlolliyes, it depends I think which country code is used, so I just multiply some random number to sizeof(int), should give me a more likely way of number that purely random number generator, I think?#2019-06-1611:25hlolliOr I could choose 1 counry code and substring to a specific length#2019-06-1611:25gfredericks(* Integer/MAX_VALUE (Math/random))
is going to be pretty skewed towards certain lengths#2019-06-1611:26gfredericksif it's easy to get a length associated with each country code, then you could do it much better#2019-06-1611:27hlolliyes, these rules are difficuly, despite all lengths being right, sometimes the first digit must fit some standard#2019-06-1611:27hlolliI can read more about this ofc#2019-06-1611:27hlollibut I thought the generator could try again and again until it's valid#2019-06-1611:27hlollilike with regex specs#2019-06-1611:27gfredericks(gen/for [[country-code length] (gen/elements country-codes-and-lengths)]
number (apply gen/tuple (repeat length (gen/choose 0 9)))]
(str "+" country-code " (0) " number))
is the sort of thing I would aim for if I Had that list#2019-06-1611:28hlollicool, instersting š#2019-06-1611:28gfredericksrelying on the generator trying again is not a good plan unless you're sure it's highly likely to succeed#2019-06-1611:28gfredericksbecause you can easily be in a situation where it only succeeds e.g. 0.001% of the time#2019-06-1611:28gfredericksand then you'll be burning a lot of CPU#2019-06-1611:29hlolliyes true, I thought the idea would be to give the generator a bit of help. The odds of alpha-numberic random gen hitting a valid phone number are probably astronomical, I'd assume#2019-06-1611:29gfredericksyeah, you can definitely do worse than what you have š#2019-06-1612:11gfredericks@hlolli incidentally, here's a generator that's evenly distributed between digit lengths
(gen/for [digit-count (gen/choose 5/10)
digits (apply gen/tuple (repeat digit-count (gen/choose 0 9)))]
digits)
#2019-06-1612:12gfredericksthat might work better for you#2019-06-1612:12hlolliok, but I didn't get gen/for, in which namespace is it?#2019-06-1612:13hlolliI'll try soon and let you know, I'm generating fake firebase users for testing#2019-06-1612:14gfredericksclojure.test.check.generators
#2019-06-1612:14gfredericksit's just syntax sugar for gen/fmap
and gen/bind
mostly#2019-06-1612:14gfredericksyou can do the above just using gen/bind
#2019-06-1612:17hlolliah I see, I was searching for it in clojure.spec.gen.alpha
#2019-06-1612:18hlollithanks š#2019-06-1701:21Alexander StavoninIs it possible to check that a spec argument is a function with expected signature?#2019-06-1701:24Alexander StavoninI can easily implement such check with protocols, but in one particular case function is preferable for me.#2019-06-1702:40taylorspec alpha1 āchecksā function values by invoking them; so yes, if that meets your needs you could spec function args#2019-06-1702:40taylormaybe you could do something with meta
and :arglists
?#2019-06-1703:11Alexander StavoninI have a function which accept other function as argument. Iād like to be sure, the argument is: a) function, b) function with expected signature (lets say with 2 arguments).
I suppose, fn?
could be helpful here, but what should I do with function arguments test?#2019-06-1704:33seancorfield@UKHQCLTQV clojure.spec
isn't a type system.#2019-06-1704:36Alexander StavoninI know, but sometimes type-system like guarantees are very convenient. This is especially important in cross components calls. Can you suggest any other way to which will provide such guarantees?#2019-06-1704:39seancorfieldMaybe you want to look at Typed Clojure instead, in order to analyze code and ensure that you're passing the right sort of function as an argument?#2019-06-1704:44Alexander Stavonin@U0JLGECPK, are you talking about https://github.com/clojure/core.typed?
I just looked thru readme and got feeling Typed Clojure means ā whole project should be statically typed. This is not a goal for me, as Iām more or less happy with dynamic typing, and the only needs is ā cross components interface data and types validity guarantees.#2019-06-1704:48Alexander StavoninOk, I should look deeper here: This work adds static type checking (and some of its benefits) to Clojure, a dynamically typed language, while still preserving idioms that characterise the language. It allows static and dynamically typed code to be mixed so the programmer can use whichever is more appropriate.
#2019-06-1704:48Alexander Stavoninthanks!#2019-06-1704:51Alexander Stavoninjust interesting reading about this project: https://circleci.com/blog/why-were-no-longer-using-core-typed/#2019-06-1704:57seancorfieldYeah, there are trade offs. And that's an old article. core.typed
has had a lot of work since then.#2019-06-1704:58seancorfieldWe also tried it back then and gave up for all the same reasons CircleCI gave up.#2019-06-1704:58seancorfieldBut the question you're asking is best answered by core.typed
rather than spec.alpha
#2019-06-1710:46djtangothat said - Racket was able figure out a solution to higher-order functions with their run-time contracts system, that also doesn't involve doing gen-testing on the input function#2019-06-1710:47djtangobut function equivalence is undecidable so tradeoffs are always going to have to be made in this space...#2019-06-1712:52droneTyped Clojure seems to be pretty inactive.
While spec isnāt a type system, I could see function arity being part of a contract. E.g., for a function passed to reduce. But I also think the spec designers would argue that function arity errors already exist and solve most of the problem.#2019-06-1716:02seancorfieldAmbrose just successfully defended his thesis work on Typed Clojure. It's still an active project. #2019-06-1716:10droneThere was a flurry of activity last winter, and then other than a few commits to the core.typed.analyzer.jvm
project in April (likely in preparation for defense) there has been no activity for seven months#2019-06-1716:17seancorfieldIf you've watched any of his talks you know that he has a lot of design work ongoing about dealing with the typed / untyped code boundary. Just because there have been no recent commits doesn't mean there's no recent work -- this is an extremely hard topic area that requires a lot of "hammock" time. #2019-06-1716:23dronesure, but itās also just as likely he is moving on from the project after completing his PhD program#2019-06-1719:59seancorfieldNot based on what he's talked about at conferences.#2019-06-1722:11dronethatās great if heās still working on it. but based on observable output, including: repo commits, the projectās twitter account, his patreon, and activity in #core-typed; not much is going on in the project and thatās what I meant by āinactiveā. this seems like a weird thing to argue aboutā¦#2019-06-1712:47Jakub HolĆ½ (HolyJak)Is there a shorter, nicer way to do this (without nesting if
in let
)? (defn conform! [x spec]
(let [conformed [(s/conform spec x)]]
(if (#{::s/invalid} conformed)
(throw (ex-info
(s/explain-str spec x)
(s/explain-data spec x)))
conformed)))
#2019-06-1712:52Charles FourdrignierThis looks like a lot to Stuart Halloway conform!
, so I guess you can't do better.
https://github.com/Datomic/mbrainz-importer/blob/master/src/cognitect/xform/spec.clj#2019-06-1816:13OlicalI came up with something similar but as a macro
(defmacro on-invalid [conformed fallback]
`(let [conformed# ~conformed]
(if (s/invalid? conformed#)
~fallback
conformed#)))
(defn validate [x spec message]
(when-not (s/valid? spec x)
(throw (ex-info (str message "\n\n" (expound/expound-str spec x)) {})))
x)
#2019-06-1713:32ghadi@holyjak don't do that because you'll be putting a ton of data in the exception message (via explain-str
)#2019-06-1713:33Jakub HolĆ½ (HolyJak)hm, good point, thanks!#2019-06-1713:37Jakub HolĆ½ (HolyJak)maybe I should just use (-> explain-data ::s/problems first (dissoc :val :value))
instead?#2019-06-1823:01mishaimplementing this as macro has an advantage of being able to include calling form into error data/message, which saves a lot of time for exceptions purposed for developer and repl.
or just have an error-msg arg in a second arity, with default message like "does not conform to spec <spec form>" because you'll expand exception data in repl anyway, because "first problem" is not always "the only problem"#2019-06-1916:16Jakub HolĆ½ (HolyJak)Thanks!#2019-06-1914:42hmaurero/. Quick question: is the new way of handling optionality in clojure spec (with selections etc) already usable?#2019-06-1915:14dronelike nilable
? or do you mean :opt
and :opt-un
in keys
?#2019-06-1915:18seancorfield@mrevelle He means in spec-alpha2. Right @hmaurer?#2019-06-1915:18droneah..#2019-06-1915:18hmaurer@seancorfield I think yes; what Rich Hickey talked about at this autumnās conj#2019-06-1915:19seancorfieldWe have a branch at work that runs with spec-alpha2, but itās still very much a moving target right now, and has a number of bugs.#2019-06-1915:19seancorfieldItās probably fine to āplayā with but Alex has advised against using it for any serious work yet.#2019-06-1915:24hmaurerš thanks#2019-06-1915:34ghadihttps://github.com/clojure/spec-alpha2/wiki/Schema-and-select
some docs here, everything subject to change#2019-06-1915:37hmaurerty!#2019-06-1915:34ghadi@hmaurer#2019-06-2007:56Ben HammondI am calling generate
on a complex data structure.
Sometimes I see this error.
I would normally troubleshoot this by commenting out parts of the spec to zone in on the problem; but I'm wondering if there is a smarter way to debug generators#2019-06-2007:57Ben Hammondis there a direct way to dicover the specific sub-spec that has the problem?#2019-06-2009:51Ben Hammondokay another question;
I want a generator that has two modes of operation
ā¢ when there is data available in the dynamc variable *existing-ids*
then I want it to behave like (element *existing-ids
)`
ā¢ when *existing-ids*
is empty then I want it to behave like (spec/gen string?)
this must be a fairly common use case
how has it been solved before?#2019-06-2009:55Ben HammondI'm thinking something like
(test.gen/bind
(test.gen/return nil)
(fn [_]
(if (not-empty *existing-ids*)
(test.gen/elements *existing-ids*)
test.gen/string-alpha-numeric)))
#2019-06-2010:00Ben Hammondthat doesn't seem a very smart solution though#2019-06-2010:11Ben HammondI think that's the best I can do, unless I hack directly into (defn- bind-helper
or
(defn- make-gen
but that doesn't seem like a very good idea either#2019-06-2010:15Ben Hammond(def ^:dynamic *existing-ids* nil)
=> #'dev/*existing-ids*
(def gg (test.gen/bind
(test.gen/return nil)
(fn [_] (if (not-empty *existing-ids*)
(test.gen/elements *existing-ids*)
test.gen/string-alphanumeric))))
=> #'dev/gg
(spec.gen/generate gg)
=> "9Dlv2SuFqT4Jb"
(binding [*existing-ids* [:this :that :tother]]
(spec.gen/generate gg))
=> :that
#2019-06-2010:18Ben HammondI suppose I could store the test.gen/elements
in the *existing-ids*
var#2019-06-2010:23Ben Hammond(spec/with-gen
takes a gen-fn, and I'd hope that I could use this to choose between generators, but of course it doesn't get called very often#2019-06-2010:31Ben Hammondthis might be better actually
(defn build-indirect-gen
[vargen]
(test.gen/bind
(test.gen/return nil)
(fn [_] (deref vargen))))
(def ^:dynamic *existingids-gen* test.gen/string-alphanumeric)
(def gg (build-indirect-gen #'*existingids-gen*))
(binding [*existingids-gen* (test.gen/elements [:this :that])]
(test.gen/generate gg))
=> :that
(test.gen/generate gg)
=> "Sjeu5IwBWbMUZl"
#2019-06-2015:02misha@ben.hammond afair, Gary touched on it debugging "Couldn't satisfy such-that predicate after 100 tries" here: https://www.youtube.com/watch?v=F4VZPxLZUdA#2019-06-2015:04mishathis is the timestamp#2019-06-2015:04mishahttps://youtu.be/F4VZPxLZUdA?t=976#2019-06-2015:17Ben HammondI've not seen that video.
that's very interesting#2019-06-2016:12Ben Hammondi'd not seen the size/scale explained properly before#2019-06-2016:12Ben HammondI'd assumed they represented mean/standard deviation#2019-06-2018:04Alex WhittI'm sure this has been asked and answered, but I can't find the answer...
Why can I not do the following?
(s/def ::a any?)
(s/def ::thing (let [ks [::a]] (s/keys :req ks)))
That yields Don't know how to create Iseq from: clojure.lang.Symbol
.#2019-06-2018:27Alex WhittI'm guessing that macros are the only way to dynamically define specs?#2019-06-2018:39favilapretty much#2019-06-2018:39favilaI think spec2 has some improvements to this, but still isn't going to let you plop a let
in there#2019-06-2018:40favilaI think spec2 will expose a public ast-like interface for programmatic spec creation/manipulation of specs as data#2019-06-2018:40favilabut the surface syntax will still be macros#2019-06-2018:41Alex WhittHmm, alright. I'm interested in the spec2 feature you mentioned so I'll take a look. Thanks!#2019-06-2019:23seancorfieldSpec 2 has a macro layer and a function layer that implements the macros. You can programmatically construct pretty much any spec in Spec 2.#2019-06-2019:24seancorfield(we have a branch of our code base at work that uses Spec 2 -- so we can track the sort of differences we'll need to make when we transition from Spec 1)#2019-06-2113:21Alex WhittAh, that's exciting!#2019-06-2107:48Ben Hammondfollowing on from yesterday; I think the tidiest way to achieve
> generator uses id from a database,
> when there is a database
is to use the overrides
feature of (spec/gen)
(doc spec/gen)
-------------------------
clojure.spec.alpha/gen
([spec] [spec overrides])
Given a spec, returns the generator for it, or throws if none can
be constructed. Optionally an overrides map can be provided which
should map spec names or paths (vectors of keywords) to no-arg
generator-creating fns. These will be used instead of the generators at those
names/paths. Note that parent generator (in the spec or overrides
map) will supersede those of any subtrees. A generator for a regex
op must always return a sequential collection (i.e. a generator for
s/? should return either an empty sequence/vector or a
sequence/vector with one item in it)
=> nil
#2019-06-2111:40djtangothanks for this btw - am enjoying your writeup! Shame it'll be gone whenever the history runs out#2019-06-2115:03vemvI was thinking that a spec/not
could make sense as exemplified here
(spec/def ::version (fn [s]
...))
(spec/def ::alpha-version (spec/and ::version
(fn []
...)))
(spec/def ::stable-version (spec/and ::version
(spec/not ::alpha-version))) ;; <- imaginary API
has this been considered already?#2019-06-2115:04vemv(I guess the generative part would suffer)#2019-06-2115:14Alex Miller (Clojure team)I've written a ton of specs and have never needed an s/not#2019-06-2115:15vemvI've also written a fair number of specs for a few years. Just now I think of not
, so that kind of proof is limited#2019-06-2115:16Alex Miller (Clojure team)you could do something like (spec/def ::stable-version #(spec/invalid? (spec/valid? ::alpha-version %)))
#2019-06-2115:16Alex Miller (Clojure team)it is probably inherently difficult to auto-gen#2019-06-2115:17Alex Miller (Clojure team)in any case, we have no plans to add it#2019-06-2115:17vemvš this was more curiosity than anything else. One could always write his own not
anyway#2019-06-2115:39Ben HammondI've got quite an interesting situation;
I have a spec that generates a fairly complex data structure
I sometimes want to overide some of the generators to make it use foreign keys from the database
but it only works intermittently:
I have hooked up a snitching ILookup
to tell me what is happening around
(if-let [g (c/or (when-let [gfn (c/or (get overrides (c/or (spec-name spec) spec))
(get overrides path))]
and what I see is that
When a recompile that spec directly to REPL, then the overridy will wok
When I recompile the entire namespace to REPL, it does not work
Which is intriguing behavour#2019-06-2115:40Ben Hammondso this my ILookup
(reify ILookup
(valAt [_ k]
(println (str "valAt*1: " k))
(println (str "==>" (get m k)))
(get m k))
(valAt [_ k nf]
(println (str "valAt*2: " k ":" nf))
(println (str "==>" (get m k nf)))
(get m k nf))
)
#2019-06-2115:41Ben Hammondwhen it works, I see
valAt*1: :db.generators.offers/offer
==>
valAt*1: []
==>
valAt*1: :db.generators.offers/offer_headline
==>
valAt*1: [:offer_headline]
...
#2019-06-2115:41Ben Hammondwhen it does not work, I see
valAt*1: :db.generators.offers/offer
==>
valAt*1: []
==>
#2019-06-2115:42Ben Hammondis this ringing any bells?#2019-06-2115:45Alex Miller (Clojure team)specs compile in their dependent specs so if you modify a spec, you need to reload any specs that depend on it. that's a likely reason you'd see different results for the two cases#2019-06-2115:45Alex Miller (Clojure team)I'd expect "recompile the entire namespace" to give you the more accurate answer.#2019-06-2115:46Alex Miller (Clojure team)hard for me to tell from this what the actual problem is though#2019-06-2115:47Ben Hammondperhaps I have misunderstood usage.
I looked at
(doc spec/gen)
-------------------------
clojure.spec.alpha/gen
([spec] [spec overrides])
and hoped that I could plug in some overrides that would get me proper foreign keys, and it would just work#2019-06-2115:48Ben Hammondwould you expect to re generate all of the specs to handle a overrides map?#2019-06-2115:50Alex Miller (Clojure team)you should be able to plug in overrides that way. I was responding to
When a recompile that spec directly to REPL, then the overridy will wok
When I recompile the entire namespace to REPL, it does not work
which seemed like a pretty textbook outcome from spec compilation#2019-06-2115:50Alex Miller (Clojure team)you haven't actually shown what you're doing, so I can't really comment#2019-06-2115:50Alex Miller (Clojure team)can you give a full example?#2019-06-2115:52Ben Hammond(spec/def ::offer
(spec/keys
:req-un [::offer_headline
::offer_classifiers
::offer_title
::offer_title_short
::offer_merchant
::offer_voucher_codes
::offer_description
::offer_terms_and_conditions
::offer_terms_and_conditions_url
::offer_claim_restrictions
::offer_presentation]
:opt-un [::offer_pre_claim_advice
::offer_key_terms
::offer_redemption_guidelines
::offer_taxable_value
::offer_identifiers
::offer_images
::offer_approval_required
::offer_discount_mechanic
::claim_condition_msisdn_list]))
(spec/def ::offer_merchant ::ingestion/merchant_id)
and then I have an ingestion namespace that says
(s/def ::merchant_id (s/and string? #(<= 1 (count %) 64)))
is the main offer spec#2019-06-2115:53Ben Hammondso I'm a bit suspicious about that (spec/def ::offer_merchant
#2019-06-2115:54Ben Hammondand now I want to generate a bunch off offers where the merchant ids have come out of the database#2019-06-2115:54Ben Hammondso I write
(:offer_merchant
(test.gen/generate
(spec/gen :db.generators.offers/offer
(let [m {:customer.ingestion/merchant_id (constantly (test.gen/return "OVERRIDEa"))
:db.generators.offers/offer_merchant (constantly (test.gen/return "OVERRIDEb"))
:offer_merchant (constantly (test.gen/return "OVERRIDEc"))}]
(reify ILookup
(valAt [_ k]
(println (str "valAt*1: " k))
(println (str "==>" (get m k)))
(get m k))
(valAt [_ k nf]
(println (str "valAt*2: " k ":" nf))
(println (str "==>" (get m k nf)))
(get m k nf))
)))))
#2019-06-2115:55Alex Miller (Clojure team)there is a known issue with specifying generator overrides on spec aliases#2019-06-2115:55Alex Miller (Clojure team)in that, it doesn't work#2019-06-2115:55Ben Hammondokay.
that's a simple explanation#2019-06-2115:55Ben Hammondis there a workaround?#2019-06-2115:56Ben Hammondshould I copy-and-paste it?#2019-06-2115:56Alex Miller (Clojure team)I think it should work if you specify it on the aliased spec :customer.ingestion/merchant_id
, but seems like you are?#2019-06-2115:58Alex Miller (Clojure team)I would expect OVERRIDEb and OVERRIDEc to never work here#2019-06-2115:58Ben Hammondso if I ttry to override :db.generators.offers/offer_title
instead#2019-06-2115:59Ben Hammondit still doesn'tt work though#2019-06-2115:59Ben Hammond(:offer_title (test.gen/generate
(spec/gen :db.generators.offers/offer
{:db.generators.offers/offer_title (constantly (test.gen/return "OVERRIDEb"))})))
=> "SC3T9wmvO54Q2TNx4"
`#2019-06-2116:00Alex Miller (Clojure team)what is test.gen?#2019-06-2116:00Ben Hammond[clojure.test.check.generators :as test.gen]
#2019-06-2116:01Alex Miller (Clojure team)so test.check.generators expects something different than clojure.spec.gen.alpha - namely, the spec version takes generator thunks, whereas I think test.check just takes generators#2019-06-2116:01Alex Miller (Clojure team)can you try it with spec's version?#2019-06-2116:02Alex Miller (Clojure team)I'm not sure what test.check supports as far as override marking - I'm not sure it even knows about the attributes as that's all spec registry based#2019-06-2116:03Ben Hammondwell it looks like all logic is buried inside clojure.spec.alpha/gensub
#2019-06-2116:04Ben Hammondso perhaps I don't understand what you are asking.
If I run
(:offer_title (spec.gen/generate
(spec/gen :db.generators.offers/offer
{:db.generators.offers/offer_title (constantly (test.gen/return "OVERRIDEb"))})))
I get the same outcome as when ran test.gen/generate
#2019-06-2116:15Ben Hammondah#2019-06-2116:15Ben Hammondthe good news is that I'm an idiot#2019-06-2116:18Ben Hammondso what I'm doing to sabotage the overridee#2019-06-2116:20Ben HammondThe declaring file goes like this
(ns db.generators.offers
...
(defmacro add-gen
"update spec to add generator"
[k g]
`(spec/def ~k
(spec/with-gen ~k
(constantly ~g)) ))
(defmacro update-gen
[k f]
`(add-gen ~k (~f (clojure.spec.alpha/gen ~k))))
...
(spec/def ::offer
(spec/keys
:req-un [::offer_headline
...
(update-gen ::offer (fn [g] (test.gen/fmap add-msisdn-csv-bytes g)))
...
#2019-06-2116:22Ben Hammondso after the offer spec is declared
I go and set its generator using with-gen
to mix in an extra bit of functionality#2019-06-2116:22Ben Hammondand the existence of this gfn
means that#2019-06-2116:23Ben Hammondclojure.spec.alpha/map-spec-impl
sees
(gen* [_ overrides path rmap]
(if gfn
(gfn)
(let [rmap (inck rmap id)
...
#2019-06-2116:23Ben Hammondand says
>Ooh a gfn. My work here is done#2019-06-2116:24Ben HammondThankyou for your help#2019-06-2116:24Ben HammondI'm not sure how I'm going to fix it
but at least I understand it#2019-06-2116:27Alex Miller (Clojure team)Some of this stuff really needs a rethink, itās pretty tricky#2019-06-2116:27Ben Hammondwell I thought the the next spec counts as a rethink?#2019-06-2116:28Ben Hammonddiscovering where the tripwires are is tricky#2019-06-2116:29Ben Hammondthat's true of my entire time with Clojure#2019-06-2116:29Ben Hammondyou have to trip 'em to find 'em#2019-06-2116:29Alex Miller (Clojure team)Yeah, we may get to it in spec 2#2019-06-2116:32Ben Hammondright its the weekend in my timezone#2019-06-2116:32Ben Hammondhave a goodd weekend Alex#2019-06-2116:50Alex Miller (Clojure team)later#2019-06-2118:54mishawhat is the difference between: (s/coll-of ... :kind vector?) and (s/coll-of ... :into []) ?#2019-06-2119:32Alex Miller (Clojure team)kind adds a validation on the input#2019-06-2119:32Alex Miller (Clojure team)into is about gen and conform (output)#2019-06-2119:33Alex Miller (Clojure team)(but kind also impacts the starting coll for gen/conform)#2019-06-2119:42mishaso :into []
is "list/seq is fine, but generate me a vector"?#2019-06-2120:03Alex Miller (Clojure team)Yes#2019-06-2120:03Alex Miller (Clojure team)And conform to vector#2019-06-2119:44mishathanks. had an impression, that :into is a younger replacement for :kind.#2019-06-2120:02Alex Miller (Clojure team)No, they are different purposes#2019-06-2120:35mishawhat is the best :ret
spec for "predicaty" function? boolean?
is insufficient because nil is falsey. and because everything but false/nil - is truthy. any?
#2019-06-2120:50Alex Miller (Clojure team)Depends on what the function returns#2019-06-2120:50Alex Miller (Clojure team)Spec the truth#2019-06-2120:50Alex Miller (Clojure team)(s/nilable boolean?) is useful sometimes#2019-06-2120:51Alex Miller (Clojure team)Donāt use sets for logically false values#2019-06-2120:55mishafunction is supplied by user, and is used as predicate, but return value is not compared to true or false.#2019-06-2120:56mishalimiting it to :ret boolean?
or :ret (s/nilable boolean?)
would be... limiting#2019-06-2120:58mishawhat is the best way to spec atom fn arg? #(instance clojure.lang.Atom %)
? any?
?#2019-06-2121:00mishait actually will be an atom, not just something dereferable. and fn might call swap! or reset! on this arg#2019-06-2121:49Alex Miller (Clojure team)on the boolean question, I'd just not spec it then#2019-06-2121:49Alex Miller (Clojure team)not spec the ret at all#2019-06-2121:50Alex Miller (Clojure team)nothing to check#2019-06-2121:50Alex Miller (Clojure team)on atom, I'd do the first one#2019-06-2121:50Alex Miller (Clojure team)or actually, use IAtom, the interface, not the concrete class#2019-06-2221:54carkhi have this spec : (s/cat :tag ...... :children (s/* ::child)
I have a special case where i want a single child, but i still want it to be conformed as a list of one item.... how would i express this ?#2019-06-2221:56carkhi have the dispatching ok,, been using conformer and it works, but it swallows the explaining of errors down the tree#2019-06-2221:57carkhso what i would want is a replacement for s/* i guess#2019-06-2222:57seancorfields/+
ā one or more.#2019-06-2222:59seancorfieldMaybe wrapped in s/and
to check the count
is 1?#2019-06-2222:59seancorfieldNot sure how s/and
combines with the sequence regex stuff thoāā¦ Iād have to experiment.#2019-06-2223:03carkhoh i can try that thanks#2019-06-2223:11carkhnot working for me but that was a good try =)#2019-06-2223:12carkhi'm really trying to use spec as a parser for sexp, but it's not actually one, i guess the limitations are due to the gen part#2019-06-2223:17carkhThis works, but the real thing is not doing integers, and the tree may be deep. so finding a syntax error deep down might be hard without proper explaining from spec#2019-06-2223:28carkhahhaaa ! this might work#2019-06-2223:32carkhand i get nice errors deep down the tree, nice#2019-06-2323:40misha@carkh
(s/conform
(s/cat :tag keyword? :children (s/& (s/+ integer?) #(-> % count (= 1))))
[:a 3])
=> {:tag :a, :children [3]}
#2019-06-2711:29Jakub HolĆ½ (HolyJak)Thanks! I took the liberty of adding the example to clojuredocs http://clojuredocs.org/clojure.spec.alpha/&#example-5d14a98fe4b0ca44402ef771#2019-06-2323:41mishahttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/&#2019-06-2323:41carkhgonna have to test this before turning out, thanks#2019-06-2323:43carkhyes that's exactly it, that combinator flew under my radar, thanks @misha#2019-06-2501:07miguelbHi everyone, Iām trying to to write a spect that has ārelationshipsā to other specs. (apologies for the language, Iām still new to spec). For example, a group has a total number of people, number of active people and number of inactive people. active-people + inactive-people = total num of people
. I can write a spec for the total number of people but how do I use that spec for in the spec for both inactive and active. Also how does that work with generators? Ultimately I would like to generate a group where total
, active
and inactive
are set.#2019-06-2501:18seancorfield@miguelb Are these specs all used together in a map?#2019-06-2501:19miguelbyea eventually, right now iām defining each with their own s/def
#2019-06-2501:20miguelbnot sure if this is the right way to go about it, define each part and then compose together#2019-06-2501:20seancorfield(s/and (s/keys :req [::active-people ::inactive-people ::total-people]) #(= (::total-people %) (+ (::active-people %) (::inactive-people %))))
#2019-06-2501:20seancorfieldYou can only apply relationship predicates to something that contains all the various related keys.#2019-06-2501:21seancorfieldHowever, I would question the model design: since that's an invariant that should always hold, you don't need all three values (and probably should not try to have all three). Any two gives you all the information you need.#2019-06-2501:22miguelbgood point#2019-06-2501:22miguelbmy end goal here is to make a generator that will generate groups#2019-06-2501:22miguelbI was aiming to have the person count part of a āvalidā group#2019-06-2501:22ghadiyou'll end up needing to manually control the generator#2019-06-2501:23ghadithe default spec generators will not find something that satisfies the invariant within 100 tries#2019-06-2501:23ghadiunless you're feeling really really lucky#2019-06-2501:23miguelbheh#2019-06-2501:23seancorfieldYes, that's another reason that I think it would be easier without the constrained total-people
number#2019-06-2501:24seancorfieldIf you just have active and inactive, then regular generators will work#2019-06-2501:24seancorfield(since there's no need for an additional constraint)#2019-06-2501:24miguelbIāll try that way, makes way more sense that what I was about to try#2019-06-2501:24miguelbty very much!#2019-06-2608:14kwrooijenHi, when instrumentation is enabled and I call a function with from arguments within a def, instrumentation doesnāt trigger an error. Whatās the reason for this?
Hereās an example:#2019-06-2608:14kwrooijen#2019-06-2608:15kwrooijenWhen running inc-and-format
, when itās defined as a function, instrumentation crashes
When running inc-and-format
, when itās defined as a def, instrumentation doesnāt crash#2019-06-2711:39Jakub HolĆ½ (HolyJak)Is it possible to "and the rest of the arguments should match this others spec"? Example: (s/def ::kid (s/cat :name string? :age int?))
(s/def ::input-explicit-username (s/cat
:username uuid?
:kid ::kid))
(s/def ::input (s/or :implicit-user ::kid, :explicit-user ::input-explicit-username))
The thing is that I either have a short input or something+the short input. How to spec this without copy&paste? Thank you!
Answer: s/cat do not nest, they become one so the code above actually works as desired: (s/conform ::input ["name" 12])
=> [:implicit-user {:name "name", :age 12}]
(s/conform ::input [(java.util.UUID/randomUUID) "name" 12])
=> [:explicit-user {:username #uuid"c19bc1f5-033f-4ca1-b2e0-5877657cfa8d", :kid {:name "name", :age 12}}]
#2019-06-2711:48jaihindhreddyyup#2019-06-2712:08mishaIf you want some of regex specs to match nested structure, wrap them in s/spec, @holyjak #2019-06-2713:26Jakub HolĆ½ (HolyJak)thx! I wouldn't figure that one out š#2019-06-2712:16misha(s/+ int?) matches
1 2 3
(s/spec (s/+ int?)) matches
(1 2 3)
s/def
wraps form in s/spec, so from (s/def ::x (s/+ int?))
it might appear that (s/+ int?)
matches (1 2 3)
(with parens). It does not.#2019-06-2712:28Alex Miller (Clojure team)I wouldn't say it that way#2019-06-2712:28Alex Miller (Clojure team)user=> (s/valid? (s/+ int?) [1 2 3])
true
user=> (s/valid? (s/spec (s/+ int?)) [1 2 3])
true
#2019-06-2712:29Alex Miller (Clojure team)the more accurate way to look at it is that (s/+ int?) is a regex op (which is not actually a spec)#2019-06-2712:29Alex Miller (Clojure team)regex ops combine#2019-06-2712:29Alex Miller (Clojure team)when used, regex ops are automatically "spec-ized"#2019-06-2712:30Alex Miller (Clojure team)s/spec is an explicit "spec-ize" step#2019-06-2712:30Alex Miller (Clojure team)in all cases, regex ops are matching the elements in a sequential collection though#2019-06-2714:26mishaThatās what I meant kappa #2019-06-2714:30Alex Miller (Clojure team)I realize that, it's just the wrong mental model#2019-06-2716:43jeroenvandijkIs it a good practise to use keyword inheritance for multi-spec dispatch? e.g. (defmulti example-config :entity/type)
(s/def :entity/type #(contains? (set (methods example-config)) %))
(defmethod example-config :default [_]
(s/keys :req [:entity/type]))
(s/def :entity/config (s/multi-spec example-config :entity/type))
(defmethod example-config :entity.type/generic [x]
(getx x :entity/type))
;; Use keyword inheritance to get to the right spec
(derive :entity.type/specific :entity.type/generic)
(s/def :entity.type/specific integer?)
(s/explain-data :entity/config {:entity/type :entity.type/spfecific})
#2019-06-2717:56Alex Miller (Clojure team)sounds fine#2019-06-2717:57Alex Miller (Clojure team)not sure if there are any gotchas in gen, but might be#2019-06-2718:33jeroenvandijkThanks, I'll give it a go. Removes some boiler plate code#2019-06-2810:27Daniel Hines(reposted from #clojure)
I've got a problem I think spec would be perfect for, but I'm a total spec noob and have no idea how to spec it.
I'm using pseudo set notation to describe the problem (read "m E S" as "m is a member of set S").
---
Given:
There's a set of types, T,
There's a set of attributes, A,
There's a set of specs S
There's a map such that every attribute has a corresponding spec from S.
What is the spec for 2-tuples where:
- The first element R1 is a vector of where every element e E T.
- The second element R2 is a set of tuples in the form [e a v], where e E {1..length of R1}, a E A, and the value of v conforms to the spec corresponding to a.
Any help is appreciated!#2019-06-2813:02djtangoso the first spec seems easy enough:
(s/def ::T #{:t1 :t2 ,,,})
You could map A and S manually by doing:
(s/def ::attribute-that-is-a-number number?
,,,
If you need to make the set of A to be concrete you'd probably want to look up the registry
Then the final spec for your 2-tuple probably involves quite a hairy predicate (remember that all specs are 1-arg preds)
This could then check that every first-element of the R2 is is a member of the set of ints from 1 up to (count R1)
Then a spec that looks up value of a in R2 and does (s/valid? a v)
#2019-06-2813:02djtangohopefully that's enough to get the brain flowing#2019-06-2813:08Daniel HinesLol, why did I have to pick this as my first venture into specs š. Letās try making it more concrete.
(s/def ::T #{:t1 :t2})
(s/def ::height number?)
(s/def ::name string?)
(s/def ::A #{::height ::name})
(s/def ::Goal (s/cat :R1 int? :R2 ???))
#2019-06-2813:10Daniel Hinesā¦ Thatās all I got so far! Can I get a hint on the rest of the syntax?#2019-06-2813:16Daniel HinesMaybe
(s/def ::Goal (s/cat :R1 int? :R2 (s/* (s/cat :e (set (range (count R1))) :a ??? v: ???)))
#2019-06-2814:35djtangoR1 is a vector right? not an int#2019-06-2814:37djtangoif you don't think about ::Goal
as a spec but more a function that takes the whole 2-tuple what would that function look like#2019-06-2815:16Daniel HinesYeah, youāre right, :R1 is more like (s/col-of ?int)
.#2019-06-2815:17droneand ::R2 is (s/coll-of ::R2-elem :kind set?)
#2019-06-2815:24Daniel HinesYeah, technically, t#2019-06-2815:24Daniel HinesThereās more to it though, right @UCF779XFS? I need some predicate function that @U0HJD63RN was alluding to.#2019-06-2815:26droneyouād need to a predicate that will take the R2/a
and verify the R2/v
passes s/valid?
#2019-06-2815:30droneso youād have something like:
(defn valid-val?
[[idx a v]]
(s/valid? a v))
(s/and (s/cat _ _ _ _ _ _)
valid-val?)
#2019-06-2815:30dronesomething like that/ where the empty cat is whatever youāre using for the three values in the R2 element#2019-06-2815:30Daniel HinesOk.. this is starting to come together!#2019-06-2815:31dronetake with a grain of salt, just woke up and may be stupid#2019-06-2816:44djtangowell from the description of the specification - R2's permissible values for e
depends on R1#2019-06-2816:44djtangoso you need the whole 2-tuple#2019-06-2816:45djtangoso you would want a predicate that can compare R1 and R2#2019-06-2816:45Daniel HinesYeah, that's right.#2019-06-2816:46djtango(let [[r1 r2] tup
allowed-e? (->> r1 count inc (range 1) (into #{}))]
(->> r2 (map first) (every? allowed-e?))
something like this maybe#2019-06-2817:01Daniel HinesIāll give that a shot later, thanks!#2019-06-2817:23djtangoso I guess you could split the spec for your tuple into three checks:
That R1 is valid
all e are valid wrt R1
all v conform to their corresponding a#2019-06-2817:25Daniel HinesYes! That sounds very doable#2019-06-2817:26Daniel HinesI was hoping to get generators for free, but even besides this, there are other constraints that between entities of different types that are much too complicated without some constraint solver.#2019-06-2812:08salokristianI'm working on creating a spec-based validator for creating human-readable error messages. I have a working implementation which locates errors based on the in
key in the returned spec errors for explain-data
. This works fine for errors concerning leaf-level specs, i.e. entries in a collection.
However, I have not yet found a way to create useful error messages for custom predicates that concern a whole collection. Is there a (built-in or custom) way to return some sort of metadata from a custom spec predicate, which could be accessed in the spec error message?
For example, the error message for this spec is not useful for locating the error, i.e. the duplicate values.
(s/def ::num int?)
(s/def ::list (s/and (s/coll-of ::num) #(apply distinct? %)))
(s/explain-data ::list [1 2 3 1])
I would like to be able to return some custom metadata from (apply distinct? %)
, which would tell me which elements caused the error, and allow to locate the errors precisely. Of course this is not just an error with this particular custom predicate, but all custom predicates. There doesn't seem to be a way to add metadata to the error messages generated by spec.
Is this doable? If it isn't, what kind of a workaround would you suggest for this? I would like to use spec for checking correctness and generate errors based on spec errors.#2019-06-2812:25jumarhow does it compare to https://github.com/alexanderkiel/phrase ?#2019-06-2815:33yuhanor https://github.com/bhb/expound#2019-06-2906:07salokristianI'm actually using phrase to generate end-user friendly errors. The problem is passing additional error-related metadata from custom spec predicates to spec errors (and therefore to phrase).#2019-07-0109:16djtangoI'm not convinced you can get helpful error messages from the general form of (s/def ::the-name #(apply distinct? %))
- if you consider that the spec is a one-arg predicate function, it would be hard to see how to determine which entry caused the failure.
For the specific case of duplication, you'd probably need to replay the data or reconstruct it - as spec would only recall the input unless you walk it element by element. Even the inbuilt :distinct
field in (s/every ...)
doesn't seem to know which specific member breaks uniqueness#2019-06-2815:42Daniel HinesCan test.check reverse regex specs out of the box?#2019-06-2815:47nwjsmithNo, youāll need test.chuck for that#2019-06-2815:47nwjsmithoh.#2019-06-2815:47nwjsmithregex specs#2019-06-2815:47nwjsmithI donāt think test.check is spec-aware#2019-06-2815:49Daniel HinesIs there a built in spec for UUIDās that works in CLJS?#2019-06-2815:49Daniel HinesJust need the strings.#2019-06-2815:56nwjsmithDo you need a spec for string-encoded UUIDs, or would uuid?
work?#2019-06-2816:47Daniel HinesThe former.#2019-06-2817:43dronethere was a whole thread about that somewhere. a Google search would probably find it#2019-06-2817:47dronefor reference: https://clojure.atlassian.net/browse/CLJ-1925#2019-06-2817:44droneuuid across JVM and JS#2019-06-2817:45nwjsmithI think this will work:
(s/def ::uuid-string
(s/with-gen
#(re-find #"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}" %)
#(gen/fmap str (gen/uuid))))
#2019-06-2817:46nwjsmithOn both platforms#2019-06-2817:50Joe LaneDoes that handle all UUID versions?#2019-06-2817:53nwjsmithIt should, but IIRC test.check.generators/uuid
will only generate version 4 UUIDs#2019-06-2817:54nwjsmithThe textual representation of UUIDs is the same across all versions#2019-06-2817:56Daniel HinesThanks a bunch!#2019-06-2819:38Daniel HinesAre there any spec power tools for defining large entity maps? Not really sure what Iām looking for - just something more concise if possible.#2019-06-2900:59kennyI'm encountering a strange error when using st/check. I'm getting an IllegalArgumentException
with the message Additional data must be non-nil.
(full trace here: https://gist.github.com/kennyjwilli/e7fcfb809a771db130a1ff37b3d3cb79). The stacktrace points to this line: https://github.com/clojure/spec.alpha/blob/5228bb75fa10b7b13ea088d84f4d031cce74a969/src/main/clojure/clojure/spec/test/alpha.clj#L280 which clearly has the possibility of passing nil
to an ex-info
call which would result in this error. Not sure why this is happening in the first place though. It's likely an issue in my code but this exception is not very helpful.#2019-06-2900:59kenny#2019-06-2901:45Alex Miller (Clojure team)A value that is invalid according to its spec should always produce explain data. Would be interested to know the spec/value causing this#2019-06-2918:20Daniel HinesCan anyone offer any advice on how to transform a json object into a proper Clojure object based on a given spec? The final result has a mix of keywords and strings, and Iām not sure how to say āHereās a spec of the end result, and hereās the JSON, now give me the end resultā#2019-06-2920:26jaihindhreddynot sure how to do it with vanilla spec but Metosin's spec-tools has coercion for this purpose, check out https://github.com/metosin/spec-tools/blob/master/docs/01_coercion.md#2019-06-2920:45Daniel HinesThanks. Speaking of, how do you get data specs to work with test.check and generators? Is that possible?#2019-06-2920:53Daniel HinesFor example, this confuses me:
(s/def ::foo #{:bar})
(s/def ::foo-map (s/keys :req [::foo]))
(gen/generate (s/gen ::foo-map)) ;; => {::foo :bar}
(def data-foo (ds/spec {::foo (s/spec #{:bar})}))
(s/valid? foo-map {::foo :bar}) ;; => true
(gen/generate (s/gen data-foo)) ;; => nil
#2019-06-2920:58jaihindhreddyI've only been watching spec-tools and schema from afar. Sorry#2019-06-2920:58Daniel HinesThanks anyway!#2019-06-2920:58Daniel HinesLooks like itās something to do with this. options include one :gen
. https://github.com/metosin/spec-tools/blob/master/docs/07_spec_records.md#2019-06-3001:46Daniel HinesHow is this possible?
(s/conform workspace ws-clj) ;; => :cljs.spec.alpha/invalid
(s/explain-str workspace ws-clj) ;; => "Success!\n"
#2019-06-3001:46Daniel Hiness/valid?
also returns false.#2019-06-3001:50Daniel HinesOh, itās because the invocation was erring out š#2019-06-3001:51Daniel HinesFor s/explain-str
to return āSuccess!\nā when an exception is thrown seems like a bug.#2019-06-3001:52Alex Miller (Clojure team)It is typically. Whatās the spec and the value?#2019-07-0117:18Daniel HinesI tried reproducing it and couldnāt figure out exactly what I did (I was working at the REPL). If I run into it again Iāll try to find a solid repro.#2019-07-0115:32gfredericksnew test.check release: https://clojurians.slack.com/archives/C0JKW8K62/p1561995125014300#2019-07-0201:27seancorfieldUpdated our code at work to use 0.10.0-RC1
and to switch away from the deprecated generators -- and we were using pos-int
on the assumption that it generated strictly positive numbers (although zero doesn't break our tests).#2019-07-0210:57gfredericksthat makes me glad I deprecated it š
did you switch to (gen/fmap inc gen/nat)
, or something else?#2019-07-0212:58seancorfieldFor now we just switched to gen/nat
to preserve the behavior but we will revisit that test and see what might happen if it ever generates zero. At least it is clear now!#2019-07-0212:59gfredericksI can assure you it generates zero quite often š#2019-07-0213:08seancorfieldThat test should probably fail with zero (but it doesn't) š#2019-07-0213:28gfrederickscomputers are crazypants#2019-07-0116:46seancorfield@gfredericks What's the thinking behind deprecating those generators?#2019-07-0119:29gfrederickspos-int
and neg-int
are highly misleading w.r.t. 0, and all five of them are misleading w.r.t. size; I think generators by default should generate a healthy range of things, and if you want to restrict yourself to small numbers you should have to be explicit#2019-07-0119:30gfredericksso it's basically entirely a naming issue#2019-07-0119:31gfredericksspose I could've also deprecate-renamed gen/ratio
to gen/small-ratio
#2019-07-0120:00seancorfieldAre you actually planning to remove them at some point (a breaking change) or just "strongly discourage" their usage?#2019-07-0120:04gfredericksthe latter#2019-07-0120:04gfredericksthere's already another deprecated generator of that sort#2019-07-0120:05seancorfieldI found 15 occurrences of "deprecated" in that file š#2019-07-0120:05seancorfield(seven functions)#2019-07-0120:05gfredericksentirely for naming, apparently:
(def ^{:deprecated "0.6.0"}
char-alpha-numeric
"Deprecated - use char-alphanumeric instead.
Generates alphanumeric characters."
char-alphanumeric)
#2019-07-0120:06seancorfieldNaming is hard. I struggled with deprecation/breakage in clojure.java.jdbc
and ended up breaking the API twice to get it consistent and idiomatic. With hindsight I would have done it differently.#2019-07-0120:19gfredericksyeah I've tried to figure out how to overhaul things so many times, and largely haven't#2019-07-0118:11rapskalianDoes spec have a simple way to express xor
with map keys? Meaning map m
can have key :a
or key :b
, but not both.#2019-07-0118:15Alex Miller (Clojure team)Not built in, but you can s/and any predicate you like#2019-07-0118:16akielHas someone a solution for using st/instrument
with :stub
were the function is called more than once and I like to have return values depending on the inputs? With Java mock frameworks such things are possible. With spec I donāt have an idea how to do it. Thanks!#2019-07-0118:18Alex Miller (Clojure team)You canāt with stub, but you could make a stateful :replace fn#2019-07-0118:27akiel@alexmiller Work perfect. Thanks! Not as elegant like mock libs but one could build a helper function which maps inputs to outputs.#2019-07-0208:54roklenarcicIs there a way to get around the limitation of s/merge
that it only conforms one branch of merge?#2019-07-0208:55roklenarcicThis is really really really detrimental.#2019-07-0407:09Jakub HolĆ½ (HolyJak)Hello! I get some kind of infinite loop in conform
after I have renamed and moved around a couple of spec names, any tips how to troubleshoot it? It seems deep-resolve
and reg-resolve
are involved in the loop as well as a particular spec (`:my.ns/account-invoices`) and if the debugger is correct then we regularly arrive to a point in reg-resolve
where both spec
and k
are null.
Update: I can replicate - calling (reg-resolve :my.ns/account-invoices)
never returns.
(Clojure 1.10.1, spec 0.2.44)
This is really weird - all parts of the spec resolve but not the spec itself. I have this: (s/def ::account-invoices
(s/nilable
(s/every-kv ::accid ::acc+invoice)))
and both (reg-resolve! :my.ns/acc+invoice)
and (reg-resolve! :my.ns/accid)
return a spec successfully yet (reg-resolve! :my.ns/account-invoices)
never returns....
If I change the name of the spec to :tmp/account-invoices
then (reg-resolve! :tmp/account-invoices)
succeeds.
SOLVED: This line little more down was the issue (s/def ::account-invoices ::account-invoices)
I wish Spec had a loop detection...#2019-07-0613:23misha@roklenarcic can you give an example, where/how it limits you?#2019-07-0615:02roklenarcicLet's say you define (s/def ::date-time (s/conformer to-date-time-string from-date-time-string))
. This conforming spec will print date time object into a string (or vice versa on unform call). If you define several s/keys
specs each containing some fields of this type and then you defined a merged spec with s/merge
you will end up with having a situation where after conforming an object only a third of date time properties will be conformed to strings the rest will remain datetime objects and it will fail.#2019-07-0618:22mishaThis sounds reasonable, although alexmiller stated many times that conformers are not for (this kind of) data transformation.#2019-07-0618:23mishae.g. https://stackoverflow.com/a/49056441/1128985#2019-07-0618:41misha(s/def :foo/s-int (s/conformer #(Integer/valueOf ^String %) str))
(s/def :foo/a :foo/s-int)
(s/def :foo/b :foo/s-int)
(s/def :foo/merge
(s/merge
(s/keys :req [:foo/a])
(s/keys :req [:foo/b])))
(s/conform :foo/s-int "1") ;; => 1
(s/unform :foo/s-int 1) ;;=> "1"
(s/valid? :foo/merge {:foo/a "1"}) ;;=> false
(s/valid? :foo/merge {:foo/a "1" :foo/b "2"}) ;; => true
(s/conform :foo/merge {:foo/a "1" :foo/b "2"}) ;;=> #:foo{:a 1, :b 2}
(s/unform :foo/merge {:foo/a 1 :foo/b 2}) ;;=> #:foo{:a "1", :b "2"}
#2019-07-0715:25roklenarcicIf data transformation is not the intent, then what IS the intent of conforming?#2019-07-0715:29mishaTo tell you how exactly data is valid or invalid. The line is somewhat blurry though.#2019-07-0715:31mishaRead up threads linked in SO answer#2019-07-0613:26mishayou might be able to use s/and, which "flows" conformed values as part of validation:
(s/conform (s/and (s/or :k keyword? :i int?) vector?) 1)
=> [:i 1]
(s/valid? (s/and (s/or :k keyword? :i int?) vector?) 1)
=> true
#2019-07-0613:27misha(but intent readability suffers, imo)#2019-07-0617:09Diego Melendezhow do I write a s/fdef for a function that does not receive args?#2019-07-0617:22Jakub HolĆ½ (HolyJak)I guess leave out args or eg (s/coll any? :count 0)?#2019-07-0617:22Jakub HolĆ½ (HolyJak)Or just empty?#2019-07-0618:00Diego Melendezš#2019-07-0619:07Alex Miller (Clojure team)I use (s/cat)#2019-07-0808:19StefanHi, Iād like to write a spec for an entry in a map that must always be a specific string, and also can be generated. So I have {:my-key "my constant string"}
(as part of a bigger structure). A spec like this works:
(s/def ::my-key (partial = "my constant string"))
Now I try to generate data inside my test, and this fails. Iām trying to use s/with-gen
, but I think Iām missing something.
(s/def ::my-key
(s/with-gen (partial = "complexity")
(constantly "complexity")))
yields:
Assert failed: Second arg to such-that must be a generator
(generator? gen)
How should I approach this?#2019-07-0808:34valeraukotry #(constantly "complexity")
#2019-07-0808:36yuhanYou could also use a 1-element set:
(s/def ::my-key #{"complexity"})
#2019-07-0808:37aisamuhttps://clojure.github.io/test.check/clojure.test.check.generators.html#var-return#2019-07-0808:37aisamu(although the set approach above is the preferred way)#2019-07-0808:38Stefan@UAEH11THP I tried that too, also gives an error. I got it to work though after some trial and error:
(s/def ::my-key
(s/with-gen (partial = "complexity")
#(gen/return "complexity")))
#2019-07-0808:38Stefan@U1UQEM078 Thanks! Why is that preferred?#2019-07-0808:39aisamuBecause then you get the generator for free!#2019-07-0808:39StefanOh wait thatās ALL the code Iād need š#2019-07-0808:39StefanMuch simpler indeed š#2019-07-0808:56valeraukowow every day i learn something new#2019-07-0810:15mishayeah, it is idiomatic to spec constants with a single element set. however, if that constant is either false
or nil
ā use false?
and nil?
, because
(s/valid? #{false} false)
=> false
(s/valid? #{nil} nil)
=> false
#2019-07-0810:18Stefan@U051HUZLD Iām probably misunderstanding:
(s/valid? #{false} false)
=> false
(s/valid? #{false?} false)
=> false
(s/valid? #{nil} nil)
=> false
(s/valid? #{nil?} nil)
=> false
#2019-07-0810:26misha(s/valid? nil? nil)
=> true
#2019-07-0810:26mishanil? false? instead of set#2019-07-0810:34yuhanthat's an edge case to look out for in general when using sets as predicate functions, which is really all that spec is doing here#2019-07-0810:37yuhan(if (#{1 2 3} x) ;; returns truthy values (1, 2, or 3) if x is in the set
...)
(if (#{false} x) ;; returns a falsey value no matter what x is!
...)
#2019-07-0810:43mishait is#2019-07-0810:48mishacan be worked around with contains?
though#2019-07-0913:45arohnerI have an s/keys
with a key of type uuid?
. I have a generative test where I generate a few hundred records, and insert them into postgres, where the uuid uniqueness constraint is checked. Occasionally the test fails due to duplicate uuids. Is there a way to tell spec/test.check to only generate unique uuids?#2019-07-0914:41ghadithere are ways of generating unique data within spec/t.c. @arohner but for this usecase I would dedupe between generation and inserting records#2019-07-0915:04arohnerThanks#2019-07-0915:39kennyIt seems useful to have s/with-gen
be passed the generator for the spec it is overriding. Often I find myself generating values from the spec's gen and then fmap
'ing to ensure certain constraints are met.#2019-07-0915:56kennyIt'd also be useful to have a variant of s/and that doesn't pass the conformed value to s/and
predicates. We have a lot of this in our specs:
(s/and
(s/keys :req [:metric/tags :metric/lower-bound :metric/upper-bound])
#(metric-tags-valid? (s/unform ::base-metric %))
#(metric-bounds-valid? (s/unform ::base-metric %)))
#2019-07-0916:48mishasometimes changing order inside s/and works#2019-07-0916:48kennyThese such-that errors are quite frustrating because they provide no info:
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:417).
Couldn't satisfy such-that predicate after 100 tries.
clojure.lang.ExceptionInfo: null #:clojure.error{:phase :print-eval-result}
at clojure.main$repl$read_eval_print__9068.invoke(main.clj:419)
clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {:pred #object[clojure.spec.alpha$gensub$fn__1876 0x71807ca2 "
At the very least, including the predicate form would be extremely helpful.#2019-07-0916:50misha@kenny you can sometimes enhance that by naming your anon fns#2019-07-0916:51mishae.g.
(fn f1 [x] (metric-tags-valid? (s/unform ::base-metric x)))
instead of
#(metric-tags-valid? (s/unform ::base-metric %))#2019-07-0916:52mishait might show up in stacktrace as something like
/cdn-cgi/l/email-protection instead of
/cdn-cgi/l/email-protection#2019-07-0917:04kennyYeah I suppose I could do that. It'd be great if there was a tighter integration between spec and test.check so I don't need to do that.#2019-07-0921:33kennyclojure.spec.gen.alpha/gen
should have up to 3 args (https://clojure.github.io/test.check/clojure.test.check.generators.html#var-generate) when it only has one.#2019-07-1014:23gfredericksThe second two args are new#2019-07-1014:24gfredericks@arohner the uuid generator is fully random with a uniform distribution, so you shouldn't be any more likely to get collisions than you are in Real Life, i.e. it shouldn't ever happen#2019-07-1014:25arohnerWhat about during shrinking?#2019-07-1014:25gfredericksMy guess is you're seeing the effects of shrinking, where your test gets run repeatedly with similar data#2019-07-1014:26gfredericksSo the UUIDs won't shrink, but your test is run again, so if you have state in a database that you're not resetting between trials, that'd be a problem#2019-07-1014:31gfredericksI agree it's annoying, because UUIDs otherwise seem like such an effective way of implicitly partitioning a stateful external system by trial#2019-07-1014:32gfredericksI'm curious whether, in practice, blasting lots of data into a test DB w/o deleting it is faster than truncating between trials#2019-07-1612:52eoliphantHi, Iām trying to figure out how to model this with spec.
Say I have (s/def ::rule-type #{:rule1 :rule2})
I then want to create (s/def ::rule1 (s/keys ...)
, etc where aside from the keys that are selected ::rule-type is ānarrowedā to specific value, such that I can then have a ::rule spec thatās an or of those guys#2019-07-1612:54gfredericksmight be what multi-spec is for?#2019-07-1612:55Alex Miller (Clojure team)yes, seems like exactly what s/multi-spec is for#2019-07-1613:00eoliphantah ok, actually just got it working with and s/and āpassthruā but will try the mult-spec as well#2019-07-1613:01eoliphantdid this
(s/and (s/keys :req [::rule-type ::rule-id ::rule-action ::object-locator]
:opt [::load-order])
#(= (::rule-type %) :selection)))
but yeah multi-spec seems a bit more elegant
spec2 looks exciting @alexmiller#2019-07-1613:20jeroenvandijkIt also gives much better feedback (more specific) when using with expound for instance#2019-07-1613:02Alex Miller (Clojure team)s/multi-spec is also open so you can add more rules later#2019-07-1613:04eoliphantsweet. Hey is spec2 going to have fspec support for mutimethods?#2019-07-1613:05Alex Miller (Clojure team)not something we're working on right now, but maybe eventually#2019-07-1613:18eoliphantcool#2019-07-1714:30talgiatIs there a way to reuse fspec in clojurscript? the only way to spec functions in clojurescript seems to use fdef but that canāt take fspec defined somewhere else as an argument.
My goal is to define a function spec in one place and re-use it for all functions that have the same function spec. This can be done in clojure the following way (I believe):
(def n->n-fn-spec (s/fspec :args (s/cat :n number?) :ret number?))
(s/def add5 n->n-fn-spec)
(defn add5 [n] (+ n 5))
(s/def add6 n->n-fn-spec)
(defn add6 [n] (+ n 6))
#2019-07-1714:35Alex Miller (Clojure team)instead of def
, use s/def
#2019-07-1714:36Alex Miller (Clojure team)oh, you're trying to reuse the whole fspec as a function's spec#2019-07-1714:38Alex Miller (Clojure team)not sure I know enough about the cljs impl to say if this is possible#2019-07-1714:53talgiatyes thatās what Iām trying to do. But this example should work in clojure (I think without s/def in the first expression)#2019-07-1715:18Alex Miller (Clojure team)it won't work exactly like this in spec 2 in Clojure#2019-07-1715:19Alex Miller (Clojure team)but there is a similar path#2019-07-1715:20talgiatI tested and it works both with def and s/def in this example (but with s/def it should be ::n->n-fn-spec ā¦)#2019-07-1715:21talgiatLooking at clojurescript fdef code it adds the fn-sym to specedvars atom that is used with instrument.#2019-07-1715:22talgiatso you canāt use s/def in clojurescript to spec function symbols (at least I canāt get it to work)#2019-07-1715:23Alex Miller (Clojure team)the fact that it even works in Clojure is somewhat accidental#2019-07-1715:49talgiatyou mean with def or even with s/def, or you mean reusing fspec in general?#2019-07-1715:50Alex Miller (Clojure team)being able to use s/def with function spec objects#2019-07-1715:51Alex Miller (Clojure team)that won't work in spec 2, but there is a new function s/register that will be available to do the equivalent#2019-07-1715:51talgiatwill it work with def (not s/def)?#2019-07-1715:52Alex Miller (Clojure team)that's really incidental - def doesn't do anything with the registry so no change there#2019-07-1715:52Alex Miller (Clojure team)the changes are around what s/def does#2019-07-1715:52talgiatyeah def just binds a symbol to fspec#2019-07-1715:53Alex Miller (Clojure team)importantly, an fspec object (not a form)#2019-07-1715:53Alex Miller (Clojure team)s/def doesn't work with objects anymore in spec 2, only forms#2019-07-1715:53talgiatright#2019-07-1715:54Alex Miller (Clojure team)but s/register is a new function in spec 2 that s/def (macro) uses#2019-07-1715:55talgiatI guess my question is, is reusing fspec is a valid use case for spec? or what is the correct way to have reusable function spec?#2019-07-1715:55talgiatthat works in clojure and clojurescript#2019-07-1715:58Alex Miller (Clojure team)I can't really answer the latter, but in spec 2 it's possible by registering the same object via s/register. cljs spec 2 work has not been started so tbd#2019-07-1716:02talgiatThanks Alex, Iāll chase David for clojurescript#2019-07-1716:02Alex Miller (Clojure team)I think they're going to wait until we're much further along with spec 2 before they start working on it#2019-07-1716:03talgiatIt will be great if with spec 2 things will work the same way in Clojure and Clojurescript#2019-07-1716:04Alex Miller (Clojure team)well that's certainly the intent (other than things that can't work the same way)#2019-07-2312:43henryw374I want to have generators associated with predicates, like you have already with the clojure.core preds. Are people re-binding gen.alpha/gen-builtins
or is there some other way?#2019-07-2313:12Alex Miller (Clojure team)make a spec with a custom gen#2019-07-2313:31henryw374but with string?
, I can use that both as a normal pred, and as a spec with a generator. If I make a spec with the pred, that's two separate things isn't it?#2019-07-2313:34Alex Miller (Clojure team)yes, but you can just use the spec#2019-07-2313:37henryw374ok, so I'm hearing the core pred fns are a bit special wrt specs. I'm creating a library and really it'd be nice for the preds to be used in the same way as the core fns. https://github.com/henryw374/time-specs/blob/master/src/time_specs/core.cljc#2019-07-2313:38Alex Miller (Clojure team)the recommended way to do this with spec 1 is to make a spec with a custom generator#2019-07-2313:39Alex Miller (Clojure team)future answer may be different#2019-07-2313:39henryw374ok thanks for the info.#2019-07-2313:40Alex Miller (Clojure team)rebinding gen-builtins from a lib is problematic - users have to do that for it to work, and whatever you do has to be compatible with whatever some other lib might do#2019-07-2313:40henryw374yeah... I'll avoid that I think. bound to be problems#2019-07-2313:40henryw374no pun intended#2019-07-2313:41Alex Miller (Clojure team)heh#2019-07-2313:41Alex Miller (Clojure team)spec 2 has some capabilities to create your own derived spec types with custom gens which is maybe another option#2019-07-2313:42Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#simple-spec-ops#2019-07-2313:43Alex Miller (Clojure team)but that's really trying to handle the case of parameterized things more so than pred generators#2019-07-2313:45Alex Miller (Clojure team)the example at that link doesn't show how to define a custom gen in a defop, but it has that capability#2019-07-2313:54henryw374ok cool. I'll have a look at that with an eye to the future.
on the topic of spec2, with select
, say I have a schema with a key a
the spec of which is a coll-of x
or map-of string? x
and x
is a schema with keys foo
, bar
. could you do a select
so say you want a
and in the value of a you want to select key foo
from x
? I guess not from what I've read so far, but I think that'd be useful.#2019-07-2314:01Alex Miller (Clojure team)not there now, but we have been thinking about how to add it#2019-07-2314:01Alex Miller (Clojure team)definitely a common case#2019-07-2314:01henryw374cool thanks, looking forward to spec2 definitely#2019-07-2322:03robertfwWhat is the recommended way to spec a map that uses strings for keys?#2019-07-2322:05Alex Miller (Clojure team)it's a little tricky right now#2019-07-2322:05Alex Miller (Clojure team)you can spec it as an s/map-of s/tuple for map entries#2019-07-2322:05Alex Miller (Clojure team)possible to s/multi-spec the tuple on the key to choose the val spec#2019-07-2322:32robertfwok, thanks#2019-07-2322:32robertfwWill that become easier with spec2?#2019-07-2401:23Alex Miller (Clojure team)not sure#2019-07-2406:40y.khmelevskiiHi gents! Is there any way to use spec/keys
definition for creating new spec as set of keys? For example I have spec:
(s/def ::schema
(s/keys :req-un [::id
::name
::active?]
:opt-un [::age]))
I want to create other spec based on ::schema
(s/def ::fields
#{:id :name :active? :age})
#2019-07-2412:53danierouxHow would I generate a random walk with clojure.spec.gen? Or, how would I generate a successive value based on the current value?#2019-07-2414:08respatializedI think the easiest way would be to use spec.gen
for starting values and then use iterate
to do the actual random walks. you could use specs
inside the fn you pass to iterate
to constrain the return values.#2019-07-2414:10respatializeddoes anyone have any thoughts on making spec
predicates usable through a java API? should I just use spec/conform
to define the class methods I want to expose to the Java API?#2019-07-2414:18Alex Miller (Clojure team)can you explain more what you're trying to do?#2019-07-2414:24respatializedI'm using core.spec
to define a series of data integrity/quality predicates that some data needs to conform to. I would like for those specs to be available as widely as possible, including for other consumers of the data who may be using Java instead of Clojure.#2019-07-2504:01jjttjjis there an example out there of a spec on arguments that look like
([a])
([a b])
([a b c])
([a b c d])
([a b c d e])
where later arguments are optional and can be omitted and replaced with defaults? I feel like there has to be a better way than just a big s/alt
with a bunch of s/cat
s but I can't think of it right now#2019-07-2505:22Alex Miller (Clojure team)(s/cat :a ::a (s/? :b ::b (s/? :c ::c (s/? :d ::d (s/? :e ::e)))))
#2019-07-2505:23Alex Miller (Clojure team)but also, sometimes s/alt
is clearer, just depends#2019-07-2505:28seancorfieldI didn't realize you could cascade s/?
like that... I've only seen (s/cat :a ::a :b (s/? ::b))
before I think?#2019-07-2505:30seancorfieldHmm, according to the docs/source, s/?
can only take a single pred-form
@alexmiller#2019-07-2505:43maxphow to write spec that conforms input value to positive int or to zero when value missing?#2019-07-2505:44maxplike that
(s/def ::offset
(s/or
:nil (s/and nil? (s/conformer (constantly 0)))
:int (s/and int? #(<= 0 %))))
#2019-07-2505:45maxpbut it returns vector (s/conform ::offset nil)
[:nil 0]#2019-07-2506:03Alex Miller (Clojure team)@seancorfield sorry, tired#2019-07-2506:03Alex Miller (Clojure team)should be s/cats in there#2019-07-2506:03Alex Miller (Clojure team)(s/cat :a ::a :an (s/? (s/cat :b ::b :bn (s/? (s/cat ...
#2019-07-2506:04Alex Miller (Clojure team)@maxp imo, you should not do that with specs#2019-07-2506:06Alex Miller (Clojure team)s/or always conforms to a tagged result where the tag matches the branch that was taken#2019-07-2506:07Alex Miller (Clojure team)but general data transformation is best applied outside spec, not as part of conforming anyways#2019-07-2506:08maxpthank you#2019-07-2507:21maxpis it possible to "attach" some human readable message to the spec to get it in (explain-data) ?#2019-07-2507:23carkh@maxp check the expound library#2019-07-2507:23carkhhttps://github.com/bhb/expound#2019-07-2507:24carkhnot exactly what you're asking but it helps#2019-07-2507:25maxpgreat!#2019-07-2515:55jjttjjIs there a way to get the resolved value for a given spec? I have a set I generate as a spec and I'm wondering if there's a way to access that set from the spec api:
(s/def ::sec-type (set (map str (Types$SecType/values))))
(s/describe ::sec-type) ;;=> (set (map str (values))) ;can't eval this
#2019-07-2515:58seancorfield@jjttjj Try s/form
instead.#2019-07-2515:58seancorfields/describe
returns an "abbreviated" version.#2019-07-2515:58jjttjjperfect, thanks again sean#2019-07-2519:17y.khmelevskiihey, can anybody help me with it?
https://clojurians.slack.com/archives/C1B1BB2Q3/p1563950441028500#2019-07-2616:00kennyAny good tools for debugging slow generators? We have lots of specs with tens of s/and
preds & custom gens associated. I'd like to know which predicate is causing a such-that to fail often.#2019-07-2616:05gfredericksI bet you could alter-var-root such-that
to default to a lower max-tries
#2019-07-2616:05gfrederickslooks like spec just uses the default (10) as far as I can tell#2019-07-2616:05gfredericksso if you change that to, e.g., 2, you might get more failures#2019-07-2616:06gfredericksbut maybe that's not useful if the "telling you which s/and
failed" feature isn't wired up yet#2019-07-2616:07kenny> isn't wired up yet
Is a feature like this on the roadmap?#2019-07-2616:08kennyGood idea on the such-that max-tries thing. That should help a bit. Though, the such-that errors are usually pretty obtuse, hiding the actual name of the predicate behind an anonymously named test.check pred.#2019-07-2616:09gfredericksspec's roadmap, yes, as far as I know; that feature couldn't exist prior to the 0.10.0 versions of test.check#2019-07-2616:09gfredericks> the such-that errors are usually pretty obtuse
that's exactly the problem that the aforementioned up-wiring would address#2019-07-2616:11kennyAh, very cool. Debugging these issues has been a huge time sink for us. It seems like it should be a fairly easy change to spec. What feature in 0.10.0 allows this better up-wiring?#2019-07-2616:16gfredericksfrom the such-that
docstring:
You can customize `such-that` by passing an optional third argument, which can
either be an integer representing the maximum number of times test.check
will try to generate a value matching the predicate, or a map:
:max-tries positive integer, the maximum number of tries (default 10)
:ex-fn a function of one arg that will be called if test.check cannot
generate a matching value; it will be passed a map with `:gen`,
`:pred`, and `:max-tries` and should return an exception
#2019-07-2616:23kennyI see. Would it be easy to hack this in to spec as a temporary fix?#2019-07-2616:24gfredericksI have no idea, @U064X3EF3 probably knows at least the easy vs hard part#2019-07-2616:27Alex Miller (Clojure team)probably not easy?#2019-07-2616:28Alex Miller (Clojure team)well, not sure#2019-07-2616:28gfredericksI'd think you'd want have some kind of path on hand at the point where such-that
is called#2019-07-2616:29gfredericksso might take some wiring to make that path available#2019-07-2616:29gfredericksor maybe it's already there, since spec likes to put paths in its error messages#2019-07-2616:29kennyThis would be a huge value add for us. If adding/hacking this in is a day's worth of work, it's definitely worth it for us to dive in#2019-07-2616:29Alex Miller (Clojure team)it should be - gens have the path#2019-07-2616:30Alex Miller (Clojure team)but they aren't using such-that right now#2019-07-2616:30Alex Miller (Clojure team)so I'm not sure how this stuff connects#2019-07-2616:30gfredericksthey? this is about s/and
in particular I think#2019-07-2616:31gfredericksI can't imagine the gen for s/and
doesn't use such-that
#2019-07-2616:31Alex Miller (Clojure team)I guess it does, just not directly#2019-07-2616:32Alex Miller (Clojure team)that's generically in gensub, which handles the recursion check stuff#2019-07-2616:33kennyIt appears there's only one place that such-that
is used in spec, gensub as you say. It also is passed other info that seems valuable to report back.#2019-07-2616:36Alex Miller (Clojure team)are you using deps.edn ?#2019-07-2616:36kennyYes#2019-07-2616:36Alex Miller (Clojure team)if so I could commit something on a branch and you could test it as a git dep#2019-07-2616:36kennySure#2019-07-2616:37kennyI currently don't have a repro of one of these such-that issues though, just the slow gen thing. The such-that errors are very frequent though#2019-07-2616:49Alex Miller (Clojure team)easy to repro with any restrictive s/and pred#2019-07-2616:49Alex Miller (Clojure team)my first hack did not yield anything useful though#2019-07-2616:49Alex Miller (Clojure team)I need to step away for a bit, but may play with it later#2019-07-2616:50kennyOk. Let me know if I can help in any way.#2019-07-2616:13Alex Miller (Clojure team)@kenny 95% of the time the issue with slow generators is generating large (and particularly nested large) collections#2019-07-2616:14Alex Miller (Clojure team)setting :gen-max 3
in s/coll-of
, s/map-of
etc is often a big difference#2019-07-2616:14kennyThis structure is highly nested & has lots of colls. We have these opts set:
:coll-check-limit 10
:coll-error-limit 10
:fspec-iterations 10
:recursion-limit 1
#2019-07-2616:15Alex Miller (Clojure team)none of those affect the gen'ed count#2019-07-2616:16Alex Miller (Clojure team)with a 3 level coll, you could have 10x10x10 nested elements#2019-07-2616:16kennyOh, right - that param is on the actual generator. Lemme see if we have those set.#2019-07-2616:16Alex Miller (Clojure team)well, recursion limit might affect the depth (although I think there are some scenarios where it is not getting applied right now)#2019-07-2616:18Alex Miller (Clojure team)if you have fspecs, that's another possible issue where you might want to consider simpler preds#2019-07-2616:18Alex Miller (Clojure team)like ifn?
#2019-07-2616:19kennyWe have considered that a number of times simply due to the massive amount of time debugging generators take. Every time, however, we always decide not to because those fspec generators catch valuable bugs haha#2019-07-2616:19gfredericksthe eternal test.check conundrum#2019-07-2616:19Alex Miller (Clojure team)another testing tip is that gens will get run 1000 times in check
and size/complexity increases#2019-07-2616:20Alex Miller (Clojure team)so gen/sample
by default (20) will not expose generator issues#2019-07-2616:20Alex Miller (Clojure team)but you can gen/sample
with 1000 and will see those#2019-07-2616:20Alex Miller (Clojure team)if you separate the args spec for an fdef into its own spec, it's easy to test that#2019-07-2616:21Alex Miller (Clojure team)(s/fdef foo :args ::args-spec)
, then (gen/sample (s/gen ::args-spec) 1000)
#2019-07-2616:22kennyI searched for :gen-max
in this service and didn't find any so it may not be used. Though, there are tons of custom gens which use the gen/*
functions which don't take in :gen-max
, I think.
So that's the usual way to debug the slow gens -- something like this (do (doall (gen/sample (s/gen ::db/db-control-args) 1000)) nil)
. Then that will hang forever and that's where the pain starts.#2019-07-2616:23Alex Miller (Clojure team)I would try just blindly adding :gen-max 3
to all the collection specs#2019-07-2616:23gfredericks@kenny what version of test.check are you using?#2019-07-2616:23Alex Miller (Clojure team)there is a ticket specifically for changing this default (or exposing a global default) btw#2019-07-2616:26Alex Miller (Clojure team)which I am completely failing to find#2019-07-2616:26Alex Miller (Clojure team)but I wrote the patch for it, so I know it exists#2019-07-2616:43kennyThe global default would still respect coll-of :min-count
, right?#2019-07-2617:12Alex Miller (Clojure team)there's actually a bug report about that#2019-07-2617:12Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2202#2019-07-2617:13Alex Miller (Clojure team)so be careful on that#2019-07-2616:24kennyWill try that.
We're using 0.10.0-alpha3.#2019-07-2616:24kennyYeah, that would help to figure out if the problem is collection gen or something else.#2019-07-2616:25gfredericksokay, that should avoid some exacerbatory issues with the 0.9.0 release#2019-07-2616:25kennyHaha, yeah. I remember a huge perf bump when switching from 0.9.0#2019-07-2616:25gfredericksoh good#2019-07-2616:47kennyAnother thing that slows down debugging slow gens is that the check
tests still run in the background even though I interrupted the current op in the REPL.#2019-07-2616:47kennyI often will need to restart the REPL to get it to stop.#2019-07-2616:47gfredericksis that a pmap
thing?#2019-07-2616:47kennyWe're not using that. Does test.check?#2019-07-2616:50gfredericksI think spec does#2019-07-2616:50gfredericksbut could be wrong#2019-07-2616:56kennypmap
is in the check
impl.#2019-07-2616:57kennyMaybe a call to shutdown-agents
would help?#2019-07-2617:00kennyGetting a frequencies
map returned after generating a spec with s/and
would also be super helpful. The keys would be the forms of each s/and pred.#2019-07-2617:10Alex Miller (Clojure team)well shutdown-agents will shutdown the pool never to restart again, so you'd still need to bounce your repl#2019-07-2617:11Alex Miller (Clojure team)pmap is lazily parallel#2019-07-2617:11Alex Miller (Clojure team)so when you stop using the result, up to n-1 (where n is number of processors) may still be working to compute the next few results in the lazy seq#2019-07-2617:16kennyHmm. It seems to go on forever. Iāll add prints to some functions to debug the slow gens and it never stops printing after interrupting. #2019-07-2617:24kennyPerhaps because of the doall
in this line? (do (doall (gen/sample (s/gen ::db/db-control-args) 100)) nil)
#2019-07-2617:52kennyThis is helpful :
(let [such-that gen/such-that]
(with-redefs [gen/such-that (fn
([pred gen]
(such-that pred gen 1))
([pred gen opts]
(such-that pred gen 1)))]
(gen/generate (s/gen ::db-control-args))))
#2019-07-2617:54kennyActually, @gfredericks the problem with taking the above approach is simple generators sometimes need the such-that. e.g.
(s/def ::int+-interval
(s/and (s/tuple ::m/int+ ::m/int+)
(fn [[x1 x2]] (>= x2 x1))))
#2019-07-2617:55kennyI suppose you could write a custom gen for an interval.#2019-07-2619:26Alex Miller (Clojure team)there already is an interval spec with a generator - s/int-in
#2019-07-2619:26kennyThat generates an integer. This one is an actual interval. e.g. [0 10]
#2019-07-2619:26Alex Miller (Clojure team)ah, I see#2019-07-2617:58gfredericks@kenny yeah the general rule with such-that
is that the predicate needs to be highly-likely to succeed; so for something like an interval you'd want to write your own generator, which is pretty easy#2019-07-2617:58kennyI mean, it is highly likely to succeed in the above case.#2019-07-2617:59gfredericksisn't it just 50%?#2019-07-2618:00kennyI think so#2019-07-2618:00gfredericksmore precisely what you want is "astronomically unlikely to fail 10 times in a row"#2019-07-2618:00gfredericksand something that only passes 50% of the time doesn't have that quality#2019-07-2618:00kennyFair enough#2019-07-2618:01gfredericksas such-that
retries, it's also incrementing size
, so often it's good enough that the probability of failure goes down with larger size
#2019-07-2618:01gfrederickse.g., gen/not-empty
relies on that#2019-07-2618:03kennyIt almost seems like you should be able to enable a warning that tells you when your specs use s/and and don't have a custom gen.#2019-07-2618:31kennyWriting the generator for an interval, like the above is tricky without such-that because you don't know how to generate a value that will match the spec. I was originally thinking it'd just be a gen/bind
but that doesn't work because I don't know how to generate a ::m/int+
such that the second one is greater than the first without using such-that.#2019-07-2619:28Alex Miller (Clojure team)the trick is to generate a starting value and the range size, then gen/fmap the pair from that#2019-07-2618:33bfabryone option: before doing a such-that add something like this https://github.com/bfabry/specify-it/blob/master/src/specify_it/bst_spec.clj#L268 that just reports on how often the such-that would succeed. then tune your inputs until itās a reasonable frequency#2019-07-2618:34kennyWell this is a reasonable frequency but it's certainly not "astronomically unlikely to fail 10 times in a row"#2019-07-2618:35kennyIt's fairly unlikely though.#2019-07-2618:36bfabryin the case that youāre going to throw away generations that donāt match your predicate, I would put āreasonable frequencyā at like 50%#2019-07-2618:37kennyThat seems right. This would go back to the original problem then where, in practice, overriding such-that doesn't really work as a debugging technique.#2019-07-2618:42bfabrya test.check generator mode where all of the various predicates print out their form line number and % success rate at the end would be cool (and quite hard to do)#2019-07-2618:42kennyThat's exactly what I want too š#2019-07-2618:52gfredericks@kenny w.r.t. avoiding such-that, how about fmap with sort?#2019-07-2618:52kennyOoo, I like it!#2019-07-2618:53gfredericksfmap can be used to avoid such-that in a lot of cases
another example is here: https://clojure.org/guides/test_check_beginner#_generator_combinators#2019-07-2619:27kenny@gfredericks That cleaned up this code nicely š Thanks for the idea! https://github.com/Provisdom/math/blob/488573a16947eab78ecd096c7ef71a33f94cd0d7/src/provisdom/math/intervals.clj#L14-L55#2019-07-2619:29Alex Miller (Clojure team)I would generate the starting value and the range size, then fmap to get [start (+ start size)]#2019-07-2619:30kennyI don't think that'd work because you'd need to know how to generate the range.#2019-07-2619:30kennyFor example, we have a "prob-interval" where the interval's lower and upper bounds must be between 0 and 1.#2019-07-2619:32kennyOne issue with @gfredericks approach, however, is if you have a strict-interval. You have a chance of hitting the such-that then.#2019-07-2619:33gfredericksSort+inc?#2019-07-2619:34kennyCan't be that simple because of the above interval case.#2019-07-2619:34kenny"prob-interval", that is#2019-07-2619:36kennyYou could add Double/MIN_VALUE in that case. But if the interval is integers only, that won't work.#2019-07-2619:36kennyWell, even adding Double/MIN_VALUE may not work because that may go outside the range.#2019-07-2620:01gfredericksMath/nextUp š#2019-07-2620:25kennyI think that's still problematic for the same reason though.#2019-07-2620:26kennyWhy isn't generator?
public in spec's gen ns?#2019-07-2620:34Alex Miller (Clojure team)Might be an oversight, might be a reason it wonāt work with dynaload#2019-07-2620:38kennySame with vector-distinct-by
#2019-07-2620:40kennyMight be able to add them to our own libs by using lazy-combinators
#2019-07-2620:42kennyWon't work because lazy-combinator
isn't qualified š https://github.com/clojure/spec.alpha/blob/5228bb75fa10b7b13ea088d84f4d031cce74a969/src/main/clojure/clojure/spec/gen/alpha.clj#L92#2019-07-2620:47kennyCould s/coll-of
take a new option for :distinct-by
?#2019-07-2621:01souenzzoI need something like (s/tuple (s/tuple ::a) (s/tuple ::b))
, but should allow values like [["a" "d"] ["b"]]
(ignore "extra" tuple values)#2019-07-2621:04souenzzo(s/cat :a (s/spec (s/cat :a ::a
:extra (s/* any?)))
:b (s/spec (s/cat :b ::b
:extra (s/* any?))))
^ this one works, but it's really ugly#2019-07-2621:13kennyCan you use a s/or
?#2019-07-2621:21souenzzoor
where?#2019-07-2621:23kenny(s/def ::tup1 (s/tuple string?))
(s/def ::tup2 (s/tuple string? string?))
(s/def ::tup (s/or :tup1 ::tup1 :tup2 ::tup2))
(s/tuple (s/tuple ::tup ::tup))
#2019-07-2621:26souenzzonot sure how many ::tupN
is needed.
The fact is: just the 3 first values are required/used. (each one has it own spec)
But sometimes a larger collection if passed as arg.#2019-07-2621:27kennyTuple is usually for a list of a known length. Sounds like you really want cat
, like your example#2019-07-2919:45respatializedat risk of asking an obvious question: how do I refer to a spec from another namespace?
here's an example: in namespace-0
, I have some specs defined using (s/def ::consistent ...)
. In namespace-1
, I am using (:require [namespace-0)]
, but making calls to the specs in the other namespace by doing something like (s/valid :namespace-0/consistent data)
fails with a Unable to resolve spec :namespace-0/consistent
. How do I refer to these specs from another namespace using fully qualified keys?#2019-07-2919:57respatializedactually, it works as expected using the fully qualified keys. my issue was just a typo in the spec name. š#2019-07-2921:54kennyIs this how :gen-max
is supposed to work? It appears to be a bug.
(distinct (map count
(gen/sample
(s/gen (s/map-of string? string?
:min-count 1
:gen-max 1)) 100)))
=> (1 2)
#2019-07-2921:56Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2202#2019-07-2921:56Alex Miller (Clojure team)tis bug#2019-07-2921:57kennyAh, ok. Switched everything to use :gen-max and generators got 2-2.5x slower š¬#2019-07-2921:57Alex Miller (Clojure team)doh#2019-07-2921:58kennyI think a lot of places are only generating one item now generate up to 2.#2019-07-3015:23misha@afoltzm "required" aliases work too:
(require '[clojure.spec.alpha :as s])
::s/invalid
;=> :clojure.spec.alpha/invalid
#2019-07-3015:27Alex Miller (Clojure team)they work, assuming that the qualifier actually corresponds to a real clj namespace#2019-07-3021:14dharriganIs there a spec that tests for an empty map (as a value) i.e., {:foo {}}
?#2019-07-3021:30minimal(s/def ::empty-map #{{}})
#2019-07-3107:24dharrigansweet#2019-07-3107:24dharriganta š#2019-08-0114:06ben.mumfordare there any best practices anywhere on where in the code to define specs? at the moment i'm defining function specs alongside the function they go with and don't have a consistent strategy for where to define the specs (spec/def ...) - usually in a few large files#2019-08-0114:08jjttjj@ben.mumford620 there's a discussion on this here: https://stackoverflow.com/questions/37942495/where-to-put-specs-for-clojure-spec#2019-08-0114:10ben.mumfordcheers. i'm guessing then that are no suggestions from rich and the crew?#2019-08-0114:16jjttjjThis is extremely shaky ground on which to make a decision, but here's a cognitect example of a spec only namespace:
https://github.com/cognitect-labs/day-of-datomic-cloud/blob/b4103e4a8f14518ed3f6d7f66f56cbf863117974/src/datomic/dodc/repl_ui/specs.clj#2019-08-0114:16Alex Miller (Clojure team)https://clojure.org/guides/faq#spec_location#2019-08-0114:20ben.mumfordthanks alex#2019-08-0216:40kennyI am getting this exception "Additional data must be non-nil."
when using st/check
on my function. I have gotten this before and then it mysteriously went away. I am now getting it constantly and am curious what is going on. It is coming from these Spec lines: https://github.com/clojure/spec.alpha/blob/5228bb75fa10b7b13ea088d84f4d031cce74a969/src/main/clojure/clojure/spec/test/alpha.clj#L278-L284.#2019-08-0216:43ghadiif you have a reproducible example case, please paste it @kenny#2019-08-0216:44kennyWorking on that. It's pretty coupled to this code but I should be able to pull something out.#2019-08-0216:44ghadicool -- thanks!#2019-08-0216:44ghadithis has been reported before, but AFAIK there's not a deterministically reproducible case#2019-08-0216:45kennyHopefully I can get one! It's definitely reproducible for me right now.#2019-08-0216:49Alex Miller (Clojure team)the error occurs when you try to create an ex-info with nil data. the case where this happens here is when spec validation fails, but explain-data succeeds (which ideally should never happen). usually, this is a bug in spec. occasionally, it's a bad predicate in user code.#2019-08-0216:50kennyThis error is quite confusing. Seems like a bug either way š#2019-08-0216:50Alex Miller (Clojure team)well the error should never happen#2019-08-0216:50ghadiright, it's an invariant violation#2019-08-0216:52kennyhaha, the best kind of bugs#2019-08-0216:53kennyI pulled some code out to get a repro case and was running st/check on it. It failed with a regular specification error. I thought that was weird because it hasn't failed yet. Turned on instrumentation and ran again and got the "Additional data must be non-nil." message. May be related to instrument.#2019-08-0216:57Alex Miller (Clojure team)this won't help you, but I've made a change in spec-alpha2 to better report this case#2019-08-0217:22kennyAlright, here's a repro: https://gist.github.com/kennyjwilli/f324b94eaadc404cb72fdfe41067b469. Not minimal but I've gotta go make some food. Within 3 or so runs of (st/check
formula->fn)`, it will get the exception.#2019-08-0218:23Alex Miller (Clojure team)so I misdiagnosed above - even more subtle... s/conform and s/valid? are returning different answers when run on the same input#2019-08-0218:24Alex Miller (Clojure team)which is because the spec is an fspec, which generates its own source of random inputs#2019-08-0218:25Alex Miller (Clojure team)this is a known issue and generally we've been suggesting people don't use fspec because it has all these unexpected consequences#2019-08-0218:50kennyfspec has proven to be quite valuable though. What is the alternative? Just fn?
#2019-08-0218:56Alex Miller (Clojure team)ifn?
#2019-08-0218:57Alex Miller (Clojure team)unfortunately it's not even easy to override in a spec alias or anything#2019-08-0218:58kennyWon't that just generate any old function during gen tests and pass it to my function?#2019-08-0219:07Alex Miller (Clojure team)yes, it is significantly harder to use it in check. I'm not saying any of this is good.#2019-08-0219:09Alex Miller (Clojure team)this is a known area I'm hoping to spend some rethink time on in spec 2, but we're going to overhaul the function return value stuff first#2019-08-0219:16kennyOh ok. Any workaround for now to avoid this error?#2019-08-0219:21Alex Miller (Clojure team)I don't think I have any simple advice for you, other than to remove your :ret spec :(#2019-08-0219:25kenny:ret on the fspec of the top level fn?#2019-08-0219:27Alex Miller (Clojure team)no, the :ret on the fdef#2019-08-0219:27Alex Miller (Clojure team)that is an fspec#2019-08-0220:10kennyI was trying to add some gen improvements to clojure.spec.alpha
but I cannot load the ns in a REPL. I get
CompilerException java.lang.Exception: #object[clojure.spec.alpha$and_spec_impl$reify__2171 0x4a332076 "
Is there some magic that needs to happen to have a REPL with Spec itself?#2019-08-0220:49y.khmelevskiihi gents! How I can define spec of vector where I know only first element and donāt know length of vector. For example [:in 1 2 3 4]
#2019-08-0220:56y.khmelevskiiit would be great to use something like
(s/tuple #{:in :or} & nat-int?)
#2019-08-0220:58Alex Miller (Clojure team)(s/cat :in #{:in :or} :nums (s/* nat-int?))
#2019-08-0220:59Alex Miller (Clojure team)this is exactly the job the regex ops are made for#2019-08-0221:01y.khmelevskiiI see, thank you!#2019-08-0317:54hjrnunesHi. Do predicates need to accept MapEntry
arguments to be used with s/or
?
This predicate
(defn str-date-time? [v]
(try
(some? (f/parse (f/formatters :date-time) v))
(catch IllegalArgumentException _
false)))
Causes ClassCastException: clojure.lang.MapEntry cannot be cast to java.lang.String
when used in an or
spec.
If I change the catch
to Exception
I get a failed validation with
Part of the value
{:post/updated "2017-05-05T11:55:00.000Z", ...}
when conformed as
[:str-date-time "2017-05-05T11:55:00.000Z"]
with spec
(s/def :post/updated (s/or :date-time ::tspec/date-time
:str-date-time str-date-time?))
used as :req
in a s/keys
form.
Any thoughts? Thanks#2019-08-0318:21seancorfield@hjrnunes That sounds like you're passing a conformed value into str-date-time?
#2019-08-0318:23hjrnunesYes it does. But it's utterly puzzling. I'm merely calling s/valid?
on a map#2019-08-0318:23hjrnunesDoes s/or
conform values?#2019-08-0318:23seancorfieldThe result of s/or
is a tagged pair, yes.#2019-08-0318:24hjrnunesso, I should cater for this in the predicate?#2019-08-0318:24seancorfieldNo.#2019-08-0318:24seancorfieldI need to see more of your specs to see where you're flowing a conformed value into that spec.#2019-08-0318:24hjrnunesactually, I have the same result with string?
or int?
#2019-08-0318:25seancorfieldShow the context.#2019-08-0318:26hjrnunesyes, I'm trimming stuff down. Thanks for the help#2019-08-0318:27seancorfieldDo you have s/and
somewhere? That's often where a conformed value ends up flowing into another predicate.#2019-08-0318:30hjrnunesYes I do#2019-08-0318:30hjrnunes(s/def :post/published (s/or :date-time ::tspec/date-time
:str-date-time str-date-time?)) ;; should be DateTime or :date-time format
(s/def :post/updated (s/or :date-time ::tspec/date-time
:str-date-time str-date-time?)) ;; should be DateTime or :date-time format
(s/def ::post-found (s/keys :req [:post/published
:post/updated]))
(s/def :post/missing coll?)
(s/def ::post-missing (s/and ::post-found
(s/keys :req [:post/missing])))
#2019-08-0318:31seancorfieldYou probably want s/merge
not s/and
there.#2019-08-0318:32hjrnunesHmm, perhaps. So the idea is to re-use ::post-found and add another key - which is what actually happens to the data#2019-08-0318:32hjrnunesso s/merge
is the right way then?#2019-08-0318:33seancorfieldThat's what s/merge
is for -- to blend two s/keys
specs.#2019-08-0318:33hjrnunesI see, I read somewhere, the mailing list I think, that s/and
would work. Perhaps it was in a previous version#2019-08-0318:34hjrnunesYep. That does it. Thank you very much!#2019-08-0318:35hjrnunesBTW, I listened to the defn podcast you were on recently - very interesting! Best of luck and thanks for all the work!#2019-08-0318:39seancorfields/and
is specifically for flowing a conformed value into another spec. For example, you might s/and
one s/keys
-based spec and then a predicate that checks two keys have distinct values perhaps.#2019-08-0318:39seancorfields/merge
is for extending an s/keys
-based spec with additional key specs -- like merge
for maps.#2019-08-0505:27akondexplain-data
produces a list of problems, but i've only seen a single item in it. can somebody show a case when the :problems vector contains more than one item?#2019-08-0512:39Alex Miller (Clojure team)(s/explain-data (s/or :i int? :k keyword?) "abc")
#2019-08-0512:46akond@alexmillerthank you#2019-08-0512:49akondbut is it still possible to produce more then 1 problem without s/or's?#2019-08-0512:50Alex Miller (Clojure team)yes, there are several places where it can happen#2019-08-0604:21dorabCan someone please explain this behavior?#2019-08-0604:22dorab$ clj -Srepro
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.gen.alpha :as sg])
nil
user=> (require '[clojure.spec.test.alpha :as st])
nil
user=> (s/def ::vec vector?)
:user/vec
user=> (s/def ::str string?)
:user/str
user=> (s/def ::str-vec (s/coll-of ::str :kind ::vec))
:user/str-vec
user=> (sg/generate (s/gen (s/coll-of string? :kind vector?)))
["2m4A8TKjUU5TJ61KkKRzbnk2z21" "xAvkdc78i5qo6O8Gf6" "cHbZwfz88d6xZj5qp54L5g" "K6MdrEaiFWm2P27BCD1913h9r1" "2jpUYE74Vvf" "PG9SrB729W5pWo6G5xt7K28tR5M33" "DTAJijHs2t9mEC" "6zkimTFtp51q698N" "eAgGEU08gJF7R5C2tb6ZdcTRO" "Ewx60Cb94NWxMMXjQJn"]
user=> (sg/generate (s/gen ::str-vec))
Execution error (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
user=>
#2019-08-0604:23dorabWhy does the sg/generate
work in one case, but not the other?#2019-08-0604:54seancorfield@dorab :kind
should not be a spec, it should be a predicate#2019-08-0604:54seancorfieldIf you define ::str-vec
with :kind vector?
it works just as expected.#2019-08-0604:55seancorfieldIf you read the docstring for s/every
(which applies to s/coll-of
), it says :kind - a pred that the collection type must satisfy, e.g. vector?
(default nil) Note that if :kind is specified and :into is
not, this pred must generate in order for every to generate.
#2019-08-0604:56seancorfieldPredicates can be treated as specs, but specs are not predicates (in general).#2019-08-0615:29dorabThanks#2019-08-0903:42valeraukois there any roadmap for spec 2? we plan to add spec validations to our codebase and if spec 2 is reasonably close we'd go straight with that#2019-08-0903:49Alex Miller (Clojure team)roadmap in terms of things to do or timing#2019-08-0903:49Alex Miller (Clojure team)I guess you mean timing#2019-08-0903:50Alex Miller (Clojure team)I would not expect it to be "done" for at least another month or two#2019-08-0903:51valeraukowe could work on adding it from this fall (october <) so that sounds promising to me#2019-08-0903:51Alex Miller (Clojure team)well, you might be fine then#2019-08-0903:51Alex Miller (Clojure team)you can use it right now as a git dep, but it is definitely a work in progress#2019-08-0903:52Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2#releases-and-dependency-information#2019-08-0903:54seancorfield@vale We have a branch of our code base running on Spec 2. There are definitely some interesting differences from Spec 1, and if you do anything where you need to share specs between 1 & 2 (perhaps because you use specs from a library that depends on Spec 1), you'll need to jump through some hoops.#2019-08-0903:55Alex Miller (Clojure team)I've landed two big sets of changes in the last week there, will have a writeup on some new stuff tomorrow#2019-08-0903:56seancorfield(just in time for me to go to England for a week so I won't get to play with it until I get back!)#2019-08-0903:57valeraukoi heard you mention that on the defn podcast, but i'm not adventurous enough to use it in production code#2019-08-0903:58valeraukowe're adding spec from zero, and that 1/2 difference thing is exactly the reason i'd prefer to go straight to 2 if it's feasible#2019-08-0904:01valeraukoon a similar note, any plan on making it not-alpha?#2019-08-0904:11seancorfieldJust reviewing those commits @alexmiller -- love the new closed spec approach!#2019-08-0904:11seancorfield@vale Oh that branch isn't in production -- just dev -- until Alex green lights production use (even in alpha).#2019-08-0904:12seancorfieldWe started using Spec 1 in early alpha in production. We take most Clojure alpha builds to production. But Spec 2 isn't yet at that point.#2019-08-0904:13seancorfieldBased on past experience, I'd expect Spec 2 to stay alpha for quite a while before it moves to a beta. tools.deps
is still alpha and it's core to the CLI / deps.edn
tooling.#2019-08-0904:33Alex Miller (Clojure team)yeah, we're still actively changing the api in spec 2 and I don't think that's done yet. we're not going to have a "release" until that settles down#2019-08-0904:34Alex Miller (Clojure team)my hope is that it can be stable enough to be non-alpha and so it will never release as spec-alpha2 at all, but will depend on timing#2019-08-0904:34Alex Miller (Clojure team)tools.deps edges ever closer to me being comfortable with taking it off alpha#2019-08-0904:35Alex Miller (Clojure team)the api is pretty stable there#2019-08-0904:43seancorfield@alexmiller Good to hear that about Spec 2. Targeting "release" by Conj?#2019-08-0904:43seancorfieldRe: tools.deps
-- any decision on add-lib
yet?#2019-08-0905:41valeraukooh nice! it could be tricky to explain to stakeholders how something labelled alpha is production ready#2019-08-0905:46seancorfield@vale Stakeholders outside the Clojure world, yes. But inside the Clojure world there's much more stability than almost any other language.#2019-08-0905:47seancorfieldWe went to production originally, back in 2011, on Clojure 1.3 Alpha 7 or 8, and we've run almost every single pre-release build of Clojure in production ever since.#2019-08-0905:49seancorfieldQuite a few Contrib libs have been in pre-release mode for ages on various revisions and it's "expected" that folks will use them, reporting any problems, but on the assumption that they're "solid". There is, after all, a mindset -- driven from the top down -- of accretive/fixative change rather than breakage.#2019-08-0905:50seancorfield(that said, some libs have had breaking changes several times -- including clojure.java.jdbc
that I have maintained for about eight years!)#2019-08-0908:41dharriganupvote for add-lib
in tools.deps please š#2019-08-0911:37ikitommi@alexmiller Closed spec looks so much better now (also read the commits)! As the settings open up more options to do things, it would be great if you would time to re-check the proposed patch in https://clojure.atlassian.net/browse/CLJ-2116, as it had similar approach, but I believe, would be even more generic way to do this, for even to apply coercion. Or if thatās an bad idea, that should be declined as itās now partially overlapping with the new design.#2019-08-0911:39Alex Miller (Clojure team)Well, all spec patches in jira are probably long broken given that Iāve rewritten pretty big chunks of it multiple times#2019-08-0911:40Alex Miller (Clojure team)Once we get through the majority of new feature type stuff, I will go through them all but no point now #2019-08-0911:40Alex Miller (Clojure team)As Iāve said in the past, I think itās highly unlikely that weāre going to take clj-2116 regardless#2019-08-0911:43ikitommiI donāt think itās a good way anymore either (2116), wrote a the spec walker issue later that could be the one generic way to walk over specs + values, and could be used for both.#2019-08-0911:44Alex Miller (Clojure team)Well, weāre working on the walking stuff too#2019-08-0911:44ikitommicurrently spec-tools & spec-coerce walk the spec forms which is IMO not a right way to do that.#2019-08-0911:45Alex Miller (Clojure team)The other half of what Iām working on now is dataficatiom of specs#2019-08-0911:45ikitommicool. just for browsing or also transforming?#2019-08-0911:46Alex Miller (Clojure team)Either - the input side is there now#2019-08-0911:46Alex Miller (Clojure team)Once you have the output side, then transformation is just normal clojure stuff#2019-08-0911:48ikitommione common case for closed specs has been the āstrip out extra keysā functionality, is that doable with the new closed feature? e.g. before saving to the database / when reading input from untrusted clients / intenet.#2019-08-0911:49Alex Miller (Clojure team)Havenāt really considered it. Why not just use select-keys?#2019-08-0911:50ikitommiselect-keys for a deeply nested structure mean double-maintaining the keysets. some helper would be nice#2019-08-0911:51Alex Miller (Clojure team)I think weāre giving you a lot of new tools to build your own spec that does something like this (flag passed through settings) but we have not talked about any plans to add this#2019-08-0911:53Alex Miller (Clojure team)This again feels like a coercion/transformation use case which we donāt believe is a job for conform#2019-08-0911:53ikitommiok. the data is there (keysets) already to feed a select-keys. would love to throw away stuff from spec-tools, when they are doable with spec itself. currently:
(s/def ::name string?)
(s/def ::street string?)
(s/def ::address (s/keys :req-un [::street]))
(s/def ::user (s/keys :req-un [::name ::address]))
(def inkeri
{:name "Inkeri"
:age 102
:address {:street "Satamakatu"
:city "Tampere"}})
(st/select-spec ::user inkeri)
; {:name "Inkeri"
; :address {:street "Satamakatu"}}
#2019-08-0911:53ikitommispec-tools does that nowadays with form walking btw.#2019-08-0911:55ikitommiI think that would use the upcoming āwalkerā, so a good test case when you develop it š#2019-08-1013:26norton@alexmiller Hello. I have a question regarding spec-alpha2. Specs and tests written with specs stopped working from this commit.
https://github.com/clojure/spec-alpha2/commit/2bd68ffee3a775e6fa0f925260b94a9c421787c0
I am reviewing the diff and I spotted one item that Iām curious about.
The third argument to conform* on this line 220
https://github.com/clojure/spec-alpha2/blob/2bd68ffee3a775e6fa0f925260b94a9c421787c0/src/main/clojure/clojure/spec_alpha2.clj#L220
and on line 326
https://github.com/clojure/spec-alpha2/blob/2bd68ffee3a775e6fa0f925260b94a9c421787c0/src/main/clojure/clojure/spec_alpha2.clj#L326
looks inconsistent. Is this difference intentional ā¦ x on line 220 and spec on line 326?#2019-08-1013:29Alex Miller (Clojure team)Yeah, the first one should be spec#2019-08-1013:29Alex Miller (Clojure team)Do you have an example of whatās not working for you?#2019-08-1013:30nortonSorry, I do but not a small sample that I can share.#2019-08-1013:31Alex Miller (Clojure team)Ok, would be happy to take a look if you can describe#2019-08-1013:31nortonOr ā¦ can you share a recipe how I can test out locally by making this change?#2019-08-1013:39nortonThe essence of the test is as follows:
(doseq [spec (filter #(contains? STRING-NAMESPACE (namespace %)) (keys s/registry)))]
(let [p (prop/for-all [g (s/gen spec)] (s/valid? spec g))]
(is (= {:result true} (abbrev-result (tc/quick-check 100 p))))))#2019-08-1013:39nortonwhere STRING-NAMESPACE is a string used to filter keys from the s/registry#2019-08-1013:39nortonI might not have all of the parens correct ā¦ just pseudo-code š#2019-08-1014:45Alex Miller (Clojure team)that conform thing above was fixed in the next commit btw#2019-08-1014:46Alex Miller (Clojure team)so what fails? which spec and what's the failure?#2019-08-1015:05nortonHere is a subset of the stack trace. I bumped up to the latest version 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42'.#2019-08-1015:17nortonHere is a subset of the stack trace. I pinned it at the version '000d7d83a98ca3af58a44c20d7bd9fea0f4b03ab'#2019-08-1015:18nortonInterestingly, there seems to be a different failure(s) between these two commits.#2019-08-1015:23nortonThe commit '922b0f5886735641e0cdded58fc85011f79cc292' has the same stack trace as 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42'. This commit introduced the second failure.#2019-08-1015:24nortonAs I posted originally, the commit '2bd68ffee3a775e6fa0f925260b94a9c421787c0' introduced the first failure.#2019-08-1015:39nortonI will try to narrow down the issue to a small repository.#2019-08-1015:41Alex Miller (Clojure team)sorry, hard for me to do much without seeing the spec#2019-08-1015:43nortonUnderstand#2019-08-1015:54Alex Miller (Clojure team)Some spec updates here: http://insideclojure.org/2019/08/10/journal/#2019-08-1016:14eval2020I would expect the first (s/valid?...)
to be without the {:closed ...}
#2019-08-1017:41Alex Miller (Clojure team)yes, I whiffed some copy/pasta on the example, fixed#2019-08-1114:52nortonWorking example with 'dc960158ccb9cf8b5dc555842eea5e1af12f28c2'#2019-08-1114:53nortonFailing example with '2bd68ffee3a775e6fa0f925260b94a9c421787c0'#2019-08-1114:57norton@alexmiller Please see these two examples. The latest version 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42' behaves the same as the failing example.#2019-08-1120:48seancorfieldDefinitely related to (s/def ::bar ::baz)
since you can do this user=> (s/def ::quux (s/keys :req-un [::baz]))
:user/quux
user=> (gen/sample (s/gen ::quux))
({:baz 0} {:baz -1} {:baz -1} {:baz -3} {:baz -3} {:baz -2} {:baz 0} {:baz 3} {:baz 0} {:baz -1})
user=>
#2019-08-1121:26Alex Miller (Clojure team)fixed#2019-08-1121:36seancorfieldYup. Just confirmed that works now!#2019-08-1200:32norton@alexmiller Thank you. That issue is resolved. I have a new one. Here is a failing example taken from the spec Guide. This is with the latest version 'e58788107ee2ab765a7d46854b2cffd85429e339'.#2019-08-1200:55seancorfield@norton I'll defer to Alex for the final word but I think that's a case where Spec 2 and Spec 1 differ. I think I ran across something like that when I tried converting our code over to Spec 2...#2019-08-1200:56norton@seancorfield thank you. I will take another look.#2019-08-1201:03seancorfieldI'm trying a few things locally but it isn't jogging my memory of what I had to do here. So it may be an unintended bug.#2019-08-1201:11seancorfieldHmm, the more I look at this, the more I think it is a new bug. I pulled up the Spec 2 branch of our code and we have some specs that look very similar to the above and used to work. Right now I can't test the Spec 2 branch on my laptop so I can't dig in any further until tomorrow @norton#2019-08-1201:34norton@seancorfield Yes, I think so. This type of spec was working with the 'dc960158ccb9cf8b5dc555842eea5e1af12f28c2' commit.#2019-08-1202:54Alex Miller (Clojure team)fixed, just missed the expand-spec for with-gen#2019-08-1205:20seancorfield@alexmiller That fixes the problem reported above, but you can't do (s/exercise ::kws)
-- that produces user=> (s/exercise ::kws)
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=>
#2019-08-1209:24Alex Miller (Clojure team)That is a spec 2 difference - the gen needs to wrap s/spec around the set#2019-08-1215:55seancorfieldAh yes. That was the difference I was thinking of when I first thought this was a Spec 1/2 difference but still couldn't get that example working! /cc @norton #2019-08-1209:51norton@alexmiller @seancorfield Thank you. The issue above is fixed. There is one more.#2019-08-1209:52nortonThis is an abridged version of a larger spec that illustrates the failure.#2019-08-1214:43Alex Miller (Clojure team)fixed, added some tests to catch this and prior#2019-08-1214:49schmeeCan you alias a spec to another spec? I have a situation like this where I would like to not repeat the definition of task-state
for both old
and new
:
(s/def ::task-state #{... big set ...})
(s/def ::old ::task-state)
(s/def ::new ::task-state)
(s/def ::event (s/keys :req [::old ::new]))
#2019-08-1214:50schmeethe code above doesnāt work, but it hopefully shows the intent#2019-08-1214:52jjttjjThat code should work though?#2019-08-1214:53Alex Miller (Clojure team)should work (in spec 1)#2019-08-1214:54schmeeyouāre right, this was PEBKACā¦ facepalm sorry for the noise#2019-08-1217:13ataggartAnyone know of guidance around organizing specs and the code being specād? E.g., putting specs in the same file as the things being specād, or in separate parallel nses, or in a separate common spec ns?#2019-08-1217:25Alex Miller (Clojure team)https://clojure.org/guides/faq#spec_location#2019-08-1217:31ataggartI swear I read that doc! š#2019-08-1217:31ataggartThanks#2019-08-1218:32seancorfield@alexmiller I'm updating our branch to the latest Spec 2 at work and ran into this exception "No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: java.lang.String",
-- would that be from this spec: (s/def ::empty-id (s/or :empty #{""} :id ::pos-int))
?#2019-08-1218:33Alex Miller (Clojure team)Dunno, would have to take a closer look#2019-08-1218:34seancorfieldI'll try wrapping #{""}
in (s/spec ,,,)
and see if that fixes it... but we have a lot of set-as-predicate specs and this has all worked with earlier Spec 2 builds...#2019-08-1218:37seancorfieldNope. Problem lies deeper. Digging...#2019-08-1218:38seancorfieldSeems to be related to this (defmacro param-spec
[coerce str-or-spec & [spec]]
(let [[to-str spec] (if spec [str-or-spec spec] [str str-or-spec])]
`(s/with-gen (s/and (s/conformer ~coerce) ~spec)
(fn [] (g/fmap ~to-str (s/gen ~spec))))))
So it's around dynamically built specs.#2019-08-1218:55gfrederickshttps://clojurians.slack.com/archives/C0JKW8K62/p1565636109003400#2019-08-1219:02Alex Miller (Clojure team)@seancorfield you shouldn't have to wrap that so I'd say it's a bug if that fixes it :)#2019-08-1219:04Alex Miller (Clojure team)it's probably not that it's dynamically built but rather something about what's being built (my first suspicion would be with conformer)#2019-08-1219:17seancorfielduser=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::foo (s/keys))
:user/foo
user=> (s/valid? ::foo {"bar" 42})
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval13332$fn$G (protocols.clj:11).
No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: java.lang.String
user=>
Only fails if the map being checked has string keys, rather than keyword keys.#2019-08-1219:19seancorfield@alexmiller Smallest repro ^#2019-08-1219:30Alex Miller (Clojure team)ah, thx#2019-08-1219:30Alex Miller (Clojure team)I changed how some of that keys stuff worked, obviously did not test it very well (but not really emphasizing keys as that's going to be replaced by schema/select)#2019-08-1219:49seancorfieldI assume it's trying to check whether unspecified keys conform and not guarding that with a check for whether the key name could actually be a spec?#2019-08-1219:50seancorfieldSwitching a large code base from s/keys
to s/schema
/`s/select` will be a massive piece of work so I hope that s/keys
will continue to work while we make those changes š#2019-08-1219:56Alex Miller (Clojure team)yeah, I'm trying to keep it working, I'll fix it up#2019-08-1222:26nortonAll tests are running fine now with 6af1f3.#2019-08-1305:09murtaza52I have a sorted collection, I want to define the relationship between 2 elements of the collection ie the first element is greater than the second element. Is this possible ?#2019-08-1305:32seancorfield@murtaza52 (sort comparator coll)
lets you pass in a comparator that defines the relationship on which you are sorting.#2019-08-1305:36seancorfielduser=> (sort (reify java.util.Comparator (compare [_ a b] (cond (even? a) -1 (even? b) 1 :else 0))) (range 20))
(18 16 14 12 10 8 6 4 2 0 1 3 5 7 9 11 13 15 17 19)
user=>
#2019-08-1305:36seancorfield(a silly example, but it sorts all even numbers ahead of all odd numbers š )#2019-08-1305:37seancorfieldIs that an answer to the question you were asking? (I wasn't entirely sure what you were asking)#2019-08-1305:39murtaza52@seancorfield thanks for the example š. However , I am already using a custom comparator for my sorting, however how do I use that in my spec definition ?#2019-08-1305:39seancorfieldI guess I don't understand what you're trying to do...#2019-08-1305:39murtaza52I have a function that does the sorting. Now I want to spec the fn, to specify that the output collection is sorted#2019-08-1312:21Alex Miller (Clojure team)We are kicking around some ideas for stating this constraint in spec 2#2019-08-1312:22murtaza52cool thanks#2019-08-1305:39seancorfieldAh...#2019-08-1305:40seancorfieldThis is one of those cases where the spec would almost need to be the implementation. I think that's over-specification to be honest.#2019-08-1305:40seancorfieldI very rarely specify the result to that level of detail.#2019-08-1305:41seancorfieldAfter all, it's not like instrumentation checks :ret
or :fn
so this is really only about generative testing and I think that can be better served by property-based testing, not direct :ret
/`:fn` spec testing.#2019-08-1305:42seancorfieldFor example, for any collection with count > 1, good properties to test are that the comparator produces the expected result for the first and second items, and for the first and last items.#2019-08-1305:43seancorfieldIt's tempting to "spec everything" when you get started but it's really not very productive in my experience.#2019-08-1305:44murtaza52ack makes sense from a testing perspective. however what I like about specs is also the documentation it produces. So when I spec a relationship, it also helps other devs to know what the fn is going to return.#2019-08-1305:44seancorfieldSpec should be "just enough" to sanity check what you're writing -- unless you are specifically using it to validate input data (or output data), rather than specific functions.#2019-08-1305:48seancorfield@murtaza52 I'm not sure that using Spec to that level of detail, just for documentation, is productive. I would say "most functions do not need to be Spec'd" but you could describe the behavior in the docstring without adding the overhead of Spec...#2019-08-1305:49seancorfieldSpec makes the most sense on boundaries and for the "critical" functions. Spec'ing everything makes code brittle.#2019-08-1305:49seancorfield(and verbose)#2019-08-1305:49murtaza52@seancorfield thanks for the insight#2019-08-1306:09seancorfieldIn my mind, this is why type systems can make code brittle -- you end up with types everywhere rather than just where they "make sense". It's one of the things I really like about Clojure: you can write a lot of generic code and a lot of abstractions without introducing that brittleness -- but you also have Spec for defining data structures and for certain functions on the edges of modules.#2019-08-1306:09seancorfieldIn my mind, this is why type systems can make code brittle -- you end up with types everywhere rather than just where they "make sense". It's one of the things I really like about Clojure: you can write a lot of generic code and a lot of abstractions without introducing that brittleness -- but you also have Spec for defining data structures and for certain functions on the edges of modules.#2019-08-1306:40murtaza52yup makes sense#2019-08-1310:19murtaza52@seancorfield so trying to think through, in what situations does specāing a fn makes sense, so that it could instrumented later ?
Bcoz as u mentioned specāing all fns dont make sense. The ones on the boundary I am already validating. So where does the optional specāing come in ?#2019-08-1316:29seancorfieldSpec'ing functions can be useful for development / testing (instrumentation / generative checking) but I would generally only spec functions that are part of module APIs or "important" functions. There's no hard and fast rules here -- just spec what is "useful" to spec. For example, in next.jdbc
, there are optional instrumentation specs for the public API functions -- for users of the library -- and those are instrumented during test runs too, for me as the developer of the library (but it's also important run the tests from time to time without instrumentation to see what behavior/errors crop up in known bad input data).#2019-08-1316:29seancorfieldI think you just have to find what works for you in terms of the amount of checking you need during development/testing and in terms of what you want to provide for clients of your code.#2019-08-1316:30seancorfieldIt's a bit like the question "Which functions should I unit test?" -- the answer is "Not all of them" but there are no hard and fast rules there either.#2019-08-1306:39murtaza52In the below spec -
(s/def ::id (s/and string? #(not (clojure.string/blank? %))))
(s/def ::my-event (s/keys :req [::id]))
this is valid - (s/explain ::my-event {::id "a"})
this is invalid - (s/explain ::my-event {:id "a"})
So do all my keys in the input have to be namespaced ? Because I am not able to validate data when the keys are not namespaced.#2019-08-1306:46seancorfieldIf you use :req
then, yes, the keys must be namespaced. If you use :req-un
then you have have simple keywords.#2019-08-1306:47seancorfieldWith :req-un
, you can have ::foo/bar
as the spec for the key -- but the key will be :bar
. That allows you to have different specs for the same key name in different contexts.#2019-08-1306:56murtaza52thanks, I was confused bcoz when I used gen
, it generated data without namespaced keywords. So my assumption was that it will also me to validate ones without ns.#2019-08-1306:56murtaza52So moving ahead is it more idiomatic to have namespaced keys in data too ?#2019-08-1307:42schmeeI have run into what I find to be a tricky modeling problem with spec. I have data that looks like this (excluding namespaces on the kws for brevity)
:event-type :state-transition
:old-state :running
:new-state :exception
Those three keys are always included, but if the event type is :state-transition
and the new state is :exception
, then there will be an :exception
key as well (there are a couple of more special cases like this).
What I would like is some form of āwildcardā, so that I can dispatch like so:
defmethod foo [:state-transition :exception]
defmethod foo [:state-transition _] <-- wildcard for the default case
Without the wildcard, I need a defmethod for all possible combinations state-transition/new-state which will lead to a lot of duplication.
Is there a better way to accomplish this, in either spec1 or spec2?#2019-08-1308:51jaihindhreddyName the special cases and use s/multi-spec
with a dispatch fn that returns these names, then impl them by extending the multimethod.#2019-08-1312:42murtaza52Is there a pred
for specifying only 1 param in a collection. I find this pattern when I want to spec the input args of a fn. Lets say if the fn expects a hash-map, I usually define the spec for args as :args (s/? ::map-spec)
. This pattern works, however is not correct bcoz it allows no values too.#2019-08-1312:44Alex Miller (Clojure team)s/cat is usually the best top level spec for args#2019-08-1312:44Alex Miller (Clojure team)(s/cat :m ::map-spec) #2019-08-1312:45Alex Miller (Clojure team)You can match up the tags and the args too#2019-08-1312:52murtaza52thanks, yup that is better#2019-08-1313:09Alex Miller (Clojure team)if you haven't seen the guide, it has several examples https://clojure.org/guides/spec#2019-08-1313:26murtaza52yup have gone through it, and its open on machine as a reference, however sometimes tend to miss a detail.#2019-08-1313:37murtaza52what is a good way to generate date time, I have a spec - (s/def ::start inst?)
, however the dates that it generates are very similar, and most are the same.
(gen/sample (s/gen ::ks/start) 5)
=>
(#inst "1969-12-31T23:59:59.999-00:00"
#inst "1969-12-31T23:59:59.999-00:00"
#inst "1969-12-31T23:59:59.999-00:00"
#inst "1970-01-01T00:00:00.001-00:00"
#inst "1970-01-01T00:00:00.000-00:00")
#2019-08-1313:41Joe Lane@murtaza52 https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/inst-in or write a custom generator.#2019-08-1313:43Alex Miller (Clojure team)inst-in will generate only dates near the start range in the first few samples but will then range farther#2019-08-1313:45Joe LaneIs that why sometimes I see a pattern to (drop 1000 ...)
with some of the generators?#2019-08-1313:50ghadimy general pattern for this stuff is to generate a bunch of deltas, and add them to a constant "anchor" value, in this case a date#2019-08-1313:51ghadi(But since inst-in exists, use it)#2019-08-1313:51Alex Miller (Clojure team)which is exactly how inst-in generator works but it "grows" from the start date#2019-08-1313:51Alex Miller (Clojure team)so test.check "shrinking" will shrink towards the start date#2019-08-1313:54Joe LaneIs that a property of the Rose tree or is it designed explicitly for that?#2019-08-1313:56murtaza52changed it to inst-in
and this worked (drop 990 (gen/sample (s/gen ::ks/start) 1000))
#2019-08-1313:57Alex Miller (Clojure team)the rose tree is designed to shrink from greater to lesser "size" and inst-in generator builds from start date + int generator (which has its own "size")#2019-08-1313:58Alex Miller (Clojure team)whether or not this is good is a separate question :)#2019-08-1314:00Joe Lanehahah cool, thanks for the insight alex. It's great to read your insideclojure blog, always a joy to see what you're working on.#2019-08-1315:56murtaza52is there a way to merge
distinct s/every-kv
like s/keys
can be done ?#2019-08-1318:14murtaza52I have speced a few fns with :args
and :ret
, however it throws an error only when the :args
does not conform, no error is thrown when the :ret
spec does not conform. Any ideas what I am missing ?#2019-08-1318:18seancorfield@murtaza52 Per our thread discussion earlier: instrument checks :args
but you need generative checking for :ret
(and :fn
).#2019-08-1318:19seancorfieldGenerative checking is about testing whether the function has the expected behavior (given generated conforming arguments).#2019-08-1318:19seancorfieldInstrumentation is about checking that other code calls this function correctly during development/testing.#2019-08-1318:25murtaza52thanks @seancorfield#2019-08-1404:12murtaza52just trying to understand why does a empty seq satisfy this spec (s/valid? (s/every integer?) [])
, shouldnāt it fail ?#2019-08-1404:32Alex Miller (Clojure team)because every element in the collection is an integer#2019-08-1404:32Alex Miller (Clojure team)for all none elements#2019-08-1405:19murtaza52could you please expand on none elements
, did not grasp it#2019-08-1406:14Alex Miller (Clojure team)(s/every? integer?) specifies a collection where each element is an integer. In this case []
meets that spec, there just happen to be no elements.#2019-08-1406:17murtaza52thanks @U064X3EF3 that explains it#2019-08-1405:22murtaza52When I run (stest/summarize-results (stest/check))
, I see the following error printed in my repl -
No implementation of method: :specize* of protocol: #'clojure.spec.alpha/Specize found for class: nil
all the tests are passing and the :sym
names are being printed correctly, however at the end it prints the above too.#2019-08-1406:15Alex Miller (Clojure team)don't know, can't tell from just that#2019-08-1415:17kennyI get this often. Typically it's from forgetting to call s/gen
on a spec.#2019-08-1415:20murtaza52There was a typo in my s/fdef
association with the sym, once that was corrected, the error disappeared#2019-08-1406:14Alex Miller (Clojure team)if you require a non-empty collection, you can use (s/every? integer? :min-count 1)
#2019-08-1421:33ataggartconformer
doc states:
> takes a predicate function with the semantics of conform i.e. it should return either a (possibly converted) value or :clojure.spec.alpha/invalid
And yet defining such a function gives me an error (minimal reproducible example):
user=> (defn f [] :clojure.spec.alpha/invalid)
Syntax error macroexpanding clojure.core/defn at (/private/var/folders/m9/5hxwnkyj0bxf6plprrbf5vzw0000gn/T/form-init9446902011342467351.clj:1:1).
:clojure.spec.alpha/invalid - failed: map? at: [:fn-tail :arity-1 :body :prepost+body :prepost] spec: :clojure.core.specs.alpha/defn-args
:clojure.spec.alpha/invalid - failed: any? at: [:fn-tail :arity-1 :body :body] spec: :clojure.core.specs.alpha/defn-args
Am I missing something silly?#2019-08-1421:35Alex Miller (Clojure team)defining that function fails the spec for defn itself#2019-08-1421:35Alex Miller (Clojure team)as it has the magic "invalid" value#2019-08-1421:37Alex Miller (Clojure team)a workaround is#2019-08-1421:37Alex Miller (Clojure team)user=> (def i :clojure.spec.alpha/invalid)
#'user/i
user=> (defn f [] i)
#'user/f
#2019-08-1422:01ataggartThanks!#2019-08-1422:04ataggartI think I have managed to avoid running into this by having predicates before the conformer
(surrounded by s/and
), thus obviating the need for the forming fn to explicitly yield ::s/invalid
.#2019-08-1505:16valerauko>magic
š¢#2019-08-1507:06Jakub HolĆ½ (HolyJak)Hello, when I run (require '[clojure.spec.test.alpha :as st])
(alias 'stc 'clojure.spec.test.check)
(st/check `my-awesome-fn
{::stc/opts {:num-tests 50, :max-size 3}})
I expect the random generated data, whenever there is a collection, to have max three elements (including collections nested in collections, speaking of lists/vectors/maps) but that is not the case. What do I wrong? Thank you!!!#2019-08-1512:18Alex Miller (Clojure team)What are the specs?#2019-08-1513:02Jakub HolĆ½ (HolyJak)The problematic argument to the function is ::account-invoices
, the core of its specs is: (s/def ::invoice-row-one (s/keys :req [::serviceType ::totalAmount ::period]))
(s/def ::invoiceRow (s/coll-of ::invoice-row-one :min-count 1 :kind sequential?))
(s/def ::invoice-group-single (s/keys :req [::name ::invoiceRow]))
(s/def ::invoiceGroup (s/coll-of ::invoice-group-single :min-count 1 :kind sequential?))
(s/def ::invoice-subscription-one (s/keys :req [::invoiceGroup ::invoiceSubscriptionUser]))
(s/def ::invoiceSubscription (s/coll-of ::invoice-subscription-one :min-count 1 :kind sequential?))
(s/def ::invoice (s/keys :req [::invoiceNumber ::billingDate ::invoiceSubscription ::period]))
(s/def ::acc+invoice (s/keys :req-un [::account ::invoice]))
(s/def ::account-invoices (s/nilable (s/every-kv string? ::acc+invoice))
#2019-08-1513:27Alex Miller (Clojure team)off the top of my head, not sure. those :min-count's may be causing something with size to be set in the constructed generator, not sure#2019-08-1513:29Jakub HolĆ½ (HolyJak)Yes, that seems likely based on the docs. The question is why the generated data is larger than that....#2019-08-1514:00Alex Miller (Clojure team)not something I have time to look at atm, feel free to file a ticket if needed#2019-08-1518:12zlrthis it possible to make a spec for map that has two keys, and if the value of one key matches, then the value of the other has to match something else? but if the value of the one key doesnāt match, then the spec is valid, regardless of the second value#2019-08-1519:04Alex Miller (Clojure team)can you write a function that says yes or no to that?#2019-08-1519:04Alex Miller (Clojure team)if so, that predicate is now a spec#2019-08-1519:04Alex Miller (Clojure team)specs are limited only to functions ... which can do anything#2019-08-1519:05zlrthah of course--thanks very much#2019-08-1519:05Alex Miller (Clojure team)perhaps a better questions is whether this is a good idea :)#2019-08-1519:05Alex Miller (Clojure team)ime, you're probably doing too much#2019-08-1519:05Alex Miller (Clojure team)don't strangle the code, just check it#2019-08-1609:16metametadataLooking at the latest blog post about spec2: (s/valid? ::s {::f "Bugs" ::l "Bunny" ::x 10} {:closed #{::s}})
.
Does this mean that if ::f
is also a map then in order to close it the user will have to explicitly put it into set too? I.e.: (s/valid? ::s {::f {:a 1 :b 2} ::l "Bunny" ::x 10} {:closed #{::s ::f}})
. If yes, then it seems to be inconvenient for cases with nested maps because when validating a top map I don't want to remember and manually specify the "closed-ness" of all the maps used in validation of every particular key.#2019-08-1610:51Alex Miller (Clojure team)Yes, thatās what it means, but it doesnāt preclude some kind of :all setting#2019-08-1611:54metametadataI see, thanks. I was wondering how easy it will be to migrate my current codebase to this approach. As currently the knowledge of closedness is tied to each spec (I have a custom macro: (sp/speced-keys :closed? true :req ...)
). And "parent" specs are not aware of this. :all
is fine if all the specs are closed but will not work in case some specs are closed and others are open.#2019-08-1612:01metametadatae.g.
::foo = {::a <vector of closed ::x maps> ::b <open ::y map>}
(s/valid? ::foo myfoo {:closed ???})
#2019-08-1612:10Alex Miller (Clojure team)Keep in mind also that s/keys itself is being replaced with s/schema and s/select#2019-08-1612:10Alex Miller (Clojure team)The closed checking is currently implemented only in s/schema, not s/keys#2019-08-1614:26sundbpIāve got this use-case: Iāve got a couple of macros/fns that I use to dynamically create a few āconceptsā and associated helper fns (stuff like serialization/deserialization etc). As part of that Iām also creating specs. Iām looking at spec-alpha2 and getting stuck wanting to do something like this:
(s/register k (s/spec (fn [x] ..something using a captured local value..)
This doesnāt fly, the local value (from e.g. an encapsulating let form) ends up not scope captured as Iād expect and instead breaks everything. Any tips how I can accomplish what I need?#2019-08-1614:29Alex Miller (Clojure team)the (fn [x] ...)
part here is a symbolic form (not evaluated)#2019-08-1614:30Alex Miller (Clojure team)so if you want to use a local value, you need to do something to construct the right symbolic form#2019-08-1614:30Alex Miller (Clojure team)you could for example use
`( ... ~foo ... )
#2019-08-1614:33sundbpah. ok. iāll give that a go. the symbol stuff still doesnāt really compute very well with me - it feels like even though spec2 allows being driven by code (as opposed to manual s/defās), I still donāt find that Iām able to do so doing things the way I expect. The macros still āget in the wayā for me a lot of the time.#2019-08-1614:35Alex Miller (Clojure team)there are no macros involved in your code above. how can they be in the way?#2019-08-1614:35Alex Miller (Clojure team)well I guess s/spec#2019-08-1614:35sundbpso I tried (s/register k (s/spec*
(fn [x] .. ~foo ..))` - no cigar.#2019-08-1614:36sundbpMeant the spec2 macros#2019-08-1614:36Alex Miller (Clojure team)can you share more of your code?#2019-08-1614:37sundbpthe 2nd register is what Iām struggling with.#2019-08-1614:38Alex Miller (Clojure team)what is entity?#2019-08-1614:39Alex Miller (Clojure team)I guess a symbol? kw?#2019-08-1614:40sundbpentity is a kw#2019-08-1614:42Alex Miller (Clojure team)and in how does this not work? error or spec doesn't do what you expect?#2019-08-1614:43Alex Miller (Clojure team)is there some reason you're using s/spec and not s/spec* in last line?#2019-08-1614:43Alex Miller (Clojure team)I think you'd want the latter there#2019-08-1614:44sundbp#2019-08-1614:45Alex Miller (Clojure team)try spec* in that last line#2019-08-1614:45sundbp#2019-08-1614:46sundbp(sorry, for screenshots - I donāt have slack access on a remote machine where the code is and canāt copy across)#2019-08-1614:50Alex Miller (Clojure team)that's actually a bug in spec 2 function explication#2019-08-1614:50Alex Miller (Clojure team)the [x] in the fn is getting ns'ed#2019-08-1614:50sundbpthank god for that - I was starting to run out of ideas š#2019-08-1614:51Alex Miller (Clojure team)well, maybe#2019-08-1614:51sundbpcan I work around that?#2019-08-1614:51Alex Miller (Clojure team)in that fn, instead of x
can you do ~'x
everywhere?#2019-08-1614:52Alex Miller (Clojure team)I'm not sure if the error here is in your expansion or inside spec 2#2019-08-1614:53sundbpcool. that compiles at least, so hoping that means Iām on my way. wont be able to confirm overall functionality without some more work.#2019-08-1614:53Alex Miller (Clojure team)basically you're fixing this issue:#2019-08-1614:53Alex Miller (Clojure team)user=> `(fn [x] (+ x 1))
(clojure.core/fn [user/x] (clojure.core/+ user/x 1))
#2019-08-1614:54Alex Miller (Clojure team)I have seen places where spec 2 has trouble with this, but here I think it's in your expansion#2019-08-1614:54sundbpmore context#2019-08-1614:55Alex Miller (Clojure team)the code you now have is not using any macros and there is no "magic" here - it's just passing forms to functions#2019-08-1614:57sundbpI think that macro+adapt-spec wrapper was a trial and error solution to similar issue I had with passing in a fn (e.g. string?). if I already had the fn and not the symbol pointing to the fn at the time I want to s/register
I couldnāt work out any way to proceed.#2019-08-1615:10sundbpbtw, that seems to get me back on track and define the sort of spec I have in mind. thanks a lot for the help - and let me know if you want any more info in terms of it being me vs spec2.#2019-08-1615:46Alex Miller (Clojure team)if you already have a function object, then you can't go through the symbolic API (spec*)#2019-08-1615:46Alex Miller (Clojure team)as it's not a symbol#2019-08-1615:46Alex Miller (Clojure team)but you can directly create a spec object that implements the protocol#2019-08-1615:47Alex Miller (Clojure team)w/the caveat that you're not going to be able to s/form or nest that kind of thing in other symbolic forms#2019-08-1615:47Alex Miller (Clojure team)which is true to some degree in spec 1 too#2019-08-1616:29sundbpyeah. compared to a more data driven API both spec1 and spec2 throws up some speed bumps if one is generating specs by code. but spec2 seems to have enough flex to make it possible to get what I need even if itās not yet intuitive to me to work in the symbolic space (as compared to more data composition).#2019-08-1616:29sundbpso Iām a bit split on the API so far. it doesnāt feel like ānormalā clojure when one comes at it from generating specs in code. however, it does compose fine as long as youāre manually specāing things. net-net Iād love a more data-driven approach. Iāve seen https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md and perhaps what Iām looking for is doable on top of the symbolic api.#2019-08-1617:30Alex Miller (Clojure team)well there is now a map-focused format as well (as of last week)#2019-08-1617:30Alex Miller (Clojure team)that is almost certainly going to change formats so I wouldn't touch it yet#2019-08-1617:33Alex Miller (Clojure team){:clojure.spec/op `s/keys
:req (conj (map first required) ::entity-type)
:opt (map first optional)}
#2019-08-1617:33Alex Miller (Clojure team)is something you could pass to s/spec* for example and get the identical spec#2019-08-1617:34Alex Miller (Clojure team)and there is an s/expand-spec that can do form->map#2019-08-1617:34Alex Miller (Clojure team)but this should be much more amenable to doing spec->map->transform->spec type things#2019-08-1617:37Alex Miller (Clojure team)there are now 3 or 4 different ways to tap into spec creation/transformation and we're looking at some meta stuff on top of that which might make some of them more accessible, but really there's a lot of options now#2019-08-1820:33sundbp@alexmiller cool. yeah, Iām not saying there isnāt ways to hook things up. For manual use, which obviously is the major use case, things have felt natural and I havenāt had to read source etc. For manipulating specs programatically Iām convinced all I want is doable, but it hasnāt felt ānaturalā and Iāve had to read the implementation to try to grok the explicate/spec/spec* machinery. The map syntax you hint at looks likely to be more intuitive to me, so I like that. Thanks for both helping me across my stumbling block and for keeping on improving spec!#2019-08-1823:28cflemingWhat are the current workarounds for closed spec-like behaviour? Iām trying to retrofit specs to some existing code, and Iād like to ensure that I havenāt missed any cases during development.#2019-08-1823:28cflemingIn particular, how can I specify a map that should only have the keys Iāve specified?#2019-08-1823:46taylorhereās an example https://github.com/gfredericks/schpec/blob/b2d80cff29861925e7e3407ef3e5de25a90fa7cc/src/com/gfredericks/schpec.clj#L13#2019-08-1823:49cflemingAwesome, thanks!#2019-08-1823:42cflemingAlso, Iād like to spec a map which has some standard keyword keys, but also might have string keys with values specified by a spec. Can I do that? Neither s/keys
nor s/map-of
seem to fit that.#2019-08-1823:47Alex Miller (Clojure team)You can, but not easily. You can do an s/coll-of :kind map? and then spec the possible mapentry types#2019-08-1823:48Alex Miller (Clojure team)Itās tedious#2019-08-1823:50Alex Miller (Clojure team)Another path (if you care less about gen) is to use a conformer to keywordize the map then s/keys#2019-08-1823:50cflemingYeah, I donāt care about gen in this case.#2019-08-1823:50cflemingJust trying to spec an existing system.#2019-08-1900:07cflemingThis seems to work:
(s/def ::entry (s/or :project (s/cat :file string? :data ::project)
:profiles (s/cat :key (s/and keyword? #(= (name %) "profiles"))
:data (s/coll-of string? :kind vector?))))
(s/def ::projects (s/coll-of ::entry :kind map?))
#2019-08-1900:07cflemingNot pretty, but it does the trick.#2019-08-1900:24metametadata@cfleming there's a support for closed maps in spec-tools
(https://github.com/metosin/spec-tools/blob/master/CHANGELOG.md#092-alpha1). Also I posted my take a while back: https://gist.github.com/metametadata/53a847cd3b02056e8e4c124e63d9ae5a.#2019-08-1900:26cfleming@metametadata Thanks! Iāll try yours out since it looks like the error messages might be more helpful.#2019-08-1900:32metametadatacool! we use an evolved version in company projects (which e.g. is more convoluted because of https://github.com/metosin/compojure-api/issues/417), but didn't have time to publish it yet#2019-08-1914:32NickIām trying to write a spec for a payload wrapper map.
For simplicity, the spec Iām trying to write is for a map with two keys, :query
and :payload
.
For the purposes of this example, Iām using extremely simple examples of the payload with just maps of one key.
`
(s/def ::query (and keyword? #{:avalue :anothervalue :thirdvalue}))
(s/def ::email string?)
(s/def ::user-id integer?)
(s/def ::avalue (s/keys :req-un [::email]))
(s/def ::anothervalue (s/keys :req-un [::user-id]))
{:query :avalue
:payload {:email ā<mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>ā }}
{:query :anothervalue
:payload {:user-id 123}}
`
I can use multi-spec and a defmulti to key off the :query object,
but that doesnāt let me use the shared :payload
keyword. Is the only way around this to define ::avalue and ::anothervalue in different namespaces ::avalue/payload ::anothervalue/payload? I havenāt created a lot of namespaced keywords in clojure yet.#2019-08-1914:39Alex Miller (Clojure team)yes, you would need to use different namespaces in this case#2019-08-1914:39Alex Miller (Clojure team)the ::query spec is redundant btw, just (s/def ::query #{:avalue :anothervalue :thirdvalue}) would be sufficient#2019-08-1914:43NickDo I need to create a separate valid namespace, or can I define the specs with a a namespace that doesnāt correspond to an existing clojure file? ie, does there need to be a (ns anothervalue)
somewhere before I can do a (s/def ::anothervalue/payload string?)
#2019-08-1914:51Alex Miller (Clojure team)technically here, it's better to use the term "qualifier"#2019-08-1914:52Alex Miller (Clojure team)which may correspond to an actual namespace, but doesn't have to#2019-08-1914:52Alex Miller (Clojure team)you can use a keyword with any qualifier#2019-08-1914:54NickThanks, that's quite helpful.#2019-08-1914:55Alex Miller (Clojure team)a lot of docs in Clojure are not particularly clear on this point#2019-08-2019:29johanatanis this guide still the latest info on spec? https://clojure.org/guides/spec#2019-08-2019:29johanatanhave things changed sufficiently that one should follow a different guide?#2019-08-2019:30johanatanit still seems almost identical to ~1.5-2 years ago when I first used spec and I thought things were changing dramatically for v2#2019-08-2019:30Alex Miller (Clojure team)v2 has not been released yet, so that is still up to date for v1#2019-08-2019:31johanatanhm, are there certain things we could do to "future proof" our specs in prep for v2 ?#2019-08-2019:31Alex Miller (Clojure team)given how things are in flux, I would not worry about it much yet#2019-08-2019:31Alex Miller (Clojure team)but you can read more about v2 at https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha and https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-08-2019:31johanatanok, thanks!#2019-08-2121:02johanatani have the following deps in my deps.edn:
{:deps {org.clojure/clojure {:mvn/version "1.10.0"}
org.clojure/clojurescript {:mvn/version "1.10.520"}
...
...
and yet i'm getting the following errors when trying to use clojure.spec.test.alpha:
Use of undeclared Var clojure.spec.test.alpha/instrument
5 [clojure.spec.alpha :as s]
6 ;[clojure.spec.test.alpha :as stest]
7 [clojure.test.check.generators :as gen]))
8
9 (defn instrument-all-syms []
10 (clojure.spec.test.alpha/instrument (clojure.spec.test.alpha/instrumentable-syms)))
^---
from figwheel-main#2019-08-2121:02johanatanUncommenting line 6 causes a cannot resolve dependency error#2019-08-2121:02johanatanis there something else that has to be done to get clojure.spec.test.alpha
to be available?#2019-08-2121:03johanatanlooks like this: https://cljs.github.io/api/cljs.spec.test.alpha/instrumentable-syms indicates that it should be there#2019-08-2121:04johanatan[i have tried subbing cljs
for clojure
as well]#2019-08-2200:08seancorfield@johanatan Is there a reason you're calling instrumentable-syms
there, rather than just (stest/instrument)
? The latter seems to work with figwheel, based on my limited testing, but the former seems to behave strangely (based on my very limited testing).#2019-08-2201:57johanatan@seancorfield it was just copypasta from a previous project. it was the first thing that worked. no particular reason.#2019-08-2201:57johanatandoes (stest/instrument)
have the same behavior as I'm attempting to get there?#2019-08-2202:24seancorfieldIt instruments all functions... which should be the same as all instrumentable-syms but maybe @alexmiller can shed light on that?#2019-08-2202:54johanatan[btw, that change did get my code to compile, strangely]#2019-08-2320:11johanatanis there a way to ask spec: "give me all embedded values in the nested map satisfying spec ::x
?" (either in the std alpha or in 3rd party extensions). I'm thinking it wouldn't be too hard to do this with specter + spec but just wondered if something like it already exists...#2019-08-2320:20Alex Miller (Clojure team)not in spec#2019-08-2320:20Alex Miller (Clojure team)but it's just something like (defn deep-match [s x] (->> x (tree-seq coll? seq) (filter #(s/valid? s %))))
#2019-08-2320:20johanatanš yea. cool, thx#2019-08-2420:07tylerAre unqualified keywords not supported for closed specs? Closed specs seem to work as expected with namespaced keyword specs but the unqualified keyword form doesnāt seem to work with closed.#2019-08-2420:14johanatanare you referring to s/keys
req-un
or opt-un
specifically? are there other aspects of specs that support "unqualified" ?#2019-08-2420:26tylerRefering to the examples here https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#literal-schemas the unqualified form doesnāt seem to be recognized by the closed option.#2019-08-2420:26tylerThis is using schema, not s/keys
#2019-08-2420:33johanatanAh, specs2 gotcha . Specs1 didnāt support unqualified except via keys
#2019-08-2420:58Alex Miller (Clojure team)No, not supported right now#2019-08-2615:33unbalancedokay so I'm doing this pattern a lot and I'm wondering if there's a more concise way to do it, perhaps to register a s/def
with another s/def
at creation time?
(s/def :platform.confirm/display-attribute keyword?)
(s/def :platform.confirm/temp-attribute string?)
(s/def :platform.confirm/id string?)
(s/def :platform.confirm/value string?)
(s/def :platform.confirm/edit-attribute keyword?)
(s/def ::confirm (s/keys :req [:platform.confirm/display-attribute
:platform.confirm/temp-attribute
:platform.confirm/id
:platform.confirm/value
:platform.confirm/edit-attribute]))
#2019-08-2615:34unbalancedand, while I'm at it ... I'm wondering if anyone has any best practices in combining spec with datomic? Since we already go to all the trouble of making the schema for datomic, has anyone looked into using that for spec?#2019-08-2617:26seancorfield@goomba This popped up as the second result on a Bing search for using clojure.spec to create datomic schema
for me: https://github.com/Provisdom/spectomic -- seems to be what you're looking for?#2019-08-2617:27unbalancedoh man, and great name too#2019-08-2617:28unbalancedthat's interesting ... they went the opposite way I would've expected, i.e., s/def
-> schema#2019-08-2617:28unbalancedI would've expected schema-ish -> s/def#2019-08-2617:29seancorfieldThat makes sense to me. We treat spec as the "system of record" and generate stuff from it.#2019-08-2617:29unbalancedmakes sense. I'm usually just paranoid about "losing" my schema.#2019-08-2617:30unbalancedI like to see them all written out explicitly. But that would be easy enough with eval-and-replace#2019-08-2617:30seancorfieldI recently answered a Quora question about using clojure.spec
https://twitter.com/seancorfield/status/1164714132647530496 (don't know if you can read the answer without a Quora account?)#2019-08-2617:30kenny@goomba clojure.spec can express more info than Datomic schema can so it makes more sense to go from spec -> schema.#2019-08-2617:30unbalancedwheeee yess! That was another question I had!#2019-08-2617:31unbalanced@kenny solid point. I was just expecting something more like a hybrid spec/schema, uh... spec#2019-08-2617:31unbalancedhave it all in one place#2019-08-2617:32unbalancedthis is totally fine though#2019-08-2617:33unbalancedalso got a lot of good answers out of it and experienced @seancorfield do a "let me Bing that for you" which is fantastic š#2019-08-2617:35seancorfieldI hope you didn't think I was being mean with my comment about Bing? You've said you're sometimes not finding search results for stuff so I figured sharing the search string I used might be helpful. Quite a few of my friends ping me about finding stuff because apparently I'm better at searches than them... š#2019-08-2618:20unbalancedno hahaha I thought it was hilarious š I loved it#2019-08-2617:33Alex Miller (Clojure team)I've seen people do both directions, and also start from a 3rd "source of truth" and generate the Datomic schema and spec too#2019-08-2617:33unbalancedmmm chimeratomic#2019-08-2617:39kennyAttaching schema data directly to the specs is something that interested us too @goomba. From what I remember when we wrote that lib, it wasn't immediately obvious what the best path to doing that would be. Ideally it would've been something simple like attaching metadata to the spec's name (e.g. (def ^{:db/isComponent true} my-spec map?)
). That wouldn't work for specs because they are typically defined as keywords which do not accept metadata.#2019-08-2617:42kennyThere were 2 other options IIRC. 1) Create a new s/def which accepts a new 3rd parameter. 2) add a function similar to s/with-gen
which would attach metadata to the spec itself.#2019-08-2617:42unbalancednod... in my lazy imagination it would be something like
{:db/ident :boot.user/string
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one
:spec/id ::string
:spec/spec string?}
and then specs and schema would be gen'd from that. But listen I'm a lazy application layer dev asking for handouts, what you did is great it's just not what I would've thought of immediately#2019-08-2617:43kennyI mean, that's basically a complete schema map.#2019-08-2617:43unbalancedI'm not even sold on the :spec/spec
with a function there because now that's not even a value anymore#2019-08-2617:44unbalancedyeah I'm excited to check out spectomic!#2019-08-2617:44unbalancedI do a ton of datascript/spec work so this is great#2019-08-2617:49kennyI haven't looked at this at all but I'm assuming that this sort of thing would be easier to do with Spec2, given Spec2 allows specs to have docstrings.#2019-08-2619:18ikitommihello all. is there a reason that pos?
and neg?
donāt have built-in generators?#2019-08-2619:28Alex Miller (Clojure team)yes, they are not specific to one numeric type#2019-08-2619:28Alex Miller (Clojure team)use (s/and int? pos?)
or whatever numeric type you need#2019-08-2619:39ikitommiok, thanks!#2019-08-2619:51Alex Miller (Clojure team)or something like pos-int?
#2019-08-2715:44joefromctHi,
Iāve read the spec guide and couldnāt really come up with the correct way to do the following; Apologies if i had missed something.
I am looking to define a spec for a ::paragraph that consists of ā::wordsā , ā::space(s)ā , and āless oftenā ::punctuation. It (of course) does not need to be grammatically correct.
It seems like i need something like s/cat however i want the results to be not a vec but a larger string (that consists of smaller āspecā strings...ā).
Iām interested in having a spec for ::paragraph as well as a generator to play with a natural language processing app iām working on.
Any tips appreciated,#2019-08-2820:03ataggartI managed to do something like that by using conformer
to tokenize the string first. E.g.:
(s/and string?
(s/conformer #(str/split % #",") #(str/join "," %))
(s/cat ...))
But it wonāt be as fast as a real parser. #2019-08-2820:04ataggartTyped that on my phone from memory, so apologies if I got things incorrect. #2019-08-2715:57Alex Miller (Clojure team)I think youāre probably asking for more out of spec than it is designed to provide #2019-08-2716:01joefromctok, that makes sense. thanks, iāll come up with something.#2019-08-2720:37vemvIs there a version (in some library, etc) of s/keys
that allows me do manipulate the spec definition before proceeding?
My current pattern for that is:
(eval `(spec/keys :req ~(conj dependencies ::foo-options)))
...Generally it works fine, but now my needs also include clojurescript, so I can't use eval
#2019-08-2721:57Alex Miller (Clojure team)I don't think there is an easy pattern for it in spec 1. CLJ-2112 has a sketch of specs for specs which lets you conform, modify then unform, but it's pretty cumbersome due to the structure of s/keys.#2019-08-2721:58Alex Miller (Clojure team)spec 2 has several ways to do this and a new symbolic map form in work in particular#2019-08-2723:16vemvAppreciate the confirmation, thanks!#2019-08-2721:38johanatanwhat would the regex to match "one or more forward slashes" be?
#"\/+"
doesn't seem to compile or work#2019-08-2721:38johanatanalso tried several other permutations#2019-08-2721:40johanatanthis seems to have worked: (js/RegExp. "\\/+")
#2019-08-2721:40johanatanbut pretty sure #"\\/+"
did not#2019-08-2721:54jahsonuser=> (re-seq #"\/+" "///////asd////")
("///////" "////")
It definetely compiles.#2019-08-2804:44johanatanhuh? brain fart apparently. this works:
cljs.user=> (clojure.string/replace "/////blah/blah/blaz///" #"\/+" "/")
"/blah/blah/blaz/"
cljs.user=>
#2019-08-2818:12johanatanhmm, ^^ that was in lumo.#2019-08-2818:12johanatanwhen it's in my source file and compiled by figwheel-main I get the following js error:
Uncaught SyntaxError: missing ) after argument list
#2019-08-2818:12johanatanso, yea, a compilation error apparently#2019-08-2818:14johanatanand i just realized this is the very wrong channel for this topic lol#2019-08-2818:14johanatan[sorry about that]#2019-08-2819:37chancerussellI noticed that alpha2 supports s/def ing symbols instead of keywords and got really curious about itā¦then realized that the first version supports the same thing#2019-08-2819:38chancerussellReally curious about the use case for that though#2019-08-2819:40Alex Miller (Clojure team)that's how function specs are registered (s/fdef)#2019-08-2819:40chancerusselllol duh#2019-08-2819:40chancerussellthanks#2019-08-2822:46agcan someone throw at me with an example of a spec for a string that has to be of specic length (within range) and should only allow letters. The catch though: it should not fail generation with Couldn't satisfy such-that predicate
message.
I swear, I used to have those, somehow Iām feeling stupid and canāt make them anymore.#2019-08-2822:51Alex Miller (Clojure team)you could use test.chuck's support for making generators from a regex#2019-08-2822:54agah test.chuck. Yeah it has a few nice gems there. But I feel Iām missing something fairly small#2019-08-2822:59agmy best effort for now is to use with-gen
with gen/such-that
with max-tries
jacked-up to a huge number. But it seems to be slow and still not guaranteed to not to fail to satisfy#2019-08-2823:05seancorfield@ag Another "vote" for test.chuck here since you can use a (string) regex such as #"[a-zA-Z]{1,32}"
for example, for pure alpha strings up to 32 chars in length and test.chuck provides a generator that works out of the box. We use it to generate email addresses and a bunch of other stuff.#2019-08-2823:11seancorfieldFYI https://www.dropbox.com/s/tvvby5apbzyra7j/Screenshot%202019-08-28%2016.11.34.png?dl=0#2019-08-2823:14agLOL#2019-08-2823:12agOn related note: Gary is awesome! He was at defn podcast (that I started listening but couldnāt finish this morning)#2019-08-2920:06joefromctis there a way to make a recursive spec to generate the output from clojure.data.xml ?#2019-08-2920:07joefromctnot sure what iām doing wrong there#2019-08-2920:09Alex Miller (Clojure team)you might try limiting the collection sizes#2019-08-2920:10Alex Miller (Clojure team)(s/def ::attrs (s/map-of keyword? string? :gen-max 3))
#2019-08-2920:10Alex Miller (Clojure team)(s/def ::content (s/coll-of ::a-content :kind vector? :gen-max 3))
#2019-08-2920:11Alex Miller (Clojure team)oh, it's probably that the unqualified keys are foiling the recursion limiter#2019-08-2920:11Alex Miller (Clojure team)try it with :req and see if that works#2019-08-2920:14joefromcthmm.. no luck with req and gen-max 3
(s/def ::tag keyword?)
(s/def ::attrs (s/map-of keyword? string? :gen-max 3))
(s/def ::a-content (s/keys :req [::tag ::attrs ::content] ))
(s/def ::content (s/coll-of ::a-content :kind vector? :gen-max 3))
(->> ::content
s/gen
gen/generate
(binding [s/*recursion-limit* 1]))
;;=> Execution error (StackOverflowError) at clojure.test.check.generators/tuple
;;(generators.cljc:534).
#2019-08-2920:27Alex Miller (Clojure team)We have some open tickets about SO on recursive gen. I have not dug into any of them recently#2019-08-2920:33joefromctok sounds good, iāll keep an eye out.#2019-08-2920:34joefromcti appreciate how you always try to respond when iām sure your busy with strange loop very soon.#2019-08-2920:34joefromcti hope i donāt miss it this year because iām stuck in a tube in the museum. j/k#2019-08-2920:35Alex Miller (Clojure team)me too :)#2019-08-3005:10johanatandoes spec-alpha2 support clojurescript?#2019-08-3012:23Alex Miller (Clojure team)No, we will probably hold off on that till churn slows down #2019-08-3019:25johanatanthx#2019-08-3008:31hkjelsGenerating from the spec: (s/inst-in #inst "1984" #inst "2019")
will just return dates from 1984
Whatās the idiomatic way of generating random dates within a range?
#inst
only works with string literals and not computed strings#2019-08-3012:24Alex Miller (Clojure team)Samples are biased to beginning of the range, but āgrowā as you make more#2019-08-3012:24Alex Miller (Clojure team)So if you sample a larger number youāll see more variety #2019-08-3010:06mpenet(java.util.Date. x) with some random number within the range#2019-08-3010:47hkjelsproblem was actually the string I created. Your absolutely correct @mpenet#2019-08-3120:30seancorfield@j.m.frith Can you share the spec and the function that you're having problems with? (perhaps in a cut down version if it's a lot of code)#2019-08-3120:32JazzerThanks @U04V70XH6 I'm just working something up. In the actual code, the problem bit is fairly deep in a map, so I'm going to try to simplify#2019-08-3120:32JazzerBack in a few minutes...#2019-08-3120:31JazzerHi everyone, I'm getting issues when running stest/check
while a function is instrumented. It seems to be conforming the input before passing it to the function. I'll try and condense my code into a minimal example...#2019-08-3120:31JazzerHi everyone, I'm getting issues when running stest/check
while a function is instrumented. It seems to be conforming the input before passing it to the function. I'll try and condense my code into a minimal example...#2019-08-3120:37JazzerNuts! My minimal example has no problems#2019-08-3120:37JazzerI'll try expanding the code until I see an issue#2019-08-3120:39JazzerThe symptoms I'm seeing are: if I run (stest/unstrument)
and then (stest/check 'my-fn)
(not sure how to write a backtick in code but my-fun is quoted with a backtick not a single quote) I get tests all passing#2019-08-3120:41JazzerIf instead I run (stest/instrument)
followed by (stest/check 'my-fn)
then I get errors that the input has not conformed to spec.#2019-08-3120:42Alex Miller (Clojure team)Could you just share the error?#2019-08-3120:45Jazzer#2019-08-3120:46JazzerI'm not sure that's quite as helpful as it might be, which is why I'm trying to scale down to get to the root of the problem#2019-08-3120:47JazzerThere's a whole bunch of data in there which isn't very helpful#2019-08-3120:49JazzerAlthough right at the front of line 4 it's the part [:decision #:log{:decision {}}]
this is definitely a conformed value - the spec has an (s/or ...)
with :decision being one of the options#2019-08-3120:49seancorfield@j.m.frith Are you sure that function works with instrumentation on when called manually from the REPL?#2019-08-3120:50JazzerOne moment, will re-try now#2019-08-3120:51JazzerIt's working fine for me now#2019-08-3120:51JazzerAnd it gives and returns a map as expected without the conform path#2019-08-3120:53JazzerHave also just checked that (gen/generate ...)
gives me a map that conforms fine to the spec.#2019-08-3120:54JazzerI think it'll be easiest if I take a bit of time to whittle away all the nonsense so that things are a bit easier to follow.#2019-08-3120:57seancorfieldOne approach that I strongly recommend when working with Spec is to write and check each spec as you write each function -- that way you don't get a bunch of code for which you are trying to write a bunch of specs all at once and then testing all at once. The REPL allows you to take small steps and get feedback immediately on each piece of code as you write it.#2019-08-3120:59seancorfieldFor example, I have a (comment ,,,)
block in each file where I'm working on code + spec and I have (s/exercise ,,,)
calls in that comment that I eval to check each spec will generate (not all specs need to generate but I think it's important to check the ones that should as you write them).#2019-08-3120:59seancorfieldThat "Rich Comment Form" also includes instrument and check calls as needed.#2019-08-3121:01JazzerThat is definitely good advice, which I should have taken before getting to this point. I was too eager to learn the language first and built a working prototype and am now going back to learn how to use spec.#2019-08-3121:02seancorfieldAlso bear in mind that "nothing everything needs a spec" š Spec is great for boundaries and for "key functions" that you really need to be sure of. Spec isn't a type system, so you don't need a spec on every function.#2019-08-3121:03seancorfieldAt work, we've used Spec since the first alpha builds in the 1.9 cycle and we mostly spec data, rather than functions. We use s/conform
and s/invalid?
in production code (and s/explain-data
to help drive error message generation).#2019-08-3121:04JazzerI guess I have a "base layer" of functions which directly interact with my state-map (in a functional way) and I'm currently only speccing those functions. The higher layers I may get to at some point#2019-08-3121:04JazzerAnd the spec is mostly so that I can run generative tests on that portion of the code#2019-08-3121:05seancorfieldWe use instrument
to support development work, mostly. We use check
sparingly.#2019-08-3121:05JazzerThe problem is that one part of the system now has an (s/or
in the spec and that is throwing off the earlier function which worked alright when I didn't have that in the spec...#2019-08-3121:07JazzerAgain, that sounds about right. I was really using check just to try and find edge cases. Maybe I should stick to either using instrument or check at a time. I'm getting no problems at all when using one but not the other#2019-08-3121:08seancorfieldI'll be interested to see the cut down example that breaks because of s/or
-- I've never run into that (in several years of working with Spec).#2019-08-3121:12JazzerI'll work it out. It seems even stranger at the moment because the spec is actually saying it's failing on a function that shouldn't even be called. It's failing on game/player-count
and I'm checking game/set-lead-player
. set-lead-player
has two lines, neither of which call player-count
. It looks like I've got a bit of digging around ahead of me... I'll be back in the next few days once I've simplified things down#2019-08-3121:14seancorfieldIf you instrument
a function that takes a function as an argument, and you have a fspec
for that argument, it will be generatively tested when it is called.#2019-08-3121:15seancorfield(that sometimes surprises people)#2019-08-3121:16JazzerI don't think that is the case here, but that is good to know. I have to head out now, but I'll be back š#2019-08-3121:17JazzerThank you for your time - I have to say that I'm impressed both by clojure and the community here#2019-08-3123:46JazzerI have got almost to the bottom of this! The issue was that I was calling a function within the spec definition, and not realising that spec passes a conformed value into whatever you give
Code looks like this:
(s/fdef game/set-lead-player :args (s/and
(s/cat :game :game/game
:lead-player :player/player-no)
#(<= (:lead-player %) (game/player-count (:game %))))
:ret :game/game)
I hadn't realised that (:game %)
returns a conformed value of the argument passed in, rather than just passing on the value itself#2019-08-3123:49JazzerTwo questions would follow from this:
1. Is there any way to use a function in this way and pass in the original argument, rather than a conformed value
2. Should I in fact be avoiding this conflation of spec and function? i.e. any specs should depend purely on the data they are presented (and core clojure functions)#2019-08-3123:53JazzerAs to my question 1 above, the answer is s/unform
. Worked perfectly with that in there. Yay! Answered my own question š#2019-09-0100:24seancorfieldWhen you're writing compound predicates like that, it's expected that you take into account the shape of the conformed value -- calling s/unform
is kind of a sledgehammer...#2019-09-0100:25seancorfieldI'm a bit surprised you need a function to get the player count from a game tho'... isn't that just an element in the game's map? Or are you calculating it on every call from something inside the game map?#2019-09-0100:28seancorfieldBut your :game/game
spec has s/or
at the top-level? What are the alternatives there? (sorry if you posted it elsewhere, I'm only looking at this thread right now)#2019-09-0111:39JazzerThanks @U04V70XH6. Agreed that s/unform
seems overkill, I just thought Iād share that it does indeed achieve my (probably misguided) aim of being able to use the original argument rather than a conformed version.#2019-09-0111:41JazzerThe reason for the function is that, as I developed things, I kept changing up the structure of the map. Rather than having to change everywhere in the code which refers to this element, I made a function to pull it out and then only need to update that one method if/when I changed my mind again.#2019-09-0111:46JazzerAnd no, the :game/game
spec at the top level is just s/keys
. One such key is :game/log
which is a s/coll-of :log/log-entry :kind vector?
and finally :log/log-entry
is where the s/or
happens to choose between various different types of log entry#2019-09-0111:47JazzerSome log entries are for player actions, others are for rules being applied and each has a different structure.#2019-09-0312:04Vesa NoriloHi guys! Is there a way to destructure and walk specs? Suppose I have (update-in document [some path] transform)
. Is there a way to derive a spec for transform
output by reaching into the spec for document
?#2019-09-0312:28Alex Miller (Clojure team)Not right now#2019-09-0312:53Vesa Noriloright, thanks!#2019-09-0418:01johanatani have a function from ::a
-> ::b
and i am testing it with stest/check
. i have set :num-tests
to 1 via: (stest/check 'the-sym {:clojure.spec.test.check/opts {:num-tests 1}})
. does it make sense that i would see two ::a
s generated?#2019-09-0418:03johanatan[i've verified that both generated ::a
s are non-trivial/non-empty so it shouldn't be stest
peeking at the first and considering it "not good enough" then proceeding to the second].#2019-09-0418:07johanatanhmm... i guess there could be a hidden such-that
or some other "retry" mechanism behind the scenes.#2019-09-0418:07johanatanhmm... i guess there could be a hidden such-that
or some other "retry" mechanism behind the scenes.#2019-09-0418:07taylorgot a small repro example?#2019-09-0418:13Alex Miller (Clojure team)you may see it gen more than once#2019-09-0418:14Alex Miller (Clojure team)even in a single test#2019-09-0511:28pvillegas12I have a spec which reuses another, something like
(s/def ::a
(s/and #(s/valid? ::sub-a %)
(s/keys :req [:a/only-attribute]))
#2019-09-0511:29pvillegas12However, my problem is that s/explain
does not show the ::sub-a
spec violations, it just says that ::sub-a
is invalid#2019-09-0511:29pvillegas12is there a way for spec to āwalkā down ::sub-a
in this case?#2019-09-0511:33pvillegas12I could, of course, copy over the (s/keys)
in ::sub-a
to the spec of ::a
but I was looking for a solution that reuses the ::a
declaration#2019-09-0512:01Alex Miller (Clojure team)Right now, this is what I would recommend. Much better options coming for this in spec 2#2019-09-0512:10pvillegas12I actually got good error messages by doing
(s/def ::a
::sub-a
(s/keys :req [:a/only-attribute]))
#2019-09-0512:10pvillegas12It this bad @U064X3EF3?#2019-09-0512:22Alex Miller (Clojure team)That looks invalid, what are you seeing?#2019-09-0512:26pvillegas12Iām actually getting the error messages for ::sub-a
should that not work?#2019-09-0512:41pvillegas12@U064X3EF3 got inspiration from https://gist.github.com/robert-stuttaford/e1bc7bfbcdf62020277dda1a277394ca#2019-09-0512:41pvillegas12https://gist.github.com/robert-stuttaford/e1bc7bfbcdf62020277dda1a277394ca#file-keys_with_and_question-clj-L13#2019-09-0511:48Leonhow can i describe a map where a key could either contain a normal value ::some-object
or, in an error-case, an error-value?
I don't want to add the error-case to ::some-object
as that would greatly decrease the expressiveness and reliability,...#2019-09-0511:59Alex Miller (Clojure team)Spec is about expressing the true range of values. If it can contain special error values, then that is part of that truth #2019-09-0512:00Alex Miller (Clojure team)If thatās weird, you might consider using an alternate attribute for the error or using exceptions#2019-09-0512:03Leonexceptions are an extremely ugly solution (at least for me, coming from ADT Either-type land)... and my data cannot contain errors, but the place they're transmitted can. i guess i'll create a wrapper object that can be either of type success or of type failure, and then contain the values respectively#2019-09-0512:04Alex Miller (Clojure team)Exceptions are the idiomatic solution on the jvm and in Clojure#2019-09-0512:06Leoni guess thats an idiom i dislike ^^ i really hate the complexity that braking out of my control flow adds, especially when otherwise working with pure functions that are easy to reason about ;D
and in my case exceptions aren't really a simple option, as im working in clojurescript with re-frame, where im dispatching events and then dispatching other events on success#2019-09-0512:49potetmProblem is: if you go down the path of āexceptions are bad,ā youāre fighting not just the clojure ecosystem, but the Java(Script) one. Leveraging host libs is one of the primary reasons clojure(script) exists at all!#2019-09-0512:51potetmNot to say you canāt write clj(s) w/o host libs. Many try. But know that youāre giving up a primary value-add for the lang.#2019-09-0512:53potetmMany have also tried to improve upon exceptions (e.g. slingshot). Iāve found them to be marginal gains at best. Not worth their overhead.#2019-09-0514:01Leonyea i know, i don't have any direct issue with exceptions, i just really hate using them where not necessary. and as i said, in my case, they are pretty much not a solution, as the one observing the result isn't the same thing that calls the thing that could throw. for such cases, where error-data makes more sense than exceptions, some solution would really be great. but with the current a spec only talks about the keys it gets, the type of the values HAVE to be defined as part of that key really makes something like this nearly impossible, and strongly increases the complexity in knowing what shape your data actually has, because i CANNOT actually spec it without introducing additional complexity (and removing a hell of a lot of expressivity) to my domain model#2019-09-0514:04Alex Miller (Clojure team)if your data can contain error values, then spec'ing it means spec'ing those values. spec is just exposing that complexity that you have introduced#2019-09-0514:10Leonmy data cannot contain errors. a book
doesn't contain any error. a find-book-response
can contain that error. the big problem is that spec forces me to define find-book-response
specs for everything, and by that introduces an additional layer of complexity where something like generics could really help.
the whole notion of needing to have a spec for a key in a map makes many things really complicated for no good reason.
say you have a utility function, that takes a (s/keys*)
map as arguments (maybe it does some complex math where you want the arguments to be named for readability.
now you need to create spec defs for each int?
that the function can take, thus adding extreme amounts of complexity where a
(s/kv :req {:num1 int?, :num2 float?})
would solve your problem perfectly.#2019-09-0514:12Leonenforcing best practices is a good thing, don't get me wrong. but if these best practices only apply in some areas, but you enforce them in every little situation, that really screws up what could have been an incredibly powerful and nice solution to safety and type-validation in a fully dynamic environment#2019-09-0514:12Alex Miller (Clojure team)I don't understand what's forcing you to define anything#2019-09-0514:13Alex Miller (Clojure team)keys specs are open - you don't have to spec every attribute#2019-09-0514:15Leonsay you have some complex, user-input based form, where the data doesn't really mean anything specific from a domain-model point of view.
if you want to validate that data in a map context, you'll need to define seperate, single use specs for each entry in that map that you want to validate.#2019-09-0514:15LeonI'd say that "just don't spec things that would take to much effort" isn't a desirable solution.#2019-09-0514:16Leonnot only does your safety suffer from that, but it also defeats the purpose of spec in general.
because then you could just say "just manually write (if (not (int? my-number)) (throw Exception. "foo"))
,... thats not the point#2019-09-0514:16Leonthe point is to make validations easier and expressive#2019-09-0514:17Alex Miller (Clojure team)well yes, but not for every purpose#2019-09-0514:17Alex Miller (Clojure team)given that these are not for your particular domain, but a truly dynamic problem, you're a bit off the goals of spec#2019-09-0514:18Alex Miller (Clojure team)you certainly could use a pairing of fields and specs to generically validate (but don't validate the aggregate map)#2019-09-0514:18Alex Miller (Clojure team)then no registration is needed#2019-09-0514:19Leonstill, what is the actual reason there is no spec-function that takes keys ( be they namespace qualified or not) and their types (predicates, etc) and validates a map against them?
is it just to encourage good practises?#2019-09-0514:19Alex Miller (Clojure team)I don't know that I explain it better than Rich, who has written/talked about this extensively#2019-09-0514:19Leoncould you give me a summary?#2019-09-0514:20Alex Miller (Clojure team)the whole idea is to imbue semantics to attributes, and register those for use globally#2019-09-0514:20Alex Miller (Clojure team)and to de-emphasize the role of the aggregate#2019-09-0514:20Alex Miller (Clojure team)that is, the attributes are the driver, not the aggregation (the map)#2019-09-0514:20Alex Miller (Clojure team)this is taken much further in the schema/select work in spec 2 (and s/keys is going to go away)#2019-09-0514:21Alex Miller (Clojure team)spec 2 schemas do have inline support for ad hoc un-namespaced key specs#2019-09-0514:22Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys#2019-09-0514:23Alex Miller (Clojure team)the ability to programatically create and use specs is greatly enhanced in spec 2 as well#2019-09-0514:35Leonso spec 2 will support that kind of thing i'm requesting?
as i said, i fully understand the "attributes are the driver, not the aggregation" argument, and i think as a base philosophy it's actually rather genius.
but there still are a lot of cases where these attribute-semantics are single-use function arguments. and in these cases, having to give names to things that only make sense in a very specific context is increasing complexity and hurting readability. the "attributes are the driver" argument stops working as soon as you leave your direct domain-model and go into functions that achieve very specific purposes with specific parts of that data. having to describe the shape of every little thing there really makes this a lot more complicated.
especially in my current project, a clojurescript re-frame based frontend application, i really want to spec out every view function. not strictly for validation or domain-expressivity, but actually primarily just to get any kind of good errors. knowing that a view function got its arguments in the wrong shape/type is a lot more helpful than knowing that some reagent-internal had problems....
for these purposes spec would nearly be perfect. dynamic validation, that is actually close to dependent type-systems, would be the perfect solution to greatly specify what each function expects as arguments, no matter how detached that function is from the domain model.
but the way that spec currently prohibits the generation of map-specs inline makes a lot of things hugely more complex.
having a view function that takes 4 arguments look like this:
(s/fdef foo
:args (s/cat :title string?, :subtitle string?, :rating ::domain/rating, :content string?))
(defn foo [title subtitle rating content]
...)
is great for the spec. but calling the function gets anoyying and unreadable, so you want to add keyworded args:
(s/def ::title string?)
(s/def ::subtitle string?)
(s/def ::content string?)
(s/fdef foo
:args (s/cat :props (s/keys* :req-un [::title ::subtitle ::domain/rating ::content])))
(defn foo [& {:keys [title subtitle rating content]}]
...)
this is easier to call and read from caller side.... but extremely unnecessarily complex from implementation side.
(yes, title subtitle and content could be domain specific stuff, but maybe im writing a component library that doesn't HAVE any domain specific information.)
do you see where i come from?#2019-09-0514:40Alex Miller (Clojure team)those seem almost the same. so not sure I'm getting "extremely unnecessarily complex"#2019-09-0514:41Alex Miller (Clojure team)just the extra s/defs?#2019-09-0514:43Alex Miller (Clojure team)I don't think spec 2, at this exact moment, helps in this particular situation. however, it's not really easy to judge as we are currently completely reworking how function specs are defined and used and it's probably not going to bear much similarity to spec 1#2019-09-0514:45Alex Miller (Clojure team)something like (s/schema {:title string? :subtitle string? :rating int? :content string?}) is sufficient at current moment to spec a map with those (unqualified) keys. can't currently apply that ala s/keys* - that's still a tbd right now (I don't think it's a big deal to address, just not current focus)#2019-09-0517:42LeonIn this case it's not that Bad, but in more complex cases, and especially all over the Code, it gets very anoyying to type and Deal with. It would be great if that could be Adressed in spec2, as it then would be useful in pretty much any Situation without being overly verbose#2019-09-0517:53andy.fingerhutSorry, I do not have much constructive to say, but I do recall an explicit statement made by one of the Clojure core team (probably Stu Halloway) that something can be simple (i.e. not complex) but still be verbose.#2019-09-0519:16Leonyes, thats the java philosophy. "make simple things seem 5 times as complex by introducing unnecessary abstractions, syntactical overhead and overengineered design patterns". that its POSSIBLE to make something simple overly verbose isn't a good thing, and absolutely nothing you should WANT#2019-09-0602:37potetmit seems like you want something like: https://cljdoc.org/d/metosin/spec-tools/0.10.0/doc/data-specs#2019-09-0602:38potetmthe upside of a well-broken-down design is you can always layer on syntax to taste#2019-09-0602:40potetmthe fundamental diff between ācomplex = verboseā and ācomplex = a few things glommed togetherā is the former can turn to the latter, but the inverse is not true š#2019-09-0602:42taylor>A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.#2019-09-0602:48potetmthis is aligned with my experience#2019-09-0602:48taylorit took me a long time to learn this lesson š#2019-09-0602:50taylorI think this quote resonates with my enjoyment of writing Clojure and how easy it is to compose little solutions into big ones#2019-09-0602:52potetmhonestly, the biggest win there is probably immutable data#2019-09-0602:52potetmbut Iām continuously surprised at how much simpler thing get if you design them to be run from the REPL#2019-09-0602:53potetmevery TDDer: Itās like Iāve been saying for yeeeears! troll#2019-09-0602:54tayloryeah, I felt the same way with other ML-like functional/immutable languages before Clojure. OCaml/F# is nice, but Clojure is more enjoyable for me#2019-09-0609:28LeonThats exactly what im looking for! Thanks, @potetm i was about to spin my own solution and dig into how to create custom spec validators for that (might still do, but only for the learning experience)
As i said, i fully agree that well broken down Systems are easier to reason about and work with. But originally coming from Java, and then going to expressive languages like kotlin, haskell, Rust (and now clojure), ive really started to value expressivity. I like to talk in my Code like i would talk to a Person. I like to describe what im trying to achieve, how different Things relate to each other, and what Things represent, and i like to Use expressive Tools to achieve that. The Same way that a for Loop isnt really the best way to describe the relation of a list of numbers to the Same list with every number squared, i feel about using spec. Defining Things that Look and feel like Domain Model/ more complex stuff for simple validations works, but it doesnt Show what i want to express. In my analogy, i want to actually express (map #(* % %) my-list)
. Simmiarly, in my spec i dont want to say "the arg title
is of type ::title
, where ::title
is a String." but just "this function takes a String
that will be displayed as the tite."
It's not so much about complexity in a Code sense, but just about expressivity, readability and also just typing and reading time. And yes. Lines of Code do matter. Otherwise lisps would put their closing parens on New lines#2019-09-0611:51potetmYeah that makes sense to me.#2019-09-0611:53potetmOne thing that I really disliked about spec is that it encouraged creating names where none would suffice.#2019-09-0612:17Leonyes, that is my exact point ;D glad we agree here, and i hope spec2 will change that#2019-09-0611:55mishaas a datapoint, in ~2 years I specced like 5 functions, and am really amused how much attention s/fdef and instrument get#2019-09-0612:07potetmwhyso? do you just not get a lot of value out of fdef?#2019-09-0612:13Leoni spec pretty much all my functions, but mainly because its the only way to get immediately helpful error messages. especially in clojurescript, sometimes the errors really dont help. i just get some errors in some G_1235
generated function name and have no idea what happened.
also, I'd say a big help for speccing all functions are the defn-spec
feature of orchestra or things like ds/defn
from defn-spec
that lets you add specs directly into the function signature#2019-09-0612:28mishathere are few functions which benefit from gen-testing. which somewhat rhymes with "you really need just few refs per project".
but other things too:
- read/write overhead,
- maintenance,
- "spec pushes you to have global names where none suffice",
- project size/complexity is not that out of control, so "specing 100% of the code is the only way out"
- wide contact surface with mutable DOM in cljs
- etc.#2019-09-0612:34Leonmaintainance isn't really a big problem, and the error reporting benefits are just huge#2019-09-0612:39mishaI prefer to understand what system is doing, when I can afford it. Admittedly, you are not always in position to have this as an option (project/team size, domain complexity, etc.)#2019-09-0612:45mishasome analogies would be:
- turning off code highlighter to pay more attention to code.
- writing code w/o autocomplete/autosuggest to know better whats there in the project.
these things keep your mind in a tonus#2019-09-0612:47mishaand more stuff you will know you will have to keep track of, more reluctant you will be to let bloat slide in.#2019-09-0613:15potetm@misha yes, to all that#2019-09-0613:16potetmIāve honestly had very little urge to use spec at all.
Though Iāve spent the last 9mo on a novel thing, so nothing is solidified to the point where I want to invest in pouring some concrete.#2019-09-0613:17potetmActually, there are a fair number of unit tests, and IMO even those arenāt worth enough to bother.#2019-09-0613:18potetm(They have to be touched fairly often and have caught very few/no bugs afaict)#2019-09-0613:25mishaand still, pouring concrete on every private function ā is questionable#2019-09-0613:30Alex Miller (Clojure team)from spec hq: agree#2019-09-0619:15seancorfieldFrom our code base (World Singles Networks): Clojure build/config 52 files 936 total loc
Clojure source 299 files 70282 total loc,
3207 fns, 662 of which are private,
441 vars, 29 macros, 62 atoms,
602 specs, 21 function specs.
Clojure tests 330 files 19116 total loc,
4 specs, 1 function specs.
We were an early adopter of Spec in production code (`conform`/`valid?`/`explain-data`) but we mostly spec data rather than functions, as you can see above.#2019-09-0622:48gerred@seancorfield are you using valid?
and conform
in production paths or in tests or their own specs.clj file? that is to say, are you paying the performance costs at runtime for the specs you do have?#2019-09-0623:21seancorfield@gerred We use conform
and invalid?
mostly, then explain-data
to drive heuristics for turning spec errors into codes/messages we can return to client apps. But, yes, that's all production path code.#2019-09-0623:23seancorfieldA lot of our input form data and most of our API input parameters are run through Spec. We have conforming Specs that accept strings and will conform them to simple numbers, dates, Booleans etc. Something that Alex frowns on š but it has worked exceptionally well for us.#2019-09-0623:24seancorfieldWe usually write the specs to accept either the native type we want or a string that can be parsed/coerced/conformed to that native type, and our specs have generators that produce strings (we mostly generate the underlying spec and then fmap
str
over that).#2019-09-0700:29johanatanis stest/instrument
known to work for spec'd reagent components (returning hiccup data structures)? once i enabled it, my browser started complaining that it didn't recognize the elements emitted (which were non-standard html elements which had apparently not been interpreted by react/reagent beforehand).#2019-09-0721:36borkdude@johanatan fwiw, I do have an fdef for one reagent component in our app, but that's a form-3 component: (defn foo [opts] (r/create-class ...))
#2019-09-0722:24johanatanYea perhaps it would be less of a problem for form-2 or form-3 components. @borkdude #2019-09-0919:17Nolanmay have been discussed before, but is there a way to install a custom randomness source into spec/gen?#2019-09-0919:21Alex Miller (Clojure team)gen rides on test.check#2019-09-0919:21Alex Miller (Clojure team)test.check does have pluggable randomness, although there are some caveats about it as it's designed to be repeatable#2019-09-0919:22Alex Miller (Clojure team)prob best to hit up @gfredericks for the details#2019-09-0919:22Alex Miller (Clojure team)from spec side, the big thing is that stest/check reports the seed it used and accepts a seed as an option to pass along#2019-09-0919:27gfrederickst.c isn't wired up for that, but the protocol is in place, so currently you could pull it off by monkeypatching random/make-random#2019-09-0919:28Nolanright. i traced it down to the relevant files, and quickly realized that there are a number of features about the existing random interface that id want to keep in almost every case#2019-09-0919:29gfredericksWhat's the motivation?#2019-09-0919:30Nolanusing the current spec-alpha2.gen
whats the best way to create a generator that basically calls a function and returns the value? looking for something like gen/return
that evaluates x
in each generation#2019-09-0919:30Nolanit may sound weird, but im aiming for a generator that produces a fixed number of CSR bytes#2019-09-0919:32gfredericksSounds like something better addressed at the generator or spec level, if I'm understanding you#2019-09-0919:32gfredericksOr is this some kind of security thing where you want High Quality Randomness?#2019-09-0919:32Nolanyeah, it definitely is. i was mistaken to think that it would be interesting to aim at the test.check
level#2019-09-0919:34gfredericksgen/fmap is the normal trick for making a generator from an impure function, with all the usual caveats#2019-09-0919:34Nolani do want a certain quality of randomness, but more generally im looking for something that would let me (gen/let [x (some-fn-producing-x)] x)
, but gen/let
isnt exposed in spec-alpha2.gen
. im using fmap
now, but didnāt know if that was the right way:tm:. appreciate the input!#2019-09-0919:38gfredericksThere is no right way, because it circumvents determinism, growth, and shrinking; but spec has generally made theses things a lot murkier#2019-09-0919:40Nolanmakes total sense. thats what i quickly realized when i was digging into the test.check
randomness. those are some really neat files, way above what im trying to do#2019-09-0919:40Nolanawesome work!#2019-09-0919:41Nolanam i correct in labeling something like this (partial gen/fmap f)
a ācombinatorā?#2019-09-0919:41Nolanor would that have a different name#2019-09-0919:49gfredericksfmap is a generator combinator, and partial is a function combinator; the result is also a generator combinator I suppose#2019-09-0919:51Nolanideal#2019-09-1014:21NickIād like to be able to add a docstring to my specs, to document particular points of interest about one spec versus a similar one. Looks like thereās an issue on the JIRA but it hasnāt been updated in some time. Is there a recommendation as to how to handle this yet? https://clojure.atlassian.net/browse/CLJ-1965#2019-09-1021:09thumbnailAt work we use/develop speced.def, which includes a helper exactly for this with tooling support; https://github.com/nedap/speced.def/blob/a6e4e40fa6b92cf66ca6b0f1542eeb8cbd09b039/src/nedap/speced/def.cljc#L24#2019-09-1014:22Alex Miller (Clojure team)We're planning to work on it for spec 2, but currently, no#2019-09-1014:44NickCool, thanks. Iāll just drop the comment in my code next to the spec def then.#2019-09-1118:33Nolanonce it leaves alpha of course, is it a priority for spec
to expose the same api between CLJ and CLJS?#2019-09-1118:47Alex Miller (Clojure team)yes#2019-09-1118:48Alex Miller (Clojure team)I'm not sure whether it's possible to guarantee 100% fidelity between the two, but certainly should be close. afaik it is mostly the same now (there are some necessary exceptions around the gen namespace, but those should be able to move closer)#2019-09-1119:08Nolanimpeccable!!! thanks for the info. and totallyāthey are (at least for us) entirely cross-compatible for spec1. ive gotten so fond of s/schema
and s/select
that now im almost disenchanted when working with our current specs š#2019-09-1218:21leongrapenthinim wondering whether spec2 is missing feedback/adoption because of cljs?#2019-09-1218:23hkjelsI would definitely have used spec2 if it was available in cljs#2019-09-1418:10johanatansame#2019-09-1218:21leongrapenthinid love to try it out and give feedback already, but all my projects are fullstack or cljc#2019-09-1218:29ghadi@leongrapenthin spec2 isn't even done#2019-09-1218:30ghadionce the design settles, will be ported to cljs#2019-09-1218:30leongrapenthincool#2019-09-1219:17seancorfieldHeh, we're about as leading edge as you'll find, for running pre-release versions of Clojure and Contrib libs but even we're not using Spec2 on the backend yet. Still far too "not done". We do have a branch based on it, passing all our tests, but was a bunch of work to get the Spec1 code running on Spec2.#2019-09-1223:11andy.fingerhutThis is perhaps a bit tangential to spec, but I was reminded recently that a handy technique for making generative testing more time-efficient in finding problems in your code, is if you can tweak parameters in your code to make sub-parts of your data structures smaller. For example, a 32-way branching tree structure like PersistentVector takes 32 conj's or pop's before it actually "does something" to affect the structure of the tree, but if you tweak the code to make it 4-way branching instead, you can make more interesting tree structures and test them with generative testing much, much faster, and the bugs are likely to be identical, except for the size of the test cases.#2019-09-1223:12andy.fingerhutA technique I saw used in hardware simulation tests long ago (where the simulations are much much slower than the real hardware devices), but hadn't been reminded of in a while.#2019-09-1322:26gerredi must be tired and missing something#2019-09-1322:26gerredi'm converting a spec from s/?
to requiring exactly one argument.#2019-09-1322:26gerredi'm staring at the source and can't figure out the arg spec I want.#2019-09-1322:28gerredoh my gosh#2019-09-1322:28gerrednm#2019-09-1509:20ikitommiIs it intentional that s/valid?
can throw? should it return false
in case the validation fails on exception?#2019-09-1509:20ikitommie.g.
(s/valid? empty? 1)
; Syntax error (IllegalArgumentException)
; Don't know how to create ISeq from: java.lang.Long
#2019-09-1704:07ataggartCan't speak to intention, but that predicate has an invariant assumption: that the arg is a collection. Neither true nor false would be valid answers. The error can be avoided by first checking that the invariant assumption holds:
user=> (s/valid? (s/and coll? empty?) 1)
false
#2019-09-1714:31dmarjenburghIs it possible to dynamically generate specs at runtime? For example, based on a runtime value, I call a fn that returns a vector of namespaced keys. I then generate a spec like (s/keys :req vector-of-keys)
. Since keys
is a macro, I havenāt been able to find a way#2019-09-1714:46Alex Miller (Clojure team)you'll need to eval somehow, either by explicitly using eval
or by wrapping in another macro that uses backtick#2019-09-1714:47Alex Miller (Clojure team)(in spec 2, we've made this much easier)#2019-09-1714:51dmarjenburghGreat, looking forward to spec2#2019-09-1916:11dominicmHas anyone experimented with making specs for stateful things, with the intention of using mocks for those values?#2019-09-1916:11dominicmThinking in the context of dependency injection somewhat.#2019-09-1916:13Joe Lane@dominicm Do you mean something like
(stest/instrument `invoke-service {:stub #{`invoke-service}})
found at https://clojure.org/guides/spec#_combining_check_and_instrument#2019-09-1916:15dominicmoh wow, that's fantastic!#2019-09-1916:16dominicmoh, wait. That's not quite what I was looking for. But still very interesting in what I'm thinking of š¤#2019-09-1916:17dominicmthat's really neat. I didn't know you could do that. That's actually really handy for what I'm working on.#2019-09-1916:19dominicmI'm actually thinking of the case where you have a function like invoke-service
, and you want a dumb wrapper which can figure out what to call it with (from it's registry of "stuff") based on the spec.
I'm a bit frustrated with the pattern of passing around a grab-bag of state which inevitably grows into an unrepl-able mess.#2019-09-2009:21djtangoinstrument actually lets you go even further and completely replace the function#2019-09-2009:23dominicmYeah, that's great at the repl. I don't want to do that in production though š#2019-09-2009:24djtangoah - sorry, I am still catching up with the discussion though have found the intercepting properties useful at test-time#2019-09-2009:25dominicmI'm a little unsure on that. Parallelizing tests is valuable for speed (until Rich invents his tool for running only the required subset of the tests). I don't know how much I love that idea really.#2019-09-2009:25dominicmBut yeah, if you're willing to trade those two properties, it's all good.#2019-09-2009:34djtangomm yeah fair enough.
So is your use-case this:
use fdef
to document what deps your function needs
have some kind of wrapper that then knows how to extract and inject only those deps into a calling function?#2019-09-2009:37dominicmYeah. I'm coming round to the idea that I might be barking up the wrong tree though š I'm thinking a little more time in the hammock should help me puzzle out the use cases.#2019-09-2009:37dominicmI'm trying to exploit maximum leverage without creating unnecessary verbosity. It's a hard balance š#2019-09-2009:45djtangoI guess I can in theory see how you could fetch the fspec then use a generator, to satisfy the s/keys :req
portion but feels brittle, if only as I'm not convinced it's an appropriate usage of the tools#2019-09-2009:47dominicmMe neither š#2019-09-2009:49djtangosounds tricky - would be interested to know how you get on#2019-09-2009:51dominicmI'm hoping to publish my results somewhere, I think I'm onto something, I've reduced the initial problem space into something more palatable. But I still have to handle the stateful dependencies part.#2019-09-1916:22Joe LaneI'm not sure I understand the usecase for the dumb wrapper
. What is registry of "stuff"
?#2019-09-1916:24dominicmI suppose something to the effect of, I have a "system" like: {:db db-conn :some-stateful-service statey}
and I want to direct things directly to a function defined like:
(defn add-user
[db add-user-data] ā¦)
So the "registry" is that first map.#2019-09-1916:25Joe LaneSo, in component terms, a "system"#2019-09-1916:25Joe Lanek#2019-09-1916:25Joe Lane(Or a pedestal context map)#2019-09-1916:26dominicmYeah. Exactly. But passing around systems is an anti-pattern. And I also can't figure out what's in my pedestal context map half the time, because there's lots of interceptors messing with it, and it's unclear why routeA gets :mongodb and routeB doesn't. (So I'm trying to come at this with a new angle of positional parameters)#2019-09-1916:31Joe Lanethinking#2019-09-1916:33Joe LaneSo, is the goal here to assert a property about the dumb wrapper
but you're trying to determine the best way to inject actually stateful mocks?#2019-09-1916:34dominicmI guess I could write a spec parser for s/cat? And that would let me figure out the arguments in order.
Well, actually I was thinking that it would be handy to just start by being able to call add-user
by using it's fdef to figure out that it needs a database, and getting one from the system. (and somehow connecting ::db/db
as a spec to that)#2019-09-1916:35Joe Lane(= dumb-wrapper add-user)
#2019-09-1916:35Joe Laneright?#2019-09-1916:35Joe Laneoh.#2019-09-1916:36dominicmyeah, right š#2019-09-1916:37Joe LaneI might be wrong, but it sounds like you want an integration between your DI tool (are you using component?) and the s/select
facilities in spec2. If you're using component though, shouldn't that fdef to figure out....
step be happening at system/lifecycle start time?#2019-09-1916:37dominicmBut I'd be using this in production, not just for testing. So I would know that this function only takes 1 non-stateful argument (the "event", or maybe "req" for http). So I just need to get the rest of the arguments.#2019-09-1916:39dominicmI'm not using component. The reason being that a system probably ends up with ~50 or so handlers, and the boilerplate involved is quickly tedious. Although maybe I should just use a macro for that š#2019-09-1916:39dominicmThe way I see it, I have actual stateful things (e.g. db conn) and things that want to use those things.#2019-09-1916:40Joe LaneThis is starting to feel more and more like spring style annotation DI vs data oriented component DI. And further away from a problem related to spec.#2019-09-1916:42dominicmA little, yeah. And that's somewhat intentional. I see one of two patterns in this space: 1. create a map with everything in, call it "deps" and hope the function you're passing it to understands it (and there's a bunch of bad patterns which fall out of this). 2. roll entire namespaces into being partial
applied with their state available, and passed in as arguments.#2019-09-1916:44dominicmTbh, I expect there's not a good spec function for figuring this out š As I have a lot of contextual awareness that spec doesn't have. (e.g. I know it's always a list, and I will always know everything except 1 argument).#2019-09-1916:45Joe LaneI think the 3rd pattern is component/system/mount oriented, where functions are passed positional arguments (not a deps/req/context map) which were determined by the DI tool. I don't think you can reduce that kind of complexity, only choose where to solve it, 1 large place (component) or in every dumb-wrapper
by declaring the deps at the callsite. I definitely don't think spec is going to be helpful here.#2019-09-1916:47dominicmI haven't seen 3? what does that look like#2019-09-1916:49dominicmoh, maybe I do know what you mean.#2019-09-1916:50dominicmI guess pattern 3 here is implemented in terms of my listed patterns 1 & 2.#2019-09-1919:28jaihindhreddyI just started solving some basic programming puzzles with Clojure, with the constraint that I want to spec the solutions.
I hit a snag right on the first one.
Here's the fn I want to spec:
(defn two-sum
"Returns indices of two elems in ints that add up to sum"
{::url ""}
[ints sum]
(loop [l 0
r (dec (count ints))]
(let [s (+ (nth ints l) (nth ints r))]
(cond (= l r) nil
(= s sum) [l r]
(< s sum) (recur (inc l) r)
:else (recur l (dec r))))))
And here's what I came up with:
(s/fdef two-sum
:args (s/cat :ints (s/coll-of int? :min-count 2 :kind vector?) :sum int?)
:ret (s/and (s/tuple (s/and int? #(>= % 0)) pos-int?) (fn [[l r]] (< l r)))
:fn (s/and
#(< (-> % :ret second) (-> % :args :ints count))
#(= (-> % :args :sum)
(+ (nth (-> % :args :ints) (-> % :ret first))
(nth (-> % :args :ints) (-> % :ret second))))))
#2019-09-1919:30jaihindhreddyThe puzzle is to find two (different) indices s.t. the elements in ints
in those indices add up to sum
.
The catch is: there is always exactly one solution. How do I spec my args to get gen right?#2019-09-1919:31jaihindhreddyUnless I use the fn itself in it's spec (or trusted equivalent), I can't seem to get gen. to work.#2019-09-1919:52seancorfield@jaihindhreddy That sounds like an external constraint on the data? An arbitrary vector of 2+ ints and an arbitrary sum aren't going to satisfy that condition so the behavior of two-sum
on such data will be...? Undefined? nil
?#2019-09-1919:52seancorfieldYour :ret
spec doesn't account for the function returning nil
-- which it clearly can -- so your spec isn't right as it stands.#2019-09-1919:55seancorfielduser=> (two-sum (into [] (range 10 20)) 9)
nil
So your :ret
spec needs to be s/nilable
or use s/or
and then your :fn
spec needs to accept that (:ret %)
can be nil
(or should satisfy that complex predicate).#2019-09-1919:58jaihindhreddyIt is an external constraint on the data. I thought of using s/nilable
there but actually the fn is not supposed to be called with such args and its UB, and I'm trying to get the generation to work in such a way that there exists exactly one answer.#2019-09-1919:59jaihindhreddyI'll probably look into using fmap
to generate the sum
from the ints
. Thanks for your help!#2019-09-1920:03seancorfieldAnd it's also an ascending sequence of ints, yes? (based on the <
logic in there)#2019-09-1920:05seancorfieldI think what you're attempting is a bit self-defeating: the generator and the :fn
spec together are pretty much going to be a re-implementation of the function logic at this point...#2019-09-1920:06seancorfield(i.e., you're over-spec'ing things, IMO)#2019-09-1920:07jaihindhreddyKinda reckoned that myself.#2019-09-1920:09jaihindhreddyThe fn returns a pair of indexes into the ints
arg, so I'm checking that the second one in the return value is less than the count of ints
.#2019-09-1920:45vlaaadnoob question: why spec forms sometimes have qualified, and sometimes unqualified symbols?#2019-09-1921:00vlaaadminimal example:
(s/def ::vec vector?)
(:pred (first (::s/problems (s/explain-data ::vec {:a 1}))))
; => clojure.core/vector?
(:pred (first (::s/problems (s/explain-data (s/coll-of vector?) [{:a 1}]))))
; => vector?
#2019-09-1921:03Alex Miller (Clojure team)Because bugs#2019-09-1921:03Alex Miller (Clojure team)Should always be qualified#2019-09-1921:03vlaaadagree!#2019-09-1921:04Alex Miller (Clojure team)There are some pending patches for this stuff#2019-09-1921:08vlaaadI'm fascinated by spec, sprinkling it a bit here and there to document/enforce contracts where it makes sense... do you think it's the future of strong gradually static type systems?#2019-09-1921:16vlaaadlike being able to say something along the lines of
let n: int? = (get-some-int)
let x: (and int? even?) = n ;; checks for `even?` during casting
#2019-09-1921:16Joe LaneI think it will push significantly more boundaries than just "type systems"#2019-09-1921:18Joe LaneI also think the traditional notion of static type systems look very different if you can work in a dynamic system.#2019-09-1921:19Joe LaneBut what do I know :man-shrugging:#2019-09-1922:05seancorfieldI don't consider it "like" a type system (and I think it's a bit misleading to think of it in those terms).#2019-09-1922:07seancorfieldWe've been using spec in production code very heavily since it first appeared. We don't use it much on functions (which is the only "like types" part of it -- and even that isn't much like a "type system").#2019-09-1922:09seancorfieldWe mostly use data specs and validation/conformance. We use it to generate (random, conforming) data for example-based tests. We use it for generative testing of some things. We use it via instrument
a little bit during dev/test.#2019-09-2005:41vlaaadWell, it's not like a type systems of today :)#2019-09-2017:51andy.fingerhutWell, the full generality of spec allows arbitrary code in predicates, so it is in general computationally undecidable to statically check that a program meets such a general spec.#2019-09-2017:53andy.fingerhutspec has been designed with run-time checking (and generation, etc.) in mind, not trying to limit itself to what can be statically checked, although subsets of it can be: https://github.com/arohner/spectrum#2019-09-2020:50seancorfieldBecause folks keep asking about our "coercing specs" at work, we just open-sourced them https://github.com/worldsingles/web-specs worldsingles/web-specs {:mvn/version "0.1.0"}
#2019-09-2105:20dominicmAny regrets using conformers?#2019-09-2105:26seancorfieldNope, none at all. I originally submitted a talk to Conj about it (and it was accepted) but I pulled it because Alex was so "anti" the whole conformer thing. That was years ago. But I still think we made the right decision and I'd stand by it every day.#2019-09-2107:04jumarAwesome, thanks for that.#2019-09-2209:56Sebastiano BarreraHi y'all... I'm a beginner tinkering around with spec (on CLJS) and I was wondering: is there a way to encode a spec so that a certain keyset is required only if another key has a certain value? For example, I might have {:type :assignment, :left ..., :right ...}
or {:type :identifier, :name 'some-name}
; could I express the requirement that :left
and :right
must both appear iff :type
is :assignment
?#2019-09-2210:03Sebastiano BarreraWait, I think I figured it out... I'm guessing
(defn node-is? [type] #(= (:type %) type))
(s/or
(s/keys :req-un [::type (node-is? :assignment) ::left ::right])
(s/keys :req-un [::type (node-is? :identifier) ::name]))
#2019-09-2210:59vlaaad@sebastiano.barrera_cl I think what you need is multi-spec#2019-09-2211:00vlaaadhttps://clojure.org/guides/spec#_multi_spec#2019-09-2211:02Sebastiano BarreraOh, let me take a look at that#2019-09-2517:07sundbpIāve got a question about spec2, how do I do something like this:
(s/def ::foo (partial bar 1))
- in spec2 that fails. Wrapping the partial in e.g. s/spec is no solution.#2019-09-2517:57seancorfieldI believe function literals are accepted @sundbp -- try (s/def ::foo #(bar 1 %))
#2019-09-2614:02sundbpThanks. Will give that a go#2019-09-2518:06seancorfieldYeah, that works: user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (defn bar [n y] (< n y))
#'user/bar
user=> (s/def ::foo #(bar 1 %))
:user/foo
user=> (s/conform ::foo 2)
2
user=> (s/conform ::foo 1)
:clojure.spec-alpha2/invalid
user=>
#2019-09-2519:01andy.fingerhutI can dig into the implementation and find out, but does anyone happen to know whether test.check/quick-check calls clojure.test/is or clojure.test/are internally, and thus is intended to be called inside of a clojure.test/deftest form directly?#2019-09-2519:01andy.fingerhutThe doc string says it runs multiple tests, but isn't clear about whether there is a return value#2019-09-2519:03andy.fingerhutPerhaps one reasonable way to use test.check/quick-check inside of a clojure.test/deftest would be (is (:pass? (test.check/quick-check ...)))
?#2019-09-2519:06andy.fingerhutAh, from looking at test.check's own deftest's, it appears that it often uses (is (:result (test.check/quick-check ...)))
. I will use that, too.#2019-09-2519:10andy.fingerhutIs there some existing test.check docs that describe the return value of quick-check that I could learn this from? Or perhaps a small addition to the quick-check doc string might be welcome?#2019-09-2519:16gfredericks@andy.fingerhut hi#2019-09-2519:17gfredericksThe docstring of the latest version is pretty explicit about the return value#2019-09-2519:17gfredericksYou're correct that the quickcheck function is unrelated to clojure.test; you can integrate manually as you noted, or by using the t.c.clojure-test namespace#2019-09-2519:24andy.fingerhutok, I may have been checking the quick-check doc string of an older version that core.rrb-vector is using. Let me look at the latest.#2019-09-2519:24gfredericksThe past was a terrible time#2019-09-2519:25andy.fingerhutOK, much better now. Thanks!#2019-09-2519:27gfredericksNP#2019-09-2710:09lmergenis it possible to somehow tokenize a string using specs, in a s/cat
-like way?
e.g.
(s/def :directory (fn [x] (re-matches ...))
(s/def :filename (fn [x] (re-matches ...))
(s/def :absolute-path (s/cat :d :directory :f :filename))
(which doesn't work for obvious reasons, but that's the idea of what i'm looking for)#2019-09-2710:11lmergenactually i'm probably looking for a way to compose regexes more than i'm looking for this spec-like functionality... š¤#2019-09-2710:21Alex Miller (Clojure team)You can do this (by seq-ing a string) but I generally think itās a bad idea. There is actually a regex composition library for Clojure, but Iām blanking on the name#2019-09-2710:23Alex Miller (Clojure team)Maybe https://github.com/fhur/regie is what Iām thinking of?#2019-10-1616:49rickmoynihanCould it also have been:
https://github.com/cgrand/seqexp
?
cc @U0M8Y3G6N#2019-09-2710:27lmergeni see, that actually looks interesting#2019-09-2710:27lmergenthanks#2019-09-2712:37jonpitherHi. is there a generalised pattern / approach for handling config, in that a) defining the specs for the config attributes, b) documenting the individual options, c) providing default values for options? I realise this Q is a bit wide ranging, but I'm wanting to take a step back and to see what others usually do for config input#2019-09-2713:47taylorI donāt have an answer for your exact question, but Iāve specād CLI args before which is kinda like config https://github.com/taylorwood/clojurl/blob/c69cef7c1f2b46ac9034cf75ea1c5f7d3abf6d2b/src/clojurl.clj#L13-L42
Re: documentation, spec doesnāt really support it (yet?), and I probably wouldnāt use spec to influence config default values. That said, thereās probably an opportunity to pull pieces of this together into an āopinionatedā config library :man-shrugging:#2019-09-2714:27jonpitherThanks @U3DAE8HMG will check this out#2019-09-2721:11jumar@U050DD55V I've done something similar for one of our applications.
It's a rather hairy code and I'm sure there's a better approach for validation and replacing the invalid values with defaults but here's essentially what we've had in production for at least 1 year: https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/config/config.clj#L1
See this commit which shows all relevant parts: https://github.com/jumarko/clojure-experiments/commit/35d429e1641806bba78b470e4f4e249207739e96#2019-09-3008:56jaihindhreddyspec might support docstrings but probably won't support metadata, because that might cause specs to become bloated and less suitable for a variety of usecases. Because ns-qualified keywords come with no strings attached, spec is able to associate global semantics in particular contexts (when calling s/validate
, for example). In a same way, maybe something like aero
can maintain its own global registry of documentation, defaults etc. and be useful in contexts that make sense for it, perhaps pawning off the job of validation to spec.
@U050DD55V sorry for being vague š
And pointing to your own library (`aero`) š#2019-10-1617:02rickmoynihan@U050DD55V we have an inhouse tool that we use to specify service requirements and dependencies in EDNā¦ itās meant to tie together CI services with what they expect at runtime etc, and help in pulling clojure services and service dependencies together across various environmentsā¦
e.g. in dev you want to run the services with repls and various things setā¦ in prod you want packer images built with config files written in the right places that define parameters to be injected into env files and config files etc.
And all environments want to pull in dependent services.
Anyway part of it is aero; and each service declares things like:
:omni/keys {:stardog/stardog-home {:doc "Location of the stardog home directory"
:default [:install "stardog-home"]
:env-var STARDOG_HOME}
:stardog/java-args {:doc "Arguments to the JVM when running stardog"
:env-var STARDOG_SERVER_JAVA_ARGS}
:stardog/temp-dir {:doc "Java temp directory for the stardog process"
:env-var STARDOG_JAVA_TEMP_DIR}
:stardog/logs-dir {:doc "Directory to write the stardog.log file to"
:default "."}}
#2019-10-1617:03rickmoynihanwhich includes the doc string, and where the variable comes from etc.#2019-10-1617:04rickmoynihanso services can inform things further up the chain what they need to set; and display a useful error when something is missing#2019-09-3010:08shanBasic question about spec, but I would expect clojure to complain if a spec doesnāt exist but seem it just seems to ignore the fact:
(s/def ::test-spec
(s/keys :req-un [::req-does-not-exist]
:opt-un [::opt-does-not-exist]))
(s/valid? ::test-spec {:req-does-not-exist :foo})
;; => true
(s/valid? ::test-spec {:req-does-not-exist :foo
:opt-does-not-exist :bar})
;; => true
Any idea if thereās a reason for this or how you usually handle being aware of missing specs?#2019-09-3010:16jaihindhreddyI'm guessing that's because spec is an open system. When you say I need ::a
to be present in the map, s/keys
will verify that, and if you have specified what ::a
should look like, then it will verify (and generate) that too correctly. This is what allows us to spec as little or as much as we want about our domain.#2019-09-3010:17jaihindhreddyAlso comes in handy when doing interactive development, where spec doesn't become this thing that is constantly bothering you about missing spec definitions. You get to decide how much you want to spec, and accordingly you get back as much value and precision in validation and generation. Hope that answers your question.#2019-09-3010:29vlaaadCode above looks like a bug to me.#2019-09-3010:30vlaaadby specifying :req-un
, you asked spec to check that :req-does-not-exist
key should be checked against ::req-does-not-exist
spec during a call to valid?
#2019-09-3010:31vlaaadit's one thing to have forward declarations in spec where you don't check if referenced spec is declared during "definition phase", and another to ignore required keys during "validation phase"#2019-09-3010:40shan^^ yep, thatās what I was thinking#2019-09-3010:58jaihindhreddyYeah. That does seem like a bug. Didn't read the whole thing. My bad.#2019-09-3014:05jaihindhreddyActually, s/valid is behaving correctly here. (s/valid? ::test-spec {})
correctly returns false.#2019-09-3010:42shanit seems to mean you could add a missing spec to :req/opt-un
and add it to the data but this wouldnāt actually be checked or give any indication the spec doesnāt exist#2019-09-3014:28minimalStuart H wrote a gist to check missing keys a couple of years ago https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2019-09-3023:04shanCheers dude. Youāre always a good source for clojure knowledge š#2019-09-3013:02conanThis seems fine to me. You're ensuring that a map contains a specific key, but making no claims about the value of that key. I do this frequently when receiving required but complex data from external systems - i want to check that the data is there, but i don't know enough to write a comprehensive spec for it.#2019-09-3016:55sgerguriHere is my situation: I have a top-level spec that allows for two situations - cluster updated and cluster removed. This is handled by two multispecs as follows:
(defmulti cluster-updated :cluster-type)
(defmulti cluster-removed :cluster-type)
(s/def ::cluster-updated (s/multi-spec cluster-updated :cluster-type))
(s/def ::cluster-removed (s/multi-spec cluster-removed :cluster-type))
(s/def ::cluster (s/or :cluster-removed ::cluster-removed
:cluster-updated ::cluster-updated))
#2019-09-3016:56sgerguriEach of those multispecs has two implementations. The implementations generate nested maps, where some of the nested maps have generators that generate matching input, but conforming strips away extra data as specified.#2019-09-3016:58sgerguriNow, I would like to call (s/conform ::cluster (g/generate (s/gen ::cluster)))
and get either a :cluster-removed
or :cluster-updated
data, but conformed recursively all the way through. However, conformance does not seem to propagate to the level of the multispecs, it simply stops at the level of ::cluster
.#2019-09-3016:59sgerguriI can work around it by examining the tag I get from ::cluster
, then conforming again against either ::cluster-updated
or ::cluster-removed
, which does the correct thing against the correct implementation. But ideally, I would like to conform only once and have the correct thing happen. Is there a way to do that?#2019-09-3017:00sgerguriFYI - I have two multispecs there as unfortunately multi-spec seems to expect a single field as a dispatch tag, even though the multimethods themselves can dispatch on more than one field (through the use of juxt
).#2019-09-3017:01sgerguri@alexmiller ^ Will definitely appreciate some thoughts on this situation! š#2019-09-3017:05Alex Miller (Clojure team)well on the last point first, you can use an arbitrary function for the multi-spec, doesn't have to be a keyword#2019-09-3017:05Alex Miller (Clojure team)so you could for example use juxt to produce a vector and dispatch from a single multimethod#2019-09-3017:05Alex Miller (Clojure team)maybe that solves your whole problem, not sure#2019-09-3017:06sgerguriWhat would I use for a retagging function though?#2019-09-3017:06Alex Miller (Clojure team)it's used to drive gen#2019-09-3017:06Alex Miller (Clojure team)it will pick a random multimethod, gen from that, then "retag" to something passed to the multimethod#2019-09-3017:07Alex Miller (Clojure team)here it would need to go from the dispatch value (the vector output of juxt) to modify the generated defmethod map to get back to your original expected input#2019-09-3017:08Alex Miller (Clojure team)depending on your method generators, there may be nothing to do and you could just use identity#2019-09-3017:08Alex Miller (Clojure team)or you could assoc back in the keys and the values from the juxt vector#2019-09-3017:08sgerguriWith juxt
I'd be ideally looking at something as follows:
(defmulti cluster (juxt :cluster-type :action))
(s/def ::cluster (s/multi-spec cluster ?))
I'm just not sure what to put in instead of the ?
.#2019-09-3017:09Alex Miller (Clojure team)as above, identity
might work, or you could use (fn [[type action]] (assoc % :cluster-type type :action action))
#2019-09-3017:09sgerguriAh, identity
might just do the trick. Let me give that a try!#2019-09-3017:10Alex Miller (Clojure team)or maybe I've got the signature wrong there - does retag fn take 2 args?#2019-09-3017:10Alex Miller (Clojure team)yeah, it's value and dispatch value#2019-09-3017:10sgerguriI think it only takes one? It can also be a keyword, which would support that argument.#2019-09-3017:10Alex Miller (Clojure team)so should have been (fn [gen-val [type action]] (assoc gen-val :cluster-type type :action action))
#2019-09-3017:10sgerguriAh no, it actually takes two. But apparently one can supply a keyword as well.#2019-09-3017:12Alex Miller (Clojure team)I think that's right. I haven't done one of these in a while.#2019-09-3017:12sgerguriWhy is the retagging even necessary, when the multimethods return a spec that is narrow enough to always generate the right data? I guess that's one thing that's confusing me.#2019-09-3017:20sgerguriHm, this did not really solve the problem - but thanks anyway. I'll post here if I manage to solve this in a satisfactory fashion.#2019-09-3017:25sgerguriActually, I take that back - I had a dangling redef on the multispec elsewhere. Doing it as per Alex's suggestion:
(defmulti cluster (juxt :cluster-type :action))
(s/def ::cluster (s/multi-spec cluster (fn [gen-val [type action]] (assoc gen-val :cluster-type type :action action))))
does the trick. Thanks for the help, @alexmiller!#2019-09-3017:26Alex Miller (Clojure team)Np#2019-10-0123:01Oliver GeorgeIt'd be nice if defn did s/instrument when you're working at the repl.#2019-10-0123:02Oliver GeorgeAny chance something like that is on the radar?#2019-10-0123:14Alex Miller (Clojure team)I think thatād be wrong for everyone sometimes#2019-10-0123:15Alex Miller (Clojure team)Iād prefer to have a lot more control over things like that#2019-10-0123:17seancorfieldAgreed. I definitely would not want auto-instrument at the REPL (and how would the compiler tell the difference between REPL and regular running since the semantics are the same?).#2019-10-0123:18seancorfield(remember that not everyone AOT's code!)#2019-10-0211:29Jakub HolĆ½ (HolyJak)@alexmiller I have a tiny improvement to gensub
to provide a meaningful error when such-that
fails, is it meaningful for send a patch to https://clojure.atlassian.net/browse/CLJ-2097? Essentially (gen/such-that #(valid? spec %) g 100)
-> (gen/such-that #(valid? spec %) g {:max-tries 100, :ex-fn (fn [{:keys [max-tries]}] (ex-info (str "Couldn't satisfy such-that predicate after " max-tries " tries.") {:path path}))})
(Since path
is the only thing that looks meaningful, contrary to test.check's gen, pred
and spec's spec, form
)
I have updated the issue, let me know there whether to send a patch. Thank you!#2019-10-0212:39Alex Miller (Clojure team)test.check 0.10.0 has some new hooks for this case too#2019-10-0212:41Alex Miller (Clojure team)if you have a patch, go for it, but I'm not sure whether we're going to go back and patch anything on spec.alpha. this is tbd, but my current thought is that once we get close to a spec 2 release I would reassess all the spec tickets (I know some have been fixed) and try to do a clean-up wave. this will require re-working any of the patches as I'd expect none of them to apply to spec 2.#2019-10-0215:05Jakub HolĆ½ (HolyJak)Thanks, @alexmiller! Question 2: I discovered that it is even more useful to include an example of why the spec failed on sample data - knowing which spec failed to match is good but knowing why is even better. In my case the spec failed was something like ::person
because of invalid :person :address :zip
. So, to troubleshoot my issue, I include this in the thrown error: {:max max-tries
:path path
:sample-explain (->> (first (gen/sample g 1))
(explain-data spec)
:clojure.spec.alpha/problems)}
- which was extremely useful for me but not sure whether it would be an appropriate general solution.#2019-10-0215:06kenny@holyjak YES. The failing args and ret should be easily shown in the error message. I don't think that particular approach will work because there's no guarantee you'll hit the failing case with one sample but I like the idea.#2019-10-0215:08Alex Miller (Clojure team)showing an example which isn't the one that actually failed seems more confusing to me#2019-10-0215:09kennyYes - it needs to be the failed one.#2019-10-0215:09Jakub HolĆ½ (HolyJak)ALL the examples fail#2019-10-0215:09kennyWe have internal code that does this already but it's a hack on gensub.#2019-10-0215:10kennyNot always @holyjak.#2019-10-0215:10Jakub HolĆ½ (HolyJak)Well, in theory you are right. What we call is `(gen/such-that #(valid? spec %) g {:max-tries 1000 ...}) - none of the 1000 generated samples failed. What is the chance that the one we generate in the error does not fail?#2019-10-0215:11kennyDepends on how you're controlling the size.#2019-10-0215:12Jakub HolĆ½ (HolyJak)What do you mean?#2019-10-0215:12Jakub HolĆ½ (HolyJak)Of course I could add a check and include :sample-explain
only if it really fails. Would that be satisfactory? If not - what is a good way of capturing some generated, failing data?#2019-10-0215:12kennyIIRC, gen/sample
does not use all 200 size values.#2019-10-0215:13kennyUsing quick-check
or st/check
will.#2019-10-0215:13Jakub HolĆ½ (HolyJak)I am not sure how such-that
invokes the generator but I would assume that it does so in a way very similar to sample
?#2019-10-0215:15Jakub HolĆ½ (HolyJak)Anyway, what do you think about the "best effort" approach - generate a sample, if it fails the spec, include :sample-explain
in the error. I think that would work in 99.9% cases?#2019-10-0215:15kennyAnyway, the piece I find valuable is being able to easily see why a check failed, whether it be by generator failure or check failure. When a generator fails, I would like to see the failed samples. This usually lets me figure out which predicate is causing the issue pretty quick. In the case the check fails, I'd like to see the args and the ret.#2019-10-0215:17kennyYou should know which samples failed while generating @holyjak, no need to generate new ones.#2019-10-0215:18Jakub HolĆ½ (HolyJak)Well, I do not generate anything, it is such-that
that does it - and in its failure-handling function I have no idea what values were actually tried. Or? BTW here is the code https://gist.github.com/holyjak/8cadc0d939c8e637ef6bf75b070d28b4#file-clojure-spec-alpha-clj-L17#2019-10-0215:20kennyOur hack (not acceptable for a patch but let's us debug generators without going crazy) is to store invalid values in an atom and display that in the error message.#2019-10-0215:20kennySo we just change the such-that predicate to do that.#2019-10-0215:22Jakub HolĆ½ (HolyJak)Smart! The question is: What approach would be acceptible for a patch (and useful for the users!)?#2019-10-0215:23kennyI don't know. It definitely feels too hacky for a patch. It may be the only way to do it though. When I was looking at this, I don't think I saw any test.check hook to grab the failed values.#2019-10-0215:24kennyJust speculating but I imagine it'd be somewhat easy to add that hook to test.check if it isn't already there and is the proper solution.#2019-10-0215:25kennyI would definitely be in favor of adding some sort of patch to spec1 alpha given we don't know when spec2 is coming. Debugging generators is super painful as it is.#2019-10-0215:25gfredericksCheck the such-that docstring in the latest version#2019-10-0215:26kennyDoesn't appear to have anything that stores failed samples.#2019-10-0215:27gfredericksWould you want all of them? Just the latest?#2019-10-0215:27gfredericksHaving the generator means it should be easy to generate more#2019-10-0215:27kennyProbably all.#2019-10-0215:28kennyIf I have all, I can easily only have the latest if I want.#2019-10-0215:28Jakub HolĆ½ (HolyJak)I believe a single value is enough (preferably the 1st as it is the simplest one)?#2019-10-0215:29kennyTrue.#2019-10-0215:29gfredericksKeeping all could be a GC issue#2019-10-0215:29Jakub HolĆ½ (HolyJak)> Having the generator means it should be easy to generate more
That is what I do in my solution (generate a new value using sample
) but it has been argued that using one of the actually failed would be better.#2019-10-0215:29kennyI guess I only ever work with one of the failed samples anyway.#2019-10-0215:31Jakub HolĆ½ (HolyJak)@gfredericks As I explained above, this popped up when working with specs - a custom generator generated value that did not conform to the spec. I can get the name of the spec but want to see why the custom gen failed to provide anything valid. There, having a sample value and running s/explain-data
on it is sufficient and very useful.#2019-10-0215:31kennyLooks easy to add in such-that-helper
.#2019-10-0215:31Jakub HolĆ½ (HolyJak)I am creating an issue on test.check to pass a sample failed value to ex-fn. Stop me if I should not š#2019-10-0215:32gfredericksYou shouldn't not#2019-10-0215:36Jakub HolĆ½ (HolyJak)Conclusion: I create a patch for Spec to include the spec name (or rather path
) in the such-that failures and we wait for test.check to expose a sample failed value to the :ex-fn
before adding explain-data
of it.#2019-10-0215:39Jakub HolĆ½ (HolyJak)@alexmiller is there any point in providing also a similar patch for Spec 2 or is it too much in flux?#2019-10-0215:40Alex Miller (Clojure team)I'm not going to do the sample thing, so not interested in a patch for that#2019-10-0215:41Jakub HolĆ½ (HolyJak)no, not that, just including path#2019-10-0215:41Alex Miller (Clojure team)that's fine#2019-10-0215:41Alex Miller (Clojure team)I don't think there's any point for spec 2#2019-10-0215:41Alex Miller (Clojure team)one possible outcome is that we repackage everything in spec 2 to remove alpha designation, which will (again) break any patches#2019-10-0215:45Alex Miller (Clojure team)I think in test.check, while generating in such-that, it would be possible to just retain the first or last gen'ed every time (before the pred check) and report that if you hit the retry limit. that way you're reporting an example that was actually tried.#2019-10-0215:50gfredericksAn example generated after the fact is just as meaningful, philosophically#2019-10-0215:53Alex Miller (Clojure team)if only a percentage of them fail, and you happen to generate a sample that doesn't fail, that seems strongly less useful than one of the actual examples that didn't pass the predicate#2019-10-0215:53Alex Miller (Clojure team)so I'ma disagree with you on that#2019-10-0215:53gfredericksI was assuming you'd filter on failure š#2019-10-0215:53Alex Miller (Clojure team)well you failed to mention that :)#2019-10-0215:53Alex Miller (Clojure team)but that would be fine#2019-10-0215:53gfredericksWhich, ironically, could fail š though presumably is unlikely to#2019-10-0215:54gfredericksBut that might be a good enough reason not to do it#2019-10-0215:55gfredericksAdding the size to the failure data would assist with that though#2019-10-0215:57Jakub HolĆ½ (HolyJak)@gfredericks Will you consider adding a sample failed value to the arguments of such-that's :ex-fn
or some alternative improvement?#2019-10-0216:04gfredericks@holyjak I'm not going to be doing active work on test.check for the foreseeable future, so that'll be up to somebody else; but a jira ticket is definitely the right way to make sure it gets looked at#2019-10-0216:25Jakub HolĆ½ (HolyJak)Hm, I was about to create it but you told me I should not https://clojurians.slack.com/archives/C1B1BB2Q3/p1570030324189400 I guess it was just misunderstanding.#2019-10-0216:25Jakub HolĆ½ (HolyJak)I'd be happy to provide a patch to test.check - and even happier to get any pointers regarding the best solution.#2019-10-0216:29gfredericksNo I meant you should create the ticket#2019-10-0216:29gfredericksSorry, I was being cute with English double negatives, it was probably a bad idea#2019-10-0216:34Jakub HolĆ½ (HolyJak)I see š My bad, I should have read more carefully. Also, I believed that English normally doesn't permit double negatives. Always learning š#2019-10-0216:35Jakub HolĆ½ (HolyJak)Any pointers regarding the best way to implement this? Is failed-value
a good key name to pass to :ex-fn
?#2019-10-0216:35Jakub HolĆ½ (HolyJak)FYI I have created https://clojure.atlassian.net/browse/TCHECK-156 Provide sample failed value to such-that's ex-fn for better error messages#2019-10-0216:54Jakub HolĆ½ (HolyJak)FYI I have sumbitted a patch for āļø#2019-10-0223:10nopromptWhat is the proper way to lawfully copy/paste source code from core.specs.alpha
?#2019-10-0223:10nopromptThis patch has caused me some headache: https://github.com/clojure/core.specs.alpha/commit/938c0a9725a284095baa2387dff1a29c3f1e26ac#2019-10-0223:11nopromptWhat Iād like to do is copy ::defn-args
and its dependencies into my own project.#2019-10-0223:14nopromptDo I just put the copy right information above the code I want to copy?#2019-10-0223:14nopromptAlso, my project is MIT licensed, so Iām not sure how all that works.#2019-10-0223:18noprompt> What are my obligations if I copy source code obtained from http://Eclipse.org and licensed under the Eclipse Public License and include it in my product that I then distribute?
> Source code licensed under the EPL may only be redistributed under the EPL.#2019-10-0223:19nopromptIām guessing I need to handwrite my own version of spec.#2019-10-0223:33seancorfield@noprompt I'm curious as to how you're depending on specs from that namespace since they are intended to support clojure.core
macros?#2019-10-0223:34seancorfield(and, yes, you're going to be on some pretty shaky ground if you copy'n'paste that code into your system and then distribute it)#2019-10-0223:34noprompt@seancorfield Imagine you want to write a macro that has the form as defn
.#2019-10-0223:36nopromptAnd imagine that I want to use conform
to parse ::core.specs/defn-args
.#2019-10-0223:36nopromptThats my situation.#2019-10-0223:37seancorfieldAh, so your code was relying on the internal names used in the those specs?#2019-10-0223:37nopromptIām noticing youāre using the word āinternalā.#2019-10-0223:37seancorfieldOnly two top-level specs changed their names.#2019-10-0223:38seancorfieldIf you conform
, you have to rely on the implementation of the spec.#2019-10-0223:38nopromptYeah but specs are globally publicā¦#2019-10-0223:38seancorfieldWe've used alpha versions of Clojure libs in production for over eight years and sometimes Cognitect change stuff while it's alpha and break your code. That's the risk of using alpha libs.#2019-10-0223:39nopromptI hope you can appreciate the irony in this.#2019-10-0223:39nopromptš#2019-10-0223:39nopromptTo me its a bit surprising given the critique of SemVer, etc.#2019-10-0223:39nopromptSo suddenly calling it āalphaā makes it okay to make breaking changes?#2019-10-0223:40noprompt::defn-args2
?#2019-10-0223:40seancorfieldThey've been very clear that alpha means "can change/break". Things are only guaranteed additive/fixative once the alpha label goes away.#2019-10-0223:40nopromptIts not on the README.#2019-10-0223:40nopromptWhere were they clear?#2019-10-0223:41seancorfieldThey've said it repeatedly publicly. Pretty sure even Rich has mentioned that in talks...?#2019-10-0223:41nopromptPublicly?#2019-10-0306:57jaihindhreddyRich says it in the same talk where he contends SemVer is broken.
Search for alpha in https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md
Funny thing though, Rich also says something like:
But, that is not to say just leave your thing 0.0.967. At a certain point, you are going to have users, and whether you change it to 1.0 or not, they are going to be depending on your stuff.
And that is where spec is going right now IMHO. But his Maybe Not talk and Alex Miller's blog eased my fears of spec remaining in this state forever.#2019-10-0223:41nopromptLike on Twitter?#2019-10-0223:41nopromptThe ML?#2019-10-0223:42nopromptI rarely, if ever, watch talks.#2019-10-0223:42nopromptThe README and the source code are far more public.#2019-10-0223:42nopromptThe disclaimer should be there.#2019-10-0223:43taylorwhat would it say#2019-10-0223:43nopromptWhat Sean said.#2019-10-0223:44nopromptHow about
> Alpha means this library ācan change/breakā do not rely on it.#2019-10-0223:44nopromptBecause I obviously misunderstood the situation.#2019-10-0223:45seancorfieldI'm sorry you've missed it being discussed on the mailing list and in Rich's talks and here on Slack (where it has come up several times in several situations).#2019-10-0223:46nopromptI just want answers to my questions.#2019-10-0223:47nopromptCan/How do I copy the source code?#2019-10-0223:48seancorfieldNot legally into anything you are distributing under an incompatible license.#2019-10-0223:49seancorfieldIf you want to use EPL code in your OSS project, consider changing your license to EPL so it's compatible.#2019-10-0223:49nopromptOkay, thatās what I gathered by reading the EPL website.#2019-10-0223:50nopromptIts fine, I can write the specs myself in this case. I was just hoping I could take a short cut.#2019-10-0223:50nopromptš#2019-10-0223:50seancorfieldGiven the day I've had working on our Spec 2 branch, I do sympathize (even if it may not sound like it).#2019-10-0223:51nopromptHonestly, it didnāt feel that way but you saying that makes me feel like you do.#2019-10-1415:49andy.fingerhutSomething like a{1,2}
can be hand-expanded into what it means, which is a|aa
. The expansions can get pretty long, of course.#2019-10-1414:00Alex WhittHi! I have some s/keys
specs in which the keys' specs use s/or
, and the map is only valid for certain combinations of s/or
paths. I ended up writing a function, keys-conform?
, which allows me to give a map of keywords to spec path, and check those spec paths against the conformed values:
(s/def ::foo (s/or :int int?
:string string?))
(s/def ::bar (s/or :branching ::foo
:kw #{:a :b}))
(s/def ::map (s/keys :req [::foo ::bar]))
(deftest keys-conform?
(are [?conform-test ?input ?result]
(= ?result (keys-conform? ?conform-test (s/conform ::map ?input)))
{::foo :int ::bar :kw} {::foo 1 ::bar :a} true
{::foo :int ::bar :kw} {::foo "hi" ::bar :a} false
{::foo :int ::bar :branching} {::foo 1 ::bar "hi"} true
{::foo :int ::bar [:branching :int]} {::foo 1 ::bar :a} false
{::foo :int ::bar [:branching :int]} {::foo 1 ::bar 1} true))
My typical use-case involves using keys-conform?
in other specs:
(s/def ::specific-map
(s/and ::map
(s/or :both-int
#(keys-conform? {::foo :int ::bar [:branching :int]} %)
:string-and-kw
#(keys-conform? {::foo :string ::bar :kw} %))))
(s/valid? ::specific-map {::foo 1 ::bar 1})
;=> true
(s/valid? ::specific-map {::foo 1 ::bar "hi"})
;=> false
(s/valid? ::specific-map {::foo "hi" ::bar :a})
;=> true
(s/conform ::specific-map {::foo "hi" ::bar :a})
;=> [:string-and-kw #:user{:foo [:string "hi"], :bar [:kw :a]}]
This works, but as I've been using it, I'm finding I also want access to more subtle conditions for valid paths, like "or" and "not", and the logic is getting more complex.
So I'm wondering if there's already a built-in way to do what I'm doing, or if someone has written a library like this? I can't think of a good way to use multi-specs for this, but maybe someone else can?
(Also, I'll preempt any possible objections that ::foo
and ::bar
should be split up into different keys. In my use case, they really do have to be the same key with different possible paths.)#2019-10-1420:04seancorfield@alex.joseph.whitt Is there any sort of disciminant you could use to turn this into a multi-spec?#2019-10-1420:06Alex WhittThe problem seems to be that the branching at the level of ::specific-map
is sort of multi-dimensional. So there's not necessarily one keyword that changes everything. The options are sort of mix-and-matchable.
Do you think there might still be a way?#2019-10-1420:08Alex WhittThe domain in question is parsing a rather nuanced binary protocol called BACnet. The packets have a number of valid shapes, but just as many invalid combinations of fields.#2019-10-1420:08Alex WhittAnd the fields are obnoxiously polyvalent#2019-10-1420:12Alex WhittPerhaps there could be a solution in nested multi-specs? Like if the flag in the header says this is a foo-packet, check foo-spec, and foo-spec has a number of other options under it based on the given values?#2019-10-1420:22Alex WhittWell actually, it seems like multi-specs would just be replacing the use of s/or
in ::specific-map
... I don't think that would change the use of keys-conform?
...#2019-10-1420:44misha@alex.joseph.whitt do you have specification document url?#2019-10-1420:45Alex WhittDo you mean you want to see my real use case?#2019-10-1420:45mishayes#2019-10-1420:47mishabut I think the issue here is your attribute tries to be too much different things. I'd try to model it with many attributes instead, e.g.
(s/def : int?)
(s/def :bac.str/foo string?)
,,,
#2019-10-1420:48mishamight save you some if
s down the road#2019-10-1420:50mishaanother question is: "why exactly do you want to spec that?"#2019-10-1420:50mishaand "how many valid combinations there are vs. how many possible ones?"#2019-10-1420:55mishalol, it's 150$ for pdf#2019-10-1420:58Alex WhittI can put a portion of my code up in a gist in a bit.
I have a few main reasons why I want to spec it:
1. To validate packets and my parsing logic
2. To have the ability to generate random but valid packets for automated test purposes (testing an embedded device that supports the BACnet protocol).
3. Generative testing for code that touches BACnet
How many valid combinations: Depends. For most parts that would use this pattern, probably less that 6 for each level of abstraction. The possible combinations are probably 2-3 times that number.
Heh, yeah it's a closed standard.
As for splitting the keys, that's a pattern I can and do utilize, but at some point it does boil down to one key representing multiple possibilities due to the nature of the fixed positions in the binary protocol. I'll keep noodling on it though.#2019-10-1421:00misha> BACnet currently defines 35 message types, or "services," that are divided into 5 classes.
sound like you can put em into multispec#2019-10-1421:04mishaif messages are sequential, did you try to write reg-ex specs for them, instead of map ones?
https://clojure.org/guides/spec#_sequences#2019-10-1421:06Alex WhittI'm working at a different level than the message types at the moment. At the message type level, absolutely a multi-spec works (and that's what I use.) Here's my code for the generic header:
https://gist.github.com/WhittlesJr/4571fed1596400e242724972ee39b2d4#2019-10-1421:06mishaso the general advice is to reduce combinatorial explosion, and spec only things you need to spec. how exactly ā depends on that protocol, and what exactly you are doing with it.#2019-10-1421:07mishacan you show raw header/message/whatever?#2019-10-1421:07Alex Whittyeah, one sec#2019-10-1421:08mishaI think this
(s/def ::class
(s/or :application #{:application}
:contextual #{:contextual}))
can be just
(s/def ::class #{:application :contextual})
#2019-10-1421:09mishasorry, Sean kappa#2019-10-1421:10Alex WhittHere's an example capture of BACnet packets. Open in Wireshark and apply the bacnet
filter. You'll probably also need to add UDP port 47824 as a BVLC decoding. (Analyze -> Decode as... -> +)#2019-10-1421:11Alex WhittI used the s/or
there for testing branching in ::attributes
#2019-10-1421:12Alex WhittAlternatively I suppose you could use a multi-spec, but seems to be 6 of one 1/2 dozen of the other#2019-10-1421:21mishawell, looking at raw data did not help opieop#2019-10-1421:22Alex WhittMerp. The BACnet spec is really dense and nuanced, takes a ton of contextualizing to grok the issues involved.#2019-10-1421:25mishathis is what I dislike about it: context. spec works great when attributes are the same regardless of context. and it (still) seems to me, flattening out amount of valid attribute combinations with different attribute names would be simpler solution, than trying to replicate 1to1 all the branchyness in specs (don't forget to write custom generators hehe)#2019-10-1421:27mishalooking at spec names (all ::/
) namespaces of attributes don't mean much for you, so I'd try to encode types in those, to keep attribute name "polymorphic" (if you need for something)#2019-10-1421:28Alex WhittYeah, I have to wrestle with the high-context nature of the spec when writing the parsing logic. It's horrible.
I'll look into breaking it up, but something tells me that it won't be a fully satisfying solution. Part of the purpose of my code is to represent the packet fully, much like Wireshark does. It will make it less obvious what the actual binary fields are saying if I have three different optional keys rather than one key that can have three different types. But that's more of a human-level issue rather than a code-level issue.#2019-10-1421:30mishawhat are you parsing exactly? what is your input to spec?#2019-10-1421:33Alex WhittIf you want to look at the capture, check out packet 337 -> "Building Automation and Control Network APDU" -> "list of Values:" -> "{[4]". You should be able to recognize the header in there, which is the part of the spec that my gist is dealing with.#2019-10-1421:34Alex WhittEverything under the "list of Values:" has one of those headers (or is a standalone "header" acting as an opening or closing tag)#2019-10-1421:38Alex WhittAnyway, the approach I'm taking is working, I was just curious if there was already a solution out there or if I was coming at this from the wrong angle. Depending on how this goes, I may just make a library out of keys-conform?
with some well-defined syntax. If my employer will let me open source anything, that is... :c#2019-10-1421:40mishadoes it have generator tho? :)#2019-10-1421:43Alex WhittYeah, actually I'm doing pretty well with generators, surprisingly. I'm finding that the coercion logic a la spec-coerce
is synergizing really well with generators.#2019-10-1421:44Alex WhittEspecially as you climb up the abstraction chain and you get more and more fields that are interdependent, the coercion approach is helping to keep generation sane#2019-10-1421:45Alex WhittBut maybe I have yet to hit a real barrier and it's lurking somewhere in the thickets, waiting to spring an ambuscade on my productivity#2019-10-1421:46Alex Whitt::attributes
in the gist generates out of the box#2019-10-1511:52mpenetis there a way to express negation of other specs in a regex spec other than something like (s/+ #(not (some (fn [spec] (s/valid? spec %) [::foo ::bar])))
#2019-10-1511:52mpenetI am sure I am missing the obvious#2019-10-1512:23Alex Miller (Clojure team)Not missing anything#2019-10-1512:23Alex Miller (Clojure team)Negation is fairly unusual in specs #2019-10-1512:24mpenetyeah it doesn't even work for my use case actually, I am better of with a simple predicate#2019-10-1514:06misha@mpenet is that pseudocode or actual code?#2019-10-1514:13mpenetpseudo#2019-10-1514:13mpenetI went with something else btw. I was specing a form similar to try/catch/finally#2019-10-1514:14mpenetgetting the body accurately was the target#2019-10-1617:22rickmoynihanIn spec 1 keywords are privileged with regards to s/keys
. Iād love for spec 2 to let me define other types of values to name key specsā¦ e.g.
(s/defkey "foo" integer?)
(s/keys :req-key ["foo"])
Iām not actually so bothered by string keys, as what Iād really like to do is use URIās as keys. My use case is that we work with RDF, and itās really annoying having to convert between URI predicates and clojure keywordsā¦ Iād like to essentially do:
(def rdfs:label (URI. "http://,,,,#label"))
(s/defkey rdfs:label (s/or :xsdstring string? :langstring langstring?)
(s/def ::resource (s/keys :req-key [rdfs:label]))
Any ideas on whether this sort of thing might be possible in spec2?#2019-10-1617:23Alex Miller (Clojure team)no#2019-10-1617:24rickmoynihanimplementing my own s/keys seems like itāll be hard work#2019-10-1617:25rickmoynihanmight there be an easy way to do it that can leverage stuff in spec?#2019-10-1617:25Alex Miller (Clojure team)the idea of keywords as names for specs is deeply embedded#2019-10-1617:25Alex Miller (Clojure team)not just s/keys but every spec impl relies on that#2019-10-1617:25rickmoynihanI wouldnāt mind having a layer of indirection between the name of the spec and the URI/key#2019-10-1617:25Alex Miller (Clojure team)we have no plans to make that#2019-10-1617:28rickmoynihanso would doing:
(s/def :rdfs/label (URI. ""http://,,,,#label"))
(s/def ::resource (rdf/keys :req [:rdfs/label]))
not be feasible?#2019-10-1617:29Alex Miller (Clojure team)no#2019-10-1617:30Alex Miller (Clojure team)it is possible that we might allow arbitrary metadata for specs. if we did that (and I'm not committing to it yet), then you could hang your mapping to a uri on that.#2019-10-1617:35Alex Miller (Clojure team)in the meantime you could create such a mapping outside spec if desired#2019-10-1707:17vlaaad@alexmiller can you share rationale behind decision to limit spec names to keywords? (and symbols, right? for fdef
s...)#2019-10-1713:53jaihindhreddyI believe, it's more like: spec allows us to spec data and functions. We spec functions by their name and pieces of data by their name (keywords), symbols are already used to name vars in the context of the language, which is why symbols can't be used. All other stuff (strings, characters, aggregates, etc) is not namespace qualified and hence those cannot be "precise" names. That causes collisions and is a recipe for brittleness IMHO.#2019-10-1708:04rickmoynihan@vlaaad: Though Iām asking whether itās possible or suggestions on how to do it; I totally understand the reason to name specs with keywords. I suspect the main reason is that they support namespaces; and can therefore be unambiguous and named in a global context, but donāt require a dependency on being loaded like vars. Allowing the use of a key to be decoupled from its spec/definition, whilst also allowing the same key to have multiple specs. Like with RDF it means itās open for extension, anyone can spec anything etc.
The reason Iād like to do it, is because when I convert RDF into a clojure structure, itād be nice not to have too coerce URIās (which are used to identify properties & classes in the same way spec uses keywords) into keywords. However itās probably not so bad, maintaining that mapping. I was just hoping itād be possible to write my own version of s/keys
that used URIās directly.#2019-10-1714:45mpenetit's super easy to make your own in the meantime. Just use atom as metadata registry + write few fns (alter-meta, doc, meta, etc). then it's possible to chain (-> (s/def ::foo string?) (stg/with-doc "bla bla bla"))
since specs returns their key.#2019-10-1718:48eskemojoe007I'm trying to create a spec around a time characterstic in a cljc
file and getting confused. I generally try to use tick
to manage time within cljs and clj. But their api doesn't have any tests for type of time that I'd like to put into the spec. What do others do for cljc spec that involve times?#2019-10-1803:59jjttjjHave you seen https://github.com/henryw374/time-specs#2019-10-1720:23dominicmDoes inst? work?#2019-10-1817:54agwhatās the idiomatic way to assert results of stest/check
?#2019-10-1818:07taylornot sure how idiomatic this is but (deftest foo-test
(is (= 1 (-> (st/check `foo)
(st/summarize-results)
:check-passed))))
#2019-10-1818:09agokay, but letās say it returns nil
and I want to print that out with expound, I guess Iām gonna have to run st/check
one more time?#2019-10-1818:10taylornot sure about the expound part#2019-10-1818:12ag(expound/explain-results (stest/check `foo))
would print out the reason in human readable form, right? But it doesnāt return anything, so no way to assert it.
and I canāt do something like:
(let [chk (stest/check `foo] ,,,
to re-use it for both#2019-10-1818:13agfor assertion and for human-readable output#2019-10-1818:13taylorIāve never tried to use expound with test.check output#2019-10-1818:13taylorwhich part doesnāt return anything?#2019-10-1818:14taylorexplain-results
? you could use (doto (stest/check `foo) expound/explain-results)
if so#2019-10-1818:19agah yeah, nifty doto
. Cool, thanks!#2019-10-1818:41agah, the problem with doto approach though - stest/check
throws on failureā¦ damn#2019-10-1818:52taylorhmm really? maybe Iām thinking of something else but I thought it returned a map with useful failure info#2019-10-1818:52tayloror maybe thatās in the exception#2019-10-1818:06agcan someone show me an example of deftest
where combination of stest/check
used with expound
#2019-10-1818:13bbrinckhttps://gist.github.com/kennyjwilli/8bf30478b8a2762d2d09baabc17e2f10#gistcomment-2682609#2019-10-1818:14bbrinck@ag -^#2019-10-1818:16agOhā¦ wowā¦ this is pretty cool.#2019-10-1818:16bbrinckAlso apparently Kaocha can automate some of this https://cljdoc.org/d/lambdaisland/kaocha/0.0-554/doc/automatic-spec-test-check-generation#2019-10-1818:16bbrinckHavenāt tried it yet, but itās on my list. #2019-10-1818:21agReally awesome ideas, thank you guys!#2019-10-1818:36kszabowe use Kaochaās feature, it works very well#2019-10-2012:48abdullahibraHell everyone,#2019-10-2013:08abdullahibracan i pass the spec to function?
(defn validate [data spec]) ?, or this isn't right, because the spec is globally defined so i can use it inside the function directly ?#2019-10-2015:07rascioyes you can, it is what the default validate does.
specs are just values if you want to do a function that can work with any kind of spec you should do it.#2019-10-2418:48agdo we have any libraries that can generate specs based on DB schema, e.g.: postgress DDLs?#2019-10-2420:27aghas anyone dealt with type of a problem where you need a spec for a map with underscored keys (for data coming from DB) and absolutely same spec with the only difference being that keys are hyphenated? Maybe someone wrote a macro?#2019-10-2420:37ikitommiThere is no way to close a Spec2 Select? e.g. dontā allow any extra keys, right?#2019-10-2420:43ikitommiSo, if i have s Schema with ::id
, ::name
, ::description
and ::secret
keys and I would like to make an api where you must send ::id
& ::name
, optionally ::description
, but not ::secret
, I would first have to make an (api) Schema with just ::id
, ::name
and ::description
close that (so the ::secret
becomes illegal) and make a select for that schema with just ::id
and ::name
defined so the ::description
becomes optional. Right?#2019-10-2420:55seancorfieldSpec 2 uses an option hash map to specify which specs are closed -- at checking time (`s/valid?` s/conform
etc)#2019-10-2420:56seancorfieldselect
is always open. It just says what is required.#2019-10-2421:02ikitommiThanks, thatās my understanding too. So, to support a use case that one canāt send the ::secret
, I need to create a concrete new Schema to be able to mark is as closed.#2019-10-2421:03seancorfieldThe checking code can determine which specs are treated as closed -- independent of schema
/ select
.#2019-10-2421:04seancorfieldhttps://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#closed-spec-checking#2019-10-2421:08ikitommi(s/def ::id int?)
(s/def ::name string?)
(s/def ::description string?)
(s/def ::secret boolean?)
;; the internal schema
(s/def ::user (s/schema [::id ::name ::description ::secret]))
;; the api schema
(s/def ::user2 (s/schema [::id ::name ::description]))
;; the api select (id & name mandatory, description optional)
(s/def ::user3 (s/select ::user2 [::id ::name]))
;; closed validation
(s/valid? ::user3 {::id 1, ::name "kikka"} {:closed #{::user2}})
#2019-10-2421:28vlaaadi wonder why closed specs are implemented as options instead of as another type of spec#2019-10-2421:38seancorfield@vlaaad Alex talks about that on the Inside Clojure blog as Spec 2 went down the "closed spec" path first and then changed to "closed checking".#2019-10-2421:41vlaaadyou mean this http://insideclojure.org/2019/04/19/journal/ ?#2019-10-2421:42vlaaad(s/close-specs ::s)
this is some mutability thing, I was talking about having closed specs as different specs, no mutability attached#2019-10-2421:42vlaaadlike that:
(s/def ::open-keys (s/keys :req-un [::some-key]))
(s/def ::closed-keys (s/closed ::open-keys #{::open-keys}))
#2019-10-2421:45vlaaadI think that makes sense: being able to refer to your data specification by name, instead of by combining a name and an arguments to s/valid?
#2019-10-2421:47seancorfieldI'll leave it up to @alexmiller to respond. I prefer the path they've taken where "closedness" is just an option on checking.#2019-10-2421:48vlaaadIf you have it as another spec it's still opt-in, but now it's also composable#2019-10-2421:50Alex Miller (Clojure team)Not going to do that#2019-10-2421:51Alex Miller (Clojure team)Itās not composable - negation is inherently problematic as specs evolve#2019-10-2507:20vlaaadI don't get it, can you explain? I thought about closed specs for a bit, and the way I see them, they can be just a s/select
counterpart, where s/select
describes lower bound of data shape ("be at least that"), while s/closed
describes upper bound ("be at most that").
By composable I mean using same specs both as open and as closed in same s/valid?
etc. checks. I think it might make sense, for example, if I were to spec a function that cleans user input before putting it into db, it's spec would look like that:
(s/fspec
:args (s/cat :user ::user)
:ret (s/closed ::user))
Speaking of specs evolution, do you mean being able to tell if change of spec from a to b is accretive or not? I don't see problems here as well:
- changing (s/closed (s/keys [::name]))
to (s/closed (s/keys [::name ::age]))
in return position provides more ā it returned ::name
, now it also can return ::age
;
- changing (s/closed (s/keys [::name]))
to (s/closed (s/keys [::name ::age]))
in argument position requires less ā it didn't accept ::age
, now it accepts it;
- changing (s/closed (s/keys [::name]))
to (s/keys [::name])
in return position provides more ā it used to return only ::name
, now it may return more.#2019-10-2512:39Alex Miller (Clojure team)those words do not match what you're saying in the specs. the closed version says "MUST NOT contain ..."
1/3. is probably ok from the consumers point of view but is contextual to use
2. if you provide age on the first one, then forbid on the second one, you've broken users that previously passed age#2019-10-2512:42Alex Miller (Clojure team)this notion of context is critical and is driving the changes in schema/select/fdef, so I wouldn't rule out some way to make a more restrictive statement at the point of use, although we currently think of that again as more of a "check", not part of a spec#2019-10-2513:04vlaaad> those words do not match what you're saying in the specs
what words?#2019-10-2513:07vlaaadmaybe we have different definitions of closed? My (s/closed [::name ::address])
says that map may be empty, or contain ::name
, or contain ::address
, or contain both, but no extra keys are allowed#2019-10-2513:11vlaaadlike select
, that requires at least this and allows more, but in different direction: requires at most this and allows less. in different contexts select
, closed
and a combination of both might be useful#2019-10-2513:12vlaaadso in "2" you could not provide ::age
before: this is invalid, and version after allows it#2019-10-2816:56vlaaad@alexmiller bump#2019-10-2421:57ikitommiWhat if they were part of the select?#2019-10-2422:32Alex Miller (Clojure team)well, again the principle here is that "closedness" is part of the checking, never part of the spec#2019-10-2504:43ikitommiI agree on that, but have thought that select is a utility of making checking context (values!) from schema. It would allow one place & syntax to easily close any of the nested maps too. The current syntax of closing specs by name in s/valid?
with different (nested maps) syntax adds more things for the developer to keep in his/her head. Need to jump to both schema and select definitions to see what subkeys are including in the select and need to be closed. Also, if schemas & selects get refactored, the closed options might get out-of-sync.#2019-10-2421:58ikitommi(s/select ::user [::id ::name s/closed])
#2019-10-2422:01ikitommiAlso:
(s/select ::user [::id ::name [s/optional ::description] s/closed])
#2019-10-2422:05ikitommiIn the code I pasted above, keysets need to be introduced in three places: the root schema, the api schema and the select. Also, partially in the s/valid?
call to make it closed.#2019-10-2422:15seancorfieldI was a bit surprised by this: user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::thing (s/schema [::a ::b ::c]))
:user/thing
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::c 3} {:closed #{::thing}})
true ; fine -- ::c is optional but present
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2} {:closed #{::thing}})
true ; fine -- ::c is optional and omitted
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::d 4} {:closed #{::thing}})
true ; huh? -- ::d is not in ::thing -- why doesn't this fail?
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
false ; because ::b is required but omitted... see next...
user=> (s/explain-str (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
"#:user{:a 1, :d 4} - failed: (fn [m] (contains? m :user/b))\n"
user=>
@alexmiller is that expected or a bug?#2019-10-2422:17seancorfielduser=> (s/def ::sub-thing (s/select ::thing [::a ::b]))
:user/sub-thing
user=> (s/valid? ::sub-thing {::a 1 ::b 2 ::d 4} {:closed #{::sub-thing}})
true
user=> (s/valid? ::sub-thing {::a 1 ::b 2 ::d 4} {:closed #{::thing}})
true
(still puzzled)#2019-10-2422:29Alex Miller (Clojure team)there's some lingering tech debt in the schema/select code where some things are duplicated rather than having select lean on schema's validation code. I think once that's cleaned up, these will fail as you expect#2019-10-2723:01kennyIs there a way to write s/and
s without having the conformed value passed to the predicate? All of our s/and
predicates call s/unform
on the value before calling the actual predicate function. This seems unnecessary. For example:
(s/def ::base-metric
(s/and
::metric/metric
#(metric-bounds-lower-less-than-or-equal-to-upper?
(s/unform ::base-metric %))
#(metric-default-value-within-bounds?
(s/unform ::base-metric %))))
#2019-10-2800:15Alex Miller (Clojure team)currently, no. we will probably have both flowing and unflowing forms in spec 2#2019-10-2807:53mishayou might get away with moving spec to the last position in the s/and, depending on resilience of your predicates to random input. @kenny
or, if this is the only place you use predicates ā make them expect conformed value#2019-10-2816:05kennyHmm. Can't do the latter because the predicates are used on the regular form of the data too.#2019-10-2816:19Alex Miller (Clojure team)you could wrap s/nonconforming
around each pred#2019-10-2816:20Alex Miller (Clojure team)(s/def ::base-metric
(s/and
(s/nonconforming ::metric/metric)
(s/nonconforming metric-bounds-lower-less-than-or-equal-to-upper?)
(s/nonconforming metric-default-value-within-bounds?)))
#2019-10-2816:20Alex Miller (Clojure team)something like that#2019-10-2816:24kennyAh, that's a lot nicer.#2019-10-2816:26kennyThat doesn't appear to work for all cases.#2019-10-2816:27kennyOh, I missed the first one.#2019-10-2816:31kennyThis is going to make things so much cleaner! Thanks @alexmiller š
(defmacro and2
[& pred-forms]
(let [pred-forms (map (fn [pred-form]
`(s/nonconforming ~pred-form)) pred-forms)]
`(s/and
#2019-10-2817:15ataggartPer the description of every
, the :kind
pred "must generate in order for every to generate". I see that preds like set?
can generate, e.g., (gen/sample (s/gen (s/spec set?)))
, but I'm at a loss as to how to provide generators for custom preds.#2019-10-2822:51Alex Miller (Clojure team)there is no open mechanism to bind generators to custom preds currently#2019-10-2822:53Alex Miller (Clojure team)you can provide the :kind pred as a separate spec outside s/every too
(s/and (s/with-gen my-pred ...) (s/every ...))
#2019-10-2822:54Alex Miller (Clojure team)that said, what kind of pred are you using that's not a core pred?#2019-10-2915:32ataggartThe specific case of wanting to use every
over a priority queue.#2019-10-2915:51Alex Miller (Clojure team)prob easier to attach a generator to the overall spec#2019-10-2916:30ataggartYup#2019-10-2919:59ataggartI assume the same goes for predicates like uuid?
#2019-10-2920:00ataggartIn that we can't provide generators directly for custom predicates#2019-10-2920:00Alex Miller (Clojure team)I think uuid? has a generator? or maybe that's just one we've talked about#2019-10-2920:01Alex Miller (Clojure team)yeah, uuid? gens#2019-10-2920:01Alex Miller (Clojure team)you can generally wrap specs with custom generators around preds with s/with-gen or s/spec#2019-10-2920:01Alex Miller (Clojure team)the particular case of :kind in s/every is a place where you don't have that opportunity#2019-10-2920:46ataggartI've been fighting to successfully use with-gen for about an hour now#2019-10-2920:47ataggartAnd I just figured it out.#2019-10-2920:48ataggartspec.gen
wants fns for everything, but I'm used to test.check
which doesn't.#2019-10-2920:48ataggartI'm good now, thanks for your help#2019-10-2920:51Alex Miller (Clojure team)yep, it's thunked to make the load step dynamic and optional#2019-10-2920:54ataggartYeah, makes sense#2019-10-2920:55ataggartIt's a one-time problem#2019-10-2920:55ataggartNow I know#2019-10-2907:46jumarLooking at spectrum, I can't figure out how to use it: https://github.com/arohner/spectrum
I know it's (still) not recommended for any kind of use, but I was curious about what it can do.
However, I don't even know what kind of output can I expect from it.
Trying the most basic example doesn't produce anything useful:
(defn foo [i]
(inc i))
(f/infer-var #'foo)
;;=> couldn't find analysis for #'clojure-experiments.spectrum/foo
;; throws AssertionError
#_(f/infer-form '(foo 3))
Using spectrum 0.2.5; code is here: https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/spectrum.clj#L11#2019-10-2907:58jumarIs anybody using this tool and if so how?#2019-10-2909:11danielnealas far as I know itās still all in development and not usable yet#2019-10-2921:36arohnerItās not ready yet. master
has much newer code, but is also not ready.#2019-10-3005:13jumarHmm. looking forward to hearing more in the future then#2019-10-2921:34ataggartI'm digging into it now, but does anyone have a guess as to what might be causing this?
user=> (require '[clojure.tools.logging :as log])
Syntax error macroexpanding clojure.core/defn at (clojure/tools/logging/impl.clj:241:1).
THIS - failed: vector? at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
THIS - failed: (or (nil? %) (sequential? %)) at: [:fn-tail :arity-n :bodies] spec: :clojure.core.specs.alpha/params+body
The line of code referenced above: https://github.com/clojure/tools.logging/blob/tools.logging-0.5.0/src/main/clojure/clojure/tools/logging/impl.clj#L241#2019-10-2921:34ataggartI'm not yet sufficiently fluent in spec-error-ese.#2019-10-2921:48andy.fingerhut@ataggart Most likely an older version of clojure.tools.logging from before Clojure 1.9 was released, that contains some old syntax that Clojure 1.9's and later's stricter checking rejects, and someone has since fixed with a later tools.logging release.#2019-10-2921:49ataggartI'm using 0.5.0, which is the latest version. And that "someone" would be me. š#2019-10-2921:49ataggartdeps hell once again#2019-10-2921:49andy.fingerhutSorry, I was going for an answer appropriate for 99% of similar questions. Looking at the line of code now...#2019-10-2921:50ataggartlol np#2019-10-2921:50seancorfield(! 775)-> clojure -Sdeps '{:deps {org.clojure/tools.logging {:mvn/version "0.5.0"}}}'
Clojure 1.10.1
user=> (require '[clojure.tools.logging :as log])
nil
user=>
Something in your environment @ataggart?#2019-10-2921:51seancorfield(and I did wonder if a version conflict was bringing in an older version)#2019-10-2921:52seancorfieldFWIW, we use tools.logging 0.5.0 at work with Clojure 1.10.1 -- which includes all the Spec checking on defn
-- and we don't see any problems.#2019-10-2921:52seancorfieldWhat other dependencies have you got in play?#2019-10-2921:53ataggartYep, I'm confident it's a deps issue. I'll keep digging. Aside, any guess on what that error means?#2019-10-2921:54seancorfieldIt's a badly formed defn
. Expecting an argument vector or sequence of pairs of arg vectors / bodies.#2019-10-2921:54andy.fingerhutIt mentions param-list
, so my first guess would be it thinks the defn parameter list is not a vector#2019-10-2921:54ataggartweird#2019-10-2921:56seancorfieldAre you using lein/boot/clj? What does your deps tree show? Could it be a badly-behaved plugin patching stuff?#2019-10-2921:56ataggartI'll find out and report back.#2019-10-2922:03ataggartSo, I changed the version to 0.4.1, same error. Changed it back to 0.5.0, and the error is gone....#2019-10-2922:04andy.fingerhutPlease keep your gremlins away from my computer.#2019-10-2922:05ataggartThe vision of a world of spec'd libraries without versions can't come soon enough.#2019-10-2922:06ataggartBack to work....#2019-10-2923:05Alex Miller (Clojure team)maybe a cached classpath? changing the deps file would invalidate the cache#2019-10-2923:05Alex Miller (Clojure team)so any change would have forced an update#2019-10-2923:06Alex Miller (Clojure team)if you're ever not sure with clojure
, adding an -Sforce can force that regardless#2019-10-3013:26ataggartIs there a canonical way to generate only a subset of the ātypesā from a multi-spec?#2019-10-3013:33sgerguriYou can force a multimethod clause to be selected with the following:
(s/gen (<multispec> <dispatch-arg>)))
#2019-10-3013:34sgerguriTo pick a subset of the multispecs you can make a collection of the valid dispatch args, then randomly sample that.#2019-10-3013:35ataggartThanks!#2019-10-3014:22ataggartHmm... doing that is yielding maps that have the correct shape for the type, but the value of the dispatching type key is random, not the dispatch value.#2019-10-3014:24ataggartI'm guessing it's because the result of (s/gen ...)
isn't flowing through the multi-spec's retag.#2019-10-3014:25sgerguriThat all depends on how tight your multispec clauses are. You can override what you get with fmap
like this:
(clojure.test.check.generators/fmap #(assoc % ...) (s/gen (<multispec> <dispatch-arg>)))
#2019-10-3014:26ataggart>That all depends on how tight your multispec clauses are.
What do you mean?#2019-10-3014:28sgerguriFor example, I have a multispec where I dispatch on two keys in the map I generate. The keys are unqualified, meaning I can strictly enforce that for a particular clause, I always get a particular value. Like this:
(s/def ::type #{:foo})
(defmethod multispec-fn :foo
[_] (s/keys :req-un [::type ...]))
#2019-10-3014:29sgerguriSo even if the s/keys
is part of a multispec for a dispatch value that involves :foo
, because the dispatch is done based on the value of a function applied to the data this way I make sure that when the above clause is selected, :foo
will be the value of the key :type
.#2019-10-3014:29sgerguriIn general though, if you do the fmap
above it should be fine.#2019-10-3014:30sgerguriIrrespective of whether you do something similar to what I just described or not.#2019-10-3014:33ataggartUsing the code here: https://clojure.org/guides/spec#_multi_spec
user=> (map :event/type (gen/sample (s/gen :event/event)))
(:event/error :event/search :event/search :event/search :event/error :event/search :event/error :event/search :event/search :event/error)
user=> (map :event/type (gen/sample (s/gen (event-type {:event/type :event/search}))))
(:+/- :*/. :*/gq :IZ/K :./.35 :YT/* :d/B :*Pk/q2L6 :Lf/J! :+/M-)
#2019-10-3015:27ataggart@U064X3EF3 Continuing the multi-spec example from the official guide, does this seem reasonable for the simple case of a kw retag?#2019-10-3014:33ataggartThe intention of the second one is to only yield :event/search
.#2019-10-3014:35ataggartThat's why I think just calling the multi-method isn't enough. Generating via the multi-spec would flow through retag, but the code above doesn't.#2019-10-3014:37ataggartThe master approaches...#2019-10-3014:37Alex Miller (Clojure team)I think going back to your original question, the answer is no, there is no way to (easily) generate a subset of the multispec values right now#2019-10-3014:38sgerguriIf you fmap
, however, you should still get the right thing.#2019-10-3014:38ataggartMan, I was all braced to receive your wisdom, Alex.#2019-10-3014:38Alex Miller (Clojure team)there are a variety of hacks :)#2019-10-3014:38ataggartYeah, I can manually retag#2019-10-3014:39Alex Miller (Clojure team)you can choose a particular one and retag, or you can dive into the guts of the multimethod dispatch value map (which is what the multispec generator does)#2019-10-3014:39sgerguriAlso :event/type
is defined as a keyword in the example, whereas in mine above I restrict that particular spec to a single value.#2019-10-3014:39Alex Miller (Clojure team)or you could s/and and only take events of a particular type#2019-10-3014:39sgerguriIt just so happens that I have the same unqualified key in all the multispecs defined difrerently per multispec.#2019-10-3014:40ataggartEvery time I try to use s/and for this issue I end up with such-that errors#2019-10-3014:40sgerguris/with-gen
is your friend there.#2019-10-3014:40sgerguriDon't be afraid to use it - I do in strategic places, typically around complex s/and
cases.#2019-10-3014:40ataggartproblem is I can't tell what's causing the such-that error#2019-10-3014:41ataggartWell, I can eventually, but it's not obvious when it happens#2019-10-3014:41ataggartok, I'm going to beat my head on this a little more#2019-10-3014:41sgerguriWell if it's an s/and
then simply provide your own generator by wrapping that in s/with-gen
.#2019-10-3014:41ataggartThe issue tends to be using s/and when I should use s/merge#2019-10-3014:42sgerguriPossibly. Though if you rely on custom conformers, s/and
is a very frequent thing to end up using.#2019-10-3014:43ataggartyep, just wasn't helping in this case#2019-10-3014:44ataggartretagging via fmap over the generated values from the multimethod is working though.#2019-10-3014:46ataggartFound the such-that:
user=> (s/gen :event/event)
#clojure.test.check.generators.Generator{:gen #function[clojure.test.check.generators/such-that/fn--6656]}
#2019-10-3014:46ataggartš#2019-10-3015:06orestisI know it's a vague question, but what's the upgrade path from spec -> spec2 look like? I know there's changes coming in s/keys
and I guess the functions that validate/conform/explain might change, but I was wondering how much of the spec definitions will change?#2019-10-3015:08orestisI'm trying to figure out if we should start with spec2 directly, and deal with the breakage as it comes (I guess it's pre-alpha), or start with spec and deal with the upgrade breakage when it comes.#2019-10-3017:05seancorfieldWe've been trying to keep a branch of our codebase at work up to date on Spec 2 but it's required quite a few changes to some of our Spec 1 code and at this point, we've essentially "given up" on a straightforward migration. We plan to adopt Spec 2 for new code once it is out of pre-alpha status and run a mixed Spec 1 / Spec 2 codebase for a good long while.#2019-10-3015:16Alex Miller (Clojure team)I would use spec right now#2019-10-3015:17Alex Miller (Clojure team)the api functions - validate/conform/explain etc have gained some capabilities, but I expect will continue to work in spec 2 as is#2019-10-3015:19mpenetany news on eta? we're also waiting on v2 release for some new features (looking in particular at new data spec format for "interpretation" of specs)#2019-10-3015:19Alex Miller (Clojure team)the spec op forms will mostly be the same, although keys -> schema/select will be a significant change and there are some differences around the use of sets, preds, functions, etc at thee top level#2019-10-3015:20Alex Miller (Clojure team)I don't have an eta but I'd guess it's on the order of months#2019-10-3015:32Alex Miller (Clojure team)I guess?#2019-10-3015:34ataggartheh, ok, thanks#2019-10-3016:59orestisThanks for your reply Alex, much appreciated. #2019-10-3017:07seancorfield@orestis As Alex says, use Spec 1 for the time being. Conversion to Spec 2 may be easy or hard depending on how you use Spec 1.#2019-10-3017:17orestisThe thing is I need to do closed map checking, recursively. I think that it can be hacked on top of spec1, but not sure if itās already done and if not, if worth the effort to do myself. #2019-10-3017:22seancorfieldIs the project a production-level one, or likely to be one within the next couple of months?#2019-10-3017:22seancorfieldIf not, I'd say give Spec 2 a go. Otherwise, stick to Spec 1.#2019-10-3017:23Alex Miller (Clojure team)I wouldn't, I'd say stick to spec 1 :)#2019-10-3017:25Alex Miller (Clojure team)there is no release of spec 2 available, because I do not think it is release-worthy yet. until then, I would not recommend for anything other than tire-kicking and feedback (which is very welcome)#2019-10-3017:28seancorfieldRight, that's what I meant about not production-level -- if this is just an exercise project, go kick the tires. Otherwise, stick to Spec 1.#2019-10-3017:28seancorfieldI really like where Spec 2 is going but it's a different beast in many ways from Spec 1...#2019-10-3018:26orestisProduction-level. Ok, Iām hearing you loud and clear :)#2019-10-3020:59mloughlinThe book Secure By Design from Manning writes about using Java classes with immutable properties and data contracts in the constructor, then only using this defined class (instead of using, say, a string for a bank account number) in order to prevent business logic bugs. The nice thing about this is the type system enforces a degree of defensive programming.
My question: is there an clojure.spec
analogue, or convention for this?#2019-10-3021:29Alex Miller (Clojure team)mostly I think that's trying to patch problems that are endemic to use of Java, rather than an actually good idea. however, I'd say the closest thing is to separate specs into two levels - domain definitions (::domain/bank-acct-no) and uses in specific contexts that alias those domain definitions#2019-10-3021:30Alex Miller (Clojure team)but this is really mostly just making complicated definitions once rather than repeating them#2019-10-3022:40mloughlinwould you validate as a pre-condition every time you were to use ::domain/bank-acct-no? I suppose it's super cheap computationally#2019-10-3022:41Alex Miller (Clojure team)No, I would not#2019-10-3022:41Alex Miller (Clojure team)I would use instrument to check during dev/test #2019-10-3022:43mloughlinthanks for clarifying#2019-10-3022:43mloughlinš#2019-11-0109:51Nazralhi#2019-11-0109:52NazralI'm trying to write a macro that among other things needs to generate new specs by wrapping or merging existing ones, and it doesn't work. I made the simple following example
(defmacro wrapspec
[base-name base-spec]
(let [n-name `(keyword (str ":wrapped-" '~base-name))
new-spec (s/coll-of base-spec)]
`(s/def ~n-name ~new-spec)))
I'm not sure why doesn't this work? I tried with a symbol instead of a keyword too, to no avail#2019-11-0110:12NazralI also tried with n-name `(symbol (str (.name *ns*)) '~base-name)
to no avail#2019-11-0112:51Alex Miller (Clojure team)did you look at the expansion?#2019-11-0112:52Alex Miller (Clojure team)just as a general technique it's important to (pprint (macroexpand-1 '(wrapspec 'abc int?)))
or whatever to see if it's doing what you want#2019-11-0112:53Alex Miller (Clojure team)here s/coll-of is going to be a spec object, which is not what you want#2019-11-0112:53Alex Miller (Clojure team)new-spec `(s/coll-of ~base-spec)
#2019-11-0112:54Alex Miller (Clojure team)probably closer but I'm just eyeballing this#2019-11-0121:25FloAnyone aware of issues with clojure.spec.alpha/gen
in cljs (using shadow-cljs)? This snippet works well in clj repl:
(s/def ::foo string?)
(s/gen ::foo)
but fails in cljs with: Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required at eval (/js/compiled/cljs-runtime/cljs.spec.gen.alpha.js:1854:8)
#2019-11-0121:26FloMy project.clj
has a dependency entry for [org.clojure/test.check "0.10.0"]
#2019-11-0121:28FloNevermind, after I explicitly required [clojure.test.check.generators]
in the same file I'm (s/gen ..)
ing, it works.#2019-11-0208:35kurt-o-sysI'm having a pretty extensive spec, with quite some s/or
parts. When any data structure doesn't conform, it shows the issues, as expected. Conforming structures show that they conform... so all is well so far.
However, what I'd like to have, is to know which 'path' has been taken to conform:
(s/def ::spec (s/and string?
(s/or :path-A ::path-A
:path-B ::path-B)))
Is there a way to know, if a structure conforms, to know if it took path-A
or path-B
?#2019-11-0209:07vlaaadIsn't s/conform showing it?#2019-11-0209:11kurt-o-sysoh yeah, it is... thx.#2019-11-0209:11kurt-o-systhat was easy š#2019-11-0218:49coder46334Hi everyone!
Let's say I have a map like this
(def example-map {0 {:children [1 22 55] :parent nil}
22 {:children [14] :parent 0}
14 {:children [] :parent 22}})
How can I ensure with spec that a) a :parent is never the same key as the current line or ideally b) that :parent is a valid key in the map? I read through the website and some tutorials and I don't know if that's even possible with spec. Any advise would be super helpful#2019-11-0220:02andy.fingerhutYou can write arbitrary functions that return true or false for a given value, and use those in your specs, so if you can write a normal Clojure function to do it, you can write a spec to do it. Some of those reduce the amount of help you get from random generation of values, etc.#2019-11-0220:04coder46334This sounds useful, can you hint me what this is called in the docs so I can look it up? Also, I thought it would be cool to have the test cases auto-generated, so that won't work?#2019-11-0220:05andy.fingerhutThis property of functions is mentioned very early in the spec guide article here: https://clojure.org/guides/spec under the heading "Predicates"#2019-11-0220:07andy.fingerhutI am not a knowledgeable enough user of spec to give you the best answer to how to auto-generate things with custom predicates, but there are places in the docs that describe how to create custom random value generators and use those instead of the default built-in ones.#2019-11-0220:10coder46334I guess where I'm stuck at is how I bring that into the s/def of a map-of. I can't put it onto the smaller pieces, because they'd have to reference each other#2019-11-0220:11coder46334Ok, I thought of looking into the generators, but was hoping there is an easier solution to define it on the specs. There are not really a lot of examples out there that are more complex than one-liners. Thanks for your help!#2019-11-0220:12coder46334And if anyone has an idea about how to link it with the s/def of the map-of I'm still curious. Will be back at the computer later and will try to find out#2019-11-0220:14andy.fingerhutI believe s/and can combine arbitrary specs into one?#2019-11-0220:17coder46334I'll try later - thanks!#2019-11-0221:57coder46334Eventually I found some good examples here: https://clojuredocs.org/clojure.spec.alpha/map-of.#2019-11-0318:03alex-dixonAnyone interested in spec as a parser combinator library?#2019-11-0410:06acronIs there a preference to using either s/*
or s/coll-of
when we just want 'a collection of x`? I realise coll-of
has more tuning, but besides that, is there difference between (s/* ::foo)
and (s/coll-of ::foo)
?#2019-11-0410:15jumarProbably the most visible difference will be when you're nesting them - the regex based specs will make it work like it was "flattened" => I'd normally use coll-of
;
see also https://stackoverflow.com/questions/58636813/i-cant-understand-the-following-clojure-spec-error/58637063#2019-11-0410:17acronThanks, I think for sanity sake then I will ask for coll-of
in these situations#2019-11-0412:57Alex Miller (Clojure team)Yes, should use coll-of by default#2019-11-0420:43manutter51Hmm, I thought this would work: (s/fdef page-header-ribbon
:args (s/cat :title string? :extra (s/* any?))
:ret vector?)
(page-header-ribbon "Manage Users")
but I get Call to #'page-header-ribbon did not conform to spec:āµ
val: "Manage Users" fails at: [:args] predicate: (cat :title string? :extra (* any?))...
#2019-11-0420:44manutter51What am I doing wrong?#2019-11-0420:46Alex Miller (Clojure team)is there more to that ... ?#2019-11-0420:46manutter51Just the rest of the stack dump stuff, the JS objects etc#2019-11-0420:47manutter51Call to #'page-header-ribbon did not conform to spec:āµ
val: "Manage Users" fails at: [:args] predicate: (cat :title string? :extra (* any?))āµ
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha44661]āµ
:cljs.spec.alpha/value "Manage Users"āµ
:cljs.spec.alpha/args "Manage Users"āµ
:cljs.spec.alpha/failure :instrument
#2019-11-0420:48Alex Miller (Clojure team)oh, cljs#2019-11-0420:49Alex Miller (Clojure team)at a glance it doesn't look wrong to me#2019-11-0420:50manutter51Ok, good, Iām not nuts then.#2019-11-0420:50manutter51Probably some kind of dependency conflict or something.#2019-11-0420:56Alex Miller (Clojure team)I couldn't repro in either clj or cljs. maybe old repl statee?#2019-11-0420:58manutter51Itās surviving multiple shutdown/clean/restart cycles. :shrug:#2019-11-0420:59Alex Miller (Clojure team)#2019-11-0421:00Alex Miller (Clojure team)in case that's different than what you're doing...#2019-11-0421:02manutter51Not [cljs.spec.alpha :as s]
?#2019-11-0421:15Alex Miller (Clojure team)either should work#2019-11-0421:15Alex Miller (Clojure team)the clojure one aliases to the cljs one#2019-11-0506:54murtaza52I am trying to specify that atleast one key should be present, ex -
(def ::city string?)
(der ::name string?)
(def my-input (s/keys :opt-un [::city ::state])
The above makes both of them optional, I want to specify that atleast one should be present in the input.#2019-11-0510:00misha(s/def ::a string?)
(s/def ::b string?)
(s/def ::m (s/keys :req-un [(or ::a ::b)]))
(s/explain ::m {:a "1" :b "2"})
;; Success!
(s/explain ::m {:a "1" :b 2})
;; 2 - failed: string? in: [:a] at: [:b] spec: :user/b
(s/explain ::m {:a "1"})
;; Success!
(s/explain ::m {})
;; {} - failed: (or (contains? % :a) (contains? % :b)) spec: :user/m
#2019-11-0704:57murtaza52thanks @U051HUZLD#2019-11-0513:41jeremysHey guys I ran into something Iād like to ask you about. I wanted to āmergeā to s/keys specs using s/and. It didnāt work as I thought it would. Here is an example showing what I wanted to do:
(ns example
(:require [clojure.spec.alpha :as s]))
(s/def ::type (s/or :s string? :i int?))
(s/def ::a ::type)
(s/def ::b ::type)
(s/def ::t1 (s/keys :req [::a]))
(s/def ::t2 (s/keys :req [::b]))
(s/def ::v1 (s/and ::t1 ::t2))
(s/def ::v2 (s/and #(s/valid? ::t1 %)
#(s/valid? ::t2 %)))
(def example {::a "1" ::b 2})
(s/valid? ::v1 example)
;=> false
(s/valid? ::v2 example)
;=> true
(s/explain ::v1 example)
;[:s "1"] - failed: string? in: [:example/a] at: [:example/a :s] spec: :example/type
;[:s "1"] - failed: int? in: [:example/a] at: [:example/a :i] spec: :example/type
;[:i 2] - failed: string? in: [:example/b] at: [:example/b :s] spec: :example/type
;[:i 2] - failed: int? in: [:example/b] at: [:example/b :i] spec: :example/type
;=> nil
Do I use s/and
in an unintended way? Is ::v2
the correct way to do it? Did I miss something in specās api?#2019-11-0513:48sgerguri@jeremys Sounds to me like you want s/merge
.#2019-11-0513:57jeremys@sgerguri as usual I missed something in the api... Thanks a lot mate#2019-11-0514:07jeremysItās so logical also, as you merge maps, you merge specs about maps...#2019-11-0517:46colinkahnIs it a good idea to define your generators in the same file as your specs? Sometimes I have generators that need my specs and vise-versa. Couldnāt tell if it was good practice to keep them together though.#2019-11-0602:02sgerguriI (and my colleagues) tend to do that, then things are nicely collocated. Doubly useful when your spec is a complicated s/and
- if you wrap with s/with-gen
it's all neatly in one place.#2019-11-0621:21telekidIs there a functional difference between (s/spec some-pred :gen some-gen)
and (s/with-gen some-pred (constantly some-gen))
?#2019-11-0701:52Alex Miller (Clojure team)No#2019-11-0721:30pabloreis there an expected date for spec 1.0.0 ?#2019-11-0721:32Alex Miller (Clojure team)no#2019-11-0721:39vlaaadwill there be spec 1.0.0?#2019-11-0721:39vlaaadand tools.deps without alpha#2019-11-0721:42Alex Miller (Clojure team)yes#2019-11-0722:18seancorfieldspec 1.0.0 will be Spec 2 when it's "baked", right @alexmiller ?#2019-11-0722:36Alex Miller (Clojure team)most likely#2019-11-0722:37Alex Miller (Clojure team)I don't know that it will be 1.0.0 exactly but it will be some non-alpha version#2019-11-0800:45pablorewhat are the main differences between spec 2 and alpha?#2019-11-0800:48kszabomainly how they handle aggregates of attributes. You only specify required/optional attributes at the context of the usage site#2019-11-0800:48kszaboābagsā of attributes are now the primary way of mapping domain entities in spec2, they donāt talk about requirements#2019-11-0800:49kszaboalso some programmatic convenience improvements#2019-11-0800:49kszaboof course I am oversimplifying#2019-11-0803:57Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-11-0803:57Alex Miller (Clojure team)and https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-11-0816:23wilkerluciohello, with spec1, I'm having issues to define a spec that is not loaded, I get Unable to resolve spec: ...
, consider this is file B
, the spec in question is defined by A
, but A
requires B
, so I can't load A
before loading B
(circular reference), with that constraint, is there a way to tell spec to ignore that the spec is not defined at that time?#2019-11-0816:26Alex Miller (Clojure team)most spec types delay resolving references so don't run into this, but there are a few that don't#2019-11-0816:26Alex Miller (Clojure team)for those that don't, you'll need to break that cycle somehow#2019-11-0816:43wilkerluciothanks, I got rid of the cycle by moving those spec definitions to a new namespace (since I we can define things from other namespaces, so still have the same name, just defined in a different file)#2019-11-0821:01didibusWhat's the best way to restrict a string to a min/max length, so that the generator still works for it ?#2019-11-0821:04Alex Miller (Clojure team)I don't think there is a general way to do that without writing a generator#2019-11-0821:06Alex Miller (Clojure team)(and to do that, I would generate strings of length up to (max-min) and a string of length min, then concat)#2019-11-0821:06didibusHum, alright, I'll use a custom generator then#2019-11-0821:06didibusthx#2019-11-0917:17cddrI made a thing to capture one side of that constraint as part of a toolkit for generating test-data to match a SQL schema....
(defn string-up-to [max-len]
(s/with-gen string?
#(gen/fmap (fn [x] (apply str x))
(gen/bind (s/gen (s/int-in 0 max-len))
(fn [size]
(gen/vector (gen/char-alpha) size))))))
#2019-11-0920:18bortexzIs there any way to add a docstring to specs?#2019-11-0920:56Alex Miller (Clojure team)no, not currently#2019-11-0923:51bortexzAre there any plans to add it? I think it could be interesting to have docstring when using spec to model the domain of an application#2019-11-1000:28Alex Miller (Clojure team)yes, it is the highest voted ticket in the tracker :)#2019-11-1000:29bortexzgreat, thanks :+1:#2019-11-1000:30Alex Miller (Clojure team)it's on the list for spec 2, but we haven't gotten to it yet#2019-11-1200:49didibusFYI, its pretty easy to generate a string generator of min/max length using: #(sgen/fmap str/join (sgen/vector (sgen/char-alphanumeric) min max))
#2019-11-1214:18coder46334Hi! When I start shadow-cljs with :browser-test , insead of :app, my spec instrumentation doesn't work. The specs work with (exercise fully-qualified-ns/spec), but I can't instrument functions either through code or repl. Any ideas?#2019-11-1214:22aisamuIs something from the test namespaces requiring the files containing the specs?#2019-11-1214:24coder46334Yes. I also have an instrument command in that namespace and also tried it through REPL while in that namespace#2019-11-1214:26coder46334I can even copy the specs into REPL, run the instrument command there and it's not working#2019-11-1214:47aisamuThis is probably something with your code.
We have specs on test ns's running with :browser-test
, and there's nothing special about the invocation/setup:
(stest/instrument `ns/component)
#2019-11-1214:49aisamuIt's worth noting that the instrument
call must happen after the fdef
, so mind the namespaces loading order#2019-11-1215:00coder46334I just drop-in replaced it with orchestra-cljs.spec.test and it works#2019-11-1215:01coder46334yes, the instrument calls are after the fdefs#2019-11-1215:02coder46334So with orchestra it works, but no idea why it doesn't with normal spec. It's hard to debug when you don't get an error, it just doesn't work š#2019-11-1222:51Oliver GeorgeI find myself wringing my hands when writing specs in my CLJS code. Particularly choosing namespaces for my specs. I'd like something like (s/def ::list-options/options (s/coll-of ::option))
to mean (s/def :current-ns.list-options/options (s/coll-of ::option))
where the alias isn't defined in the current ns.#2019-11-1222:51Oliver GeorgeI find myself wringing my hands when writing specs in my CLJS code. Particularly choosing namespaces for my specs. I'd like something like (s/def ::list-options/options (s/coll-of ::option))
to mean (s/def :current-ns.list-options/options (s/coll-of ::option))
where the alias isn't defined in the current ns.#2019-11-1301:12ataggartI've found it convenient to always define specs as ::foo
, and reference specs with ::foo
or ::x/foo
. It eliminates a lot of bikeshedding and subtle bugs. I want it to fail when the ns alias isn't defined.#2019-11-1301:21ataggartOh, I see what you mean. You could just move the dot segments to the other side of the slash, e.g., ::list-options.options
.#2019-11-1304:58Oliver GeorgeThat works sometimes but for s/keys you need the name part to match so it's the namespace part which needs to be adaptable/unique.#2019-11-1315:46ataggartAh, yeah, if you're using unnamespaced keywords#2019-11-1505:00valeraukoi wanted a similar thing for db enums so i made it.
(defn enum
[value]
(let [key-ns (namespace value)
last-dot (.lastIndexOf key-ns ".")
model (subs key-ns 0 last-dot)
spec-name (subs key-ns (inc last-dot))
type-name (str/replace (str model "-" spec-name) "-" "_")
spec-kw (keyword (str "my-app.spec." model "s") spec-name)
enum-value (name value)]
(if (s/valid? spec-kw enum-value)
(convert-enum type-name enum-value)
(throw (ex-info "invalid enum value"
{:type ::invalid-enum
:data (s/explain-data spec-kw enum-value)})))))
this validates keywords like :my-model.status/ok
against ::my-model/status
to see if "ok" is a valid value#2019-11-1222:51Oliver GeorgeOr something similar and less complected of course.#2019-11-1222:53Oliver GeorgePerhaps something like clojure.core/alias for CLJS would solve the problem.#2019-11-1404:10danielcomptonIs there a ticket for spec to bring in the rest of the generators in clojure.test.check.generators
?#2019-11-1405:05Alex Miller (Clojure team)we did not have a general goal to replicate all aspects of test.check.generators in spec.gen#2019-11-1405:06Alex Miller (Clojure team)so, no as we do not intend to do so#2019-11-1405:06Alex Miller (Clojure team)you can just use them from test.check if you need them?#2019-11-1416:33kennyThe problem is that you want the lazy loading spec.gen ns provides.#2019-11-1417:02Alex Miller (Clojure team)is there something specific you're looking for?#2019-11-1417:24kennyNo runtime dependency on test.check. #2019-11-1417:25Alex Miller (Clojure team)I mean which generator functions?#2019-11-1417:41kennyThe ones that we have copied over are: generator?, let, vector-distinct-by. I believe there may be some others scattered throughout the code though.#2019-11-1417:42Alex Miller (Clojure team)is the dynaload stuff in gen exposed enough to just use?#2019-11-1417:44kennyWe had to copy over lazy-combinator
macro because dynaload
was private.#2019-11-1417:44Alex Miller (Clojure team)lazy-combinator is public though, right?#2019-11-1417:44Alex Miller (Clojure team)ah, so you can't get to the expansion, right#2019-11-1417:44kennyRight#2019-11-1417:47Alex Miller (Clojure team)let's a macro - so I presume that's not something you can dynaload?#2019-11-1417:47Alex Miller (Clojure team)there actually is a ticket about that one in particular#2019-11-1417:47kennyYes. But let
is used a ton so having that in the spec gen ns would be very helpful.#2019-11-1417:47Alex Miller (Clojure team)vector-distinct-by could be done through (s/gen (s/coll-of ::foo :distinct true :into [])) ?#2019-11-1417:48Alex Miller (Clojure team)oh, distinct-by#2019-11-1417:48Alex Miller (Clojure team)nvm#2019-11-1417:49Alex Miller (Clojure team)generator? is prob weird given the thunking done in gen isn't it?#2019-11-1417:50kennyThis is what we are using:
(def generator? #'gen/generator?)
#2019-11-1417:50kennywhere gen
is clojure.spec.gen.alpha
#2019-11-1417:52Alex Miller (Clojure team)so these each seem like distinct issues#2019-11-1417:52Alex Miller (Clojure team)there's a ticket for let already, generator? could just be made public, and vector-distinct-by could be added to the dynaload list#2019-11-1417:54kennyYep. @U051KLSJF may be interested in some other missing ones. I'm pretty sure there's a few others I've run into.#2019-11-1418:02Alex Miller (Clojure team)I've made the generator? and vector-distinct-by changes in spec-alpha2#2019-11-1418:03kennyThanks!#2019-11-1420:04danielcomptonI was missing byte
which prompted me to ask, there are also more generators around ints which arenāt exposed #2019-11-1415:31ataggartI find that the things in clojure.spec.gen.alpha
are sufficient to define generators for specs, and I only need the things in clojure.test.check.generators
for generating "interesting" data for tests.#2019-11-1416:03Alex Miller (Clojure team)you can get pretty far by using (s/gen <some-spec>) too#2019-11-1416:03Alex Miller (Clojure team)often I find that's easier that constructing the generator from scratch#2019-11-1518:14dpkpHow do folks usually choose between req and req-un ? Namespaces are great for organizing, but do folks usually drop them after validation w req-un? Or maybe coerce to namespaced keys? Or require upstream users to pass namespaced keys?#2019-11-1519:27seancorfield@dpkp "It Depends". We tend to use :req-un
for input validation specs (coming from JSON or URL/form input) but :req
for quite a bit of internal stuff, including database-related data (where we either use clojure.java.jdbc
's :qualifier
option to add a generic "namespace" to all columns, or next.jdbc
's default table-qualified column names).#2019-11-1519:29seancorfieldIt does mean that at system boundaries you have to decide where (or even whether) to switch from unqualified keys to qualified keys -- but I think that's reasonable since I would expect input data to be fairly "flat" and domain data to be a more interesting shape, i.e., I would expect a mapping from input -> domain to exist anyway in your code, even in the absence of Spec.#2019-11-1522:12cjmurphyWhen you believe a spec is not being picked up for whatever reason (I'm using guardrails/ghostwheel - therefore expound and spec are picked up), is there a way to clear the registry, just to test the assumption?#2019-11-1522:18ghadiwhat do you mean by "not being picked up"?#2019-11-1522:20kszabojust use: https://clojuredocs.org/clojure.spec.alpha/form to verify#2019-11-1522:50cjmurphyThanks that helps. But I just verified they are as they are written in the file that defines them. The problem is unlikely to be with spec itself. But if I could somehow 'invalidate the cache' I'd be able to force the other tools to recalculate, hopefully...#2019-11-1522:33cjmurphy@ghadi 'not being picked up": I have a spec for essential-rule
and another for rule
. One of the function parameters has been changed from rule
to essential-rule
, yet there's still a spec failure and the expound message thinks that the function parameter is rule
rather than the new and more lax essential-rule
. I've touched various files, restarted the JVM/REPL. But still somehow 'it' still thinks the function is different to the way it is written.#2019-11-1522:52kszaboif you restarted the JVM then itās not a cached thing#2019-11-1523:02cjmurphyI worked it out, and its not the first time I've fallen for this š. The function in question is multi-arity and I was changing one of the arities and not the other. Thanks for your help kszabo and ghadi.#2019-11-1713:37tianshuin spec2, should {:closed true}
be available for schema spec?
(s/def ::user (s/schema [::name ::age]))
;;; contains? not supported on type: java.lang.Boolean
(s/valid? ::user {::name "cat" ::age 10} {:closed true})
(s/def ::complete-user (s/select ::user [*]))
;;; ok
(s/valid? ::complete-user {::name "dog" ::age 15} {:closed true})
#2019-11-1715:40Alex Miller (Clojure team)Itās a set of schema names, not a boolean#2019-11-1722:28coder46334Hi! Is there a way to spec a return type of Promise<something> ? I would need this in quite a few places using cljs#2019-11-1723:19Alex Miller (Clojure team)Not usefully, no#2019-11-1723:22coder46334Thanks!#2019-11-1804:28tianshu@alexmiller thanks!#2019-11-1821:05colinkahnIāve used with-redefs
around an st/check
in ClojureScript code and it has worked fine, but trying in Clojure code it doesnāt. I had to redefine a my-function-with-redefs
and spec that with the redefs inside the body. Is that expected? Iām assuming itās something to do with multiple threads vs single thread in JS#2019-11-1821:24Alex Miller (Clojure team)might be - st/check uses pmap which is multi-threaded in Clojure (or possibly it's really the laziness of pmap)#2019-11-1821:25Alex Miller (Clojure team)make sure you realize the result of pmap with doall or something inside the lexical scope of the with-redefs#2019-11-1916:41colinkahnAwesome! That worked#2019-11-1917:09Lukasz HallData generation question -
I have a top-level (map) spec composed of many nested specs and their associated generators at a leaf level. Some of the leaves of the nested structures use the same spec as other leaves, and I need those that share a spec to produce the same values when they generate. Is there any way to have this happen without explicitly writing a generator for the top level spec?#2019-11-1917:37Alex Miller (Clojure team)No#2019-11-1917:37Alex Miller (Clojure team)To guarantee properties at the aggregate level, you need to control generation at the aggregate, as a general principle #2019-11-1920:21telekidIām curious how people integrate instrumentation with testing. Do they create an instrumentation fixture and call it in a :once
block at the top of every test ns? If so, how do you restore instrumentation to its initial state at the end of that fixture?#2019-11-1920:39Alex Miller (Clojure team)unstrument
exists to do that#2019-11-1921:43seancorfield@jake142 I instrument directly at the top of a test file (after the ns
form) and don't bother undoing it. I do it separately from my test fixtures because those are often functions I reuse across multiple test namespaces, whereas the instrumentation that I do (if any) is usually limited to the functions under test in a specific namespace.#2019-11-2010:21abdullahibrahello everyone#2019-11-2010:21abdullahibrawhat is the spec for checking file data type ?#2019-11-2010:53vlaaadyou mean like that #(instance? java.io.File %)
?#2019-11-2013:33DaoudaHey Folks, I am learning spec
and need help to understand the registry
parte. They say that it help to avoid name collision, but since we already have namespaces, they aren't enough to avoid that? I don't get to see the real advantage of using ::
in spec declaration. Can someone help me to understand please with example will be wonderful#2019-11-2013:58Alex Miller (Clojure team)::keywords are āauto-resolvedā to have the current namespace so itās just a useful tool for succinctly making qualified keywords#2019-11-2013:59Alex Miller (Clojure team)They create the exact same result as using :my.ns/keyword so use that if you prefer!#2019-11-2014:00Alex Miller (Clojure team)Youāll see them in a lot of examples as it just makes the example shorter#2019-11-2016:13DaoudaThank you very much @alexmiller#2019-11-2017:00DaoudaI think I`ve got the big picture now. When you want to use a spec/def
, first spec
look in the spec-registry
if exist then take from there. Else it create it and store it in the spec-registry
. A very naive description of course hehehe#2019-11-2019:33DaoudaHey Folks, when should I use qualified
or unqualified
keys
in clojure spec
?#2019-11-2019:43Alex Miller (Clojure team)qualified#2019-11-2019:44Alex Miller (Clojure team)unless you are working with existing data, don't want to transform it, and it has unqualified keys#2019-11-2111:12DaoudaHey folks, can you help me to understand why I can't gen/generate
or gen/sample
:
(def email-regex #"^[a-zA-Z0-9._%+-]
#2019-11-2111:18guyHave you tried exercising your specs?#2019-11-2111:18guyCouldn't satisfy such-that predicate after 100 tries
#2019-11-2111:18guyThats the key part to the error message#2019-11-2111:18guyI would just go through each def and see if they can produce a result#2019-11-2111:19vlaaadmy bet is on email and phone#2019-11-2111:19guyyes i would agree#2019-11-2111:19guyIf the predicate is too hard to er fufill? based on random string input, don't you usually just create a generator for those complicated predicates?#2019-11-2111:20guyalso i'm not all that sure how (spec/def ::phone #"^\d{12}$")
works#2019-11-2111:26guyIf you exercised ::phone
what would you get? thats a pattern isnt it š¤#2019-11-2111:28guyUnable to construct gen at: []
#2019-11-2111:28guyThats what i get#2019-11-2111:28guyfor (s/exercise ::phone)
#2019-11-2111:29DaoudaJust saw now, the answers, will be right back after doing what you said š#2019-11-2111:29guySo then I think for the email, its just as simple as, the string generator can't create a random entry that matches your regex#2019-11-2111:29guywhich makes sense right? as even after 100 times its going to be difficult to randomly create an email i think haha#2019-11-2111:30guyhttps://clojure.org/guides/spec#_custom_generators#2019-11-2111:32DaoudaAfter trying to exercise each attribute, it appears that ::phone
and ::email-type
can't be exercised#2019-11-2111:35acron@quieterkali Use https://clojuredocs.org/clojure.spec.alpha/with-gen to provide alternative generators#2019-11-2111:36acronAlso useful for composing generators: https://clojure.github.io/test.check/clojure.test.check.generators.html#2019-11-2111:43guyahh thanks that was the link i was looking for. š#2019-11-2117:19seancorfieldhttps://github.com/gfredericks/test.chuck has a generator for regex strings that I've found very useful.#2019-11-2122:18sgerguriI am trialling offloading some extra logic onto s/or
and would like some collective opinion, if y'all were so kind.
Suppose I have a spec defined like this:
(s/or :x :x/item
:y :y/item)
Now suppose I add x1
and want to combine it with x
at some point; so let's say I start making the semantic distinction between what this thing is and what it can be combined into:
(s/or [:additive :x] :x/item
[:additive :x1] :x1/item
[:multiplicative :y] :y/item)
Now, suppose further that I want to encode the order in which the respective operation should be defined. Now I'm thinking it would be really useful to be a bit more descriptive in my tags, so why not use a map?
(s/or {:type :additive
:operand-type :x
:precedence 1}
:x/item
{:type :additive
:operand-type :x1
:precedence 2}
:x1/item
{:type :multiplicative
:operand-type :y}
:y/item)
Now I'm thinking, should I simply add this via a conformer, keeping the tags simple (and thus transforming them at a higher level elsewhere in the spec) or do I keep this?
This might all seem like a contrived example - and it is, because my use case isn't algebraic operations, but rather data merging, and thus it's really important to define what order each data types should be merged in, and what the parent type is, so that the correct merge strategy is applied.
My question is - am I really using s/or
in a way I shouldn't be or would you still consider this ok?#2019-11-2206:27jaihindhreddyMy 2c is that: this is trying to do too much with spec. I'd keep the tags simple keywords, and do the manipulation later, using spec only to suss out the implicit ordering or types into explicit data (via cat
and or
respectively).#2019-11-2209:59sgerguriI'm always torn between that and doing more. Spec feels like it's one step away from automatic transformations, with conformers being the key enabling element for this.#2019-11-2210:42jaihindhreddyI can sympathise 100%#2019-11-2210:43jaihindhreddyBut spec's neither a kitchen sink, nor a silver bullet š#2019-11-2211:06sgerguriI think if specs supported metadata one could use simple tags and just indicate code-level stuff there. Perhaps we'll get that functionality in version 2.#2019-11-2211:38jaihindhreddyI dunno if spec wants to do that. Because the awesome thing in Clojure is that, we already have namespaced keywords, which are (when correctly used) globally precise names.
And this means just like spec maintains a global registry of specs attached to these names for the purpose of specification, we can write our own things which maintain global registries of other stuff attached to the same names, for use in other contexts. I think this is one of those instances where making things global makes sense, and alleviates parochiality. For this kind of a thing to be realistic though, I feel like the idea of providing and requiring contexts needs to be first class and written down in code, so that automatic verification of breakage can be detected and avoided, making way for an ecosystem that can integrate various things seamlessly while still being dynamic and "growable"#2019-11-2213:48Alex Miller (Clojure team)conform is designed to tell you why something was validated and separate data into parsed components#2019-11-2214:00Alex Miller (Clojure team)It is not a general ādata transform meat grinderā and youāll be disappointed if you try to use it that way#2019-11-2516:58vemvGiven a ns that goes like this:
(defmulti event-type ...)
(spec/def :some-event (spec/multi-spec event-type ::type))
(defmethod event-type :some-dispatch-value [_]
(spec/keys :req-un [::login ::email]))
;; ...many other defmethods...
...how to generate :some-event
s only with :some-dispatch-value
type? i.e., exclude all other dispatch values (i.e. the specs coming from other defmethods)#2019-11-2523:28sgerguri(g/generate (s/gen event-type {::type :some-dispatch-value}))
#2019-11-2600:36vemvawesomesauce!
thanks#2019-11-2711:34doesntmeananythingwhat's the best way to spec a nested map? Say, I have a customer
map that looks something like this:
{:email "
how would you go about validating these nested fields? Simply write a couple of spec registries and then collect them in an entity map? So you'd do:
(s/def ::customer (s/keys :req-un [::email ::password ::first-name ::last-name ::organization]))
and ::organization
would be specced to have the namespaced nested data? I'm a bit unsure on how to proceed since the official guide doesn't really have explicit examples of nested data as far as I can see#2019-11-2711:51vemvI'd say you're going in the right direction
try the pattern you have posted, and verify if you are getting the desired validations.
Else post the exact failed attempt here#2019-11-2712:08vlaaadhttps://clojure.org/guides/spec#_a_game_of_cards#2019-11-2712:09vlaaadthis is an example of nested data structures: players/deck in a game#2019-11-2712:09doesntmeananythingcool, thanks for the help, guys#2019-11-2722:39bbloommini-survey: are folks using https://github.com/danielcompton/defn-spec or something similar? how do you like/dislike it?#2019-11-2800:24vemvI authored https://github.com/nedap/speced.def and we're happy with it at work. Since it compiles to a clojure/script defn
it has zero limitations or new syntax, guaranteed.
it doesn't preclude creating generic/reusable specs: you can author those and still use them here.
It has no :fn
equivalent (only :`args` :ret
ones), but honestly I have yet to see how that is not equivalently achieved with the occasional :pre
condition.#2019-11-2800:25vemvI haven't campaigned much about it so far, but I plan to post something to #news-and-articles later this week :)#2019-11-2722:56seancorfield@bbloom I don't use it -- I often put fdef
s in a separate namespace in libraries so they are optional and the code can then run on older Clojure versions.#2019-11-2722:57seancorfield(and I don't like that syntax so I wouldn't want to use it even when I would put fdef
s above the function for which they exist)#2019-11-2722:58seancorfieldSeems like a lot of limitations as well https://github.com/danielcompton/defn-spec#limitations#2019-11-2722:58bbloomthanks for the thoughts - iām not using anything like it yet b/c i like to kinda use the primitives until i grok them fully and then and only then abstract more#2019-11-2722:58seancorfieldThose limitations alone would make it non-viable for most of my fdef
use cases I think.#2019-11-2722:59bbloomiām still trying to figure out where/when to utilize spec#2019-11-2722:59seancorfieldhttps://corfield.org/blog/2019/09/13/using-spec/ is about how we use it at World Singles Networks#2019-11-2722:59bbloomiāve been doing a lot of TypeScript for work lately, and all the inline annotations are simultaneously: laborious, helpful, and hurtful#2019-11-2723:07bbloomthanks @seancorfield, that all makes sense to me - to some extent iām trying to decide how far to do w/ development time checking#2019-11-2723:09bbloommy current thinking is basically just-in-time specing, where i spec stuff as part of my debugging process and leave the specs around for future use#2019-11-2723:13didibusI've got a library like that in the works, probably 80% done at this point. Don't know if or when I'll finish it. But my focus is on fn specs, since those are the painful ones to write in my opinion. As well as multi-arity, var-args, et all. Those are the hard ones to write, so my DSL intends to make these way quicker to spec, as to encourage more spec writing. Right now the machinery attaches itself as metadata on defns, so it also is compatible with older Clojure versions.#2019-11-2723:14didibusSome edge cases can be tricky, which is why it's taking me longer than I initially thought to put together.#2019-11-2723:15didibusBut I do think I'd write more fn specs if defn had a easier syntax for writing them. So that's what I'm trying to achieve. I also recognize it is hard to come up with one that is simpler but not limited. Which is why there probably isn't an official one.#2019-11-2800:29Alex Miller (Clojure team)Thatās on the table for spec 2#2019-11-2800:29Alex Miller (Clojure team)But they are likely to be significantly different than spec 1#2019-11-2802:01bbloom@alexmiller Could you please clarify: āThatāsā == what exactly?#2019-11-2802:04Alex Miller (Clojure team)specs integrated into defn#2019-11-2802:05bbloomah, ok, just wanted to be sure thatās what you meant š very cool. looking forward to seeing what you folks come up with#2019-11-2802:34didibusWhy is there no support for and/or inside of s/keys :opt ?#2019-11-2802:35didibus(s/keys :req-un [::a]
:opt-un [(and ::b ::c)])
#2019-11-2802:35didibusIn my case, ::b and ::c are optional, but if anyone of them is there, the other has to be as well#2019-11-2802:45Alex Miller (Clojure team)Because optional things are ... optional#2019-11-2802:45Alex Miller (Clojure team)Youāll need to s/and another predicate if you have additional constraints#2019-11-2802:46didibusHum.. its just more annoying, because the combinations to list out with s/and can quickly grow#2019-11-2802:46didibusbut that's what I'll do for now#2019-11-2802:48didibusI guess I need s/or#2019-11-2802:49didibuss/or (s/keys :req-un [::a]) (s/keys :req-un [::a ::b ::c])
#2019-11-2803:02didibusHum... its a lot trickier then I thought#2019-11-2803:02didibusOr I'm tired#2019-11-2803:03didibus(s/or :one (s/keys :req-un [::a]) :two (s/keys :req-un [::a (and ::b ::c)])
This means that the following is valid as well:
{:a :b}
Because the first s/keys is open.#2019-11-2816:14gwsYou could do something like this, which doesn't require you to list all possible optional combinations#2019-12-0200:16fredericHi. Do you have guidelines, or rules of thumb for naming specs in the following two situations
1. do you rather choose a name that describes the role that the data plays in the enclosing structure (say, ::confirmation-url
), or a name that describes the intrinsic kind of data that key holds (say, ::url
). I tend towards the former, because I think the more descriptive keys read better, but I'm wondering if by doing so I'm not inadvertantly creating private DSL (fragmenting keys when I could have one large overarching key instead and maximise the number of functions that can operate on the data)
2. when do you use symbols rather than keywords to name specs? I tend to use symbols as a kind of zero-arg spec-op, specs that I think of as building blocks, reserving keywords for specs that I would expect to actually appear in maps. Is that a personal quirk of mine?#2019-12-0200:21Alex Miller (Clojure team)1. I would say both. Name the generic thing, then have the specific one refer to the generic one.#2019-12-0200:22Alex Miller (Clojure team)2. You should never name specs with symbols (those are reserved for function specs)#2019-12-0200:22Alex Miller (Clojure team)Lots of places in spec make that assumption and you are likely to be burned by using symbols.#2019-12-0200:27fredericThank you! That answer was fast and very clear cut, I appreciate that š#2019-12-0202:09favilaBy āsymbolsā do you mean vars with def or literally (s/def āsymbol ...)?#2019-12-0202:11favilaI go back and forth on whether I should register specs or just def the spec object. Iām on a ādonāt registerā swing now#2019-12-0202:23seancorfield@favila (s/def ::keyword ...)
vs (s/def 'symbol ...)
-- the code assumes that only function specs use symbols (s/fdef 'foo ...)
#2019-12-0202:26favilaI know this, I am asking OP what he meant#2019-12-0202:31seancorfieldBoth Alex and I made the same assumption and the OP didn't correct Alex so... ĀÆ\(ć)/ĀÆ#2019-12-0211:12fredericYou were correct about what I meant, @seancorfield. Thanks for suggesting using a plain def
when there's no need to s/def
a keyword, @favila.#2019-12-0309:13QuestAnyone have a convenient solution for defining clojure specs at runtime? Channel logs say either wrap with another macro OR to use eval.
Macro approach requires everything else to become a macro & still doesn't allow full runtime extension.
Eval works but creates more complexity to get it working in CLJS.#2019-12-0309:18QuestInterestingly, I can simply copy my own version of res
(private FN from clojure.spec.alpha) & suddenly the below works.
(s/def-impl k (res v) v)
While hackish it proves that dynamic definition works just fine, but seems to have been intentionally disallowed? Though I still prefer over the eval nonsense.
(Which for completeness sake is included below:)
(eval (list 's/def k v))
#2019-12-0309:31QuestLooks like CLJS still hiccups on the def-impl
approach due this a difference in how clojure.core/resolve
works. It's a macro in CLJS but a FN in Clojure? (I'm curious how the impl differs under the hood if anyone knows...)
The offending line:
(symbol? form) (c/or (-> form resolve ->sym) form)
In my case I'm just passing namespaced keywords at runtime, so this part is unnecessary for me. Simply commenting it out obviates the issue.
I'm sure this breaks many best practice recommendations, but if this is the most direct solution then I'm happy with it. I understand that spec2 should support this more easily & development is focused there, meaning that spec1 is unlikely to change & break this approach.#2019-12-0310:33Alex Miller (Clojure team)Correct. There are multiple non-macro ways to make this work now in spec 2#2019-12-0413:38Vincent CantinHi, I am trying to define a cross-recursive spec (i.e. 2 specs defined using the other one) using spec-alpha2. How to do that?#2019-12-0413:40Vincent CantinMy repl complains on the first definition, saying that it does not find the second one (which is defined after)#2019-12-0413:57Alex Miller (Clojure team)That wonāt work at the moment#2019-12-0413:58Alex Miller (Clojure team)Thereās a pass needed to ensure every spec properly delays resolution of named specs#2019-12-0414:11vlaaadwill it work in spec-alpha2?#2019-12-0414:20Alex Miller (Clojure team)yes#2019-12-0414:20Alex Miller (Clojure team)it's just work I haven't done yet#2019-12-0414:20Alex Miller (Clojure team)it (mostly) works in spec 1#2019-12-0502:08Lukasz HallHow fully is support for unqualified keywords baked into spec2? (Last example of https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys-1 fails with latest codebase)
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
#2019-12-0503:22seancorfieldI expect that's just a bug in master -- or a doc bug.#2019-12-0503:23seancorfield@lukaszkorecki Spec2 is very much a work-in-progress right now.#2019-12-0503:24seancorfield@hall ^#2019-12-0509:35QuestTook a closer look at spec2. Really like ease of programmatic definition & similar API is a plus. Good vision & look forward to moving to it after CLJS support šš#2019-12-0510:21rickmoynihanIn lieu of spec 2, is there any advice on how to structure spec 1 specs, so that theyāre more likely to be easily portable to spec 2's schema / style?#2019-12-0510:21rickmoynihanFor example, defining essentially the schema (leaf) keys, and then essentially defining what would be the select
ions on the fdefs.#2019-12-0510:21rickmoynihanObviously that can lead to some duplicationā¦ is there a way in spec 1 to dissoc an s/key
:req
from an s/keys
spec?#2019-12-0510:22rickmoynihanI canāt see such a thing; so Iām thinking itās better to build the composites from multiple s/keys
at the fdef
sites with s/merge
?#2019-12-0510:28rickmoynihani.e. it seems you can kind of simulate spec2's idiomatic structure with spec1#2019-12-0512:52kszaboI had this discussion with Alex before, your ārecord typesā should all use :opt
/`opt-un` , at usage sites (endpoints/fdefs) you can tighten them down by (s/merge ::user (s/keys :req [:user/name :user/age]))
#2019-12-0513:16rickmoynihan@thenonameguy: great, thatās what Iāve been doing. Thanks for the clarification. Is ārecord typesā a phrase Alex mentioned, or is it one youāve coined (hence the bunnies?)
I ask because I wouldāve thought he wouldāve said āschema typesā or just āschemasā, given that is the terminology used in spec2.#2019-12-0513:17rickmoynihanOr is there some more nuance to that?#2019-12-0513:24kszaboI just came up with it, case class/record type/schema, there are lots of names for arbitrary groupings of attributes. That is my conclusion from these spec2 developments: Aggregates are incidental and are context-dependent.#2019-12-0513:33kszaboAnd you can see the industry trend towards systems that are attribute-focused and not aggregate focused. Think of SQL vs Datomic for instance, folks appreciate the multi-dimensional flexibility of EAV* stores vs predetermined tables. There is also REST vs GraphQL where the predetermined āresourceā is inferior to getting just the attributes that you want.#2019-12-0513:34rickmoynihanYeah, I do a lot of work with RDF; which is pretty much the poster child for property-based open-world thinking.#2019-12-0514:08Alex Miller (Clojure team)and the inspiration for parts of spec#2019-12-0514:24rickmoynihanyeah and it really showsā¦ same for clojure too actually š#2019-12-0514:31rickmoynihanIncidentally Iām not sure if you or Rich had seen SHACL or SHEX; but theyāre remarkably similar to spec (but for RDF):
https://www.w3.org/TR/shacl/
https://shex.io/
Iāve often wondered if they were part of the inspiration behind spec, or whether spec and SHACL/SHEX just ended up in the same sort of place because of the nature of RDF / OWL.
Either explanation seem plausible to me.#2019-12-0514:34rickmoynihanIncidentally this (free) online book describes both of them and their usage, and their differences: https://book.validatingrdf.com/#2019-12-0513:35kszaboAlthough you can take one more step and go to Pathom which is even more attribute-focused, there are no predetermined ātypesā at all, very RDF-like#2019-12-0513:37rickmoynihanI should check out pathomā¦ itās been on my radar for a while ā but Iāve not really dug into it, so thanks for the pointer.#2019-12-0513:37kszabohttps://www.youtube.com/watch?v=IS3i3DTUnAI highly recommended#2019-12-0513:38kszaboWe are in the process of adding Pathom to all of our APIs so that we can later have a unified āmaximalā graph, it has been great so far#2019-12-0514:58eggsyntaxI thought there was a way to say in a map spec that a key could be either qualifed or unqualified. I misremembered that :req-un
behaved that way, but apparently not:
all-test> (s/def :foo/bar string?)
:foo/bar
all-test> (s/def ::baz (s/keys :req-un [:foo/bar]))
:dw-domain-specs.specs.all-test/baz
all-test> (s/valid? ::baz {:bar "hi"})
true
all-test> (s/valid? ::baz {:foo/bar "hi"})
false
Is there any good way to say that either qualified or unqualified is fine? I'd like to be able to reuse the same spec both in internal contexts where keys are qualified and at the system boundary where keys come in unqualified. The simplest thing I've thought of is to use :req-un
and always strip namespaces before checking for validity. But I thought that spec itself had a way to express that...#2019-12-0515:04Alex Miller (Clojure team)it doesn't#2019-12-0515:05eggsyntaxOh well. Thanks, Alex!
Might be an interesting feature to consider at some point, it'd make spec reuse easier and encourage people to use qualified keys when possible rather than defaulting to the least common denominator of unqualified keys (in cases where they have to deal with unqualified keys somewhere in their system).#2019-12-0515:31Alex Miller (Clojure team)the intent with spec is to strongly encourage you to use qualified keys#2019-12-0515:32Alex Miller (Clojure team)spec 2 has more support for working with unqual in schemas though#2019-12-0515:32Alex Miller (Clojure team)unqualified that are not tied by name to qualified, but just directly to specs#2019-12-0516:23eggsyntaxMeaning something like this?
(s/def :foo/bar string?)
(s/select {:bar :foo/bar} [:bar])
#2019-12-0516:24Alex Miller (Clojure team)yeah#2019-12-0517:46eggsyntaxIn a spirit of spec 2 brainstorming -- it seems like this case is similar to closed spec checking: something that it's better to avoid, but in some specific contexts turns out to be necessary. It would be cool if rather than write the boilerplate I gave above (which would be tedious and noisy in the case of an edge function that takes a map with a lot of unqualified keys) you could use an :unqualified
setting with conforming API calls (just like with :closed
):
(s/valid? ::foo {:bar "a"} {:unqualified true})
This also somewhat parallels the use of :keys
in map destructuring for the common case where you want the binding names to be the same as the key names -- ie it handles the common case where the unqualified keys are the same as the names of the qualified keys.
Just a thought š#2019-12-0518:05Alex Miller (Clojure team)again, trying to encourage qualified keys#2019-12-0518:06eggsyntaxFair enough š#2019-12-0518:06Alex Miller (Clojure team)the settings are really for conform ops (valid? / conform / explain)#2019-12-0518:07Alex Miller (Clojure team)unqualified would also need to be in play for gen#2019-12-0518:10eggsyntaxThat makes sense. I was only suggesting it for valid? / conform / explain, but I see how gen breaks the parallelism with :closed
in an important way.#2019-12-0516:19eggsyntax> the intent with spec is to strongly encourage you to use qualified keys
For sure, and I'm 100% bought into that idea. But sometimes at the edges you have to accept unqualified keys, and it'd be nice to be able to say, even on a case-by-case basis, "I'm looking for qualified keys, but I'm willing to accept unqualified" rather than (as currently) either
a) making a separate unqualified spec for the same sort of data structure (eg a user), for use at the edges; or
b) using a :req-un
spec across the board and internally stripping namespaces from keys before you check for validity.
What would you say is the best way to handle that situation?#2019-12-0516:23Alex Miller (Clojure team)I think I'd either do a) or go the other way and add a pre-transformation to qualify the attributes so they match the qualified specs#2019-12-0516:23Alex Miller (Clojure team)b is stripping good information so I don't think I'd do that#2019-12-0516:24Alex Miller (Clojure team)if you're going to change the data, at least make it better :)#2019-12-0516:24eggsyntaxGood point, and I should have thought of that since I do have places where I already do that š#2019-12-0611:55FloHow can I check whether the (optional) keys are within a set? e.g. valid keys are #{:foo :bar :qux}
and would (s/valid? :myspec {:foo true})
#2019-12-0611:58Floah! map-of
is my friend#2019-12-0614:45bortexzWhen using spec in the context of domain modelling, sometimes I see that some of the require
I do are only for specs, as some entities would have properties from a different entity (sometimes to define relationships). This makes it very easy to end up with circular dependencies, is there any way on the namespace declaration to have only ns aliases, without requiring them to avoid circular dependencies?#2019-12-0614:46Alex Miller (Clojure team)no, but you don't need to require them at all to use keywords with a certain namespace#2019-12-0614:46Alex Miller (Clojure team)you just have to write the full namespace out w/o aliases#2019-12-0614:48bortexzOk, I thought about that, I wanted to know if there was any way to avoid having to write the full namespace#2019-12-0614:49Alex Miller (Clojure team)this is an area where we may add something in the future (https://ask.clojure.org/index.php/2817/lighter-weight-aliasing-for-keywords)#2019-12-0618:34telekidWeāve been using this pattern at Flexport: https://ask.clojure.org/index.php/2817/lighter-weight-aliasing-for-keywords?show=8918#a8918
Not sure if itās a good idea necessarily, but it works for us.#2019-12-0618:59seancorfieldThat looks a lot more work and complexity than a simple alias
/`create-ns` like I pasted in the main channel. Could you explain what your macros are trying to do, beyond just the alias
/`create-ns` call I showed?#2019-12-0620:43telekidYeah, I think my def-synthetic-ns
fn is overly complex, probably because of a combination of āmy first macroā confusion and not noticing that create-ns
returns the ns
#2019-12-0614:50bortexzNice, thanks @alexmiller !#2019-12-0617:14vemvnot sure if I already mentioned this one to you last time we talked, but I tend to use a namespace called kws.clj
(for "keywords") which contains only specs, and zero implementation details (other than very simple predicates)
since kws definitionally depend on nothing, you can always depend on a "kws" ns, and no circular deps will arise#2019-12-0617:59bortexzThat could be useful, but I prefer to have different namespaces as I want to keep ns keywords on the namespace that contains function that operate on them, for now I am using keywords under namespaces that do not correspond to the files, like :product/id
instead of ::product/id
, even if I define itās spec in ::product namespace. I prefer this to writing the full namespace, for that Iāll probably wait until there is a better way of doing so#2019-12-0618:09vemvI see!
In case it wasn't clear, I actually use one kws
ns per model (or 'module', depending on the chosen arch)
e.g. product.clj
+ product/kws.clj
#2019-12-0618:18bortexzOh ok, I thought you meant one big kws. Although this still does not solve interdependency between models where you want to reference each other through the specs, as you might need to import to define specs from another place. Image orders from a product, you might have ::product/id
inside an order, and have ::orders
referencing orders from product#2019-12-0618:24vemvthat design sounds problematic per se tbh: the DB design might not be sound, and in any case the generative part of spec might choke on the mutual recursion#2019-12-0618:30bortexzBut databases have relationships of this kind, by the product_id of an order you are able to get all orders from a product (::orders) . It would be the case when you want to hydrate a one-to-many relationship, the many
has id of the one
, and the one
has an array of the many
#2019-12-0618:33bortexzI am not very familiar with the generative part of spec, but a problem I can see in generative is that it wouldnāt generate the same :product/id for all orders when generating a set of orders, aside from that the specs per se are not recursive, no spec end up calling itself, but both reference things from the other#2019-12-0618:36vemvone-to-many is easily modeled as:
product/kws.clj # zero references to orders; since products exist independently from orders
order/kws.clj # references product specs; this is a simple (non-cyclic) dependency
and even a fancy many-to-many relationship can be modeled as:
product/kws.clj # zero references to orders
order/kws.clj # zero references to products
order_product/kws.clj # depends on orders and products
#2019-12-0618:38vemvI wouldn't let a trivial concern like ids (normally ids have the same type across all SQL tables) introduce cyclical deps
IMO that's true regardless of the way you decide to use spec#2019-12-0618:55bortexzI still think could be useful to have some product info on the order itself (without directly referencing a product). Maybe this scenario is not the ideal one for this use case, but I still think you might want to reference inside your schema specs from other ns, I donāt think itās bad to have cyclical namespaced keywords inside specs, as itās not a dependency itself, and allows a lot of flexibility for how you retrieve your domain entities. In my case the spec are not directly translated to database tables or columns (at least not a database I control), and sometimes properties from a parent entity are hydrated directly on the child entity, not always referencing by id#2019-12-0618:58bortexz> I wouldnāt let a trivial concern like ids (normally ids have the same type across all SQL tables) introduce cyclical deps
Namespaced keywords do not need a require, so itās not strictly a cyclical dep, thatās only for aliasing them, and I think it is better to have the flexibility of being able to do that (or at least as it is now just using full namespace)#2019-12-0619:00vemvCould be, I'd have to check it out carefully
Still, keep in mind that specs are composable.
so you can have non-cyclical, discrete product
and order
specs, and then completely aside, a hydrated-order
spec that composes both.
keeps things non-cyclical, without compromising DRY either :)#2019-12-0619:02vemvalso keep in mind that at write time, order
should only have thin references to products. so a full-blown spec could be an impediment, as it could require more product keys to be present.
the hypothetical hydrated-order
spec keeps read/write concerns separate#2019-12-0619:04bortexzschema/select allow to handle those contextual dependencies#2019-12-0619:05bortexzBasically I am going for these specs with schema like āThis is everything an order might containā, then by context I could select some properties#2019-12-0619:05vemvyeah. it's in the radar for spec 2 as you probably know#2019-12-0619:06vemvmust be said, considering graphql-style responses, a monolithic spec can end up being problematic. there can be N "read models" of a given thing#2019-12-0619:06bortexzYeah, as this is a personal project I am already using it, so my concern was on that context#2019-12-0619:07bortexz> must be said, considering graphql-style responses, a monolithic spec can end up being problematic. there can be N āread modelsā of a given thing
I am not familiar with graphql, what do you mean N read models?#2019-12-0619:09vemvthere are N possible schemas ("models") in which a client might want a response
e.g. [[1 2] [3 4]]
or {1 2 3 4}
(same data, different representation)
a monolithic spec cannot reasonably define all schemas that client could request#2019-12-0619:16bortexzWhy not? And what do you mean by monolithic schema?#2019-12-0619:18vemvmonolithic: a single spec that can be used for describing both reads and writes, relying on select
for flexiblity#2019-12-0619:19vemvproblem with that:
different clients (e.g. web, android, ios) might want different data shapes, pagination, camelCase, referenced models, etc
probably a single spec defining all possible combinations of those would not be maintainable#2019-12-0619:55bortexzWhat would be best in that case, one per read model and one per write?#2019-12-0619:56bortexzI think I would have 1 spec for the read you retrieve from the DB, and then map it to whatever the client expects, probably not event having a schema for the specific data shape of the client, or if you want, then a spec for each of those too#2019-12-0619:59bortexzItās far from my use case though#2019-12-0703:29vemv> What would be best in that case, one per read model and one per write?
at work I advocate the following:
* there's one write model (since there should be only 1 acceptable way of persisting things)
* by default, that also serves as the read model (since one always can read a subset of what he wrote)
* there also can be N additional read models, as your needs grow
so, rule of thumb is 1:N#2019-12-0703:31vemv> Itās far from my use case though (edited)
yeah I don't even particularly like graphql, and in a personal project you likely won't have 10 different clients ;p
but I like finding patterns that scale, that are universal. that way I don't have to worry later, or have different patterns per-project#2019-12-0708:45bortexz> or have different patterns per-project
I disagree here, depending on the scope of the project this could make sense#2019-12-0708:56vemvmorning :)
sure thing. Personally I enjoy having a minimal amount of patterns in my head. The tradeoff being that sometimes 'scalable' solutions can certainly feel as overkill#2019-12-0709:13bortexzMorning š
I think though that scalable could mean different things on different projects, to justify different patterns to achieve such scalability#2019-12-0618:12seancorfieldYou can also do this to get a local alias without needing the full namespace to actually exist as code:
(alias 'm (create-ns 'ws.domain.member))
#2019-12-0619:03bortexzThanks! This looks really good and something that solves my use case š#2019-12-0618:12seancorfieldThen ::m/country
will expand to :ws.domain.member/country
for the spec name.#2019-12-0618:13seancorfield(in this particular case, that ns does exist, but we don't need to require
it everywhere we use the specs)#2019-12-0703:49vemvGenuine question (and not an exercise in snobbery ;p), I wonder if many people's seeming disfavour for ns-qualified keywords comes from lack of (properly setup or understood?) IDE tooling.
With cljr-slash
(https://github.com/clojure-emacs/clj-refactor.el/blob/50d2d8aad5e0bd8002173b300f8419d72ceab7af/clj-refactor.el#L1968 ), I type ::foo/
and the right :require will be inserted in my ns declaration, and the requiring will be actually performed into the runtime
that's an instantaneous op, and after that, tab completions for ::foo/
will be 100% accurate
I can imagine that if lacking this tooling, using ns-qualified kws could be more tedious.#2019-12-0703:53eggsyntaxOooh, I hadnāt stumbled on cljr-slash
before, thanks for pointing that out!#2019-12-0703:54eggsyntaxI can say that for me personally, I found it annoying using the more verbose namespaced keys, until I started to feel the benefits directly. Now Iām the person whoās pushing people toward namespacing keywords everywhereā¦#2019-12-0703:54seancorfieldI don't find auto-completion/hinting to be any obstacle at all to using ns-qualified keywords, to be honest. My set up is Atom/Chlorine with its bare-bones completion (without Compliment). Stu Halloway says he doesn't use auto-complete/hinting at all in his editor setup -- and he seems to use ns-qualified keywords just fine š#2019-12-0703:57seancorfieldAs I'm converting code at work from clojure.java.jdbc
to next.jdbc
I'm working more and more with (table)-qualified keywords and I like it. I'm even changing some old code that still uses c.j.j to use the :qualifier
option to add a table-qualifier to the column keywords š#2019-12-0703:59vemvinteresting design, thanks for sharing!#2019-12-0704:01seancorfieldIt was this that caused me to update my CFML/Clojure bridge library to support namespace-qualified keys in hash maps for interop, BTW š#2019-12-0822:25fredericIs this supposed to work in spec 2?
(s/def ::some-keyword ::some-other-keyword)
When I try to validate against ::some-keyword
I get this exception:
java.lang.IllegalArgumentException: No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.Keyword
I can fix the symptoms by doing this instead of s/def
, but the fact that I have to do a special dance makes me suspect that I might be working against the tool's grain
(s/register ::some-keyword (s/resolve-spec ::some-other-keyword))
#2019-12-0822:34fredericI get that spec2 is stricter than spec1 in terms of separating symbolic specs from spec objects.
From the documentation (https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha) (emphasis mine)
> s/def
is a helpful wrapper macro for s/register
that understands how to interpret all kinds of symbolic specs (not just spec forms), including symbols and sets, which will be wrapped in the helper s/spec
spec op.
I was hoping that ::some-other-keyword
would qualify as a symbolic spec, but maybe not.#2019-12-0822:34seancorfieldI believe aliased Specs are supposed to work but there are situations where you can forward reference in Spec 1 but you cannot yet forward reference in Spec 2.#2019-12-0822:34seancorfieldI assume ::some-other-keyword
is defined after ::some-keyword
here?#2019-12-0822:35fredericActually, it's imported from another ns in my real code, shouldn't have tried to simplify that away#2019-12-0822:35seancorfielduser=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::one pos-int?)
:user/one
user=> (s/def ::two ::one)
:user/two
user=> (s/valid? ::two -1)
false
user=> (s/valid? ::two 1)
true
user=>
#2019-12-0822:35frederichmm#2019-12-0822:35fredericinteresting#2019-12-0822:35fredericThere must be something else going on then#2019-12-0822:36fredericLet's see if I can build a minimal example that reproduces the behaviour I observe#2019-12-0822:36seancorfieldIn that simple case, it does let you forward reference FYI -- I tried it in a fresh REPL session with the defs of ::one
and ::two
swapped and it still worked.#2019-12-0822:40fredericInteresting. Thank you for the handholding š, will try to figure out what I'm doing differently and report back#2019-12-0822:52seancorfieldWith the caveat, again, that there are numerous bugs in Spec2 still at this point. And it sounds like function specs will change dramatically before they're done, according to Alex's latest blog point (and s/and
may become non-flowing -- which would be another departure from Spec1). While I was working against the Spec2 branch for a while, things would break between various updates on master, sometimes intentionally but often unintentionally. It's very much a moving target right now.#2019-12-0822:54seancorfieldI had hoped to migrate our (large) codebase from Spec1 to Spec2 but, in reality, I think we're "stuck" with Spec1 in our existing code at this point and we'll just adopt Spec2 for new code (depending on how Spec1 and Spec2 interop -- because they don't, right now).#2019-12-0822:54seancorfieldEventually, we'll convert everything over but I suspect we'll do it piecemeal over time.#2019-12-0822:58fredericMine is just a hobby project, but yes, I might have been over eager there š#2019-12-0823:00fredericMaybe I should just switch back to spec 1, and follow the advice above (<https://clojurians.slack.com/archives/C1B1BB2Q3/p1575541261064300>) to make all my specs :opt
rather than :req
, merging in :req
at the last minute to emulate schema
and select
.#2019-12-0823:00fredericThat way I'd also get orchestra and expound, which would be nice#2019-12-0823:01fredericAnd probably other libraries and tooling integration that I'm not aware of yet.#2019-12-0823:04eggsyntax> make all my specs :opt rather than :req, merging in :req at the last minute to emulate schema and select.
I hadnāt seen the discussion of that above, but thatās pretty much what Iāve come to independently for a large-ish set of domain specs Iāve been working on. Seems to be a decent approach.#2019-12-0823:06eggsyntaxAlong with a couple of helper functions to eg build a selection from a schema less verbosely by passing in the base schema and the list of attributes that should be required (in the simple cases), and by passing in the base schema and a full tree of requiredness (for complex cases).#2019-12-0901:51fredericConverting to spec 1 fixed my problem. I must have run into some corner case that wasn't covered yet.#2019-12-0915:52arohnerI have a multi-spec which works fine, but for a test I want to gen messages of a single type. If I do (defmethod my-spec ::foo [_] ::foo)
, then the messages are missing retag
. If I include the retag
key in ::foo
, the keys are auto-generated and donāt correspond to the message type. Whatās the best way to make the correct retag show up in the individual messages?#2019-12-0917:48Alex Miller (Clojure team)if you hit the backchannel here, several people have walked through this recently - zulip has the archive#2019-12-0917:49Alex Miller (Clojure team)https://clojurians.zulipchat.com/#narrow/stream/180378-slack-archive/topic/clojure-spec is the link there I think#2019-12-1009:33bortexzIn spec2, can you create a subschema of a schema, like doing select-keys
on schema keeping the optionality?#2019-12-1010:21vlaaadisn't it what select
is for?#2019-12-1010:32bortexzselect
returns the required keys from a schema on a specific context, I am looking for something to generate schemas of optionals from a subset of keys of another schema#2019-12-1013:44Alex Miller (Clojure team)Schema keys are all optional. I donāt understand what youāre trying to do.#2019-12-1014:05bortexzImagine I have an entity, for which I have a spec/schema with a set of keys that it can have.
Now, I would like to spec the request parameters to modify the given entity. You can only modify half of the parameters of that entity, although each of them is still optional.
How would I approach this case? Does it make sense?#2019-12-1014:16Alex Miller (Clojure team)a select
is exactly the case of specifying a more constrained structure based on a schema
, intended specifically for "in a context", like request parameters#2019-12-1014:17Alex Miller (Clojure team)if the keys are optional, then the existing schema suffices#2019-12-1014:41bortexzI see, in terms of validation and instrumentation it makes sense, just use the ābig schemaā with all possible keys and select if there is any requirement. I guess what I was trying to achieve was more aligned with a āspecification / documentationā use of schema, so I could tell which of the keys are supported
by a specific function, rather tan required
#2019-12-1014:52bortexzIs there in spec any concept or would it make sense to have something to specify that a property is supported? with a meaning of The property will be used if it exists, but it's not required
, or is it something that should be left to documentation / outside of the code specification?#2019-12-1014:59Alex Miller (Clojure team)nothing like that in spec, no#2019-12-1015:12bortexzThanks alex. Would it make sense in the context of spec? I think it could be something very close to the concept of select
. As I imagine it, would be:
ā¢ schema: All keys that travel together, outside of function context (all are optional, although wouldnāt optional also be contextual? optional
meaning supported or used
)
ā¢ select: All keys that are used in a specific context (With the slight change that it would allow to select optional and required keys for the case mentioned above)
Is this something that makes sense? Am I missing something in how I conceive spec?#2019-12-1015:17Alex Miller (Clojure team)schemas are all optional, select is all about what's required#2019-12-1015:58bortexzI (think I) understand, what I am trying to achieve maybe fits more into the metadata space (saying what is used if specified, for auto generated doc or generative test) rather than in spec.
I am trying to wrap my mind around the philosophy behind the schema/select approach, I already find it quite useful how it is now, although sometimes I find myself in a 3-concepts space rather than two (schema/select) when thinking about the information model:
1. What travels together (big schema with all related kw, e.g. domain entity)
2. What a context uses (subset of kw used by the fn. Not everything used is required, but itās not ignored)
3. What a context requires to work properly (required set of kw for the fn to work)
Maybe it makes sense to have smaller schemas in 2. with shared kw with 1. , or to have 2. just as metadata of the fn outisde spec. Any ideas on this ?
Seems a good time to re-watch Maybe not
, I might be missing something about the philosophy of the tool#2019-12-1016:01Alex Miller (Clojure team)On 2, you don't have to use a predefined schema, you can just restate specifically what attrs are allowed, which may be a smaller set#2019-12-1016:02Alex Miller (Clojure team)or you can just make a schema that is a smaller set#2019-12-1016:17bortexz> or you can just make a schema that is a smaller set
Iāll go for that. Which brings me to the beginning, create schema from subset of another schema, (or join schemaās together). Probably something that can be done as tooling outside spec2.
Looking forward for a final release š cheers!#2019-12-1017:28seancorfields/merge
could merge smaller schemas to make a bigger schema, yes?#2019-12-1017:31bortexz^ Indeed !#2019-12-1017:33bortexzBy <https://github.com/clojure/spec-alpha2/wiki/Schema-and-select> I think s/union
would be the one to use on schemas#2019-12-1017:45bortexzThinking about my previous comment of generating schema from subset of schemas, it doesnāt make any sense :man-facepalming: Schemaās syntax is just the set of keys (when ns-kw at least)#2019-12-1017:46Alex Miller (Clojure team)We may actually re-purpose āunionā for nonflowing s/and so donāt get too attached to that name :)#2019-12-1018:07seancorfieldUsual caveat: Spec2 is a moving target whose function names may change and/or whose semantics may change, potentially several times š#2019-12-1018:26bortexzYeah, luckily it is a personal project with a small codebase, refactors arenāt very painful (I do lots of them to try new approaches). Having that into account, I was avoiding to go much further than defāing schemas, which it feels already useful to define my domain entities and properties (and understand them) as a common ground between a set of external services that all of them use similar entities but each one of them with different set of properties and property names and formats#2019-12-1011:12mpenetfound a bug ? https://gist.github.com/mpenet/fa611105b4a0a38126c5ba8ffe60d8a7#2019-12-1011:13mpenet#2019-12-1011:13mpenetbasically spec will happily validate against stuff that was never declared, just checking for presence of the key as if any?#2019-12-1011:15mpenetthe docstring is not really clear about that one#2019-12-1013:45Alex Miller (Clojure team)s/keys checks that required keys exists and that all registered keys have values that conform#2019-12-1013:47Alex Miller (Clojure team)So, not a bug. I think youāll get exceptions if you try to gen though#2019-12-1014:03mpenetyes, it throws#2019-12-1020:35danielcomptonIs there a way to validate a "closed select"? I can see how to check a schema with closed settings, but it doesn't seem like that works for selects?
(s/def ::a (fn [x]
(Thread/sleep 2000)
true))
(s/def ::b string?)
(s/def ::b-sch (s/schema [::b]))
(s/def ::b-sel (s/select ::b-sch [::b]))
(comment
(s/valid? ::b-sel {::b "a" ::a nil})
; => <waits> true
(s/valid? ::b-sch {::b "a" ::a nil} {:closed #{::b-sch}})
; => <immediately> false
(s/valid? ::b-sel {::b "a" ::a nil} {:closed #{::b-sch}})
; => <waits> true
(s/valid? ::b-sel {::b "a" ::a nil} {:closed #{::b-sel}})
; => <waits> true
)
#2019-12-1100:44Alex Miller (Clojure team)I think the select linkage is not fully implemented yet, long story, wip#2019-12-1020:36danielcomptonThe use case is for API boundaries, where you want to make a selection of keys that you will accept, fail if extra keys are provided, and especially to not validate the extra keys before failing#2019-12-1102:56bbrinckSlightly off-topic, but I have a question about what people would prefer in their error messages. Iād appreciate a vote if you have a minute https://twitter.com/bbrinck/status/1204595098207444993#2019-12-1122:43kennyDoes anyone have a keys
spec but for a map with string keys?#2019-12-1123:00seancorfield@kenny Not sure how you'd do that -- s/keys
requires keywords I believe? You could use s/map-of
for string keys (and add some custom predicates depending on how much checking you need)...#2019-12-1123:00kennyYeah that's what I have. Was hoping someone had written something already that did that & generated valid values.#2019-12-1123:17Alex Miller (Clojure team)not that I'm aware of#2019-12-1123:17Alex Miller (Clojure team)you can get some of what you want by combining s/and and an s/conformer that keywordizes keys#2019-12-1123:18Alex Miller (Clojure team)not sure it works on all the api operations the way you want though#2019-12-1201:05Oliver GeorgeHi Alex, Iām super excited about the new spec. Just trying to moderate my expectationsā¦ Do you think it will be a complex port to clojurescript? From memory the first version took a while (months?) to come out. #2019-12-1201:10Alex Miller (Clojure team)ĀÆ\(ć)/ĀÆ#2019-12-1201:12Alex Miller (Clojure team)I think there were a couple tricky parts to the cljs port in gen/instrument/check - I suspect those tricky bits aren't going to change, at least in the tricky parts. And I think the increased clarity around symbols vs objects is probably likely to make the cljs code larger but simpler#2019-12-1201:13Alex Miller (Clojure team)so I don't see any reason it should be particularly hard, but there is a lot of internal structural change, enough that if it were me doing it, I'd probably start fresh and pull from the old code as needed#2019-12-1201:14Alex Miller (Clojure team)that said, we're not done yet :)#2019-12-1201:24Oliver GeorgeThanks #2019-12-1201:25Oliver GeorgeWhile Iāve got you can I ask about the idea of including specs in defn. Do you think we will see :args and :ret keys added to the pre/post condition map?#2019-12-1203:11Alex Miller (Clojure team)Havenāt looked at that yet#2019-12-1203:12Alex Miller (Clojure team)Ret and fn specs are likely going to change a lot#2019-12-1203:20Oliver GeorgeFair enough. Thanks.#2019-12-1216:13gariepyalexI am trying without success to generate clj-time
dates using specs in clj-time.spec
. (s/exercise ::spec/date-time)
always returns dates close to January 1rst 2011 by a few millis. The spec in the lib is using:
(spec/def ::past-and-future (spec/int-in (to-long (date-time 2011 1 1 00 00 00))
(to-long (date-time 2030 12 31 23 59 59))))
or simplified
(s/exercise (s/int-in 1293840000000 1924991999000))
which always returns numbers close to 1293840000000.
Under the hood, it looks like (gen/large-integer* {:min 1293840000000 :max 1924991999000})
. When I run this directly, I have the same issue.
Is there a way to generate dates that are more spread from 2011 to 2030?#2019-12-1216:22ghadiare you sure that it doesn't start near 2011, then move away progressively? @gariepyalex#2019-12-1216:22ghadiif you sample more values#2019-12-1216:28gariepyalexYou are right. With more samples some are different. Still, most date are very close to the minimum value. This is expected behavior? If I want something else, I need to write a custom generator?#2019-12-1216:30ghadiThis is expected.#2019-12-1216:30ghadithere's a control -- I don't have a link handy, but see the test.check wiki I think#2019-12-1216:31ghadihttps://clojure.github.io/test.check/growth-and-shrinking.html#2019-12-1216:32gfredericks@gariepyalex the expected behavior of large-integer is that many values are close to 0#2019-12-1216:33gfredericksSo any date/time generator naively implemented with large-integer will have the sort of behavior you're describing#2019-12-1216:33gfredericksThere's no datetime generator that doesn't privilege some span of time#2019-12-1216:35gariepyalex> From https://clojure.github.io/test.check/growth-and-shrinking.html
> gen/sample
starts with very small sizes in the same way that the quick-check
function does. This can be misleading to users who donāt expect that and take the first ten results from gen/sample
to be representative of the distribution of a generator. Using gen/generate
with an explicit size
argument can be a better way of learning about the distribution of a generator.
Thanks @ghadi and @gfredericks!#2019-12-1216:36gfredericks:+1:#2019-12-1320:32kvltIs it not possible to use with-redefs
inside of defspec
?
I have a relatively simple test that seems to suggest that it's not working:
The function I'm testing is here:
(defn send-email2
[domain]
(gen-uri domain "/messages"))
The test using deftest
:
(deftest abc-test
(with-redefs [sut/gen-uri (constantly "")]
(is (= ""
(sut/send-email2 "String is ignored because of the redef")))))
(abc-test) => nil
The test using defspec
:
(defspec send-email-test 1
(with-redefs [sut/gen-uri (constantly "")]
(prop/for-all [domain (s/gen string?)]
(is (= ""
(sut/send-email2 domain))))))
(send-email-test) =>
FAIL in () (email_test.clj:24)
expected: ""
actual: ("")
{:result false,
:seed 1576269139182,
:failing-size 0,
:num-tests 1,
:fail [""],
:shrunk {:total-nodes-visited 0, :depth 0, :result false, :smallest [""]}}
#2019-12-1320:42gfredericksNot to override generators like that, no#2019-12-1320:43gfredericksI'm assuming gen-uri is a generator#2019-12-1320:44kvltgen-uri
is a function that generates a url to a service I'm using to send email:
(defn gen-uri
[domain route]
(-> domain
base-url
(str route)))
#2019-12-1320:44kvltI see how it was poorly named for this example, sorry for that#2019-12-1320:44kvltWhat I really want to mock is the http call to the service that sends the email. BUt wanted to keep things simple#2019-12-1320:45gfredericksThen try reversing the order of prop and with-redefs#2019-12-1320:45kvltOh I feel silly#2019-12-1320:45kvltI didn't expect that to throw things off#2019-12-1320:46kvltThanky ou#2019-12-1320:47gfredericksThe prop call is essentially creating a function that gets called later, at which point your with-redefs has already exited#2019-12-1320:47gfredericksSimilar to if you put it around a defn#2019-12-1320:47kvltThat makes sense.#2019-12-1320:47kvltThank you again!#2019-12-1321:40Alex Miller (Clojure team)I don't know if you've looked at it, but clojure.spec.test.alpha/instrument has the ability to mock and stub#2019-12-1411:32kwrooijenHey, I was wondering if there was a shorthand for creating fdefs
, For example I have:
(s/fdef new-person
:args (s/cat :name :person/name
:age :person/age)
:ret :app/person)
(defn new-person [name age]
{:person/name name
:person/age age})
Which works, but itās quite a lot noise (seeing that the fdef
is larger than the function itself
I know I shouldnāt be comparing spec to type signatures, but thatās basically what Iām doing here. Something like this would be pretty useful for me:
(s/sig new-person [:person/name :person/age] :app/person)
(defn new-person [name age]
{:person/name name
:person/age age})
#2019-12-1413:27taylorI think Iāve seen a few libraries that have macros combining defn and fn specs#2019-12-1413:29taylorhereās one https://github.com/danielcompton/defn-spec#2019-12-1413:29taylorhttps://github.com/danielcompton/defn-spec#alternatives#2019-12-1414:19kwrooijenThanks, these sort of do what I want. But they replace defn which I donāt like.
I couldnāt find anything so I ended up writing a quick macro:
(defmacro defsig
[fname fargs fret]
`(clojure.spec.alpha/fdef ~fname
:args (clojure.spec.alpha/cat
#2019-12-1413:53Alex Miller (Clojure team)Stay tuned for spec 2.... :)#2019-12-1414:21kwrooijenLooking forward to that!#2019-12-1416:02Alex Miller (Clojure team)Rich is working on the design for this right now and itās starting to look pretty good #2019-12-1514:14valeraukoI'd be interested to hear what your workflow (feedback loop?) is for developing the language. Are there any blogs about that?#2019-12-1514:48Alex Miller (Clojure team)I did a talk at Clojutre that talks about it some#2019-12-1806:30valeraukoin your Nice and Accurate Counsel?#2019-12-1416:23kwrooijenThat would be such an amazing improvement for me. Iād honestly want to spec every function at my day job (I guess thatās the static typing mindset in my head talking), but itās just too verbose at the moment#2019-12-1416:24kwrooijenitās also difficult to convince my colleagues š#2019-12-1613:46kwrooijenWrote a small library which does what I need: https://github.com/kwrooijen/spec-signature
Hopefully spec2 will make this obsolete, but for now this works š#2019-12-1716:24vemv> Hopefully spec2 will make this obsolete
I also hope the current alternative landscape goes mostly obsolete - obviously not good to have N competing solutions, or syntaxes. If browsing a given general-purpose library (e.g. a file unzipper, whatever), casual readers shouldn't struggle to understand its defns. It goes in the opposite direction of having a common language that we all speak :)
There are some interesting ideas around though, like ghostwheel's.
https://github.com/nedap/speced.def which I authored is quite minimalistic, in that regard. Although it has advanced features like spec -> type hint inference, which at this point would be painful to stop benefiting from
(at work we have essentially zero reflection warnings, other than those coming from external libs. Our lib makes this easy and meaningful)#2019-12-1718:44zane> obviously not good to have N competing solutions
Rich said something interesting about this a few months ago: https://www.reddit.com/r/Clojure/comments/crnq9f/joy_clark_interviews_rich_hickey_problem_solving/ex7fitv/#2019-12-1719:01vemvThe money quote being, I presume:
> I'm not gonna go code up this big thing, because you know, some of the things that you're talking about as being standard, they also have a bunch of known shortcomings
Fully agreed. I don't look forward to a big solution that solves everyone's problems
But I do advocate small-yet-full solutions that actually solve one specific problem.
Else the landscape you have is a Lisp Curse
http://www.winestockwebdesign.com/Essays/Lisp_Curse.html .
There's actually a pretty strong analogy between Common Lispers doing OOP and Clojurists wrapping spec :)#2019-12-1821:46aghow can I ns-resolve symbol
a spec sitting in a different namespace based on a given string?#2019-12-1821:47Alex Miller (Clojure team)can you more clearly state inputs and output?#2019-12-1821:49agI want something like:
(ns-resolve
'my-specs
(symbol "foo"))
but for a spec. How can I resolve ::my-specs/foo
when given two strings āmy-specsā and āfooā#2019-12-1821:49Alex Miller (Clojure team)resolve to what?#2019-12-1821:50Alex Miller (Clojure team)a spec object?#2019-12-1821:50Alex Miller (Clojure team)a fq keyword?#2019-12-1821:51agyeah, letās say I want to validate a spec, but instead of spec I have its name as a string#2019-12-1821:51Alex Miller (Clojure team)my-specs
is an alias only meaningful in the context of a namespace. is that the current namespace or some other one?#2019-12-1821:54agso letās say I have a bunch of fields that I extracted from e.g. SQL query, itās a vector of ["foo" "bar"]
. I have a namespace my-specs
with two specs in it: (s/def ::foo string?)
and (s/def ::bar boolean?)
ā¦ now I want to validate data, or generate data, basically I need to āgetā those specs#2019-12-1821:55agall that programmatically#2019-12-1821:55Alex Miller (Clojure team)do you know that all of these specs are in my-specs
?#2019-12-1821:57agsureā¦ letās assume they are 100% there#2019-12-1821:57Alex Miller (Clojure team)if so (s/get-spec (keyword "my-specs" "foo"))
should work?#2019-12-1821:58agOMGā¦ thereās a literally a function called get-spec
ā¦ LOL#2019-12-1821:58agThank you Alexā¦ sorry for being so lame at explaining#2019-12-1821:58Alex Miller (Clojure team)np, just trying to impedance match :)#2019-12-1821:59aghoweverā¦ what if I want to check if the spec is indeed there? š#2019-12-1821:59Alex Miller (Clojure team)well if you get nil, it's not there :)#2019-12-1821:59Alex Miller (Clojure team)you can also call (s/registry)
to just get the full map too#2019-12-1821:59agahā¦ okayā¦ awesomeā¦ exactly what I needed. Thanks again!#2019-12-1822:00seancorfield@ag expectations.clojure.test
does a dynamic lookup like that to let you "expect" values conform to specs: https://github.com/clojure-expectations/clojure-test/blob/master/src/expectations/clojure/test.clj#L34-L40#2019-12-1822:01seancorfield(the dynamic require/resolve is so the code can run on Clojure 1.8)#2019-12-1822:01agOhā¦ thatās nice. Iāll give a gander as well. Thank you Sean!#2019-12-1822:48aghey friendsā¦ another dynamic lookup related question:
if I have a (s/keys)
spec and string representation of one of the fields how do I get-spec of that?
e.g: I have:
::foo-spec/foo
defined as:
(s/def ::foo
(s/keys :req-un [::bar-spec/bar]))
and in bar-spec ns I have:
(s/def ::bar (s/keys :req-un [::name]))
and I have strings āfoo-specā, āfooā and ābar.nameā whatās the best way to get-spec of ::bar/name ?
How can I make it work for multiple levels of nesting?#2019-12-1822:53agmeaning I can get-spec/foo
but now I need to āanalyzeā its :bar
field, I have no idea where it sits, is there a way to find it out?#2019-12-1822:53agdynamically?#2019-12-1822:54Alex Miller (Clojure team)there are multiple independent questions here#2019-12-1822:55Alex Miller (Clojure team)re understanding the structure of a keys spec, you can use s/form to get a fully resolved form representing the spec (so you'd see (clojure.spec.alpha/keys :req-un [:bar-spec/name])
)#2019-12-1822:57Alex Miller (Clojure team)finding the subspecs inside that spec is a matter of fishing for it (made somewhat complicated by the and
/ or
support in s/keys). There are a variety of ways to tackle that, all a little meh, that's something we're looking at having better answers for in spec 2.#2019-12-1822:58Alex Miller (Clojure team)if you go that path, you have only fully-qualified subspecs, so you can just pass them to s/get-spec#2019-12-1822:58Alex Miller (Clojure team)and re nesting, there is no such thing as ::bar/name in this case - you've just smooshed together two independent things there#2019-12-1822:59ag> youāve just smooshed together two independent things there
ehmmā¦ Iām just trying to illustrate that I needed nested lookupā¦#2019-12-1823:01Alex Miller (Clojure team)just saying that the specs themselves are not "nested" and have no naming relationship. spec references are always via fully qualified keywords (even if you use autoresolved names to specify them)#2019-12-1823:02agyeah, it seems this isnāt as simple as I thought it would beā¦ so basically I wanted to figure out specs for a vector of strings like:
["account.id" "account.contact.first-name" "account.contact.address.city"]
given that all specs are there with the relations set between them#2019-12-1823:07Alex Miller (Clojure team)what do those strings mean?#2019-12-1823:08Alex Miller (Clojure team)those are nested keys in map data or something?#2019-12-1823:10agso thereās:
(s/def account (s/keys :req-un [::id ::contact])
(s/def contact (s/keys :req-un [::first-name ::address])
(s/def address (s/keys :req-un [::city])
#2019-12-1823:11agand they all may be sitting in different namespaces#2019-12-1823:13agnow without knowing the ns of city
or contact
but knowing where account
resides and given a string āaccount.contact.address.cityā I want to get-spec of ::address/city
#2019-12-1823:20agoh.. well, it gets a tad bit crazier when some fields are s/nilable#2019-12-1823:30Alex Miller (Clojure team)the whole point of having namespaces is being able to differentiate things. seems like you're working really hard to rebuild what that gives you.#2019-12-1823:31Alex Miller (Clojure team)the whole idea behind spec is that attributes are the main source of semantics and maps are weaker aggregations of those#2019-12-1823:31Alex Miller (Clojure team)you are working significantly at odds with that idea#2019-12-1823:33agSo if you want a spec that describes a relational data, how would you do it?#2019-12-1823:35Alex Miller (Clojure team)generically, relational data is sets of maps#2019-12-1823:37seancorfield@ag it would probably help you to remember to use explicit qualifiers (on keywords) in your spec definition and stop using ::
at least until you get your head around this...#2019-12-1823:37agso the above snippet with account -> contact -> address is okay?#2019-12-1823:38seancorfield(s/def :person/account (s/keys :req-un [:account/id :account/contact])
(s/def :account/contact (s/keys :req-un [:contact/first-name :contact/address])
(s/def :contact/address (s/keys :req-un [:address/city])
for example ^#2019-12-1823:39seancorfieldThat also gets you close to the "relational" model if you look at something like next.jdbc
querying a database since it will use qualified keywords for column names, based on the table they are in...#2019-12-1823:40seancorfield(although there you have to reify the primary keys so you'd most likely have :account/id
:account/contact_id
and the latter would be the FK into the :contact
table and its :contact/id
column etc)#2019-12-1823:43seancorfieldIf you did the join
with next.jdbc/execute!
you'd end up with a flat map containing
{:account/id 123, :contact/first-name "Ag", :address/city "Wherever"}
#2019-12-1823:43agyeah, I see how this can simplify a few things.#2019-12-1916:23valeraukoare there any tools to use spec for static checks?#2019-12-1916:50Alex Miller (Clojure team)there's https://github.com/arohner/spectrum#2019-12-1916:51Alex Miller (Clojure team)it's something that's inherently going to have limits, but I was surprised at how far it went#2019-12-1916:53valeraukoawesome! and so many stars too! gonna dive in during the holidays#2019-12-1916:54Alex Miller (Clojure team)there's a talk about it from Conj a couple years ago too#2019-12-1916:55Alex Miller (Clojure team)https://www.youtube.com/watch?v=hzV7dFYmbAs#2019-12-1918:06joefromctif i just type into a repl a fn and a fdef, is there any reason it wouldn't be enforced for a given project?
I have an older project I'm jumping into and can't understand why my fdef's aren't throwing errors... I'm sure check-asserts is true although i didn't think that would even matter.#2019-12-1918:22Alex Miller (Clojure team)fdef on macro or function?#2019-12-1918:22Alex Miller (Clojure team)if function, you have to instrument it for it to do anything#2019-12-1918:24joefromctok, so runtime checks have to be inside the form... either with an assertion, :pre check, or conform.
I guess i missed that in the gude... I think i'll plug in an assertion so i can turn it off when i'm done with my test/dev/exploration on trying to figure out these data structures.#2019-12-1918:24joefromct^ fdef on a function#2019-12-1918:51Alex Miller (Clojure team)note that there is a s/assert#2019-12-1918:54joefromctthanks as always for your help Alex#2019-12-2612:52norton@alexmiller There appears to be a typo on line 16 and line 17 in this commit https://github.com/clojure/spec-alpha2/commit/7c708d063b6ea925fd406f87e08f508b7ed8c91d#diff-04c6e90faac2675aa89e2176d2eec7d8R16#2019-12-2613:55Alex Miller (Clojure team)Can you be more specific?#2019-12-2613:56Vincent Cantinon line 17, a i
instead of [
#2019-12-2613:58Vincent Cantin@alexmiller on line 16, .gen
is missing#2019-12-2614:00Alex Miller (Clojure team)Fixed, thx#2019-12-2717:15Alexis Vincentwhy does the last example not work?
(def s-id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(gen/generate (s/gen s-id)) ;; => works
(s/def ::id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(gen/generate (s/gen ::id)) ;; => works
(s/def ::id s-id)
(gen/generate (s/gen ::id)) ;; => doesn't work
#2019-12-2717:16Alexis Vincentare these not equivalent?
(s/def ::id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
;; and
(def s-id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(s/def ::id s-id)
#2019-12-2717:18Alexis VincentDoes s/def do some macro magic?#2019-12-2717:18Alexis VincentThis is on the latest spec2 btw#2019-12-2717:22seancorfields/def
converts the symbolic form to an actual Spec expression and then registers the spec name. So in (s/def ::id s-id)
you do not have a symbolic form of a spec. It probably should give an error.#2019-12-2717:23Alexis Vincentthanks! so would I then rather do (s/def ::id (s/spec s-id))
or something similar?#2019-12-2717:33Alex Miller (Clojure team)Use s/register instead of s/def#2019-12-2717:33Alexis Vincentawesome! thanks!#2019-12-2717:33Alex Miller (Clojure team)s/register is a function and takes a spec object #2019-12-2717:36Alexis VincentIf I have an s/cat expression but would like to generate/validate vectors not arbitry seqās, how would I express it?#2019-12-2717:37Alexis Vincent(s/def :order/pair
(s/with-gen
(s/cat :base ::asset :counter ::asset)
#(vec
(gen/generate
(s/gen
(s/cat :base ::asset :counter ::asset))))))
This worksish. But obviously is not ideal#2019-12-2717:42Alex Miller (Clojure team)If youāre on spec 2, you can use the new s/catv #2019-12-2717:43Alex Miller (Clojure team)Itās difficult to do easily on spec 1#2019-12-2717:43Alexis VincentPerfect, thanks š#2019-12-2717:55seancorfield@alexmiller Should that (s/def ::id s-id)
give an error? Or is it valid but just does something unexpected?#2019-12-2718:05Alex Miller (Clojure team)Prob error but Iāll look at it next week#2019-12-2718:20Alexis VincentThis is probably the same issue as before, but, are these not equiv in spec 2?
The first one works nicely, but the second one blows up when generating maps from schemas that contain these keys, but not when generating them directly
(s/def :order/id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
;; vs
(s/def ::id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(s/def :order/id ::id)
#2019-12-2718:33seancorfield@mail024 That should probably work but there are bugs in Spec 2 that affect aliasing of specs like that, I believe.#2019-12-2720:17Alexis Vincent@seancorfield ok thanks!#2019-12-2721:25Alexis VincentIf I have a schema with a qualified key Iām trying to override inline, it doesnt seem to want to use my override. It just picks the spec from the registry
(s/union
(s/schema {:event/type #{:event.type/snapshot}})
(s/schema
[:event/nanotime]))
Since event/type
already exists in the registry and since (I assume) it is qualified. It picks the registry definition not the one iām declaring inline#2019-12-2721:31seancorfieldYou can only use the inline specs for unqualified keys.#2019-12-2721:32seancorfieldQualified keys are intended to be globally unique so overriding them doesn't make sense in that context (since their meaning is supposed to be globally fixed).#2019-12-2721:33Alexis Vincentfine. Although in this case itās the generator I want to override. Would s/with-gen work for this?#2019-12-2721:33seancorfieldYes, possibly. Depending on exactly what you are trying to do.#2019-12-2721:34Alexis VincentI have a set of valid event types. Defined as a spec set. I want to define a particular message. Which is some schema, alongside a specific item from the set#2019-12-2721:34seancorfieldIf you want the keys in a hash map to "depend on" the value of a particular key, it sounds like you want multi-specs#2019-12-2721:35seancorfieldIf you define this as a multi-spec, it should generate correctly automatically.#2019-12-2721:35Alexis VincentThanks š Ill check them out.#2019-12-2721:51Alexis VincentPerfect thanks. Working well#2020-12-2907:17Ho0manHi, everyone is using spec/assert
in production a good practice ?
I want to produce detailed exceptions like spec/assert
does for mismatches
but do not want to allow some erroneous code to be able to disable the asserts by calling spec/check-asserts
.
Am I getting something wrong ?
Thanks a alot#2020-12-2908:25vlaaadI use spec/assert
in dev and custom function which is a mostly a valid?
+ throw
in production#2020-12-2911:01Ho0manHi @vlaaad but this way I won't have descriptive exceptions like those thrown by spec/assert
. Am I right ?#2020-12-2911:13vlaaad@ho0man that's configurable by you. when I throw, I store spec/explain-data
as error data and use spec/explain-str
as error message ā all the bits are there#2020-12-2911:18Ho0manThanks a lot @vlaaad, I didnāt know about them
Thanks#2020-12-2914:34unbalancedCurious if there's a way in spec to do the following:
{:a (s/coll-of int?)
:b (s/coll-of int? :count <3 times (count (:a this))>)}
or for that matter is it even possible to specify multiples instead of :count
in coll-of
?#2020-12-2914:36unbalanced(need 3 coordinates per vertex ID, ideally, is what I'm going for)#2020-12-2915:38codonnell@goomba Any predicate function is a spec, so you could use s/and
to combine a spec like you typed above (but without the bit about counts) with a predicate function checking that the counts are valid. If you want generation to work, you'll likely want to provide a custom generator that does something like generate a map with the proper types and then truncate things to make the counts work out.#2020-12-2915:45unbalancedahhh okay thank you! I'll look into this.#2020-01-0203:16EddieI know that you can pull the :ret
and :arg
spec from a function spec.
(def my-fn-spec
(s/fspec :args (s/cat :x int?)
:ret int?))
(s/valid? (:args my-fn-spec) [5]) ;; true
(s/valid? (:ret my-fn-spec) 5) ;; true
Is there a way to decompose/query other kinds of specs? For example, I would like to do something like the following to get the spec for the individual fn argument named x
.
(:x (:args my-fn-spec))
;; or maybe
(first (:args my-fn-spec))
#2020-01-0203:20seancorfieldI think the TL;DR is not easily with Spec 1 @erp12 but Spec 2 offers more facilities for taking specs apart and programmatically building them.#2020-01-0203:25seancorfieldIn Spec 1, you can get the form of a Spec and break it apart, but it isn't easy to turn that back into Spec objects that you can use tho'...#2020-01-0203:42Eddie@seancorfield Good to know, thank you! Based on that, would you agree that currently the best option would be to keep the sub-specs in a map and use some utility functions to "materialize" real specs from them. Just spitballin' here but something like ...
(s/def ::spec (s/spec s/spec?))
; Deconstructed Function Spec
(s/def ::arg-specs (s/coll-of ::spec))
(s/def ::ret-spec ::spec)
(s/def ::d-fn-spec (s/keys :req [::arg-specs ::ret-spec]))
; Deconstructed Collection Spec
(s/def ::coll-kind ::spec)
(s/def ::element-spec ::spec)
(s/def ::d-coll-spec (s/keys :req [::coll-kind ::element-spec]))
; Deconstructed Map Spec
(s/def ::key-spec ::spec)
(s/def ::value-spec ::spec)
(s/def ::d-map-spec (s/keys :req [::key-spec ::value-spec]))
(defn construct-spec
[m]
...)
#2020-01-0203:43EddieOr do you know of any other pattens followed by the community for stuff like this?#2020-01-0203:59seancorfieldI don't really understand what problem you are trying to solve here... It doesn't look like the sort of thing I've seen anyone trying to do with Spec.#2020-01-0204:00seancorfieldHave you looked at Spec 2? That's much more amenable to programmatic manipulation of specs...#2020-01-0220:10rafaelHi. I'm struggling with generating data from a simple (s/schema)
use case.
(s/def ::x int?)
(s/def ::baz (s/schema [::x]))
(s/def ::bar ::baz)
(s/def ::foo (s/schema [::bar]))
(gen/sample (s/gen (s/spec ::foo)))
#2020-01-0220:10rafaelThe call to (gen/sample)
throws a No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword
exception.#2020-01-0220:12rafaelWhile a straightforward translation to s/keys
seems to work fine:
(s/def ::x int?)
(s/def ::baz (s/keys :opt [::x]))
(s/def ::bar ::baz)
(s/def ::foo (s/keys :opt [::bar]))
(gen/sample (s/gen (s/spec ::foo)))
I'm probably getting something wrong in my schema definitions, but I can't figure out the problem.#2020-01-0220:32Alex Miller (Clojure team)code looks fine, prob just a bug#2020-01-0220:32Alex Miller (Clojure team)in spec#2020-01-0220:38rafaelCool, I'll stick with the (s/keys ..)
version for a while.#2020-01-0220:43rafaelChanging the :bar
definition to
(s/register ::bar (s/resolve-spec ::baz))
appears to work around the issue.#2020-01-0220:48Alex Miller (Clojure team)that makes sense - you're basically copying the spec object rather than relying on resolving through the alias#2020-01-0220:48Alex Miller (Clojure team)I have a pretty good hunch on where that bug is#2020-01-0220:57rafaelAwesome, thanks!#2020-01-0303:42Eddie@seancorfield I spent some time today with spec 2. It seems like it's design goals are exactly what I need. I attempted to migrate and it was pretty rough (not complaining, I totally understand that it is still in development and also a new major version so I expected many breaking changes). I am not sure if we will be jumping on board with spec 2 yet, but thank you for the pointer!#2020-01-0303:46EddieThere are a few things that we are doing in spec 1 that I cannot figure out how to recreate in spec 2, even though it smells like it should be possible.
For example, I would like to implement a custom generator that simply pulls a random element from a set.
I can get this to work if the set satisfies (every? constant-val? ...)
but otherwise, I cannot.#2020-01-0303:53Alex Miller (Clojure team)are you using a set spec?#2020-01-0303:56Alex Miller (Clojure team)user=> (s/def ::s #{1 2 3})
:user/s
user=> (gen/sample (s/gen ::s))
(2 1 3 2 2 1 1 1 2 2)
#2020-01-0303:58Alex Miller (Clojure team)if you're trying to do this outside s/def (which has some magic), you can do something like
user=> (gen/sample (s/gen (s/resolve-spec #{1 2 3})))
(1 2 1 1 3 1 3 3 1 1)
#2020-01-0304:00Alex Miller (Clojure team)one of the caveats you might be seeing is that set specs are constrained to constant values only which is part of the new symbolic spec direction#2020-01-0304:01Alex Miller (Clojure team)there are a couple of options but you can just use the built-in generators directly if you have something else#2020-01-0317:08EddieThank you! gen/elements
is exactly what I was looking for.
However I am running into some strange behavior on the with-gen
.
user=> (s/with-gen (s/spec int?)
(fn [] (gen/elements [-1 0 1])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval190$fn$G (protocols.clj:11).
No implementation of method: :with-gen* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.PersistentList
user=> (s/exercise (s/with-gen int? (fn [] (gen/elements [-1 0 1])))
10)
([-1 -1] [0 0] [1 1] [-1 -1] [-1 -1] [-1 -1] [0 0] [1 1] [-1 -1] [1 1])
Neither of those seem right to me. Am I missing a change from Spec 1?#2020-01-0317:14Alex Miller (Clojure team)yeah, seems fishy. I'd expect the first one to work#2020-01-0304:02Alex Miller (Clojure team)user=> (def a 1) (def b 2)
#'user/a
#'user/b
user=> (gen/sample (gen/elements [a b]))
(1 2 2 2 1 2 2 2 2 2)
#2020-01-0316:40A.J. Gardnerhello! it seems like @rafael and I ran into the same issue. I posted about it in the google group: https://groups.google.com/d/topic/clojure/rcuWmqyGWzs/discussion
my specs:
(s/def ::id int?)
(s/def ::tag-id ::id)
(s/def ::child-tag ::tag-id)
(s/conform ::child-tag 22)
;; 22
(s/conform ::child-tag "a")
;; :clojure.alpha.spec/invalid
(s/conform (s/schema [::child-tag]) {::child-tag 22})
;; Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1458$fn$G (protocols.clj:11).
;; No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword
(s/conform (s/schema [::tag-id]) {::tag-id 22})
;; Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1458$fn$G (protocols.clj:11).
;; No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword
(s/conform (s/schema [::id]) {::id 22})
;; #:foo{:id 22}
#2020-01-0316:44A.J. Gardnerand the stack trace, just in case thatās useful:
(clojure.repl/pst)
IllegalArgumentException No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword
clojure.core/-cache-protocol-fn (core_deftype.clj:583)
clojure.core/-cache-protocol-fn (core_deftype.clj:575)
clojure.alpha.spec.protocols/eval1458/fn--1556/G--1439--1567 (protocols.clj:11)
clojure.alpha.spec.impl/schema-impl/reify--2753 (impl.clj:435)
clojure.alpha.spec/conform (spec.clj:245)
clojure.alpha.spec/conform (spec.clj:237)
clojure.alpha.spec/conform (spec.clj:241)
clojure.alpha.spec/conform (spec.clj:237)
foo/eval6028 (form-init13722384443750232197.clj:1)
foo/eval6028 (form-init13722384443750232197.clj:1)
clojure.lang.Compiler.eval (Compiler.java:7177)
clojure.lang.Compiler.eval (Compiler.java:7132)
#2020-01-0316:50Alex Miller (Clojure team)yeah, this is just a bug in spec#2020-01-0317:10A.J. Gardnerwhew#2020-01-0615:52borkdudeI'm in the process of writing a version of spec(1) that can be used with babashka. For now it's an interpreted lib:
$ export BABASHKA_CLASSPATH="$(clojure -Sdeps '{:deps {spartan.spec {:git/url "" :sha "104129aae9eab6dd4622937d0f46abeed9c9c537"}}}' -Spath)"
$ bb -e "(require '[spartan.spec :as s]) (s/explain (s/cat :i int? :j string?) [\"foo\" 1])"
"foo" - failed: int? in: [0] at: [:i]
I'm unsure what parts I could be adding to babashka as a built-in. Are there any assurances what parts of the API will be unchanged, and what the future namespace of spec will be ?#2020-01-0706:15Jakub HolĆ½ (HolyJak)Why not spec 2? Though not finished, I guess it bears completion (just the last 10% that use 90% of the time :rolling_on_the_floor_laughing:)...#2020-01-0615:52borkdudeIf I have some more assurances, I could adopt parts of this lib as built-ins which would make it much faster.#2020-01-0615:55Alex Miller (Clojure team)I can assure you that the future namespace will be different :) The current spec 2 is clojure.alpha.spec
, clojure.alpha.spec.gen
, etc. We're expecting this to eventually be clojure.spec
, clojure.spec.gen
#2020-01-0615:56Alex Miller (Clojure team)most of the existing spec forms will probably be the same, with some additions and some fine details changing around a few things (and deprecation/removal of s/keys)#2020-01-0615:57Alex Miller (Clojure team)most of the spec operations will be the same although we have added arities in all of the conforming ops (valid?, conform, explain, etc) to support spec checking modes#2020-01-0616:04borkdudethanks#2020-01-0617:51seancorfield@alexmiller Any sense yet of whether there will a migration path/tooling at launch of clojure.spec
?#2020-01-0617:54Alex Miller (Clojure team)as long as there are no follow-up questions, yes :)#2020-01-0623:27lgesslerhi, I want to use s/fdef
with protocol functions. In one namespace, I'm calling fdef
and defining and implementing a protocol, but the protocol's methods appear to be unaffected by fdef
. If I understand [this issue](https://clojure.atlassian.net/browse/CLJ-2109) right, this is expected behavior. Is that all right, and if so, is there a recommended workaround for speccing protocol functions?#2020-01-0623:33Alex Miller (Clojure team)Thatās correct - fn specs donāt work with protocol methods. Only workaround Iād suggest is wrapping the call to the protocol function in a method (which I often do as a matter of practice anyways)#2020-01-0718:39telekidIāve got a spec(ish) design question that Iāve been thinking about for a while. I finally decided to throw it in a gist: https://gist.github.com/telekid/0f276bfa3b3f0d395a7a71158adbb35d
Iām curious to know how other people approach this problem?#2020-01-0720:14vemvIn a given (ns models.geography.continent)
, I tend to solve this with:
(spec/def :models.geography.continent.penguins/animals ...)
(spec/def :models.geography.continent.lions/animals ...)
i.e. I don't use ::
syntax for these cases, and I don't create additional Clojure namespaces (files).
I call it the "synthetic ns pattern"
It's a bit verbose, and losing the 1:1 mapping between cljs namespaces and spec namespaces isn't ideal either.
But it's fairly occasional so I've been happy to use these for a couple years#2020-01-0720:15vemvbtw, not sure if this solution is already described in your gist. sometimes I'm impatient š#2020-01-0720:48telekidyeah, I like that#2020-01-0720:49telekidIāve experimented with that a bit in a few places (but I forgot about it when I was writing the gist - thanks for bringing it up)#2020-01-0720:56vemvāļø!
I realised, a more accurate nickname would be "synthetic sub-ns pattern".
so, while it's not a 1:1 mapping, one is still reasonably close to that ideal#2020-01-0721:08telekidfunnily enough, Iāve been using the same name: https://ask.clojure.org/index.php/2817/lighter-weight-aliasing-for-keywords?show=8918#a8918#2020-01-0718:51Eddie@jake142 If EDIT: Gist has been updated. Disregard! :)#2020-01-0718:51Eddie@jake142 If EDIT: Gist has been updated. Disregard! :)#2020-01-0718:52telekidOh, I just realized that I made a typo#2020-01-0718:52telekidfacepalm#2020-01-0718:52telekidone moment#2020-01-0718:52telekidL20 how has ::animals/lions#2020-01-0718:54EddieI see. Thanks for the clarification.#2020-01-0719:05telekidSorry for the confusion!#2020-01-0721:37kennyWhy were multi-specs implemented using multi-methods? I have never found the dynamism of multimethods useful in defining multi-specs. In fact, I think I'd prefer all of the multi-spec types to be declared statically.#2020-01-0721:44ghadi@kenny so that they're open to extension#2020-01-0721:44ghadiin fact I can't think of anything in clojure that's closed#2020-01-0721:45kennyRight. At least interally, I've never found that useful. All type->keys are declared upfront. Multimethods make it hard to determine the input data.#2020-01-0721:45ghadi> All type->keys are declared upfront.
?#2020-01-0721:46kennyDispatch functions#2020-01-0721:47ghadiI'm sorry I still don't understand#2020-01-0721:49kennyFor a particular entity, I know it has types A, B, C. It can never and should never be extended except directly in the definition.#2020-01-0721:49kennyAll the variants of an entity are known upfront.#2020-01-0721:50ghadiok that's your use case#2020-01-0721:50Joe LaneYou might not be the one defining the multispecs, the users of your library may be.#2020-01-0721:50ghadi^^#2020-01-0721:50ghadihttps://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions.clj
look at t.d.a.extensions ^#2020-01-0721:50ghadiand all the extensions in the extensions folder#2020-01-0721:53kennyOh right, I see the use for libs. This internal use case is much different. Perhaps some sort of new macro is what we need:
(multi-spec2 {:type1 (s/keys :opt [])
:type2 (s/keys :opt [])})
#2020-01-0722:32kennyFor those interested...
(defmacro closed-multi-spec
[name dispatch-key dispatch->spec]
(let [defmethods (map (fn [[dispatch spec-form]]
`(defmethod ~name ~dispatch
[~'_]
~spec-form))
dispatch->spec)]
`(do
(defmulti ~name ~dispatch-key)
Don't know if this is where it'll land but it's a start.#2020-01-0721:54ghadiwouldn't it be amazing if you can write that macro today?#2020-01-0721:54ghadiš#2020-01-0800:19kennyHas anyone written a version of spec2 select that works with spec1? Deciding to have keys required or optional for all use cases is so painful š£#2020-01-0800:45seancorfield@kenny Not sure what you mean by "works with spec1"? The two libraries use different registries so you cannot combine them (at least, not easily)#2020-01-0800:45kennyI mean a macro for spec1 that imitates spec2's select.#2020-01-0800:47kennyWhen using nested maps, spec1 doesn't let you change which keys in the nested map are required.#2020-01-0800:47seancorfieldThat's exactly why Spec2 is coming -- that required/optional thing in Spec1 is painful š#2020-01-0800:47seancorfieldIn Spec1 that is complected and baked into how s/keys
works. In Spec2 the set of keys and the requiredness are decomplected.#2020-01-1316:31aviYou know, it just occurred to me āĀ this is going to make it hard to create an interop schema using spec ā in other words, to fully describe a data structure that a system expects/requires, including the keys that are required in each mapā¦
I suppose we might need to write some new plumbing to create a schema based on some spec2 specs, with additional annotation, expectations, etc. š¤#2020-01-1316:49seancorfieldI'm not sure I follow. Spec 2 still let's you specify which keys are required, including nested keys. It just teases the two concepts apart. #2020-01-1317:20aviHmm for some reason I was thinking that the new schema
function was designed solely for the case of defining function specsā¦ but I havenāt looked super closely š¬#2020-01-1317:20aviis it schema
? or maybe Iām thinking about select
āĀ I should just review the docs š sorry#2020-01-1317:43seancorfieldschema
defines the possible set of keys. select
defines the required subset of a schema.#2020-01-1317:44seancorfieldHence, decomplecting specifying requiredness from specifying possible š#2020-01-1317:44aviSounds pretty exciting, honestly!#2020-01-1317:45aviI just did a walkthrough of a pretty involved schema using spec1 with some members of my team who arenāt familiar with it, and they asked a bunch of questions that I answered with ānot right now, but thatās coming in spec2ā#2020-01-1318:10seancorfieldYup, I'm looking forward to Spec2 becoming non-alpha and the "standard" way to work with Spec.#2020-01-1318:12seancorfieldFor a while, I had a branch at work tracking Spec2 but with the number of bugs and changes as it has evolved, it was proving a bit much trying to keep our codebase current on it... so I gave up a while back and we'll take another look once it is stable. I doubt we'll migrate, but we'll probably start using Spec2 for new code (and may convert old code over as we maintain it over time).#2020-01-0800:48kennyYa. Was hoping for a workaround for now. It seems potentially possible to get it to work.#2020-01-0800:49seancorfieldI don't know if s/merge
will let you override an all-optional s/keys
with a subset of keys that are now marked required. Doesn't help you much with nesting I suspect.#2020-01-0800:50seancorfieldYou could just start using Spec2 I suppose š But not for any production work as it's still changing a lot.#2020-01-0800:53kennyIt's gotta be production worthy unfortunately#2020-01-1001:52telekidWe ran into a question about how to spec multimethods while maintaining their open nature. There wasnāt much good documentation on this topic floating around the internet, so I went down the rabbit hole a bit and came back up with this:
https://gist.github.com/telekid/f2e588718dbdfe48306d64e5388bdc15#2020-01-1411:57mpingis there a way to spec a map according to its key? example: if key is a number, value should be a number, if key is something else then value should be a string#2020-01-1413:34Alex Miller (Clojure team)Yes, although itās a little complicated#2020-01-1413:36Alex Miller (Clojure team)The structure is to spec it as a s/coll-of s/tuples, where each tuple is an s/or of whatever key/value is allowed#2020-01-1413:36Alex Miller (Clojure team)The coll-of should also have an :into {}#2020-01-1413:47favilasomething like
(s/coll-of (s/or :num-val (s/tuple number? number?)
:string-val (s/tuple (s/and any? (complement number?)) string?))
:into {})
#2020-01-1413:57Alex Miller (Clojure team)yep#2020-01-1416:59mpinggot it, tks!#2020-01-1417:03colinkahnI ran across something unexpected when using spec alpha v1:
(s/def ::foo number?)
(s/def ::bar-ret ::foo)
(gen/generate (s/gen ::bar-ret {::bar-ret #(gen/return 100)}))
;; => -2123123
this seems to be because when overrides are applied it determines the key to pull from the overrides map using something like this:
(let [s (@#'s/specize ::bar-ret)]
(@#'s/spec-name s))
;; => ::foo
and like that shows ::bar-ret
returns ::foo
.
My question is whether this is by design or perhaps a bug, and is it too hacky using something like (s/def ::bar-ret (s/spec ::foo))
to get this to work?#2020-01-1417:41Alex Miller (Clojure team)this is a known bug (there's a ticket for it) and we plan to fix in spec 2#2020-01-1417:41Alex Miller (Clojure team)your workaround is fine#2020-01-1417:41colinkahnAwesome, thanks!#2020-01-1518:54nullptri have a dumb spec question that unfortunately is in the "difficult to google" category - say i had a sequence with the following structure:
[1 "a" 2 "b" "c" 0 3 "d" "e" "f"]
the pattern here is a number which indicates how many following items there are directly in the sequence -- i adjusted spacing above to hopefully make that clearer
this is of crouse trivial to express if the following items are in another sequence, but scanning the docs i can't see a way to spec the flat version of this -- note that this isn't motivated by any important use case, more just part of me trying to learn spec so feel free to treat this as pure curiosity#2020-01-1519:16Alex Miller (Clojure team)regex specs combine to spec one flat sequence#2020-01-1519:20Alex Miller (Clojure team)so you could do something like (s/* (s/& (s/cat :c nat-int? :s (s/* string?)) #(= (:c %) (count (:s %)))))
#2020-01-1519:23nullptrthanks! that is extremely straightforward -- i feel like i had something similar but surely missed something basic and made a wrong assumption. will compare and learn -- thanks again.#2020-01-1519:24Alex Miller (Clojure team)it's very important to use s/&, not s/and there#2020-01-1519:24Alex Miller (Clojure team)s/and is not a regex spec and will expect a new nested collection boundary#2020-01-1519:24nullptryeah i had the &
but i think i had a surrounding cat
there that was throwing it all off#2020-01-1714:33yuhanIs there any way of relating a neighboring spec's value into the :count
option of s/every
?#2020-01-1714:34yuhaneg. I have a map spec with keys :width
and :things
#2020-01-1714:35yuhanand I want to ensure that things
has width
number of elements#2020-01-1714:35Alex Miller (Clojure team)you need to s/and with a predicate that can do that at the containing level#2020-01-1714:36yuhanokay, I have something like this
(s/def :ctx/region
(s/and (s/keys :req [:ctx/width
:ctx/slots])
#(= (count (:ctx/slots %))
(:ctx/width %))))
#2020-01-1714:37Alex Miller (Clojure team)yep#2020-01-1714:37yuhanso I'll have to write a custom generator if I want to gen examples of the spec?#2020-01-1715:01Alex Miller (Clojure team)yes#2020-01-1715:03Alex Miller (Clojure team)this is true of any data with internal constraints like this. the general strategy is to use gen the width first, then use a combination of fmap and bind to generate the slots and assemble into the map#2020-01-1715:04Alex Miller (Clojure team)you might also ask what the width is buying you in this data structure when it is the same as the count of the slots#2020-01-1715:04Alex Miller (Clojure team)(and as an aside, I find that pushing on data that is hard to spec often improves the shape of the data and the code that uses it)#2020-01-1715:15yuhanMy idea was to have a degree of "memoization" or redundancy baked into the data structure - in this case slots
may be a lazy sequence that is generated on demand, so I don't want to keep calling count
on it unnecessarily#2020-01-1715:16yuhanof course this introduces the risk of width
getting out of sync with the data, which is why I'm using spec to assert the constraints during dev time#2020-01-1715:18Alex Miller (Clojure team)fair enough#2020-01-1715:19yuhanthere are a few other "derived" keys similar to this that I'm storing in the same data structure, just wondering if this isn't considered a code smell?#2020-01-1715:22Alex Miller (Clojure team)it really depends#2020-01-1715:31yuhanas with so many things š#2020-01-1715:33yuhanHere's my best attempt at that generator:
(s/def :ctx/width (s/int-in 1 11))
(s/def :ctx/slot string?)
(s/def :ctx/slots (s/coll-of :ctx/slot))
(s/def :ctx/region
(s/and (s/keys :req [:ctx/width
:ctx/slots]
:gen #(let [w (gen/generate (s/gen :ctx/width))]
(gen/hash-map
:ctx/width (s/gen #{w})
:ctx/slots (s/gen (s/coll-of :ctx/slot
:count w)))))
#(= (count (:ctx/slots %))
(:ctx/width %))))
#2020-01-1715:51Alex Miller (Clojure team)you don't want to use gen/generate in there - that will foil the test.check shrinking mechanisms#2020-01-1715:52Alex Miller (Clojure team)instead, use gen/bind with (s/gen :ctx/width) and the gen/hashmap you have#2020-01-1715:52Alex Miller (Clojure team)gen/bind lets you make a generator on a generator#2020-01-1718:06yuhangot it, thanks so much for the help!
#(gen/bind (s/gen :ctx/width)
(fn [w]
(gen/hash-map
:ctx/width (gen/return w)
:ctx/slots (s/gen (s/coll-of :ctx/slot :count w)))))
#2020-01-1722:32zaneLetās say I have a Datomic query and I want to find all the variables defined in that query. Would spec be a good tool to bring to bear on that problem?#2020-01-1722:50seancorfield@zane I'm not saying it couldn't be done but I certainly wouldn't expect to use Spec for such a problem.#2020-01-1722:51seancorfieldEven if you wrote a complete Spec of Datomic queries, if you s/conform
it so it "identifies" the ?
variables, you'd still have to walk the resulting data structure to extract them all -- you might just as well walk the original Datomic query.#2020-01-1723:15zaneYeah, I had imagined doing something clever with s/conform
and clojure.walk
.#2020-01-1723:15zaneBut I see what you mean.#2020-01-1722:57Alex Miller (Clojure team)I believe there are specs of Datomic's query syntax out there btw, don't have any links handy (but I agree that I probably wouldn't pick that as the first approach)#2020-01-1723:15zaneGood to know!#2020-01-1723:20zaneIs it fair to say that conform
is not really suited to these kinds of problems in the general case?#2020-01-1723:22seancorfieldconform
is not designed for parsing or general transformation, if that's what you're asking?#2020-01-1723:23seancorfieldconform
is intended to "tag" the resulting data with "how" the data conformed to the Spec, so downstream code can behave accordingly.#2020-01-1723:36zaneIsnāt that kind of tagging exactly what Iām after in this case? Iām not sure Iām understanding the details here.#2020-01-1723:56seancorfieldYou'd still have to walk the result to find the tagged values tho#2020-01-1800:03zaneTrue.#2020-01-1800:04zaneI guess Iām still struggling with which kinds of āparsingā are appropriate to do with spec and which arenāt.#2020-01-1800:05zaneDifferentiating between different shapes of data seems like something you could achieve with it via s/or
and s/conform
.#2020-01-1800:05zaneBut you could also do that kind of thing with match
, or with plain old Clojure.#2020-01-1723:24seancorfieldThat's not to say that some people don't (ab)use Spec to do some amount of parsing and transformation... cough ...but that's not what it was designed for: coercion is somewhat of a "side-effect" of conforming, you might say š#2020-01-1723:34zaneThat makes sense.#2020-01-1723:34zaneI find myself wanting something like Instaparse, but for EDN rather than strings, often.#2020-01-1814:45dharriganIs there an example of how I might use spec to ensure that a field is in OffsetDateTime format?#2020-01-1815:07Alex Miller (Clojure team)#(instance? OffsetDateTime %)
?#2020-01-1815:10dharriganoooh#2020-01-1815:10dharriganwill try that#2020-01-1815:20dharriganSorta gets me there, the field is a string, so I think I'll have to do some extra magic#2020-01-1815:33Alex Miller (Clojure team)probably need to use a parser then#2020-01-1815:34Alex Miller (Clojure team)which is part of the java time library#2020-01-1815:34dharrigankk#2020-01-1815:34dharriganthanks for the pointer š#2020-01-1913:01Philipp SiegmantelHello Everybody, what test runner/library do you use with clojure.spec.test.alpha/check? I found https://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29 for integration with clojure.test but the errors it prints aren't verry informative. Is there something better?#2020-01-1915:03dominicmClojure core uses something custom#2020-01-1916:07Philipp SiegmantelDo you have a link?#2020-01-1917:58respatializedhas anyone written an unofficial spec
for EDN? I'm looking for a programmatic way of distinguishing between Clojure forms and plain EDN forms.#2020-01-1918:45seancorfield@afoltzm This is the definitive word on EDN I believe https://github.com/edn-format/edn#2020-01-1918:53respatializedyeah, I figured I'd probably have to create one myself on the basis of the format's description that suits my purposes, which also serves as a useful exercise for improving my understanding of spec
itself.#2020-01-1921:57andy.fingerhutClojure core has an implementation of reading arbitrary Clojure forms, via clojure.core/read, and the EDN subset, via clojure.edn/read#2020-01-1921:58andy.fingerhutThose implementations in Clojure itself are written in Java. The tools.reader contrib library also has implementations written in Clojure.#2020-01-2101:14Vincent CantinIn spec2, is it possible to use s/conform
on a vector parsed via a sequence matcher like vcat
and then getting back a vector via s/unform
?#2020-01-2101:16Vincent CantinFrom the documentation, it does not look like it would work.#2020-01-2102:55Alex Miller (Clojure team)Yes, it works now#2020-01-2102:56Alex Miller (Clojure team)With s/catv#2020-01-2102:59Alex Miller (Clojure team)What documentation are you referring to?#2020-01-2113:05Vincent Cantin@alexmiller I had this impression after reading those 2 places:
ā¢ https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#scat which hints that vcat uses s/and-
,
ā¢ and https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new which does not say that s/unform
validates other preds in a s/and-
.#2020-01-2113:09Vincent CantinThe example about s/vcat
may potentially confuse people w.r.t. s/catv
.#2020-01-2113:24Alex Miller (Clojure team)Thx, Iāll update that #2020-01-2115:32Filipe Silvaheya#2020-01-2115:32Filipe Silvaare there any breaking changes one should be aware of between 0.1x and 0.2.x for spec?#2020-01-2115:33Filipe SilvaI'm updating a project's cljs 1.10.520
toĀ `1.10.597` , and my production build now fails when loading something spec related#2020-01-2115:33Filipe Silva| TypeError: Cannot read property 'prototype' of ...
V
$APP.$cljs$spec$alpha$t_cljs$0spec$0alpha62946$$.prototype.$cljs$spec$alpha$Spec$gen_STAR_$arity$4$ = $JSCompiler_unstubMethod$$(6, function($_$jscomp$256$$, $overrides$jscomp$11$$, $path$jscomp$62$$, $rmap$jscomp$11$$) {
return $APP.$cljs$core$truth_$$(this.$gfn$) ? this.$gfn$.$cljs$core$IFn$_invoke$arity$0$ ? this.$gfn$.$cljs$core$IFn$_invoke$arity$0$() : this.$gfn$.call(null) : $APP.$cljs$spec$alpha$re_gen$$(this.$re$, $overrides$jscomp$11$$, $path$jscomp$62$$, $rmap$jscomp$11$$, $APP.$cljs$spec$alpha$op_describe$$(this.$re$));
});
#2020-01-2115:34Filipe Silvadoesn't seem to affect the normal build though#2020-01-2116:49GHave folks ever used spec to define a transformation between namespaced, clojure maps to another format?
My use case is creating maps in clojure but transforming them to JSON, and I don't want for there to be coupling between my clojure key names (ie. namespaced keywords) and my JSON maps (which may even contain keys that aren't representable as clojure keywords)#2020-01-2121:17rapskalian@U3G51H7NY you can have a look at this little lib I wrote a while ago for a similar use case
https://github.com/cjsauer/disqualified#2020-01-2121:18rapskalianMight not be exactly what youāre needing but could serve as a possible reference #2020-01-2121:18rapskalianItās less than 50 lines of code#2020-01-2121:23Git's not exactly what I'm looking for right now (for instance, my un-namespace keys have conflicting names, although the full paths are distinct), but this is definitely useful! I've had to implement something very similar for another project, so will keep this in mind in the future š#2020-01-2116:53GI was hoping to have the resulting key name as part of the spec so as to keep all of the relevant information about that attribute in one place, but perhaps this is best done outside of spec?#2020-01-2116:55ghadiyou can store a lookup from qualifiedkey -> key in a sibling database, doesn't have to be in the spec registry#2020-01-2216:31kennyI really like how succinct spec-tools data-specs are. It takes 17 lines worth of specs for a map that has a key that has a collection of maps down to 3 lines. Will Spec2 be able to do this as well? The syntax described here https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys looks quite similar. The description makes it sound like that syntax will only be available for unqualified keys though.#2020-01-2216:32Alex Miller (Clojure team)Thatās correct#2020-01-2216:32kennyWhy not support qualified keys with that syntax?#2020-01-2216:35kenny(s/def ::error-code string?)
(s/def ::error-message string?)
(s/def ::id uuid?)
(s/def ::failed-request
(s/keys :req [::error-code ::error-message ::id]))
(s/def ::failed-requests
(s/coll-of ::failed-request))
(s/def ::result
(s/keys :req [::failed-requests]))
versus
(s/def ::result
{::failed-requests [{::error-code string?
::error-message string?
::id uuid?}]})
The latter seems so much easier to read.#2020-01-2216:36Alex Miller (Clojure team)https://clojure.org/about/spec#_global_namespaced_names_are_more_important#2020-01-2216:37kennySorry, I don't follow. The latter could easily expand to do exactly as that doc describes.#2020-01-2216:37kennyWell, I suppose the nested map wouldn't have a name.#2020-01-2216:37Alex Miller (Clojure team)the objective here is not just to validate but to build a library of reusable (ie named + registered) specifications#2020-01-2216:42kennySo Spec1 forces you to do that with unqualified keys. Why has that changed with Spec2?#2020-01-2216:43Alex Miller (Clojure team)not sure I understand the question#2020-01-2216:44kennyIf I'm understanding the syntax here https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys, you are no longer creating a reusable specification for the nested map.#2020-01-2216:46kennyWell, that I guess that applies for key & value in that ::order
spec. Nothing is reusable.#2020-01-2216:46Alex Miller (Clojure team)well, you are in the schema, but not in the sub levels. as unqualified names, they often are ambiguous#2020-01-2216:47Alex Miller (Clojure team)you still can do like you did in spec 1 (relate unqualified keys to qualified specs) but it's not constrained to the unqualified name match and it's optional#2020-01-2216:51kennyTo be clear, the issue with something like this
(s/def ::result
{::failed-requests [{::error-code string?
::error-message string?
::id uuid?}]})
Is that the nested map is not named?#2020-01-2216:52Alex Miller (Clojure team)you're inventing syntax here, it can't be just considered in isolation#2020-01-2216:53kennyFair enough. I mean like this
(s/def ::order
(s/schema {:purchaser string?
:due-date inst?
:line-items (s/coll-of (s/schema {:item-id pos-int?
:quantity nat-int?})
:kind vector?
:min-count 1
:gen-max 3)}))
but with every key qualified instead of unqualified.#2020-01-2216:53Alex Miller (Clojure team)this now occupies the "map" slot in the language of specs, which may have other uses#2020-01-2216:55Alex Miller (Clojure team)certainly, this goes against the grain of naming and registering the specifications. names have multiple utility in spec - they are used for spec reuse, but also in things like explain reporting, and I don't remember what else#2020-01-2216:56Alex Miller (Clojure team)none of this rules out the possibility of adding more support for concise definition of multiple specs in tandem#2020-01-2216:56Alex Miller (Clojure team)or even extension to support this in the future, but it's not our top concern#2020-01-2216:57Alex Miller (Clojure team)and if we were going to do so, it would be driven by a top level problem, not just "it's a thing we can do".#2020-01-2216:58kennyUnderstood.
I don't think it necessarily goes "against the grain of naming and registering".
(s/def ::order
(s/schema {::purchaser string?
::due-date inst?}))
That would register the ::purchaser
spec as string?
, ::due-date
as inst?
and ::order
as "s/keys".#2020-01-2216:38Filipe Silva@alexmiller heya, if you're around can I ping you about https://clojurians.slack.com/archives/C1B1BB2Q3/p1579620754050400?#2020-01-2216:39Alex Miller (Clojure team)what do you mean by 0.1.x and 0.2.x?#2020-01-2216:39Alex Miller (Clojure team)are you talking specifically about spec.alpha?#2020-01-2216:40Alex Miller (Clojure team)https://github.com/clojure/spec.alpha/blob/master/CHANGES.md#2020-01-2216:40Alex Miller (Clojure team)0.1.x is Clojure 1.9 era and 0.2.x is Clojure 1.10 era#2020-01-2216:41Alex Miller (Clojure team)I don't think there were any breaking changes between those#2020-01-2216:42Filipe Silvayes that is what I meant#2020-01-2216:43Filipe Silvatrying to update from cljs 1.10.520
toĀ `1.10.597` causes my app to have a runtime error for optimized builds on what seems to be something spec related, so I thought there might be a breaking change#2020-01-2216:43Alex Miller (Clojure team)what are 1.10.520 and 1.10.597 applicable to?#2020-01-2216:44Alex Miller (Clojure team)oh cljs, sorry that got lost#2020-01-2216:44Alex Miller (Clojure team)spec.alpha is CLJ only#2020-01-2216:44Alex Miller (Clojure team)the cljs spec impl is inside ClojureScript so the stuff above does not directly correlate#2020-01-2216:45Filipe Silvaah I see... so maybe I should ask about cljs.spec.alpha
in #clojurescript instead#2020-01-2216:47Alex Miller (Clojure team)yeah#2020-01-2216:47Alex Miller (Clojure team)I don't know anything on that#2020-01-2216:48Filipe Silvacoolio, thanks for the direction!#2020-01-2417:57lilactownfor direction/discussion w.r.t. cljs.spec.alpha, should I lobby that here or in #cljs-dev?#2020-01-2418:11Alex Miller (Clojure team)cljs-dev#2020-01-2417:58lilactownIām interested in exploring (and advocating for support for) the capability in CLJS to remove specs when building for deployment#2020-01-2418:05Matti UusitaloPut them in a separate source folder which isn't part of the production build?#2020-01-2418:09lilactownthey need to show up in the namespace dependency graph of the application in order to be included at dev time and be rebuilt when the implementations change#2020-01-2418:09lilactownI could manually write a namespace that includes all specs I use in my app, and add that as a preload. but that is tedious#2020-01-2612:11vemvHow about a my.spec/def
macro which expands to sth like when js/goog.asserts.ENABLE_ASSERTS (spec/def ...)
?#2020-01-2721:19favilais there any advice for speccing containers, e.g. atoms or delays?#2020-01-2816:38vemvSay you have a check!
function. I normally hook up the :validator to it, but only on *assert*
#2020-01-2817:05favilaI went with an fspec instead#2020-01-2817:05favilaMost ref types have validators, I forgot about those. you could use s/assert in there#2020-01-2817:06faviladoesnāt help with generation, but thereās manual intervention for generation all the time anyway#2020-01-2817:06faviladelays however donāt have validators#2020-01-2817:18vemv> doesnāt help with generation,
yeah it's a tradeoff :)
> delays however donāt have validators
yup, as you may know though a custom IDeref impl can be quite thin#2020-01-2721:19favilaIād like to say ātakes a delay that returns a thing satisfying some predicate when derefedā#2020-01-2721:42seancorfield@favila There's no way to do that. You can spec around the code that processes what's inside the container, but you can't spec the container itself.#2020-01-2721:44lilactownatoms do have validator functions that you can pass in on instantiation I think?#2020-01-2721:49Alex Miller (Clojure team)they do#2020-01-2721:49Alex Miller (Clojure team)but carefully consider the costs there#2020-01-2902:55agI need a spec for a map like {:a :b}
where :b
has to be a required key, but only when value of :a
is not nil
. Can someone help me with that?#2020-01-2903:27seancorfielduser=> (s/def ::ab (s/and (s/keys :opt-un [::a ::b]) #(or (nil? (:a %)) (contains? % :b))))
:user/ab
user=> (s/valid? ::ab {:x 1})
true
user=> (s/valid? ::ab {:a nil})
true
user=> (s/valid? ::ab {:a 1})
false
user=> (s/valid? ::ab {:a 1 :b 2})
true
@ag how about that?#2020-01-2903:28seancorfieldIf :a
should be required, use (s/keys :req-un [::a] :opt-un [::b])
I guess.#2020-01-2903:33agah, I was playing around, come up with something like this:
(s/def ::a string?)
(s/def ::b string?)
(s/def ::base-map (s/keys :req-un [::a ::b]))
(s/def
:foo/bar
(s/merge
::base-map
(s/and
(fn [m]
(if (= (m :a) "bobo-included")
(contains? m :bobo)
true
)))))
thereās probably better way of doing this
Basically now :foo/bar is a spec for a map that must have :a
and :b
, keys, but when value of :a
is equal to ābobo-includedā, it must have :bobo
, key, otherwise, :bobo
key is optional#2020-01-2903:37seancorfields/merge
is intended for two key specs -- I don't think you should use it for a key spec and a predicate.#2020-01-2903:37seancorfieldAnd it looks like you have s/and
with just a single predicate there?#2020-01-2903:38seancorfieldNow that you've restated the problem, it sounds like you want to look at multi-specs.#2020-01-2903:39agahā¦ yeah. let me try digging in there#2020-01-2903:39seancorfieldThose are intended to select different specs based on some function of a value, in your case, the value of the :a
key.#2020-01-2903:40agwell, yeah, I guess I didnāt exactly follow my own requirement, but the approach you showed me probably would work for me. Iām gonna refresh my memory on multi-specs anyway.#2020-01-2903:40agThank you Sean!#2020-01-2910:18lambdamHello,
I was playing a bit with spec 2 on an Advent of Code exercise and bumped into this case:
(def dam
{:firstname "Dam"
:age 35})
(s/def ::user (s/schema {:firstname string?
:lastname string?}))
(s/explain (s/select ::user [*])
dam)
;; => {:firstname "Dam", :age 35} - failed: (fn [m] (contains? m :lastname))
(s/explain (s/select {:firstname string?
:lastname string?}
[*])
dam)
;; => Success!
(s/explain (s/select [{:firstname string?
:lastname string?}]
[*])
dam)
;; => {:firstname "Dam", :age 35} - failed: (fn [m] (contains? m :lastname))
(s/explain (s/select [{:firstname string?}
{:lastname string?}]
[*])
dam)
;; => {:firstname "Dam", :age 35} - failed: (fn [m] (contains? m :lastname))
The second explain
is weird (considering https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#literal-schemas).
Is there a subtle detail that I'm missing? (ping @alexmiller)
Thanks#2020-01-2913:38Alex Miller (Clojure team)Dunno, probably a bug#2020-01-2910:37lambdamAlso, is there a specific reason not to instrument :ret
and :fn
in fdef
(even optionally) in spec 2?
I use https://github.com/jeaye/orchestra with spec 1 and find it very useful.#2020-01-2913:29Alex Miller (Clojure team)https://clojure.org/guides/faq#instrument_ret#2020-01-2914:45lambdamAh ok. Thanks.#2020-01-2911:33lambdamAlso
(s/valid?
(s/schema {:foo string?})
{:foo 1})
=> false
(s/assert
(s/schema {:foo string?})
{:foo 1})
=> {:foo 1}
... weird#2020-01-2915:22zcljI get an exception when using a schema referring a spec with an indirection to another spec. It works as expected if I just use the spec. Am I doing anything wrong here or is it a bug?
(s/def ::thing string?)
(s/def ::other-thing ::thing)
(gen/sample (s/gen ::other-thing)) ;; => works as expected
(s/def ::foo (s/schema [::thing]))
(s/def ::bar (s/schema [::other-thing]))
(gen/sample (s/gen ::foo)) ;; => works as expected
(gen/sample (s/gen ::bar)) ;; => Exception below
No implementation of method: :conform* of protocol:
#'clojure.alpha.spec.protocols/Spec found for class:
clojure.lang.Keyword
#2020-01-2915:34Alex Miller (Clojure team)this is a bug in spec 2, so won't work yet#2020-01-2920:13kennyDoes spec2 let you write a spec for a qualified keyword such that its definition changes depending on the context? For example, if I were to spec Datomic's :db/id
, it would only ever be a nat-int?
when part of the result is from a pull query. When transacting to Datomic, :db/id
could be a lookup ref (e.g., (s/tuple keyword? some?)
), a nat-int?
, or a string?
. One could spec :db/id
using s/or
but that means everything that takes a db id as an input needs to handle all of those cases, which does not always make sense.#2020-01-2920:52Alex Miller (Clojure team)in short, no#2020-01-2920:52Alex Miller (Clojure team)in long, the general recommendation is to try to give attributes specs that truly reflect the data#2020-01-2920:53Alex Miller (Clojure team)within certain contexts, you can add on additional specs that narrow the scope if needed#2020-01-2921:00kennyBy this do you mean you can add additional predicates to the :db/id
spec in different places?#2020-01-2921:05Alex Miller (Clojure team)yes, you could s/valid while s/and'ing in an additional narrower predicate for example#2020-01-2921:05kennyOh cool. Is there an example of what that looks like?#2020-01-2921:19Alex Miller (Clojure team)just what I said?#2020-01-2921:21kennyOh - that works if the :db/id
is taken as an argument itself. I was imagining a function that takes a map (perhaps nested) that has :db/id
s on the maps.#2020-01-2921:21kennyOr do you mean needing to write a predicate that walks that structure validating db/id against this new predicate?#2020-01-2921:39Alex Miller (Clojure team)you can still s/and a predicate to a map that checks a value in the map#2020-01-2921:39Alex Miller (Clojure team)it's also an option to just not spec :db/id#2020-02-0122:28kennyShould spec2's s/select
work for collections of collections? I would think this would return false.
(s2/def ::coll-next (s2/coll-of (s2/coll-of (s2/schema [::a]))))
(s2/def ::map (s2/schema [::coll-next]))
(s2/valid?
(s2/select ::map [::coll-next {::coll-next [::a]}])
{::coll-next [[]]})
=> true
#2020-02-0123:15seancorfieldGiven that ::coll-next
is a collection, not a schema, I'm not sure how that should work...?#2020-02-0220:21kennyHi all. I wrote a library that adds support for Spec2's schema & select to Spec1. My company is working on a new product and thus a new data model. Defining the data model using Spec1's :req
feels so wrong now that we see Spec2's direction. We intend to use this library to bridge the gap until Spec2 is out or official support for some sort of Spec1 schema/select is released. It's still very early but the few examples I've worked through all work as expected!
https://github.com/ComputeSoftware/spec1-select#2020-02-0400:09kenny... now with CLJS support š#2020-02-0422:56kennyWill Spec2 provide a way to select the required keys of a nested Spec based on a dispatch value? For example, say you have a map that has a key ::items
. The required keys for each map within the items list will vary based on the :type
key on each item map.
I could write add a top-level predicate to the ::items
spec via s/and
that would check the validity of each map in the items list against another defined spec. That would result in poor error messages though.#2020-02-0423:06Alex Miller (Clojure team)Tbd#2020-02-0423:11kennyGot it. You probably have a good idea of what it may look like already. In case it may be of any use, this is what I'm thinking we'll end up implementing internally for this problem.
(s/select
::report-input-data-schema
[::instances
{;; map of dispatch value to required keys
::instances {:default []
:instance.type/a [:instance.a/prop1]
:instance.type/b [:instance.b/prop1]}
;; required keys for all maps
:common [:instance/status
:instance/name]
;; dispatch fn
:dispatch :type}])
#2020-02-0506:45colinkahn@kenny couldn't you do this with a multi-spec defmethods returning select
specs?#2020-02-0515:55kennyNot when it's a nested collection.#2020-02-0516:02colinkahnAh right, agreed this would be great to have built in#2020-02-0516:08kennyDefinitely agree. Fortunately it's not that hard to build in yourself if it doesn't include it. It really does seem like something it should support by default though.#2020-02-0517:13colinkahnRight, I built a spec that did something like this for spec-1, basically āclampedā nested multi-specs. It was a bit unwieldy because the definitions had to be so long.#2020-02-0514:42Wilson Velezhi, when to use :req or :req-un? Iāve used :req-un always#2020-02-0515:44orestisIf you have maps with namespaced keywords, use :req#2020-02-0516:30Wilson Velez:+1:#2020-02-0601:06kennyI just spend a solid 2 hours trying to figure out how to make a schema -> select work in Spec1. Started diving deeper into the Spec2 code to land on this https://github.com/clojure/spec-alpha2/blob/495e5ac3238be002b4de72d1c48479f6bec06bb3/src/main/clojure/clojure/alpha/spec/impl.clj#L421. I think this makes sense. I'm curious if there is more reasoning behind it since it seems like there isn't a technical reason as to why you can't have a select within a schema.#2020-02-0601:15kennyI guess it's more of a fundamental thing -- a schema is always a set of optional attributes. If attributes become required, it cannot be a schema.#2020-02-0603:50Alex Miller (Clojure team)yes, that#2020-02-0616:57Ben HammondI'm working on generators for a spec like
(spec/keys :req-un [::interval/start
::interval/end])
but I want to create linkage between the start/end dates.
I've written a generator `
(defn sqlinterval-generator
to give me the interval boundaries that I want,
but what is the best way to poke them in to the overal keys
structure ?
In the past I've used a bind
and fmap
to overwrite the interval boundaries
AFTER the default generator has given me random ones;
seems a bit non-intentional though; is there a better way?#2020-02-0617:04Alex Miller (Clojure team)go the other way#2020-02-0617:05Alex Miller (Clojure team)I guess you kind of are#2020-02-0617:05Alex Miller (Clojure team)I don't think what you're doing is bad necessarily#2020-02-0617:05Alex Miller (Clojure team)and it might be the easiest way in this situation#2020-02-0617:06Ben Hammondso I have this to daisy-chain the generators
(defn generator-decorate-sqlinterval
"overwrites the gen-in values with values from the interval generator"
[gen-in start-key end-key interval-generator]
(test.gen/bind gen-in
(fn [o] (test.gen/fmap
(fn [[start end]] (assoc o start-key start
end-key end))
interval-generator))))
#2020-02-0617:07Ben HammondI guess I'm asking if there is something like (test.gen/tuple
but that will give me a map instead of a vector#2020-02-0617:08Ben Hammondnah that wouldn't be any easier would it?#2020-02-0617:08Ben HammondI'll keep hacking away at it then#2020-02-0617:08Ben Hammondthanks#2020-02-0619:09waffletowerDoes clojure.spec.alpha support defining a spec for a map where keys are not known in advance, but a spec exists for their values?#2020-02-0619:14seancorfield@waffletower For qualified keys, yes, if I'm understanding you correctly. Not for unqualified keys.#2020-02-0619:15seancorfielduser=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def :my/key int?)
:my/key
user=> (s/def ::bag (s/keys))
:user/bag
user=> (s/valid? ::bag {})
true
user=> (s/valid? ::bag {:my/key "a"})
false
user=> (s/valid? ::bag {:my/key 1})
true
user=>
#2020-02-0619:16seancorfieldSpec checks any (qualified) keys' values against existing specs, even if s/keys
doesn't list them. But if you generate/exercise ::bag
, you will only get empty maps.#2020-02-0619:27waffletowerThanks Sean, I need something like a wildcard key, such that every key in a map would validate against a specific spec.#2020-02-0619:33waffletower{:unknown 1
:mystery 4
:unspecified 7} etc.
#2020-02-0619:34waffletowerAnd somehow link any key to be validated against (s/def ::wildcard-key int?)
#2020-02-0619:35seancorfield(s/map-of keyword? ::wildcard-key)
#2020-02-0619:35seancorfield(instead of s/keys
)#2020-02-0619:36waffletowerlet me test that out, thanks#2020-02-0619:43waffletower(s/def ::wildcard-key int?)
(s/valid?
(s/map-of keyword? ::wildcard-key)
{:unknown 1
:mystery 4
:unspecified 7})
true
#2020-02-0619:43waffletowernice, many thanks!#2020-02-0715:47achikinHow do I check for exact value?#2020-02-0715:47Alex Miller (Clojure team)#{:foo}
is a valid spec#2020-02-0715:47Alex Miller (Clojure team)a set containing one value#2020-02-0715:48achikinThank you!#2020-02-0715:48achikinAs far as I can understand if itās a boolean I can do (s/def :mykey true?)
right?#2020-02-0716:03Alex Miller (Clojure team)prob better to use boolean?
#2020-02-0716:15denikI'm trying to get the spec (or any spec) from cljs.core.specs.alpha
but
(ns my-ns
(:require [clojure.spec.alpha :as s]
[cljs.core.specs.alpha :as cljs-specs]))
(s/form ::cljs-specs/local-name)
throws
Error: Unable to resolve spec: :cljs.core.specs.alpha/local-name
Why this is happening? the spec is defined: https://github.com/clojure/clojurescript/blob/ef32778989f7ba2311a1e8a5d99c30e6805f5719/src/main/cljs/cljs/core/specs/alpha.cljc#L16#2020-02-1006:29jrwdunhamI want a spec that accepts nilable values but does not generate them. So far I'm going with a macro like the following. Is there a better way to do this? Is wanting to do this an indicator that I'm making a bad design decision? The motivation is that I'm working with a db schema that allows NULL but my validation on data coming from users will prevent incoming nil values. I want my generators to simulate user-supplied data going into the DB and not data coming out of the existing DB.#2020-02-1013:44Alex Miller (Clojure team)seems ok to me#2020-02-1006:29jrwdunham(defmacro def-spec-nilable-non-nilable-generator
[kw spec]
`(s/def ~kw
(s/with-gen
(s/nilable ~spec)
(constantly (s/gen ~spec)))))
#2020-02-1012:54thomDoes anybody know of anything resembling linear temporal logic (or something weaker but still useful for describing sequences) for spec?#2020-02-1021:20andy.fingerhutI do not know if this is anywhere near what you are looking for, but a Google search for the terms "clojure linear temporal logic" turned up this project: https://github.com/esb-lwb/lwb#2020-02-1021:21andy.fingerhutSpec I am sure has nothing built in for this, except the general purpose escape hatch of letting you use arbitrary functions you write that return true/false as specs -- which is a huge enabler for anything you can write code for.#2020-02-1021:21andy.fingerhutIt won't necessarily help you generate random instances satisfying that function -- that is separate work you would need to do if you wanted that capability.#2020-02-1223:25telekidInstrumentation / generation question. In the following gist, an outer function calls an inner function twice with the same value (`i`). I want to write an fdef
for the outer function, using a :fn
predicate to assert that the return from the two calls to the inner fn are identical. However, when I try to stub the call to inner
with instrumentation, the generated values returned by inner
ās stub are (sensibly) different.
Is there any way to express on the fdef
of inner
that the generated values should be some known function of the input?
gist here: https://gist.github.com/telekid/01b0aed73511167c337794bc286908cf#2020-02-1304:50colinkahnWould using :replace
instead of :stub
work in this case?#2020-02-1320:45telekidOh, interesting! Iāll look into that, cool idea#2020-02-1313:37cddrHey folks,
Looking for a bit of advice about adding specs to an open source project. We have a "command syntax" where commands are basically vectors. A few examples are provided below but essentially each command's first item is an op-code and subsequent items configure the operation in an op-specific way....
[:do some-side-effect-fn]
[:write! :foo {:id 1 :payload "yolo"} optional-args]
[:watch some-pred optional-args]
Seems like there's a few ways this could be spec'd and I was just wondering if anyone had any thoughts about how I should structure it. There are some additional commands not shown here but not more than 10 overall and I don't anticipate too many more being added in the future.#2020-02-1313:48cddrAh I've just seen how crux does it.
https://github.com/juxt/crux/blob/b7fdb38357adb85627c3be5710d0aff26a796f19/crux-core/src/crux/tx/event.clj
I think our commands follow the same sort of pattern as their transactions so can probably just use that. I didn't realize that multi-spec
could be used with non-map shaped structures.#2020-02-1317:41colinkahnIf you look in the spec 2 repo currently there is still the keys
spec, will that stay? If so, with schemas/select is there a suggested limited set of usecases for keys
now?#2020-02-1317:59seancorfield@colinkahn I have seen a couple of mentions from @alexmiller indicating that keys
may not stay in the final Spec 2 version -- but Spec 2 is still very much alpha and in flux. Also the fdef
stuff is likely getting a substantial overhaul.#2020-02-1318:01seancorfieldI spent quite a bit of time tracking Spec 2 last year and had a branch of our system at work that passed all tests using Spec 2 instead of Spec 1 -- but still using keys
and there were quite a lot of changes just to get to that point (and a number of bugs and workarounds I had to deal with). I gave up in early October and decided that we wouldn't attempt a direct migration from Spec 1 to Spec 2: instead we'll adopt Spec 2 for new code and perhaps incrementally replace our Spec 1 usage over time.#2020-02-1318:02seancorfieldOne thing that needs to be figured out for a mixed Spec 1/2 codebase is how the spec systems interact, because Clojure itself uses Spec 1 (1.9, 1.10) and so you can't (easily) mix core specs with your own Spec 2 stuff right now.#2020-02-1318:08colinkahnThanks for the insight @seancorfield. In your experience do you think your refactor would have been easier if you were already using https://github.com/ComputeSoftware/spec1-select for you spec 1 code? Iām looking to port some of my code over to it.
Regarding keys
I was curious if the (and)
and (or)
syntax used within :req/opt(-un)
would exist in some form in spec 2.#2020-02-1319:28Alex Miller (Clojure team)Re and/or - no, probably not#2020-02-1319:29Alex Miller (Clojure team)Schemas support unqualified keys in spec 2#2020-02-1406:41colinkahnThanks!#2020-02-1318:09seancorfieldOur use of Spec dates back to early alphas of Clojure 1.9 so that library is moot for us.#2020-02-1318:10colinkahnRight, not saying for your specific case but Iām evaluating using it to make our transition to spec 2 eventually smoother#2020-02-1318:11seancorfieldGiven how much Spec 2 is in flux, I don't know that such a library would necessarily be a useful starting point. Most of the work for us in attempting to get our Spec 1 codebase running on Spec 2 was around the much stricter delineation of symbolic specs vs spec objects etc.#2020-02-1318:11seancorfieldWe had a lot of Spec 1 specs that simply weren't legal in Spec 2 and we had to rewrite them to use defop
and various other constructs.#2020-02-1318:13seancorfieldRemember: we stayed with keys
when we attempted that migration -- it wasn't about switching to schema
/`select`... that was considered "future work".#2020-02-1318:18colinkahnGood to know, I havenāt dived into the defop
stuff in spec 2 yet so didnāt consider that. Besides future compatability I definitely feel the pain points that schema/select are meant to solve, which is why Iām seeing that as a valuable first step.#2020-02-1319:47seancorfieldSure, they are a big improvement on Spec 1's keys
-- but I would be wary of relying on a 3rd party lib that tries to backport Spec 2 features onto Spec 1 when a) Spec 2 is still changing so much and b) Spec 2 already has stricter rules about "what is a spec" than Spec 1.#2020-02-1319:48seancorfieldI don't consider a lib like that to be a useful "future proofing" option...#2020-02-1319:50seancorfield(it's an interesting exercise, to be sure, and if folks want some of what Spec 2 offers while staying on Spec 1 for an unspecified future time period then maybe it's of some use -- but I really doubt it will make the "migration" from Spec 1 to Spec 2 any easier)#2020-02-1423:21jasonjcknA way to attach meta data to a spec or something similar enough ?
e.g.
(s/def ^:property1 ::myspec ...)
#2020-02-1423:30seancorfield@jasonjckn You can't add metadata to a keyword.#2020-02-1423:34jasonjcknyah I'm aware; thanks for the reply#2020-02-1423:35jasonjcknhttps://github.com/mpenet/spex is a 3rd party lib that offers meta data ; but unlikely to be maintained i'd guess#2020-02-1511:21mpenetYou'd be surprised ;)#2020-02-1511:22mpenetIn any case, it's just an atom and a couple of functions, easy to make a metadata registry of your own until core implements one#2020-02-1507:45ikitommiHere issue related to metadata: https://clojure.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FCLJ-1965#issue/CLJ-1965#2020-02-1507:46ikitommiThere is also spec-tools, which has a wrapper record for specs, which can hold extra (meta-)data.#2020-02-1507:46ikitommihttps://cljdoc.org/d/metosin/spec-tools/0.10.1/doc/spec-records#2020-02-1520:08EddieDoes anybody know of a java-time LocalDate
generator? My search came up empty.. I am fine with implementing my own, but it seems like something that probably exists.#2020-02-1521:18dominicmI used a few number generators.#2020-02-1521:18dominicmUsing gen/let#2020-02-1718:24EddieThanks, thats what I ended up doing. š#2020-02-1608:46abdullahibraHi everyone,#2020-02-1608:47abdullahibrais there any simple example for howto modify the error message produced by clojure.spec to be more readable ?#2020-02-1618:22seancorfield@abdullahibra You could look at Expound and Phrase, and there was a new one announced recently (which I think extends Phrase).#2020-02-1716:35ikitommiIām trying to understand best practices on using spec2 schema, select and and qualified-keys in general. Is this idiomatic use of Spec2? https://gist.github.com/ikitommi/87f9b9136167c6be2e4b9f315a212046#2020-02-1717:03ikitommioh, the closed spec donāt even work there. Can one close selects (or subslects)?#2020-02-1717:04ikitommiclosing the schema seems to work#2020-02-1717:04ikitommiā¦ but canāt make the one field optional just with schema.#2020-02-1717:12Alex Miller (Clojure team)you can only close schemas, but I think there are some impl gaps in properly validating use of a select over a closed schema#2020-02-1717:12Alex Miller (Clojure team)but otherwise, sure? I don't consider any of the schema/select stuff done, so hard to suggest "best" practices :)#2020-02-1717:14ikitommithanks. would be great to able to say, āclose all schemas/selectsā in the future.#2020-02-1717:15Alex Miller (Clojure team)that's something we've discussed#2020-02-1717:15Alex Miller (Clojure team)so tbd#2020-02-1717:15ikitommi:+1:#2020-02-1805:19bmaddyIs it possible to parse streams or lazy seqs with spec? Specifically, I'm interested in a stream where trying to consume too much data would cause it to hang forever.#2020-02-1805:21bmaddyTo clarify that a little, there would be a number at the beginning that tells me how many more numbers to consume.#2020-02-1805:21seancorfieldSpec is not designed to be used for parsing. I suspect you already know that but it's worth repeating.#2020-02-1805:24bmaddyHuh, I guess I'd forgotten that. So definitely worth repeating! Thanks!#2020-02-1805:26seancorfield(you can abuse Spec for parsing but it will often be suboptimal... at best)#2020-02-1808:44jeroenvandijk@seancorfield I understand that Spec is not meant for high performance parsing. I do find it useful and use it sometimes to parse datastructures. Do you know of a nice alternative that is performant?#2020-02-1808:58delaguardohttps://github.com/youngnh/parsatron
this library could be a good starting point
but it was designed to parse strings instead of data so might need some tweaks#2020-02-1809:02jeroenvandijkThat's useful as well as alternative to instaparse. Thank you#2020-02-1809:18delaguardoinstaparse can not be turned into the parser of data structures unfortunately. Only strings are allowed and this is deeply hidden inside of implementation details#2020-02-1809:19jeroenvandijkYeah I might have combined instaparse with clojure.spec. Not sure actually#2020-02-1809:39pithylesshttps://github.com/metosin/malli and https://github.com/noprompt/meander come to mind. They attack different problems, but are both useful when working in the domain of "I need to interpret some data structures".#2020-02-1810:59mmeixI just read this article: https://juxt.pro/blog/posts/parsing-with-clojure-spec.html (a small parser for ical-entries) and watched this video: https://www.youtube.com/watch?v=xLB8Xgt8lpA (a simple hiccup-parser). These seemed (to me, as a spec beginner) reasonable applications of spec - maybe, because, they are quite simple? Would there be a problem for using spec at this scale?#2020-02-1811:54mmeixAdditional question: is it considered idiomatic to dispatch a multimethod on the result of a s/conform? Are there drawbacks?#2020-02-1812:20jeroenvandijkI have done that. Cannot think of real drawbacks expect for that if your spec changes you have to update the multimethods#2020-02-1812:20jeroenvandijk(Changing) dispatch functions of multimethods are a bit of a hassle in a repl environment as it is hard to undefine (for good reasons). But to make this work in the repl you need to use some tricks#2020-02-1813:39mmeixok, thanks!#2020-02-1814:01dergutemoritzTemporarily placing (def my-multimethod nil)
in front of the defmulti
and then recompiling the namespace is an easy way to do it!#2020-02-1815:41jeroenvandijk@U06GVE6NR This is true, but you have to reload all the namespace or the method is not extended the way you expect. My trick is to define a dispatch function and define the method as (defmulti my-method #'dispatch-my-method)
. This has been the most reliable way for me when working in the repl for a long time#2020-02-1815:41dergutemoritzAh right, if you have method implementations in other namespaces, then this trick won't suffice, good point š#2020-02-1815:41dergutemoritzGood idea with the var#2020-02-2018:21rickmoynihanhmmā¦
Iām wanting to spec some data that is coming from a 3rd party library, the Ataraxy router.
It always uses the namespaced keyword :ataraxy/result
to contain the parsed parameters for the route; however because ataraxy is configured on a route by route basis; the exact data structure differs, yet the ring request map always contains the same namespaced keyword.
It strikes me that this is an unfortunate mistake; in that the same global keyword is being used to name different data.
What is the best way to spec such a thing?
Iām thinking a multispec is really the only option, that could dispatch on the first
argument. :ataraxy/result
is a vector of the form [:route/id ,,,route-params,,,]
#2020-02-2018:22rickmoynihanor I suppose I could coerce it into something else first š#2020-02-2018:26rickmoynihanactually thinking itās better to just spec the functions after it, that donāt depend on the :ataraxy/result
key.#2020-02-2018:45Alex Miller (Clojure team)you could just do something like (s/cat :route-id ::route-id :params (s/* any))#2020-02-2018:45Alex Miller (Clojure team)but it depends a lot what route-params is#2020-02-2018:45Alex Miller (Clojure team)if it's kwargs, then keys* would work great#2020-02-2021:47jasonjcknmy data looks like {::type "foobar" ::a ... ::b ... ::c ...} I want different s/keys validation based on the value of ::type
whats the feature i'm looking for#2020-02-2021:47jasonjcknit's like ~ conditional spec validation polymorphic to field value#2020-02-2021:47jasonjckni guess some sort of mulitmethod on the spec#2020-02-2021:48jasonjcknreading about multi-spec now, this looks like it#2020-02-2021:57jasonjcknfor the :default case#2020-02-2021:57jasonjcknwhat would be a canonical way to fail spec validation#2020-02-2021:57jasonjckndefmethod tagmm :default
#2020-02-2021:57jasonjcknhttps://clojuredocs.org/clojure.spec.alpha/multi-spec#2020-02-2021:58Alex Miller (Clojure team)(s/keys*) would automatically handle all registered specs for the tail of that#2020-02-2021:58jasonjckni want default case to fail#2020-02-2021:58jasonjcknvalidation#2020-02-2021:59Alex Miller (Clojure team)you can definitely use multi-spec for this if that makes sense#2020-02-2021:59jasonjcknwhat is a canonical way to implement :default#2020-02-2021:59jasonjcknbesides a conformer that always throws exceptions#2020-02-2021:59Alex Miller (Clojure team)I don't understand enough what you're doing#2020-02-2021:59Alex Miller (Clojure team)what does :default mean?#2020-02-2022:00jasonjckn(s/def ::tag #{:a :b :c :d})
(s/def ::example-key keyword?)
(s/def ::different-key keyword?)
(defmulti tagmm :tag)
(defmethod tagmm :a [_] (s/keys :req-un [::tag ::example-key]))
(defmethod tagmm :default [_] (s/keys :req-un [::tag ::different-key]))
(s/def ::example (s/multi-spec tagmm :tag))
(gen/sample (s/gen ::example))
here's the example#2020-02-2022:00jasonjckncan I do :default ::s/invalid ?#2020-02-2022:00Alex Miller (Clojure team)oh, you mean multi-method default#2020-02-2022:00jasonjckn(defmethod tagmm :default [_] ::s/invalid)
will test that out#2020-02-2022:00Alex Miller (Clojure team)the multimethod returns specs so you need to return a spec that always fails#2020-02-2022:01Alex Miller (Clojure team)so probably more (constantly false) would work#2020-02-2022:01jasonjcknok great#2020-02-2022:02Alex Miller (Clojure team)or #{}#2020-02-2022:02jasonjcknthat seems more canonical#2020-02-2022:02Alex Miller (Clojure team)a set of no values :)#2020-02-2022:02jasonjcknparticular in the spec explanations#2020-02-2022:02jasonjcknit'd be nice to get something that made sense#2020-02-2023:45vemva q out of curiosity more than anything else:
(spec/valid? (spec/coll-of char? :min-count 1 :max-count 20) "abc")
doesn't work (and a straightforward impl would underperform anyway?)
are there spec-related libraries offering a coll-of
-like API that efficiently works on strings?
I just like coll-of
's API, in that it doesn't resemble regex#2020-02-2100:08seancorfield@vemv The standard answer is that Spec isn't designed for parsing strings š#2020-02-2100:09seancorfieldMaybe Instaparse?#2020-02-2100:28vemvI don't want to parse them (e.g. do sth useful with them) - just validate them#2020-02-2421:53gariepyalexWhen conforming a sequence spec, can we guaranty key order? For instance:
(s/def ::a neg-int?)
(s/def ::b zero?)
(s/def ::c pos-int?)
(s/def ::sequence (s/cat :a ::a, :(s/? ::b), :c ::c))
I would like to use conform
to know if the second element of the sequence is :b
or :c
(labels given in cat
). conform
returns a clojure.lang.PersistentHashMap
, which does not key the keys in the same order as in the sequence.#2020-02-2422:49Alex Miller (Clojure team)s/tuple is a better match#2020-02-2421:57seancorfields/conform
just tells you how it matched. If you call s/valid?
and get true
then the input sequence is valid as is.#2020-02-2421:59gariepyalexI would have liked to use conform
to parse the sequence and end up with {:a -2, :c 10}
#2020-02-2422:02seancorfieldSpec isn't really a "parser" but if you know your elements should be in a given order, you can easily manipulate the data you get from s/conform
to produce a more appropriate structure.#2020-02-2422:03seancorfield(you know you're going to get :a
first and :c
last so the only question is (contains? conformed-data :b)
to see if a zero is present...#2020-02-2422:04gariepyalexok thanks for your help!#2020-02-2514:12Filipe Silvaheya#2020-02-2514:12Filipe Silvais it possible to rename a key with s/keys
?#2020-02-2514:13Filipe SilvaI'd like to do something like this
(s/def ::msg (s/and string? #(< 20 (count %))))
;; TODO: declaring a spec for a limited :db/id doesn't sound right. Figure out a better way.
(s/def :db/id (into #{} (range 1 10)))
(s/def ::map-datom (s/keys :req [:db/id ::msg]))
(s/def ::simple-tx (s/coll-of ::map-datom :kind vector? :min-count 0 :max-count 3))
#2020-02-2514:13Filipe Silvabut instead of (s/def :db/id
, I'd like to do (s/def ::id
#2020-02-2514:14Filipe Silvaand then on s/keys
say that ::id
is required but should show up as :db/id
#2020-02-2514:15Filipe SilvaI'd like to do this because I don't want to be declaring the spec for the real :db/id
, but would like to have an alternative version of it for testing#2020-02-2514:27Alex Miller (Clojure team)in short, no - the whole idea with spec is that attributes are primary and have meaning and maps are just aggregations of attributes#2020-02-2514:28Alex Miller (Clojure team)if you want a broader definition of ::db/id, then your spec should reflect that#2020-02-2514:32Filipe Silvaok, thank you for explaining it to me#2020-02-2514:32Filipe Silvaif I want to just generate test data that is a subset of all possible data, should I be using something else?#2020-02-2514:34rickmoynihan@filipematossilva: It sounds like you might be complecting generating data with specing. If in your tests you want to generate known idās that are a subset of valid idās, you can probably just pass a generator for that set where you need it.#2020-02-2514:35Filipe Silvayes, I think I am#2020-02-2514:35Filipe Silvabut s/keys
does not support inline value specs (by design, according to the docs), so I don't think I can do that#2020-02-2514:36Filipe Silva(unless, of course, I am misunderstanding what you propose)#2020-02-2514:36Filipe Silvaan alternative is to generate the test data with exercise, and then map over the results to change the key#2020-02-2514:36Filipe Silvawhich sounds fair enough, but I wanted to inquire first#2020-02-2514:36Alex Miller (Clojure team)you can pass generator overrides when doing most of the functions that generate in spec, but you still have to generate data that conforms to the spec#2020-02-2514:39Filipe Silvaah, then this was just something I did not know existed#2020-02-2514:39Filipe Silvais it https://clojuredocs.org/clojure.spec.alpha/gen?#2020-02-2514:39Filipe Silvaor rather https://clojuredocs.org/clojure.spec.alpha/with-gen, as that seems to have examples#2020-02-2514:40rickmoynihanTotally! :-)
When I first started with spec I made the mistake of over speccing to get spec to try and generate test data of the right shapeā¦ but now prefer the alternative of specing a little more broadly but using :gen
ās to generate the more precise data I want.
Obviously itās all trade offs.
Anyway I was just suspecting Filipe might be falling into this same trap.#2020-02-2514:42Filipe Silvayes, I was š#2020-02-2514:45Filipe Silvathank you#2020-02-2514:45Filipe SilvaI'll try to fiddle with the :gen
option on s/keys and see where that takes me#2020-02-2515:00rickmoynihanš hope it helps#2020-02-2515:11Filipe Silva@rickmoynihan I'm trying to use the :gen
key on s/keys
, but I don't quite follow how I can use it to replace a single def#2020-02-2515:11Filipe Silvais that possible? or does :gen
need to override how everything is generated#2020-02-2515:11Filipe Silvaif you have an example of using it with s/keys
it would be super useful#2020-02-2515:23Filipe Silvaoh, I think this is the idea?#2020-02-2515:23Filipe Silva(s/def :db/id pos-int?)
(s/def ::small-pos-int? (into #{} (range 1 10)))
(s/def ::map-datom (s/keys :req [:db/id]))
(def small-pos-int-gen (s/gen ::map-datom {:db/id #(s/gen ::small-pos-int?)}))
(s/def ::small-id-map-datom (s/keys :req [:db/id]
:gen (fn [] small-pos-int-gen)))
(s/exercise ::small-id-map-datom 10)
;; => ([{:db/id 5} {:db/id 5}] [{:db/id 3} {:db/id 3}] [{:db/id 1} {:db/id 1}] [{:db/id 1} {:db/id 1}] [{:db/id 7} {:db/id 7}] [{:db/id 8} {:db/id 8}] [{:db/id 6} {:db/id 6}] [{:db/id 8} {:db/id 8}] [{:db/id 6} {:db/id 6}] [{:db/id 2} {:db/id 2}])
#2020-02-2515:23Filipe Silvamaking the real spec, then making another spec that has a generator that takes the real spec generator and adds an override#2020-02-2515:26rickmoynihanThatās not quite what I was suggestingā¦#2020-02-2515:31rickmoynihan(s/def ::foo integer?)
(s/def ::foo-map (s/keys :req-un [::foo]))
;; The general generator (generates negative integers too)
(s/exercise ::foo-map);; => ([{:foo -1} {:foo -1}] [{:foo 0} {:foo 0}] [{:foo 0} {:foo 0}] [{:foo 0} {:foo 0}] [{:foo 3} {:foo 3}] [{:foo -2} {:foo -2}] [{:foo -7} {:foo -7}] [{:foo -37} {:foo -37}] [{:foo -2} {:foo -2}] [{:foo -3} {:foo -3}])
;; An overriden tighter generator generating just 1 and 2 values for :football:
(s/exercise ::foo-map 10 {::foo-map (fn [] (s/gen #{{:foo 1} {:foo 2}}))});; => ([{:foo 1} {:foo 1}] [{:foo 2} {:foo 2}] [{:foo 1} {:foo 1}] [{:foo 2} {:foo 2}] [{:foo 2} {:foo 2}] [{:foo 1} {:foo 1}] [{:foo 1} {:foo 1}] [{:foo 2} {:foo 2}] [{:foo 2} {:foo 2}] [{:foo 2} {:foo 2}])
#2020-02-2515:33rickmoynihan@filipematossilva: You can override on s/exercise
, or st/check
as well, so you can tailor your generation there rather than in the specs if you want.#2020-02-2515:33Filipe Silvaah on exercise itself#2020-02-2515:33Filipe SilvaI see now#2020-02-2515:33Filipe Silva(s/def :db/id pos-int?)
(s/def ::small-pos-int? (into #{} (range 1 10)))
(s/exercise :db/id 10 {:db/id #(s/gen ::small-pos-int?)})
#2020-02-2515:34Filipe SilvaI understand now#2020-02-2515:34Filipe Silvathank you for taking the time to make the examples for me#2020-02-2515:34rickmoynihanš#2020-02-2515:34Filipe Silvait was very helpful#2020-02-2515:36rickmoynihanessentially s/exercise
/ st/check
are the testing entry points to spec; so you can split the concerns of specing and overriding generation/testing at those entry pointsā¦ whilst s/conform
s/valid?
and s/explain-xxx
are the checking side of spec ā and consequently donāt support generators at all (because theyāre a different concern)#2020-02-2515:36rickmoynihanat least thatās how I think of it#2020-02-2515:37rickmoynihanThere are probably more nuanced explanations š#2020-02-2515:39Filipe Silvathat sounds much more sensible than I what was attempting#2020-02-2515:39rickmoynihanits ok been there, done it, got burned! š#2020-02-2604:48yuhanI was playing around with spec2 and noticed that it doesn't allow for using local bindings in definitions#2020-02-2604:48yuhanie. this worked in spec.alpha:
(let [max-v 10]
(s/def ::foo (s/int-in 1 (inc max-v)))
(s/valid? ::foo 20))
#2020-02-2604:48yuhanbut not in spec2, where it would throw an error "Unable to resolve symbol max-v"#2020-02-2604:50yuhanIs this a deliberate design decision or was it ever officially supported in spec.alpha in the first place?#2020-02-2605:41seancorfield@qythium Spec 2 draws a much sharper line between symbolic specs and spec objects. Spec 2 has new facilities for creating specs programmatically that allow you to create such a spec, but in a different way.#2020-02-2605:46seancorfieldYou can read about the design decisions here https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha @qythium#2020-02-2605:59yuhanhmm.. that's slowly starting to make sense#2020-02-2605:59yuhanSo I would do this instead?
(let [max-v 10]
(s2/register ::foo
(s2/resolve-spec `(s2/int-in 1 (inc ~max-v))))
(s2/valid? ::foo 20))
#2020-02-2606:05seancorfieldYup, that should work.#2020-02-2606:08yuhanaha, so it's roughly like (s2/def kwd expr)
is macro sugar for (s2/register kwd (s2/resolve-spec
expr))`#2020-02-2606:11seancorfieldYes, if you look at the source https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/alpha/spec.clj you'll see def
calls register
and several macros use resolve-spec
to produce spec objects from symbolic forms.#2020-02-2606:12seancorfieldI don't have any code handy but defop
is also handy for building new spec predicates.#2020-02-2606:13yuhanThat actually makes a lot more sense than the nested macros in the original Spec š#2020-02-2606:14yuhanwhich I recall macroexpanding to trace the logic and eventually just treated as magic#2020-02-2606:14yuhanthanks!#2020-02-2618:13benIf I have written a s/fdef
for a function, is there an easy way to generate property tests for that function (as part of a test suite), without having to manually (re)write all the generators?#2020-02-2618:23dchelimskyHave you looked at https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check-fn?#2020-02-2618:28benI saw this, but my understanding is that is runs the tests (like test.check/quick-check
) rather than creating a test (`deftest`). Have I misunderstood?#2020-02-2619:33rapskalian@UFUDSN6BE maybe youāre looking for something like defspec
https://github.com/clojure/test.check/blob/master/doc/intro.md#clojuretest-integration#2020-02-2620:26kennyAlthough some don't recommend it, we do run spec's "check" in deftests using an "internal" (but public) library: https://github.com/Provisdom/test/blob/a61266ab281580af88fd394e9896cb85332cc5d7/src/provisdom/test/core.clj#L330
This will let you write something like this
(deftest my-fn-gen-test
(is (spec-check `user/my-fn)))
which will run & report st/check errors.#2020-02-2709:59benThanks @U6GFE9HS7 - I am aware of test.check, but as I understand it, I will have to write my own generators again, rather than using fdefās :args to check the function#2020-02-2710:00benThanks @U083D6HK9 - this looks like what I was imagining. Why do some not recommend it?#2020-02-2716:01kennyI think the argument is that they should run at a different time than regular unit tests because they are different. I don't have a problem with that -- run them in whatever way fits your company's workflow. These sort of tests are invaluable though. You should certainly run them before deployment. Having one test runner (https://github.com/lambdaisland/kaocha for us) is quite nice.
We have a large codebase with thousands of tests. Running gen tests with our unit tests has not caused us any substantial pain. They typically take 1s or less to run. The more complex functions with custom gens can take longer. The results of a gen test do not fit well into the expected/actual framework clojure.test uses though. That is certainly another downside.#2020-02-2718:57robertfw@UFUDSN6BE re: generating args. I use the following code to generate args from an fdef:
(defn arg-gen
"Creates an argument generator for a given function symbol"
[sym]
(-> sym s/get-spec :args s/gen))
#2020-02-2618:14benLike stest/check
but as part of my tests#2020-02-2711:40rickmoynihan@ben Kaocha has support for suites of fdefs#2020-02-2800:41robertfwI'm trying to figure out how the overrides map for (s/gen)
works, namely for overriding a generator that's down the tree a bit. That is, I have a top level spec ::data
which has a key ::things
which is a collection of ::thing
, which has a key ::properties
. I'm generating the top level - ::data
- and wanting to override both ::things
so I get only 1 thing, and ::properties
so I get an empty list. using (s/gen ::data {::things #(s/gen (s/coll-of ::thing :min-count 1 :max-count 1))})
, I'm able to force ::things
to only have 1 ::thing
, but I haven't figured out the incantation needed to have ::properties
be empty.#2020-02-2800:50robertfwThe docstring talks about providing a vector of keyword paths, but I haven't been able to find any more detail on what those paths should look like. I've been trying various combinations#2020-02-2801:29robertfwI suspect the issue is in using (s/coll-of)
- for now I'm using an spec.gen/fmap
call to remove any extra data I don't want, but would prefer to be able to just generate only the data needed#2020-02-2801:32seancorfieldYour s/gen
override for ::things
could have a third argument which is the override for ::properties
#2020-02-2801:32seancorfieldA bit like this
(s/exercise ::things 10 {::things #(s/gen (s/coll-of ::thing :min-count 1 :max-count 1) {::properties (fn [] (s/gen (s/coll-of string? :min-count 0 :max-count 0)))})})
#2020-02-2801:33robertfwgotcha. that makes sense#2020-02-2801:33seancorfieldI would have expected to be able to use [::properties]
as a key in the top-level overrides but that doesn't seem to work.#2020-02-2801:33seancorfieldI can't figure it out from the code š#2020-02-2801:34robertfwYeah I spent some time walking through some spec guts but couldn't unravel it#2020-02-2801:35seancorfields/keys
and several others build a vector of keys to index into the overrides
but I got lost trying to trace through multiple layers#2020-02-2802:54colinkahnI think it's because the nested s/gen
creates a different context that doesn't know about the overrides in the top level s/exercise
#2020-02-2803:01Alex Miller (Clojure team)^^ this, there's actually a bug around this iirc#2020-03-0119:42bbrinckIn spec2, how would I use a set spec to match against a symbol that happens to be the name of a macro?
> (s/def ::is-foo #{foo})
:expound.alpha2.core-test/is-foo
> (s/form ::is-foo)
#{foo}
> (s/explain ::is-foo 'foo)
Success!
nil
> (s/def ::is-or #{or})
:expound.alpha2.core-test/is-or
> (s/form ::is-or)
#{clojure.core/or}
> (s/explain ::is-or 'or)
or - failed: #{clojure.core/or} spec: :expound.alpha2.core-test/is-or
nil
Or should I avoid using set specs in this case?#2020-03-0120:08Alex Miller (Clojure team)there is actually a known issue around sets of symbols (kind of a collision with symbol as function reference, which need qualification)#2020-03-0120:13Alex Miller (Clojure team)a workaround for the moment is (s/register ::is-or (s/resolve-spec #{'or}))
#2020-03-0121:06bbrinckPerhaps just a different view on the same underlying issue, but it looks like spec2 is using clojure.core/and
instead of the and
symbol for combining specs in :req
?
> (s/def :keys-spec/name string?)
> (s/def :keys-spec/age int?)
> (s/def :key-spec/state string?)
> (s/def :key-spec/city string?)
> (s/def :key-spec/zip pos-int?)
> (s/def :keys-spec/user2 (s/keys :req [(and :keys-spec/name
:keys-spec/age)]
:req-un [(or
:key-spec/zip
(and
:key-spec/state
:key-spec/city))]))
> (s/form :keys-spec/user2)
(clojure.alpha.spec/keys :req [(clojure.core/and :keys-spec/name :keys-spec/age)] :req-un [(clojure.core/or :key-spec/zip (clojure.core/and :key-spec/state :key-spec/city))])
#2020-03-0121:09bbrinckI would have expected :req [(and :keys-spec/name :keys-spec/age)]
#2020-03-0122:35favilaIām pretty sure spec1 does the same? I think the form (in spec1) gets rewritten to a predicate fn in a trivial way. Iirc you can even use your own predicates instead of only and/or#2020-03-0202:36Alex Miller (Clojure team)technically, it is the clojure.core/and function in both spec 1 and 2#2020-03-0202:41Alex Miller (Clojure team)not really going to do anything with it as the plan is for s/keys and and/or to go away in spec 2 anyways#2020-03-0215:30bbrinckAh, thatās good to know. Thanks!#2020-03-0215:14norton@alexmiller I tried the following example from spec 2 wiki page but it fails with āCouldnāt satisfy such-that predicate after 100 triesā. Is this a known issue?
(gen/sample (s/gen (s/select {:a int? :keyword?} [:a])) 5)
#2020-03-0215:14norton@alexmiller I tried the following example from spec 2 wiki page but it fails with āCouldnāt satisfy such-that predicate after 100 triesā. Is this a known issue?
(gen/sample (s/gen (s/select {:a int? :keyword?} [:a])) 5)
#2020-03-0215:19nortonIām using this version:
{:git/url ""
:sha "8498f9cb352135579b6d3a0a5d15c40e5c2647ce"}
#2020-03-0215:26nortonI would like to use select with unqualified keys for schema name and select keys.
Something like the following:
(s/def :foo/bar
(s/schema {:a int? :b keyword?}))
(gen/sample (s/gen (s/select :foo/bar [:a])) 5)
#2020-03-0216:18Alex Miller (Clojure team)used to work, prob broken by later changes#2020-03-0216:20Alex Miller (Clojure team)(gen/sample (s/gen (s/select [{:a int? :b keyword?}] [:a])) 5)
should work as an alternate right now#2020-03-0216:48nortonThat works. Thank you.
Any suggestions for this? It fails with āCouldnāt satisfy such-that predicate after 100 triesā.
(s/def :foo/bar
(s/schema {:a/b string? :b keyword?}))
(gen/sample (s/gen (s/select :foo/bar [:a/b])))
#2020-03-0217:07Alex Miller (Clojure team)sorry, not sure off the top of my head#2020-03-0300:25norton@alexmiller I was able to find the reported error (outside of the generator).
The last s/def below fails with the following error:
1. Unhandled java.lang.AssertionError
Assert failed: (every? (fn* [p1__18963#] (not (select?
p1__18963#))) unq-specs)
impl.clj: 423 clojure.alpha.spec.impl/schema-impl
(s/def :foo/bar
(s/schema {:a/b string? :b keyword?}))
(s/def :foo/baz
(s/select :foo/bar [:a/b]))
(s/def :foo/buz
(s/schema
{:z :foo/baz}))
#2020-03-0716:18norton@alexmiller I tried to find a smaller failing case.
(require '[clojure.alpha.spec :as s] '[clojure.alpha.spec.gen :as gen])
;; => nil
(s/def :wf/S
(s/schema {:s/i-001 string?
:s/i-002 string?}))
;; => :wf/S
(s/def :wf/S1
(s/select :wf/S [:s/i-001]))
;; => :wf/S1
(s/valid? :wf/S {:s/i-001 "1"})
;; => true
(s/valid? :wf/S {:s/i-001 1})
;; => false
(s/valid? :wf/S1 {})
;; => false
(s/valid? :wf/S1 {:s/i-002 "1"})
;; => false
(s/valid? :wf/S1 {:s/i-001 "1"})
;; => true
(s/valid? :wf/S1 {:s/i-001 1})
;; => true
(gen/sample (s/gen :wf/S1) 1)
;; Caused by clojure.lang.ExceptionInfo
;; Couldn't satisfy such-that predicate after 100 tries.)
The result of the last 2 expressions is not expected (to me).#2020-03-0403:53jrwdunhamAre there best practices for how specs are intended to be identified with one another? I have the situation where I want to define spec ::data
via (s/def ::data ::specs/jsonb)
(where (s/def ::jsonb map?)
). I want to define a map (`spec-casters`) from specs to the functions that can be executed to transform their values (for db insertion). The spec-casters-YES-PLEASE
function (which I want to use) seems to do what I want, but I have already run into situations in development where (get spec-casters-YES-PLEASE (s/spec ::other-ns/data)) ;; => nil
. I could use spec-casters-NO-THANKS
but I would rather avoid that verbosity. This makes me wonder consider that there might be something inadvisable about this approach.
(def spec-casters-YES-PLEASE
{(s/spec ::specs/jsonb) pg-cast-jsonb})
(def spec-casters-NO-THANKS
{::specs/jsonb pg-cast-jsonb
::other-ns/data pg-cast-jsonb})
#2020-03-0404:01Alex Miller (Clojure team)I only understand about 30% of what you're trying to do, but (s/spec ::specs/jsonb) is a spec object without well-defined equality semantics so it seems like a bad choice for a key#2020-03-0404:09jrwdunhamOk, the fact that specs lack well-defined equality semantics suggests that I should just be more explicit and use the NO-THANKS
approach where the keys are just the namespaced keywords.#2020-03-0404:10jrwdunhamHowever, that seems to violate the intention of a key spec though, namely that its value should conform to it ...#2020-03-0404:11Alex Miller (Clojure team)but you're not conforming#2020-03-0404:14Alex Miller (Clojure team)you're just matching spec objects#2020-03-0404:15Alex Miller (Clojure team)not using them for anything#2020-03-0404:17jrwdunhamright, that's true.#2020-03-0522:03jrwdunham(btw, thanks for taking the time to discuss my question Alex. I do appreciate it.)#2020-03-0800:12agDo we have a repo with a collection of arbitrary specs for all sorts different cases? I often need to create a spec for something so trivial but end up spending more time than intended, only to remember later that I did it already in some other Clojure project. Sometimes I just need to perform Github search and find some gems. Would be nice to have a community driven repo of good examples.#2020-03-0800:16seancorfieldHmm, I see Specs as being tied to domains rather than being that sort of reusable -- what sort of things do you have in mind @ag?#2020-03-0800:28agthings like various datetime, timestamps, comma separated lists of currencies or/and US states and US-state abbrevs, etc.#2020-03-0800:29agYou right though, maybe not a collection, but perhaps something like a cookbook#2020-03-0800:53seancorfieldYou're talking about strings? Parsing strings? That's not a good use for Spec tho'...#2020-03-1307:59ro6I went down that road a bit once too, mostly for want of the automatic error descriptions Spec gives, but extending down into a String (eg "That's not a valid email because X, Y, Z"). Spec definitely does a good job of that for aggregates/shapes. I don't recall finding a satisfactory solution at the time.#2020-03-0800:57seancorfieldSpecs for datetime/timestamps -- inst?
, s/inst-in
.#2020-03-0801:01seancorfieldLocales, country codes, currency codes -- all easily available from Java classes but I guess ready made Specs for sets of those things would save you writing a single line of Clojure?#2020-03-0801:04seancorfieldUS states -- that's domain-specific: do you just want the 50 states, plus DC? Plus the various US territories/islands? But stuff like that belongs in databases and you can construct a set spec from that.#2020-03-0816:06derpocioushi, I am wondering if it is possible to instrument the return value of a function?#2020-03-0816:59EddieI like to use post-assertions combined with spec. They look like this.
(defn foo
[x y]
{:post [(s/valid? ::my-spec %)]}
(do-something x y))
#2020-03-0817:01EddieYou can also validate your arguments similarly using a pre-assertion.
(defn foo
[x y]
{:pre [(s/valid? ::x-spec x)
(s/valid? ::y-spec y)]
:post [(s/valid? ::my-spec %)]}
(do-something x y))
#2020-03-0907:10andy.fingerhutI have not used it before, but the orchestra library aims to do this: https://github.com/jeaye/orchestra. It is not included in spec because it is instead recommended to only do checking of return values during testing, e.g. property-based testing using randomly generated arguments.#2020-03-0918:55Gustavo GoretkinHi. Total beginner here. I've seen mention of this global registry of specs. Where can I see which specs are defined there? Also, are there any writings about spec and other format-specific Schema definitions (like XML Schema (.xsd) or JSON schema things)?#2020-03-0918:58favilahttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/registry to get the registry#2020-03-0918:59favilaitās global--to your process, not the world#2020-03-0919:00Gustavo GoretkinI see. Thanks! I think I misinterpreted. Are there clojure libraries/packages that primarily define specs?#2020-03-0919:05favilaI only know of https://github.com/cognitect-labs/anomalies Sometimes a library will include specs for its api or extension points. Iām not aware of any library that just shares spec objects the way one would share utility functions#2020-03-1011:54kszabohttps://github.com/edn-query-language/eql/blob/master/src/edn_query_language/core.cljc
is like 50% specs, 50% code#2020-03-1020:35Gustavo GoretkinThanks for the pointers.
I was expecting more packages like this, I think, to capture "ontologies", the same way there are schemas for e.g. geometry data, like https://github.com/mapbox/geobuf , http://wiki.ros.org/geometry_msgs
Is that kind of the spirit of spec at all?#2020-03-1021:33Gustavo GoretkinOr something like this: https://clojureverse.org/t/experiment-a-spec-for-maven-poms-generated-from-xml-schema/2594#2020-03-1012:06ataggartAnyone have insight into why the such-that
is used here?
https://github.com/clojure/spec.alpha/blob/spec.alpha-0.2.176/src/main/clojure/clojure/spec/gen/alpha.clj#L187#2020-03-1012:09ataggartOur tests are occationally failing with "Couldn't satisfy such-that predicate after 10 tries.", and the associated pred is :pred #object[clojure.core$ratio_QMARK_ 0x62fb2f00 "
.#2020-03-1012:13sogaiuthere is some mention of such-that failures in gary fredericks' generators video at around 16:16 -- https://www.youtube.com/watch?v=F4VZPxLZUdA#2020-03-1012:25gfredericksratios are conceptually goofy because you may or may not want to consider integers to be ratios in a given context
it's useful to think of integers as special kinds of ratios, because almost any use case for ratios should almost certainly apply to integers as well. It's hard to do arithmetic with ratios without bumping into integers inadvertently at least (e.g., (* 3/4 4/3)
)
but arguably it would surprise people if (ratio? 42)
returned true, so it doesn't; maybe there are other good reasons for this too beyond just surprise
in any case, there's a mismatch because test.check takes the first approach, but ratio?
wearing its spec hat obviously has the same semantics as ratio?
wearing its predicate hat, and so when you generate from it you don't want to generate integers
I think (such-that ratio? (ratio))
could definitely be improved; setting a higher retry threshold would stop the exceptions, but a custom generator would be the best; something like (gen/let [[n d] (gen/tuple gen/large-integer gen/large-integer)] (let [r (/ n d)] (if (ratio? r) r (/ (inc n) d))))
there might be some arithmetical edge case there I'm not thinking of, but I think that works#2020-03-1013:25ataggartThanks for that! I thought I was losing my mind.#2020-03-1013:28ataggartPerhaps spec'ing on rational?
is the way I should go.#2020-03-1012:27gfredericksalso I forgot to handle (zero? d)
#2020-03-1123:06aghow can I āunaliasā an fdef spec?#2020-03-1123:13Alex Miller (Clojure team)Fdef to nil#2020-03-2401:33QuestSpecific question about the "data specs" feature in spec-tools:
https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md
I want to specify a vector of elements where the vector's count = 2. Is this possible to specify using data-specs?#2020-03-2401:44seancorfield@quest You can certainly specify that with clojure.spec
. No idea about spec-tools
.#2020-03-2401:46QuestYeah, the :min-count option works fine in normal specs... I'm trying to cast to clojure specs from an EDN business schema I have, proving difficult#2020-03-2401:46QuestI know spec2 has much improved support for runtime specs, but I need the CLJS side unfortunately#2020-03-2401:47seancorfieldAh, so you can't eval
either...#2020-03-2502:35QuestJust to follow up on what I ended up doing:
Creating a separate "pre-runtime" step which generates a file of clojure.spec definitions from the data model.
I expect this can be eliminated once spec2 is usable everywhere.#2020-03-2502:46seancorfieldSounds like a good compromise for now!#2020-03-2401:48QuestYeah. I had some success by doing funky things using the spec private "impl" functions, but it completely mangles the error messages when data fails validation#2020-03-2418:03yonatanelIs it possible to spec a map where either a namespaced key or another unnamespaced key is required? e.g {:my/a 1}
ok, {:my-a 1}
ok, {:my/a 1, :my-a 2}
should also be fine, {:b 2}
invalid.#2020-03-2420:00colinkahnSpec V1 doesnāt really have a way to say āthis map canāt contain :bā. For aliasing :my-a
to :my/a
you could choose a single ācanonicalā version of your map and use a conformer when you expect there to be variations of it.
(s/def :my/a number?)
(s/def ::m (s/keys :req [:my/a]))
(s/def ::->normalize-keys (s/conformer #(clojure.set/rename-keys % {:my-a :my/a})))
(s/valid? (s/and ::->normalize-keys ::m) {:my-a 1}) ;-> true
(s/valid? (s/and ::->normalize-keys ::m) {:my/a 1}) ;-> true
(s/valid? (s/and ::->normalize-keys ::m) {:my-a "1"}) ;-> false
(s/valid? (s/and ::->normalize-keys ::m) {:my/a "1"}) ;-> false
Depending on what your needs are for generation though this can be troublesome.#2020-03-2420:07yonatanelYeah, itās nice, but I wanted (s/explain)
to mention the unnamespaced key if both are missing#2020-03-2420:19colinkahn@U1B0DFD25 this is kind of convoluted but:
(s/def :my/a number?)
(s/def ::my-a :my/a)
(s/def ::m (s/merge (s/nonconforming (s/or :v1 (s/keys :req [:my/a])
:v2 (s/keys :req-un [::my-a])))
(s/keys :opt [:my/a]
:opt-un [::my-a])))
(s/valid? ::m {:my/a 1}) ; true
(s/valid? ::m {:my-a 1}) ; true
(s/valid? ::m {:my/a 1 :my-a 2}) ; true
(s/valid? ::m {:b 2}) ; false
(s/explain-data ::m {:b 2})
(s/exercise ::m)
#2020-03-2420:58yonatanel@U0CLLU3QT Thanks. I wasnāt aware of nonconforming
. Gave me some options anyway.#2020-03-2418:40kszabonot directly#2020-03-2418:40kszaboyou have to spec that with additional predicates. (s/keys)
doesnāt have support for that AFAIK across req/req-un
groups#2020-03-2421:08aghow do I make a (s/coll-off ::foo)
, making sure that the elements are distinct by one given field?
e.g.: if (s/def ::foo (s/keys :req-un [:name]))
, I want that all the names in the generated collection are unique.#2020-03-2421:11agnevermindā¦ figure it outā¦ used: (s/and (s/coll-of ,,,, ) #(apply distinct? (mapv :name %)),,,
#2020-03-2511:05Janne SauvalaWhat is your recommendation using spec2 in a new project? My observation is that many are using spec1, Plumatic/Prismatic Schema or metosin malli and not migrating to spec2 yet.#2020-03-2512:17Alex Miller (Clojure team)Spec 2 is not ready for use yet#2020-03-2513:06Janne SauvalaThanks, good to know :+1::skin-tone-2:#2020-03-2606:14Ben SlessI have a bit of a philosophical question: I watched several talks about spec and it's pretty clear how you'd use it for domain modelling when starting from scratch, but several mentioned its benefits in speccing an existing codebase. Any tips on where and how to start with speccing an existing code base?#2020-03-2606:34robertfwI would start by adding fdef
s for key functions#2020-03-2614:32Timur LatypoffMaybe it's also a good idea to spec places where pieces of code, written by different people in the team, meet. To set expectations and settle future disputes.#2020-03-2607:12sogaiuthere was this bit from stuart halloway: http://blog.cognitect.com/blog/2017/6/19/improving-on-types-specing-a-java-library -- may be that's of some use?#2020-03-2622:02bortexzHi, does clojure-spec provide or will provide a way to convert schema maps (unqualified) without explicitly specifying spec on every val? e.g.
From (spec/schema {:type #{:open :closed}}) -> (spec/schema {:type (spec/spec #{:open :closed})})
Seems like a map traversal, is it something that can already be done with normal clojure on the context of spec and schemas?#2020-03-2623:39Alex Miller (Clojure team)I dont think you should even need that?#2020-03-2700:00bortexzyeah on a second thought itās not very useful#2020-03-2700:26bortexzAlso because it would only apply to spec from simple predicates, not other types of specs#2020-03-2700:34Alex Miller (Clojure team)it already does what you're asking, unless I misunderstand what you're asking#2020-03-2710:41bortexzTrue, I was confused by this in the schema and select wiki [::a ::b {:c (s/spec int?)}]
, but in the unqualified keys section is shown as I wanted#2020-03-2721:59hiredmanhttps://gist.github.com/hiredman/60e5e6f0025cb38c8a7739d21f5a8ce6 is a little thing I've been toying with for using spec to define and check stateful protocols, meaning it defines how a client and server communicate, and certain client requests are only valid when the server is in certain states. given such a definition you stick it between your client and server and it checks the communication back and forth and gives errors if either side violates the protocol.#2020-03-3020:21robertfwI'm experimenting augmenting some of our fdefs
with some additional keys, to make it easier to write fdefs
that test async functions, or fns that have some stateful dependencies injected in the args. I thought I would ask in here to see if there is any strong reason not to do this, or any related changes coming in spec2 that I should be aware of#2020-03-3020:22Alex Miller (Clojure team)none that you can prepare for :)#2020-03-3020:22Alex Miller (Clojure team)fdefs will probably be changing significantly in spec 2#2020-03-3020:23robertfwah. any dev logs or documents that discuss possible directions they will be taking?#2020-03-3020:24Alex Miller (Clojure team)no, still in ideation#2020-03-3020:36robertfwgotcha. thanks š#2020-03-3118:10arohnerIs there a way to get there-exists
behavior to compliment for-all
? Context: I have a protocol, and a bunch of implementations (30+), and a test suite that is a series of test.check properties. A broken implementation slipped through, because it returned a pathological response: i.e. (s/nilable ::foo) always returned nil
. Iād like to assert during the test, āthis can return nil or ::foo, but the test must return ::foo at least onceā#2020-03-3118:11gfredericksYou could generate 10 or 50 calls or something#2020-03-3118:12gfredericksSomething large enough that it should never fail#2020-03-3118:12arohnertrue#2020-03-3118:12gfredericksIt does seem like that would be a useful feature though#2020-03-3118:13gfredericksBecause then you could test things of that sort without paying for 50 trials each time#2020-03-3118:13gfredericksA similar structure is using such-that and just generating a single thing#2020-03-3118:18arohnerI think Iāve got it using such-that
, thanks#2020-03-3118:18gfredericksThere's a bunch of dumb tests in t.c itself that use the 50 things approach; it's slow#2020-04-0115:34abdullahibraHi everyone,#2020-04-0115:34abdullahibrais there a way to convert json schema into clojure spec in dynamic way ?#2020-04-0115:35abdullahibrawhat i mean by dynamic here is: i read json schema files and convert them into equivalent clojure spec which could be used#2020-04-0120:11dominicmI would make a single spec per schema which was a predicate which validated the data using the schema?#2020-04-0115:45EddieDepending the exact specs you need, you might be able to write a utility using spec-tools
to do accomplish your goal. For examples, https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md.#2020-04-0316:22Ben GrabowSo far I've been using spec mostly for validating data structures "in-place", meaning I have an x
, at the front door of my API I check if it's a valid format, and if it is I continue using x
inside my API. There's another way of using spec that I don't really understand yet so I'm looking for good resources on it: I have an x
, and at the front door of my API I check if it's a valid format and destructure/reformat it to the format expected internally in my API, y
. y
might have all the same data as x
but perhaps arranged in a different nested structure or with different key names. Are there any good guides to using spec for this kind of thing? I guess this is using spec as a parser, rather than just a validator. This use case is hinted at (https://clojure.org/about/spec#_conform) but I haven't found a good set of examples yet.#2020-04-0316:25Alex Miller (Clojure team)Thatās what conform is for#2020-04-0316:25Alex Miller (Clojure team)āIs this valid, and if so, why?ā#2020-04-0316:26Alex Miller (Clojure team)With the caveat that it is NOT for āperform arbitrary transformation to other structureā#2020-04-0316:27Ben GrabowI'm trying to understand the situations it's good for. e.g. https://juxt.pro/blog/posts/parsing-with-clojure-spec.html#2020-04-0316:27Alex Miller (Clojure team)If you want that, use conform + standard Clojure data transformation#2020-04-0316:27Ben GrabowIn that example, a sequence of characters is being parsed into a map that describes the components.#2020-04-0316:27Ben GrabowI have one map with a nested structure that I want to transform to another map with a flat structure.#2020-04-0316:27Alex Miller (Clojure team)Yeah, dont do that#2020-04-0316:28Alex Miller (Clojure team)(That applies to the string case)#2020-04-0316:28Alex Miller (Clojure team)If you want to transform maps, use Clojure stuff to do that#2020-04-0316:28Alex Miller (Clojure team)If you want to validate, use valid?#2020-04-0316:29Alex Miller (Clojure team)If you want to know why something is valid wrt options and alternatives, use conform#2020-04-0316:29Ben GrabowIt's that last part I'm looking for more content on.#2020-04-0316:30Ben GrabowI don't quite understand when I'm dealing with a situation that's good for conform and when I'm not.#2020-04-0316:33Ben GrabowI remember in one of Rich's talks he mentioned David Nolen rewriting part of the cljs parser in spec. Is that code publicly available somewhere? It sounds like a powerful use case.#2020-04-0316:41Ben Grabowhttp://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2020-04-0316:42Alex Miller (Clojure team)Well thatās mine :)#2020-04-0316:42Alex Miller (Clojure team)Using it to parse the input to a complex macro is a good fit#2020-04-0316:42Alex Miller (Clojure team)Itās particularly good for āsyntaxā stuff#2020-04-0316:43Alex Miller (Clojure team)If your input is mostly maps and stuff, probably not as useful#2020-04-0316:45Ben GrabowThe difference in the way my input map is structured vs how my output map is structured feels like syntax to me so I wonder where the distinction lies#2020-04-0316:47Ben GrabowThere's the idea that a map is just a seq of kv pairs. What if the spec is an s/cat
of kv-pairs instead of an s/keys
?#2020-04-0316:49Alex Miller (Clojure team)Syntax is positional (this is kind of present in the Greek roots of the word even)#2020-04-0316:49Alex Miller (Clojure team)Maps are not positional#2020-04-0316:50Alex Miller (Clojure team)Position in syntax is implicit#2020-04-0316:50Alex Miller (Clojure team)Spec regex ops describe that, and the conformed values tell you how it was interpreted#2020-04-0316:51Alex Miller (Clojure team)Must of the map specs essentially return the map if itās valid - there was no thing to figure out and tell you about#2020-04-0316:53Alex Miller (Clojure team)Specing a map as a seq of kv pairs is not the same thing as maps are inherently unordered#2020-04-0316:53Alex Miller (Clojure team)You canāt rely on position to describe the structure of the map#2020-04-0316:54Ben Grabowhmm, that's a good point#2020-04-0316:54Alex Miller (Clojure team)The regex specs like cat will even error if you try to do this, for this reason#2020-04-0316:56Ben GrabowI had in mind using a big s/or
to describe all the possible keys. Which brings up an interesting question, how does the positionality of syntax impact s/or
? I usually reach for s/or
when I have a couple different possibilities for my input, like a union type, and I'm usually frustrated that I'm pushed towards destructuring and naming the options instead of conforming to just the value.#2020-04-0316:56Ben Grabow(s/or :even even? :small #(< % 42))
from the docstring as an example.#2020-04-0316:57Ben GrabowI wish the API were (s/or even? #(< % 42))
for the use I have in mind#2020-04-0316:57Ben GrabowI feel like there's something I don't "get" in the reasons why s/or
asks for keywords to name the options.#2020-04-0316:59Alex Miller (Clojure team)or gives you back a map entry of tag (key) and value#2020-04-0317:02Ben GrabowSure, I get that. I'm wondering why it does that when it doesn't seem to be about processing the position of a thing. Instead it seems to be about processing the alternatives of a single thing.#2020-04-0317:27Alex Miller (Clojure team)well the idea is the same - tell me what choice you made when there was an alternative#2020-04-0317:28Alex Miller (Clojure team)in practice, it is often a source of issues in deeply nested data where that's the only conformed thing so I kind of would like to have some more options in spec 2#2020-04-0317:28Alex Miller (Clojure team)whether that's an or variant, or a s/nonconforming, or something else#2020-04-0317:29Alex Miller (Clojure team)but that's down the list a bit and we haven't talked about it#2020-04-0317:33Ben GrabowGlad to hear the sentiment is shared.#2020-04-0317:35Ben GrabowI often run into situations where a map is mostly just a set of specific things so s/keys
works, until we add uuids into the mix and they could be strings or uuid objects. I'm using a custom conformer to get around the s/or
destructuring for that case but it would be nice to have a more general solution.#2020-04-0317:39Alex Miller (Clojure team)one workaround right now is to wrap those s/or's in s/nonconforming#2020-04-0317:39Alex Miller (Clojure team)which is not documented, but exists#2020-04-0317:49Ben GrabowNice!#2020-04-0317:49Ben Grabow(comment
(s/conform (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)) (UUID/randomUUID))
(s/conform (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)) (str (UUID/randomUUID)))
(gen/generate (s/gen (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)))))
All these work as expected. The first two produce just the value, without the keyword tag. The third generates both strings and uuids.#2020-04-0317:51Ben GrabowMy ::uuid-string
spec is still a little complicated to make the generator work but I think it could be simplified.#2020-04-0819:06dspiteselfI see spec alpha 2 adds select. Will select have a special knowledge of coll-of so that you could select a map of a vector of maps?#2020-04-0819:07dspiteselfLike pull does across datomic cardinality many attributes.#2020-04-0819:07Alex Miller (Clojure team)Currently, no, but thatās under discussion#2020-04-0820:21dspiteselfThanks I really like what is going on over there. That is the only problem left I see that limits our uses.#2020-04-0820:29Alex Miller (Clojure team)certainly there are a lot of similarities to pull#2020-04-1113:46credulousHi! This is definitely a novice question, but Iāve struggled for a day or so trying to figure out the best way to use spec in my re-frame application. My problems stem from not grokking spec I think.#2020-04-1113:46credulousIām using my app db to store form information under a :form key. Forms are stored by form-id.#2020-04-1113:48credulous#2020-04-1113:52credulousIn each field the āerrorā is set based on a spec that gets checked on each edit:
(s/def ::id (s/and string? #(> (count %) 0)) )
(s/def ::team (s/and string? #(> (count %) 0)) )
(s/def ::nickname (s/and string? #(> (count %) 0)))
(s/def ::password (s/and string? #(> (count %) 7)) )
#2020-04-1113:52credulous(Lengths are set to zero but will change)#2020-04-1113:54credulousBut I also want to define a spec for the form as a whole, so I can pass the āsignupā form above to (s/valid?). But I canāt figure out how to define the structure above, where signup has four keys, each of which have the same structure#2020-04-1113:55credulousin :signup :team for example, I want to say signup has a map that requires a :team key, and the team key is a map that has a :value key that is defined by ::team in the snippet pasted above#2020-04-1116:22shooitIām not sure that I understand 100% but you might need to put each :value key into a different namespace so that they can be a distinct spec e.g :foo/value, :bar/value. You could also check out multi-spec for multimethod like dispatch on the submaps #2020-04-1113:56credulousI canāt define a ::value spec to include in (s/keys) for :team - because what :value is is different for each sibling key in :signup#2020-04-1116:26Alex Miller (Clojure team)spec is primarily designed to associate semantic specs with namespaced keys, so by using :value in each of these places (instead of :id/value or whatever), you are outside the sweet spot#2020-04-1116:27Alex Miller (Clojure team)you can do this with s/keys and :req-un for unqualified keys though, but you'll need to match up the name parts of the specs#2020-04-1116:32Alex Miller (Clojure team)(s/def :team/value ::team) ;; same for the others
(s/def :team/team (s/keys :req-un [::team/value])) ;; same for the others
(s/def ::signup (s/keys :req-un [:team/team ..etc..]))
#2020-04-1116:33Alex Miller (Clojure team)as I said, this is outside the expected pattern so it's more work for sure#2020-04-1118:08credulousThanks Alex. What is the preferred way to satisfy the goal I have here, which is to independently validate the form contents and the form data as a whole? Is the correct answer to my confusion āread moreā? š#2020-04-1119:17Alex Miller (Clojure team)the preferred way is to have namespaced attributes with one meaning#2020-04-1308:21AronHi, I would like to do some "spec driven development" in cljs, where should I start? Is it the same for clojure and clojurescript?#2020-04-1413:56colinkahnI think in general itās similar. For UI applications the boundries that you want to focus your specs on are different of course. I consider anything coming over http and websockets a good candidate for checking with spec and all of my component props as well. If youāre using Reagent the later is easy because your components are already functions and you can just us s/fdef
. Another consideration is I like to keep my specs separate from my other code on the F/E. Instead I write separate spec files that require the namespaces that Iām specing. Then you can make a preloads file that imports those spec files and runs st/instrument
to get development-time feedback.#2020-04-1414:07AronI am not using reagent because it has no support of modern react, I am using helix.#2020-04-1414:09AronThanks for the note on 'separate', I was just thinking about that, I saw a video with @U064X3EF3 where it was noted to not use these in production and I am not quite sure how to do that. I thought that the compiler would remove these when I build for production and I will not have to maintain two versions of my code#2020-04-1414:18colinkahnIt looks like the defnc
macro in helix eventually produces a function with [props ?ref]
that you can spec like a Reagent component: https://github.com/Lokeh/helix/blob/master/src/helix/core.clj#L80-L87#2020-04-1414:19AronHow many years before I can do what you just did @U0CLLU3QT ? : D#2020-04-1414:20AronI can read the code, but all these ^
s and ~
confuse the hell out of me.#2020-04-1414:21Aronand yeah, I am fairly sure I can spec it, it's more about the context/reducer situation that I am somewhat unsure. I think I can do state transition functions and spec those and then I can create spec for the state as input and output before and after the transition.#2020-04-1414:24colinkahnThe āseparateā approach for me works because I agree these are more development-time tools and donāt really belong in my production source code. The compiler I believe will remove most of it (Iāve seen tricks where frameworks using Google Closure features to help with this as well) but I think certain parts of the spec library itself cannot be dead code eliminated (though donāt quote me on that, and probably better to try it yourself).
For having two versions, this isnāt necessary. For example:
app.components.cljs
(ns app.components)
(defn my-component [props]
...stuff)
app.components.specs.cljs
(ns app.components.specs
(:require [app.components :as c]))
(s/fdef c/my-component
:args (s/cat ...))
#2020-04-1414:25colinkahnhttps://clojurians.slack.com/archives/C1B1BB2Q3/p1586873997105600?thread_ts=1586766083.099100&cid=C1B1BB2Q3
You have to go through your āmacrosā phase š#2020-04-1414:26AronI haven't even started that... : )#2020-04-1414:29colinkahnThis is the preloads feature I was referring too: https://clojurescript.org/reference/compiler-options#preloads
Once you have your *.specs
namespaces I make a single specs.preload
or something that is roughly:
(ns app.specs.preload
(:require [clojure.spec.test.alpha :as st]
; all your spec namespaces here
[app.component.specs]))
(st/instrument)
#2020-04-1506:53AronThis is somewhat different for shadow-cljs, right?#2020-04-1515:12colinkahnItās pretty much the same, youād specify the preloads in your build defined in shadow-cljs.edn
#2020-04-1515:13colinkahnhttps://shadow-cljs.github.io/docs/UsersGuide.html#_preloads#2020-04-1515:14colinkahnActually yeah, itās under a different key#2020-04-1410:00Jim NewtonI've never used spec before. Can someone help me with a recipe ?
for a given integerĀ `n`Ā and a given sequence, possibly heterogeneous, whether the sequence is of the formĀ `n`Ā many optional integers followed by exactlyĀ `n`Ā integers?Ā Ā some analogous expression to "a?^8a^8" which I can substitute any fixed integer for 8 ?#2020-04-1411:29gfredericksis that the same thing as "between 8 and 16 integers"?#2020-04-1418:15zaneHow common is it to use spec (specifically s/or
+ conform
+ (`case` / multimethod
/ ā¦)) to define polymorphic functions, and when would / wouldn't doing so be appropriate?#2020-04-1418:31Alex Miller (Clojure team)I have no way to judge frequency, but seems ok to me. Itās probably not the fastest option.#2020-04-1509:28flowthingI often test my specs in the REPL by doing something like (clojure.test.check.generators/generate (clojure.spec.alpha/gen ::foo))
. Sometimes I mess up the spec such that the generator can't generate a value from my spec, and I get Couldn't satisfy such-that predicate after 100 tries.
. Is there an easier way to debug cases like this than process of elimination? If the spec is fairly large, it's can be pretty time-consuming and difficult to find the offending spec. The exception and the stack trace aren't of much help.#2020-04-1512:38Aron'test in the REPL' means you type it in?
#2020-04-1512:41flowthingWell, I evaluate the forms in my editor, but yeah.#2020-04-1512:50Aronare you by chance using neovim and shadowcljs?#2020-04-1512:50flowthingCursive, but I have dabbled with Conjure a bit.#2020-04-1512:52Aronhttps://github.com/thheller/shadow-cljs/issues/508#issuecomment-565021177 YAY. so i can use conjure finaly#2020-04-1512:52Aronshould've paid attention, it's been MONTHS#2020-04-1512:52Aronthanks flowthing! : )#2020-04-1515:12Aronwell, nope.#2020-04-1512:56gfrederickssomebody needs to write up a summary of what the such-that error means for spec users#2020-04-1513:07potetm@gfredericks It looks like @flowthing knows what it means. The question is: is there an easier way to find it than manual searching?#2020-04-1513:07potetm(Iām curious your answer to that as well.)#2020-04-1513:08gfredericksI don't think so. There's a feature added to such-that that enables spec to give better errors, but that was never implemented in spec#2020-04-1513:23Alex Miller (Clojure team)s/was never/has not yet been/#2020-04-1513:25gfredericksI was uncertain whether these situation with spec2 was similar or not#2020-04-1513:48Alex Miller (Clojure team)no changes for this yet#2020-04-1513:48Alex Miller (Clojure team)I did a little spike of it a while back but it was less obvious than I expected#2020-04-1513:48mpenetsorry to ask again: any eta for spec2 ?#2020-04-1513:59Alex Miller (Clojure team)no, kind of been dormant recently but we'll be back to it soon I hope#2020-04-1517:02seancorfieldRe: such-that
-- about the only thing I've found that helps avoid that is relentlessly running s/exercise
on each spec as I write it, so that I trip over a "Couldn't satisfy" error at the earliest possible point and therefore have a really small surface of possibilities to investigate/debug. It's not perfect but it helps me, and it fits naturally in with a very tight RDD workflow that has you evaluating every piece of code as you write it.#2020-04-1517:18Frank HenardHey Everyone, I would love your input on my SO question: https://stackoverflow.com/q/61234691/59439#2020-04-1518:36Alex Miller (Clojure team)they both ultimately call the same function so I would expect perf to be pretty similar right now. we have considered fast-pathing a separate valid? path in spec 2 though and I would expect that to be the faster option.#2020-04-1519:08Frank HenardGot it, thanks!#2020-04-1519:11Alex Miller (Clojure team)the big differences are that a) with valid?, you can fail fast and b) with explain, you have to track a lot of additional info and accumulate it for reporting#2020-04-1517:43vlaaadIsn't it sort of obvious what specs might throw such-that errors? The ones that use s/and with second condition rejecting most values from the set specified by first condition?#2020-04-1606:49flowthingYes, it's often obvious, but not always, if the spec is large. An error message that points to the location where it originates would certainly be helpful. Anyway, I think the approach @U04V70XH6 proposed is the way to go for the time being.#2020-04-1518:18zaneI think it's a good idea to avoid using the word "obvious" in this kind of context. Spec is pretty complicated, and people come to it from a variety of backgrounds.#2020-04-1518:42bbucciantiCould be possible to use spec to validate lambda expressions?#2020-04-1518:42bbucciantiI mean, I'm building a lambda calculus reducer#2020-04-1518:43bbucciantiAnd I'm thinking if would be possible to use property based testing in order to see if it's reducing properly#2020-04-1518:43bbucciantiBut I never used spec and I don't know (yet) if that would be possible#2020-04-1518:48Alex Miller (Clojure team)s/fspec is the tool provided, but it's often more trouble than it's worth when using gen#2020-04-1518:51bbucciantiDo you mean than it doesn't ve of any value building such a thing?#2020-04-1518:51bbuccianti*be#2020-04-1518:59Alex Miller (Clojure team)spec is not particularly useful for checking functions as args in a deep way#2020-04-1519:01Aronout of curiosity, is there anything that can do that? Sounds like something that I call the halting problem?#2020-04-1519:14nwjsmith@U0VQ4N5EE there are several property-based testing tools that can do this, they generate functions that satisfy the function's type#2020-04-1519:20AronThen I misunderstood what functions as args in a deep way means, sorry.#2020-04-1519:18ikitommifast-path to valid?
is a good idea. Any other new ideas for Spec2? New syntax for fn specs still wip?#2020-04-1521:17Alex Miller (Clojure team)too many ideas :) fn specs dormant atm, working on other stuff, but we'll get back to it#2020-04-1521:47kennyIs this a known bug?
(s/def ::foo
(s/with-gen #(string? %) #(s/gen string?)))
(s/form ::foo)
=> :clojure.spec.alpha/unknown
#2020-04-1521:52Alex Miller (Clojure team)Yes#2020-04-1521:52Alex Miller (Clojure team)Not going to be fixed, but not an issue in spec 2#2020-04-1521:53kennyThis appears to be an ok workaround:
(s/form (s/spec #(string? %) :gen #(s/gen string?)))
=> (clojure.core/fn [%] (clojure.core/string? %))
#2020-04-1521:54seancorfield@kenny Why use #(string? %)
instead of just string?
?#2020-04-1521:55kennyFor demonstrating the bug, nothing else š#2020-04-1522:37seancorfieldAh, so it works just fine with string?
?#2020-04-1522:38kennyYep#2020-04-1522:38kennyI encountered this with a spec defined with a #(re-find ...)
#2020-04-1812:23JoeIs this the correct way to spec a non-empty collection?
(s/def ::things (s/coll-of ::thing :min-count 1))
#2020-04-1813:34shooitThat would do it. The other thing that some to mind is from the sequential specs: s/+
. It depends on context which is preferred. Just a non-empty collection of things? Your s/coll-of
solution is it. A non-empty sequence of things as part of a larger pattern? Look to the sequential specs for their regex like functionality. #2020-04-1817:32prncSpec rationale quote Iām pondering
> Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment. The best and most useful specs (and interfaces) are related to purely information aspects.
https://clojure.org/about/spec#_informational_vs_implementational
Trying to get clear on the distinction ā what would be the canonical example of the offending type of use and how far you can push the boundary so to speak? Is this mostly about trying to enforce implementation a la types (an thus cussedness/rigidity etc.) ?#2020-04-1818:00Alex Miller (Clojure team)Clojure core functions are a good example#2020-04-1818:01Alex Miller (Clojure team)You can spec a function as talking PersistentVector (bad, concrete, may change) or as satisfying vector? or seq? or coll? or sequential? (depends on the fn of course)#2020-04-1818:02Alex Miller (Clojure team)And itās common to try to spec what exactly it does now vs leaving open for future#2020-04-1901:47salam@alexmiller It seems like the Clojure Spec 1 alpha JAR is compiled against Java 1.5 (class file version 49). Is that intentional? It seemed a bit odd now that Clojure itself targets Java 1.8 (class file version 52).#2020-04-1901:48Alex Miller (Clojure team)Which version in particular?#2020-04-1901:50salamthis is what's on my class path: spec.alpha-0.2.176.jar
#2020-04-1901:51salami took a look at the pom.xml (https://github.com/clojure/spec.alpha/blob/master/pom.xml) file for spec.alpha and the absence of <target.version>1.8</target.version>
in the <properties>
section seems to confirm that.#2020-04-1901:52Alex Miller (Clojure team)Iām not at a computer but certainly older versions of the parent pom could have produced that#2020-04-1901:55salamI see. I was experimenting with https://openjdk.java.net/jeps/310 and noticed the following warning and figured I should at least let you know:
OpenJDK 64-Bit Server VM warning: Pre JDK 6 class not supported by CDS: 49.0 clojure/spec/alpha/Spec
Preload Warning: Cannot find clojure/spec/alpha/Spec
#2020-04-1901:58Alex Miller (Clojure team)There is a newer version that I suspect is newer bytecode. - 0.2.187#2020-04-1902:00Alex Miller (Clojure team)Spec was introduced in the same release that bumped to java 1.8 bytecode and it took a while for the contrib libs to catch up#2020-04-1902:04salamah, i see. it appears that the release version 1.10.1 of the clojure.jar still depends on the version that i have: https://search.maven.org/artifact/org.clojure/clojure/1.10.1/jar#2020-04-1902:33Alex Miller (Clojure team)You can add an explicit dependency on the newer spec.alpha version to override #2020-04-1902:35salamyep, i'll go with that. thank you.#2020-04-1903:17salamok, reporting back here. it turns out [org.clojure/spec.alpha "0.2.187"] is compiled against java 1.5 as well.#2020-04-1904:27seancorfieldConfirmed, if I'm reading this right. class version 49 for spec.alpha 0.2.187, but class version 52 for clojure itself. The latter is Java 8.#2020-04-1908:13practicalli-johnspec version 1 or version 2, that is the question š My default assuming is spec 1
I am starting a new project, which I will mainly use spec for data structures I design and pass to functions as arguments. Its a data science visualisation project so there is a lot of data. I will also write specs for the functions that for the API for each namespace.
Just checking that I am not missing an obvious benefit of using spec 2. Thank you.#2020-04-1908:38mpenet1 for sure. v2 seems still wip, especially fn annotations which apparently will be improved/modified#2020-04-1908:38didibusSpec 2 is experimental and subject to breaking change#2020-04-1908:38didibusSo unless you are doing this for fun, go with Spec 1#2020-04-2021:42Aronso I am reading docs and I am sure I will figure this out too, but someone could help me here a lot with a shortcut to speccing a hashmap that has 3 levels, the first is a fixed set of keys, second also a fixed set of keys, and the third is depending on the key either a string or a vector of keywors. so a valid example would be {:firstlevelkey {:label "string" :fields [:fieldone :fieldtwo]}} and I just want to say that on the first level only certain keys are allowed( one predefined set of keywords), second level always have to be both a :label and :fields and label always have to be string and the :fields always needs to be a vector of keywords from another set of predefined keys#2020-04-2021:42Aroni don't need a full solution, just tell me please what to use so I can read up on those specifics because it's taking forever to sieve through š#2020-04-2021:43AronI am trying to google
> clojure spec "-un"
and it's not going well#2020-04-2021:53Aronthe more I read the less I understand.#2020-04-2022:06seancorfield> I just want to say that on the first level only certain keys are allowed
This isn't something Spec favors -- it follows the concept of "open for extension". So you will need additional predicates with s/and
to restrict keys (and remember that it's not really "idiomatic" to do that).#2020-04-2022:07AronOk, good to know, so I won't force it if it's not expected#2020-04-2022:11seancorfieldThe whole spec is going to end up looking something like this:
(s/def :key/label string?)
(s/def :field/keys #{:fieldone :fieldtwo ,,,}) ; your set of predefined keys allowed
(s/def :key/fields (s/coll-of :field/keys :kind vector?)) ; maybe you want min and/or max counts too?
(s/def :second/level (s/keys :req-un [:key/label :key/fields]))
(s/def :top/keys #{:firstlevelkey ,,,}) ; available top-level keys
(s/def :top/level (s/map-of :top/keys :second/level))
#2020-04-2022:12seancorfieldIf your set of keys isn't known at spec-time, it's a bit harder but that's probably a good first cut outline to start with.#2020-04-2022:12Aronit's known š luckily#2020-04-2022:12seancorfield(with better names, of course)#2020-04-2022:13seancorfieldThat actually solved the restricted set of top-level keys, but at the expense of not requiring a minimum set either.#2020-04-2022:13Aronthe naming of keywords is arbitrary, right? so I could do just :label instead of :key/label, it just pays off to have very specific names of symbols so there is no collision?#2020-04-2022:14seancorfieldRight. Qualified names help avoid collisions.#2020-04-2022:14seancorfieldOnly s/keys
cares -- :req-un
treats the qualified names as just their unqualified part (`:req` would pay attention to the qualifier as well).#2020-04-2022:15Aronit's quite hard to grasp the pattern#2020-04-2022:15AronI need to experiment with this, thank you very much#2020-04-2022:17Aronone more question that I think I know the answer for but still, if I want to spec anything, I always have to give a name to both the thing I want to spec and either use the definitions in place or create a named spec, right? So there is no way to write specs that would fit existing names automatically.#2020-04-2022:21seancorfieldI'm not quite sure I'm following your question but I think the answer is "no, no way to do that in Spec 1".#2020-04-2022:22seancorfieldSpec 2 does allow unqualified keys in a schema to have inline specs (predicates).#2020-04-2022:22Aronin app namespace I define something like (def whatever "string") and then in app.specs namespace I say (s/def whatever string?) and there is some magic to match them#2020-04-2022:23Aronprobably a bad idea#2020-04-2022:25seancorfieldSpecs are in a completely separate "space" from regular Var names so you need to associate them yourself explicitly.#2020-04-2022:26seancorfieldWith functions, that's s/fdef
but there's nothing for other Vars.#2020-04-2022:26seancorfieldYou need to use s/valid?
or s/conform
to "apply" a spec to a value at runtime.#2020-04-2022:26seancorfield(does that answer your question?)#2020-04-2022:27Aronyes : )#2020-04-2101:12AlexI'm having some trouble with adding a spec for a keyword which is supposed to represent a function. I tried this
(s/fdef ::on-change
:args (s/cat :value :option/id))
(s/def ::props (s/keys :req-un [::options
::value]
:opt-un [::class
::centered?
::on-change
::variant]))
(s/fdef tabs
:args (s/cat :props ::props))
#2020-04-2101:12Alex> Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required#2020-04-2101:12Alexfrom trying to pass ::on-change
to fdef
#2020-04-2101:14AlexAfter reading the API docs, it seems fdef
can only take a symbol naming a function.
My question is... how can I create a spec for the function that is passed as :on-change
?#2020-04-2101:17robertfwIt's funny you ask, I was just arriving to ask something similar. I have a spec describing a map, of which some keys are functions. I'd love to be able to specify that those functions should conform to a given fdef, but not sure how to accomplish that. As is, I just have the keys referring to the functions specced using fn?
#2020-04-2101:19AlexNow I don't feel too silly asking š#2020-04-2101:26Alex Miller (Clojure team)use ifn?
not fn?
#2020-04-2101:27Alex Miller (Clojure team)you can use fspec
to do this as well, but there are some big caveats for generation#2020-04-2101:27Alex Miller (Clojure team)personally, I have not found it to be worth doing fspec
over ifn?
#2020-04-2101:32Alex@alexmiller Does ifn?
allow you to specify the shape of the arguments that are passed in? I can't seem to find any examples doing so#2020-04-2101:37Alex Miller (Clojure team)no#2020-04-2101:38Alex Miller (Clojure team)you can do that with fspec
(same args as fdef
)#2020-04-2101:45AlexTried a simple fspec
but that doesn't seem to compile. Hmm
(s/fspec ::on-change
:args (s/cat :value number?)
:ret any?)
#2020-04-2102:37kennyRemove the first arg:
(s/fspec :args (s/cat :value number?)
:ret any?)
#2020-04-2103:06AlexThanks! How do I associate it to the keyword which will use the spec?#2020-04-2111:40sgepigon(s/def ::on-change
(s/fspec :args (s/cat :value number?)
:ret any?))
#2020-04-2114:09AlexI get this error when I try that. Is this expected?
> Uncaught Error: Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required#2020-04-2114:09kennyfspec uses test.check to ensure the correctness of the spec'ed input.#2020-04-2123:10AlexThank you!#2020-04-2107:23Aronhow can I spec transient things like values of let bindings?#2020-04-2109:06Ben SlessIt might be terrible but I recently had a similar issue, went for this solution:
(let [x (expr ...)]
(eval
`(s/def ::my-spec
(s/keys
:req-un
[~x]))))
#2020-04-2109:06Aronš#2020-04-2109:07Aroni am not sure about terrible but it's scary#2020-04-2109:07Ben Slessit is. If you can convince yourself the expr doesn't do any IO then you can sleep well imo#2020-04-2109:09Ben SlessIn my case I had to create a qualified keyword who's name starts with a number. seems like :1foo
is a valid kw but :user/1foo
can't be read by the reader. (keyword "user" "1foo")
works#2020-04-2109:12Aronthis is supposed to run in a browser š#2020-04-2109:13Ben Slessah#2020-04-2107:33AronI see the suggestion to use s/assert in docs, but isn't that also something that I should only do in development and not make it part of the code that might be shipped eventually?#2020-04-2112:33Alex Miller (Clojure team)Spec asserts can be turned off and even compiled out in prod code#2020-04-2112:47Aronthanks, I realized there has to be some solution because there were blogposts about how to use it, but now I know where to look for it.#2020-04-2109:02Ben SlessIs it possible to spec/validate java collections? this naive example doesn't work
(s/def ::foo int?)
(s/def ::bar string?)
(s/def ::m (s/keys :req [::foo ::bar]))
user=> (s/valid? ::m {::foo 1 ::bar "2"})
true
user=> (s/valid? ::m (java.util.HashMap. {::foo 1 ::bar "2"}))
false
#2020-04-2112:32Alex Miller (Clojure team)Spec does not cover Java collections so youād have to pour one into a Clojure collection first#2020-04-2117:35Ben SlessThis is probably a terrible idea, but I guess everything is possible if you try hard enough#2020-04-2117:52Ben SlessIt would be nice, maybe in spec2, if the meanings of maps and sequences were relaxed a bit. But I'm coming at it from an esoteric use case#2020-04-2117:52Alex Miller (Clojure team)that seems way harder than converting the HashMap into a Clojure map#2020-04-2117:53Alex Miller (Clojure team)we do not have plans to support Java colls in spec 2#2020-04-2117:54Alex Miller (Clojure team)(s/valid? ::m (into {} (java.util.HashMap. {::foo 1 ::bar "2"})))
#2020-04-2117:56Ben SlessThat's the trivial case, in reality I might be dealing with an arbitrarily nested java collection š#2020-04-2118:05Ben SlessIt's also unfortunate because it creates a bit of a mismatch between the validation mechanism and the applied predicates. In this example, get
works on java.util.Map
, so does seq, so conform*
can do its work. The only "hurdle" is map?
, which is a pretty stringent requirement. Why not expand spec a bit more towards reflecting intent and less implementation?#2020-04-2118:15Alex Miller (Clojure team)the intent is to validate clojure data#2020-04-2118:16Alex Miller (Clojure team)validating java colls was out of scope for us#2020-04-2116:23eval2020What spec could validate that a map contains at least one true value?#2020-04-2116:25Alex Miller (Clojure team)any function that takes a value can be a spec#2020-04-2116:25Alex Miller (Clojure team)so rephrasing - what function could validate that a map contains at least one true value?#2020-04-2116:25Alex Miller (Clojure team)write that function and you're done#2020-04-2116:28eval2020Ah, I was overthinking this - thanks#2020-04-2201:08Aronshould spec have their own ns an import app ns? or should the app import the spec ns? I am having this issue that circular dependencies seem unavoidable#2020-04-2201:12seancorfield@ashnur The answer is "it depends". You should be able to find a partial ordering of dependencies that would allow you to split things into different namespaces -- probably three, so you have an "app" ns, a "spec" ns, and a "predicate" or "utility" ns that contains things the specs depend on, and the app can also depend on. But it may not be worth doing that analysis so having the specs in the same ns as your app code may just be easier.#2020-04-2201:13seancorfieldIf you're writing code that you want to be able to run on older versions of Clojure(Script) that don't include spec, you have to break them out into optional namespaces so that users can opt in if they want the specs -- but that's really only for library writers.#2020-04-2201:14AronI am running shadow-cljs#2020-04-2201:14seancorfieldAt work, I try to have the data specs in a separate namespace (and do the work to untangle any problematic dependencies) but the function specs will generally go in the same namespace as the functions they are for.#2020-04-2201:14AronI initially imagined having files for data/specs/components/react-app all separated#2020-04-2201:15Aroncan I have multiple files all in the same namespace?#2020-04-2201:21seancorfieldThe simple answer to that question is no. The complex answer is there are ways to spread a namespace across multiple files but I don't think it will solve the problem you have here.#2020-04-2201:22Aronok, no it is š I want to keep it simple#2020-04-2201:22AronI must be doing something silly again because I am unable to refer to my s/defs from the spec file/ns#2020-04-2201:24seancorfieldCan you share some code to illustrate the problem? Is this in a project up on GitHub?#2020-04-2201:29Aronin spec file I wanted to do something like
(s/def :my.specs/one-spec identity)
then in app file ns require
[my.specs :as myspecs]#2020-04-2201:31Aronit's closed source but I can make a new repo with completely new code, it might just take a couple of weeks before i have the time#2020-04-2201:31Aronš only half kidding though#2020-04-2202:00seancorfieldOK. And what can't you do in the app file ns?#2020-04-2202:01seancorfield(sorry for the delay in responding -- was dealing with an issue someone just opened on clj-new that I needed to repro!)#2020-04-2202:04seancorfieldIn you app file ns, after that require, you should be able to reference that spec either as :my.specs/one-spec
(i.e., its fully-qualified name) or ::myspecs/one-spec
using the auto-resolve syntax which will expand ::myspecs
to :my.specs
using the alias of the namespace... the latter requires that the actual spec name really matches the spec ns in the qualifier. (and you could just (s/def ::one-spec identity)
in the spec ns to have the spec automatically match the ns of the namespace it is defined in).#2020-04-2202:04AronI feel bad if you apologize for such stuff, I realize people have lives beyond what is visible here : )#2020-04-2202:04AronI had to do (def (s/def ... and it worked#2020-04-2202:04Aronfor some reason I thought just (s/def would be enough#2020-04-2202:05AronI am also in utter confusion about how keywords work, not just in spec but other places. I know it's a symbol that retuns itself, but apparently it also can hold specs#2020-04-2202:06seancorfields/def
updates a registry behind the scenes which is essentially a map from keywords to spec objects. The keywords don't "hold" specs -- the association is done separately.#2020-04-2202:07Arongotcha#2020-04-2202:08seancorfieldI have no idea what (def (s/def ...))
would do... but it's definitely not the right thing to do.#2020-04-2202:10Aronsorry, (def name-of-spec (s/def#2020-04-2202:10seancorfieldThat's what I assumed -- not the right thing to do.#2020-04-2202:12Aronso much black magic š#2020-04-2202:12seancorfieldThe "name-of-spec" is the keyword. That's how you refer to it in code that calls s/valid?
or something.#2020-04-2202:12Aronso specs can only be registered to keywords?#2020-04-2202:13seancorfieldYes.#2020-04-2202:13Arondocs said resolvable symbol k
#2020-04-2202:13AronI am confused, that's all#2020-04-2202:14seancorfieldLink? So I can see what the context is.#2020-04-2202:14Aronhttps://clojuredocs.org/clojure.spec.alpha/def#2020-04-2202:15Aronhow can I use :require :refer to import a spec definition?#2020-04-2202:16seancorfieldAh, probably because of function specs: s/fdef
uses the fully-qualified function name (symbol) so that's probably why s/def
will accept a symbol.#2020-04-2202:16seancorfieldYou can't refer in a single spec.#2020-04-2202:16seancorfieldOnce you've loaded the ns in which specs are defined, they are globally available.#2020-04-2202:16seancorfieldThey're not like Vars or functions.#2020-04-2202:17seancorfieldSo you could require the specs ns in whatever you app entry point is and you would have access to those specs (as keywords) everywhere in your program.#2020-04-2202:21AronI can't even require them at this point#2020-04-2202:21Aronhow to "load the ns", I usually just do :refer or :as, neither which seems to work here#2020-04-2202:22seancorfieldJust require the namespace like any other. Then the specs are in the registry and available globally.#2020-04-2202:22Aronoh ok, so i can just write the ns :as, but then i have to completely ignore it#2020-04-2202:23Aronand instead always have to read the file of the spec, because ns-publics don't list it#2020-04-2202:24seancorfieldYeah, if you're not using the ::
auto-resolve syntax, you don't even need :as
#2020-04-2202:24Aronand it's available via some (black, that is, unobservable) magic#2020-04-2202:24seancorfield(:require ... [my.specs] ...)
#2020-04-2202:24Aronbut how? just (:require [my.specs])?#2020-04-2202:24Aronok š#2020-04-2202:24Aronthanks again#2020-04-2202:25Aronwhen I think I understand something, it turns out that thing is specific to that particular area, and something else using the same syntax behaves completely differently#2020-04-2202:25seancorfieldHere's where the docs talk about the registry https://clojure.org/guides/spec#_registry -- don't know if that'll help?#2020-04-2202:27Aronit will help š#2020-04-2202:28Aronthe problem, which you have to realize is a problem, that I've had this page open for days and I have read it through from top to bottom and bottom to up several times#2020-04-2202:29seancorfieldDon't worry, sometimes it taking a lot of reading the Clojure docs before it actually makes sense and you get to see a big picture!#2020-04-2202:30AronI see the big picture, I just don't see the details.#2020-04-2202:30Aronš i've been trying to use clojure for 8 years at least#2020-04-2202:31Aronthis time I will succeed because finaly I can do stuff like this, waking up at 1:30 am, then suffering for an hour before you come to help š#2020-04-2202:31Aronoh and because I can use shadow-cljs#2020-04-2202:31Aronso the tooling is solved more or less#2020-04-2202:33seancorfieldI hear great things about shadow-cljs -- when (if) I ever go back to trying to do ClojureScript, that's the path I'll take.#2020-04-2213:58lambdamHello everyone,
I am defining some specs in a .cljc
file, some only exist in a #?(:clj (do ...))
, others in a #?(:cljs (do ...))
I get an error on a keyword at parsing time:
2. Unhandled clojure.lang.ExceptionInfo
failed compiling
file:/.../foo.cljc
{:file
#object[.File 0x5b1b23e7 "/.../foo.cljc"],
:clojure.error/phase :compilation}
compiler.cljc: 1717 cljs.compiler$compile_file$fn__3955/invoke
1. Caused by clojure.lang.ExceptionInfo
/.../foo.cljc
[line 52, col 41] Invalid keyword: ::const/env.
{:type :reader-exception,
:ex-kind :reader-error,
:file
"/.../foo.cljc",
:line 52,
:col 41}
Here the ::const
namespace is declared only for .clj parts of code (:require ... #?(:clj [... :as const]))
When I split this .cljc into two files .clj and .cljs, there is no error.
Would it be related to spec or is it a more general error? It is the first time that it happens.
I really don't have any clue.
Thanks#2020-04-2214:04Alex Miller (Clojure team)it's more general - ::const relies on having a clojure runtime namespace context to resolve it - I'm not sure what the exact problem is without more context but generally you'll want to avoid these in reader conditionals#2020-04-2214:05Alex Miller (Clojure team)you should be able to just use :foo/const or whatever instead#2020-04-2214:41lambdamThanks @alexmiller it works.
It's not as convenient as the shortcut version but it's still better than two .clj and cljs files.
If you want more info, don't hesitate to ask for. I'll send in a private message.#2020-04-2317:07friczeIām curious about specing functions. Thereās probably something I donāt understand, but why function specs made with fdef
are not reusable? I can easily imagine, and even find in my codebase, few functions that share the same spec, but still fdef
have to be called with all the arguments for each function. If anyone has some insight into this decision, Iād love to hear something š thanks#2020-04-2317:15kennyNot sure if I'm following what you mean but you can do this:
(s/fdef example
:args (s/cat :x int?)
:ret int?)
=> user/example
(s/valid? (:args (s/get-spec `example)) (list 1))
=> true
#2020-04-2317:17kennyYou can also
(s/def ::example-args (s/cat :x int?))
(s/fdef example
:args ::example-args
:ret int?)
#2020-04-2317:53friczeyeah, I can do that but can I do something like
(s/def ::example-fn-spec
{:arg (s/cat :x int?)
:ret int?})
(s/fdef example-fn ::example-fn-spec)
(s/fdef example-fn-2 ::example-fn-spec)
?#2020-04-2317:54kennyNo#2020-04-2317:54friczelikeā¦ most things in Clojure are easily movable and spec is not always like that. Iām curious why?#2020-04-2317:55friczeit seems to work a bit opposed to rest of Clojure design#2020-04-2319:42nikolavojicicI think in spec2 it will be (or it is already?) possible to transform spec -> map and vice versa.#2020-04-2323:49pbrownIn spec1 s/def
can refer to another spec by fully qualified symbol too, so this should work:
(s/fdef f1 ...)
(s/def f2 `f1)
#2020-04-2702:03arnaud_bosThere's an example in spec-alpha2's wiki to illustrate select from a literal schema with unqualified keys:
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
;;=> ({:b :C, :a -1}
;; {:b :f_/s!, :a -1}
;; {:b :J8.M+/+88, :a -1}
;; {:a -2}
;; {:b :IEcw.l?/X, :a -1})
But at the REPL this is what I get:
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
I'm on 8498f9cb352135579b6d3a0a5d15c40e5c2647ce
, which seems to be the latest commit on master.
Sorry if it's already been reported before. I don't really know where to look for stuff like this.#2020-04-2702:11Alex Miller (Clojure team)yeah, there's a bug in this area#2020-04-2714:28dspiteselfI am sure you have already seen this, but unqualified keys are not being looked at for closed specs in spec-alpha2
8498f9cb352135579b6d3a0a5d15c40e5c2647ce
(s/register
:blah/name2
(s/resolve-spec 'clojure.core/string?))
(s/register
:blah/Object2
(s/resolve-spec
`(s/schema {:name :blah/name2})))
(s/explain :blah/Object2 {:name "hi"} )
;-> Success!
(s/explain :blah/Object2 {:name "hi"} {:closed #{:blah/Object2}})
;-> {:name "hi"} - failed: (subset? (set (keys %)) #{}) spec: :blah/Object2
https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/alpha/spec/impl.clj#L452#2020-04-2714:32dspiteselfCreating Specs Programmatically is so pleasant. I have really been enjoying spec-alpha2.#2020-04-2714:42Alex Miller (Clojure team)I'm sure there are any number of bugs in the newer stuff right now, it's not done#2020-04-2714:50dspiteselfyep I was just checking if it would be valuable to report it. Thanks for the awesome work.#2020-04-2809:23arnaud_bosThere's this line in spec-alpha2's wiki page on "Schema and select" (emphasis mine):
> General form: (s/select schema selection)
> ā¢ schema (required) - can be a registered schema name, schema form (like s/union), or literal schema
But I can't get the "schema form" to work.
(in-ns 'user)
=> #object[clojure.lang.Namespace 0x54029c23 "user"]
(s/def ::a int?)
=> :user/a
(s/def ::b keyword?)
=> :user/b
;; Literal schema: OK
(gen/sample
(s/gen
(s/select
[::a ::b]
[*]))
2)
=>
(#:user{:a -1, :b :B/d}
#:user{:a -1, :b :Q/r})
;; Schema name: OK
(s/register ::ab (s/schema [::a ::b]))
=> :user/ab
(gen/sample
(s/gen
(s/select
::ab
[*]))
2)
=>
(#:user{:a -1, :b :B/_}
{})
;; Union name: OK
(s/register ::ba (s/union [::a] [::b]))
=> :user/ba
(gen/sample
(s/gen
(s/select
::ba
[*]))
2)
=>
(#:user{:b :./l?}
#:user{:a -3})
;; s/schema form: Err
(gen/sample
(s/gen
(s/select
(s/schema [::a ::b])
[*])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1814$fn$G (protocols.clj:20).
No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.lang.PersistentList
;; s/union form: Err
(gen/sample
(s/gen
(s/select
(s/union [::a] [::b])
[*])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1814$fn$G (protocols.clj:20).
No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.lang.PersistentList
Looks like I've messed something up wrt "symbolic" vs "object"?#2020-04-2812:52Alex Miller (Clojure team)No, just bugs I think#2020-04-2812:53Alex Miller (Clojure team)Spec 2 is not ready for use#2020-04-2814:32arnaud_bosYeah, sorry to make you feel like you have to repeat it again and again. I'm just toying around š#2020-04-2814:32Alex Miller (Clojure team)the whole schema/select impl is due for a rewrite, just don't have time to work on it right now#2020-04-2814:37arnaud_bosI see, thank you š#2020-04-2814:35flipmokidHi all, I'm playing around with the lastest spec alpha 2. I'm not sure if I'm doing something wrong but:
(spec/def ::tag (spec/and string? (spec/conformer clojure.edn/read-string str)))
(spec/def ::value string?)
(spec/def ::kv (spec/cat :tag ::tag :value ::value))
(spec/def ::fix-message (spec/and string?
(spec/conformer (fn [x] (map #(clojure.string/split % #"=") (clojure.string/split x #"\01"))))
(spec/+ ::kv)))
Should work with a FIX input string (a bunch of k=v pairs joined by ASCII code 01) but currently fails with
["8" "FIX.4.4"] - failed: string? in: [0] at: [:tag] spec: :cme-fix.spec/tag
However, adding an additional predicate into ::kv
makes it work:
(spec/def ::tag (spec/and string? (spec/conformer clojure.edn/read-string str)))
(spec/def ::value string?)
(spec/def ::kv (spec/and (constantly true) (spec/cat :tag ::tag :value ::value)))
(spec/def ::fix-message (spec/and string?
(spec/conformer (fn [x] (map #(clojure.string/split % #"=") (clojure.string/split x #"\01"))))
(spec/+ ::kv)))
Am I doing something obviously wrong?#2020-04-2818:31FranklinHey guys š , I'm tring to get started on using clojure.spec and I hit an unexpected/frustrating roadblock...#2020-04-2818:31Franklinuser=> (s/exercise false?)
Execution error (FileNotFoundException) at user/eval1428 (form-init5827540222428447763.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
#2020-04-2818:32FranklinThis error is thrown for any spec for which I call s/exercise
#2020-04-2818:32FranklinAny help with be appreciated#2020-04-2818:33jaihindhreddyYou need to add test.check
library as a dependency to your classpath#2020-04-2818:33Franklincool... I'll go ahead and do that#2020-04-2818:33jaihindhreddyspec uses it for generation#2020-04-2818:37Alex Miller (Clojure team)this is covered in the spec guide btw https://clojure.org/guides/spec#2020-04-2818:37Franklinoh... cool.. I didn't actually read that.. I'm following some tutorials... will go through that..#2020-04-2818:37Franklin@jaihindhreddy That worked š š#2020-04-2819:19kennyIs there a spec collection macro that works with eduction?#2020-04-2819:20kennye.g., something that returns true here:
(s/valid? (s/coll-of int?) (eduction (map inc) (range 10)))
=> false
#2020-04-2819:25shooitI donāt know if there is a spec that you can use but you could alternatively call seq
on your eduction to turn it into a LazySeq which would then pass the spec#2020-04-2819:29dominicm@kenny I'm not sure, but I don't think eduction returns a collection, it returns a reducible :). That doesn't help but it does indicate that coll-of probably isn't the right tool.#2020-04-2819:30kennyRight. I'm curious what folks use to validate against it. Hmm, I could do that @shewitt!#2020-04-2819:32Alex Miller (Clojure team)eductions are suspended computations so there is no data to validate without forcing it#2020-04-2819:34kennyThis use case is for in a test so I think @shewitt's idea solves it. In general, it probably wouldn't make sense to call valid? on an eduction for that reason, right? Even if checked with s/every, it'd still need to recompute *coll-check-limit*
when you finally use the eduction elsewhere.#2020-04-2819:42Alex Miller (Clojure team)Right#2020-04-3002:24bbrinckI noticed that instrumentation doesnāt work for certain core functions. Is this related to the inline
metadata?
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as st])
(s/fdef clojure.core/count :args (s/cat :coll coll?))
(s/fdef clojure.core/reverse :args (s/cat :coll coll?))
(st/instrument ['clojure.core/count 'clojure.core/reverse])
(reverse 1) ; gives spec error
(count 1) ; gives non-spec error
#2020-04-3003:52Alex Miller (Clojure team)yes#2020-04-3003:54Alex Miller (Clojure team)you will also find that adding specs to core functions will not be checked in calls from one core function to another as core is compiled with direct linking#2020-04-3015:16bbrinckThanks! Thatās good to know. #2020-04-3015:20bbrinckThe reason why the āinlineā limitation comes up is Iām thinking about ways to use instrumentation to help provide some different error messages when using core functions, but this means that some functions cannot be instrumented. #2020-04-3015:22bbrinckI wonder if thereās a way around this. I can also see how many functions are actually inlined. Maybe itās not a huge amount in practice.#2020-04-3015:23bbrinckI may be misremembering, but I seem to remember some mention of spec2 instrumentation changing things. Do you have any idea if this inline limitation will remain in the new instrumentation?#2020-04-3015:23Alex Miller (Clojure team)no difference in how instrumentation is implemented#2020-04-3015:33bbrinckOK, thanks. A quick search in clojure.core
indications about 85 cases of :inline
. Maybe itās not a huge deal to just skip those functions or maybe thereās a non-instrumentation-based way to help with errors. Iāll think more.#2020-04-3015:39Alex Miller (Clojure team)the biggest thing to be concerned about is probably performance. Some of the tests they did with https://github.com/borkdude/speculative made instrumented core pretty much unusable.#2020-04-3017:58bbrinckAgreed. I tried instrumentation on my test suite and it was way too slow. I have an idea about a different way to instrument that might be better wrt performance (basically by only doing spec checking if there is an error) but the inline limitation will require some additional thinking #2020-04-3018:16Alex Miller (Clojure team)Sounds like a good idea#2020-04-3016:18Frank HenardTo use spec.alpha2, do I need to clone the repo? I don't see it in maven-central#2020-04-3016:20Alex Miller (Clojure team)it has not yet been released#2020-04-3016:20Alex Miller (Clojure team)there are instructions in the readme on how to use it as a git dep#2020-04-3016:21Alex Miller (Clojure team)it is still a wip, actively changing, and buggy. you have been warned. :)#2020-04-3016:21Frank Henardunderstood, thanks!#2020-05-0320:02oskarkvI tried one of the examples on http://clojure.org, namely this one:
(defn adder [x] #(+ 1 x %))
(s/fdef adder
:args (s/cat :x number?)
:ret (s/fspec :args (s/cat :y number?)
:ret number?)
:fn #(= (-> % :args :x) ((:ret %) 0)))
And I did did
(st/instrument `adder)
If I pass in a string, I get a complaint. But as you can see, I added a 1
in the body of adder
so that the :fn check wouldn't hold. But nothing happens when I call adder. What's going on?#2020-05-0320:16Alex Miller (Clojure team)https://clojure.org/guides/faq#instrument_ret#2020-05-0322:45oskarkvThanks!#2020-05-0722:43dvingois there any way to abstract a spec?
(defmacro to-many-ref-type
[spec]
`(s/coll-of (s/or :id u/id? :ident ::ident ~(keyword (name spec)) ~spec) :type vector?)))
(s/def :habit/tasks (to-many-ref-type ::task))
i'm trying this, but I think the macroexpansion of s/def doesn't like it#2020-05-0722:45dvingoactually, i think it's working - my issue is it's not working in cljs, I'll dig into it#2020-05-0919:17aisamuCLJS macros can be a bit finicky! Check https://code.thheller.com/blog/shadow-cljs/2019/10/12/clojurescript-macros.html#2020-05-0919:22dvingoThat was my issue! I was trying to use a cljc file and updated them to separate cljs and clj files. it's working now š#2020-05-1019:49Amir EldorHello, rather new to this. How would you instrument specs only in development? I'm using Leiningen, I guess there's some way to check what profile we're running? If so, would you do this (if) at the bottom of each namespace with specs??#2020-05-1021:37seancorfield@amir In several of my projects, I have the tests run with instrumentation in place, so each test file has a test fixture that turns instrumentation on (or just a top-level form that does it as the test file is loaded). Since I tend to be loading/running tests a lot during development, that usually means I have instrumentation turned on during development -- but it's easy enough to just eval an instrument
call from a comment
in your source file.#2020-05-1021:38seancorfieldNot sure what profiles have to do with that. If I'm developing, I'm working with a REPL. If I'm testing, I can have the tests turn instrumentation on. Otherwise the code won't turn instrumentation on by default, which is the behavior I want.#2020-05-1021:39seancorfieldWe use Specs heavily in production as well -- see https://corfield.org/blog/2019/09/13/using-spec/#2020-05-1417:10dvingothis is super useful, thanks for the post! I have a use case where I want to generate some crud operations as well for some map specs - do you have any pointers for functions you used to parse the specs? I see s/describe
and s/form
did you use those or something else?#2020-05-1417:38seancorfieldJust those. To get at the primary list of required/optional keys.#2020-05-1417:41dvingoperfect, thanks!#2020-05-1109:50Petrus TheronHow do I use Spec with core.async? E.g. how do I spec a function that returns a channel where I'll put! another spec on? Or do I need to wrap it in some kind of record for this?#2020-05-1116:45jkxyzyou can spec that something is a Channel, but thatās not very useful, so you probably want to spec the values at the points where you put/take them#2020-05-1620:19Petrus TheronYes, I realized that I could spec the channel transducers that interpret incoming values on put!.#2020-05-1122:44rafaelI'm seeing strange behavior in spec alpha2 (latest github commit). If I define a schema using the map form, to specify non-namespaced keywords, refer to it from another schema, and later try to generate data from a selection of latter schema, I get an "Unable to resolve spec" exception:
; Context
(s/def ::foo int?)
(s/def ::inner (s/schema {:foo ::foo}))
(s/def ::outer (s/schema [::inner]))
; Works as expected:
(s/exercise ::outer)
(s/exercise (s/select ::inner [*]))
; Fails with "Unable to resolve spec: :foo"
(s/exercise (s/select ::outer [*]))
#2020-05-1122:44rafaelIs this expected, should I change something in my code, or is it a bug that I should report somewhere?#2020-05-1123:15seancorfieldSpec 2 is very much a work in progress and isn't ready for folks to use -- and, yes, it has numerous bugs in it @rafael#2020-05-1123:25rafaelYeah, so far I'm finding workarounds. I think I can work around this one as well with s/keys
#2020-05-1202:09seancorfieldAs long as you're not building anything with it that is more than a personal toy project @rafael...#2020-05-1209:22fmnoisehi everyone, is it possible to add meta to spec?
(s/def ::my-custom-int ^{:error-fmt #(str % " is not valid int")} int?)
when I call (meta (s/get-spec ::my-custom-int))
it doesn't contain my meta#2020-05-1209:26hkjels@fmnoise I donāt know the answer, but Iām pretty sure youāre checking the wrong thing. Try (meta ::my-custom-int)
#2020-05-1209:31vlaaad(s/def ::my-int (with-meta (s/spec int?) {:err-format #(str % " is not a valid int")}))
#2020-05-1209:32vlaaad@fmnoise then (meta (sp/get-spec ::my-int))
will work#2020-05-1209:56fmnoisethanks @vlaaad, this works!#2020-05-1212:45Alex Miller (Clojure team)While this may work, please that it wonāt work on all specs and some spec transformations like with-gen will not preserve meta. This is more accidental than intentional#2020-05-1212:49Alex Miller (Clojure team)Adding meta and doc string support to specs is the most requested thing in the tracker and I expect we will add it in spec 2#2020-05-1218:16sgerguriI have a design question - I have a Kafka Streams topology where I use conformance to automatically sanitise input, then split the stream into two - those where conformance resulted in ::s/invalid
and a happy path. In the happy path, I call a pure function that transforms the conformed message.
This transformer has a function spec attached to it, but because I have already conformed the input according to the arg spec before calling the function, it will fail the conformance check.#2020-05-1218:17sgerguriI seem to be facing two options - either remove the function spec, or create yet another spec for the already-conformed input, a "derived spec" of sorts. Neither really feels satisfactory.#2020-05-1218:18sgerguriYet another alternative would be to simply validate with s/valid?
earlier in the stream, then conform inside the pure function - which feels like the right thing to do, but it also feels like I am validating the data twice effectively. What would be the cleanest solution here?#2020-05-1218:59seancorfield@sgerguri The approach we've taken at work is to treat that initial Spec layer as a boundary and assume it maps from "external data format" to "internal data format" and is applied consistently. Thus everything "inside" is Spec'd in terms of the (already conformed) data produced at the boundary.#2020-05-1219:00sgerguriSo if I understand it correctly you have a second spec layer for the internal representation, that represents things that have been conformed, correct?#2020-05-1219:07seancorfieldYeah, an API Spec, a Domain Spec, and actually a Persistence Spec. The API Spec is conforming, from external data (often strings) to internal data (numeric, Boolean, date, etc). The Domain Spec describes the data formats used inside the API. The Persistence Spec describes the data that goes into the database (flat hash maps with JDBC-compatible types). We don't use all three in all cases, but it seems to have become a useful way to split things up -- and it clearly identifies two boundaries (API input and database output).#2020-05-1219:09sgerguriThat's excellent, thanks. I have used a similar approach in another service, the only thing I'm struggling with in this one is the fact that the internal layer is very thin and quite close to the input layer, thus it feels like duplicating the specs, but might be worth it for the greater clarity.#2020-05-1219:10mpenetspec-coerce is quite good for this too#2020-05-1219:11seancorfieldI would caution against leaning too heavily on conforming with Spec -- I don't really like what spec-coerce enables.#2020-05-1219:12mpenetBy default it will do the right thing to coerce x to your spec expeceted values, and the other way around (serializing to db type, ex keyword -> string) just requires to override those cases#2020-05-1219:12mpenetNo spec duplication that way#2020-05-1219:12mpenetNo polluted specs either (no conform abuse)#2020-05-1219:13sgerguriI have used it for some fairly tricky transformations in the past, but I've found it really works well for emulating pattern matching along with multimethods by tagging individual cases through an s/or
. In this particular example I only wanted to trim whitespace but for some reason am having trouble fitting it neatly in without overhauling one part of the service or another.#2020-05-1219:15sgerguriThis particular situation also left me wondering whether conformance should either be the final station in your transformation/validation stack or whether it should yield something that also has a spec for it (without further conformers), like @seancorfield described.#2020-05-1219:15sgerguriIn the grand scheme of things I could just trim the fields I need to but with spec around that just feels like the wrong approach.#2020-05-1219:16seancorfieldThere are folks on both sides of that decision š#2020-05-1219:17seancorfieldSome feel that input should be cleaned and parsed first, then checked for validity with Spec (and therefore next to no conforming). Others feel that input conformance to valid data is a reasonable use of Spec.#2020-05-1219:17mpenetI used to have dual specs like you mention, but it requires more work and is more brittle imho#2020-05-1219:18mpenetThen as you say ymmv#2020-05-1219:18seancorfieldI am slightly on the conforming side of center. spec-coerce is quite a lot further out on the conforming side. I think Alex (and maybe others at Cognitect) are on the non-conforming side of center.#2020-05-1219:18Alex Miller (Clojure team)I actually like spec-coerce :)#2020-05-1219:19seancorfieldReally? I was sure you had complained about it in principle in the past? š#2020-05-1219:19Alex Miller (Clojure team)I don't like spec-tools#2020-05-1219:21sgerguriFood for thought. Thank you everyone, always a pleasure to come here for another opinion on how folk do things. š#2020-05-1219:26Alex Miller (Clojure team)spec-coerce builds a separate registry of coercions that leverages specs, which I think is a good approach#2020-05-1219:27Alex Miller (Clojure team)I haven't used it in anger but that seems in line with how I would approach it#2020-05-1219:29mpenetIt's gotten better lately, it was missing multi-specs, merge & tuple until recently but now it's quite feature complete#2020-05-1219:30mpenetIt's also 300ish lines of code, easy to modify, fit to your taste if needed#2020-05-1219:43seancorfield@alexmiller Ah, thanks for the clarification. Then I suspect I'm getting the two libraries confused and maybe I should look at spec-coerce
in more detail? š#2020-05-1220:06mpenetYou contributed to it a few years ago https://github.com/wilkerlucio/spec-coerce/pull/11 :)#2020-05-1221:40seancorfieldWow, I don't even remember that. Looks like I'd started to look at spec-coerce
as a possible alternative to our existing "web specs" at World Singles Networks -- and I guess by the time that PR got accepted I'd decided not to go down that particular path for some reason...#2020-05-1221:45seancorfieldI've made a note to revisit it, but here's what we ended up doing at work https://github.com/worldsingles/web-specs#2020-05-1220:31Amir EldorHello, I hope it's ok to post some code. I'm trying to spec a function that has ehmm, this optional arguments thing? I'm not sure how it's called in Clojure. I'm having some trouble with the spec for it.
Right now I specifically use (st/instrument)
in a test as suggested and the following code breaks with the following exceptions:
(ship (:id p)) ; (:id p) is surely a uuid
|| :cause "class java.util.UUID cannot be cast to class java.lang.Number (java.util.UUID and java.lang.Number are in module java.base of loader 'bootstrap')"#2020-05-1220:32Amir EldorIt seems like it's related to the ::speed thing, being an integer. I must have made something funny in the spec, if someone can notice. Thanks!#2020-05-1220:46Amir EldorAh yes, thet exception I gave is invoked on the s/def of ::speed š#2020-05-1221:35fmnoisewhat is (random-id)
?#2020-05-1221:40Amir EldorIt's my own function, which calls (UUID/randomUUID) wth java.util.UUID#2020-05-1221:43Amir EldorThe code itself works without the spec, I must be defining something badly#2020-05-1221:45fmnoisewhat about default-ship-speed?#2020-05-1221:46Amir Eldor(defonce default-ship-speed 20)
#2020-05-1221:51seancorfieldWhat is DateTime
?#2020-05-1221:51Amir Eldor(:import [org.joda.time DateTime]
[java.util UUID]))
From Java, too#2020-05-1221:51Amir EldorThough I use clj-time, so I might be wrong here when I think of it#2020-05-1221:52seancorfield'k... Joda Time is deprecated. If you're on Java 8 or later, you should use Java Time really.#2020-05-1221:52seancorfieldclj-time
is also deprecated.#2020-05-1221:52seancorfield(for the same reason)#2020-05-1221:52Amir EldorOh, thanks for letting me know. I'll google for Java Time#2020-05-1221:53fmnoise(s/def ::speed (fn [v] (and (int? v) (> v 0))))
this definition works#2020-05-1221:54fmnoisebut dunno why it assumes uuid
there by default when doing coercion#2020-05-1221:57seancorfieldThe problem is that all three optional arguments are optional independently#2020-05-1221:58seancorfieldSo dest-planet-id
could be omitted and the speed
and departure-time
could still be provided.#2020-05-1221:59seancorfieldIn other words, it tries to check (ship src-planet-id src-planet-id)
against a signature that is essentially [uuid? #(> % 0)]
and that's causing the exception.#2020-05-1222:00seancorfieldAnd that's why adding (and (int? %) ...)
into ::speed
"works" -- it guards the >
operation from being called on non-numeric values.#2020-05-1222:01seancorfieldBecause then the ::speed
spec successfully fails to match (instead of blowing up) and Spec goes on to try the other options.#2020-05-1222:01seancorfieldSo...#2020-05-1222:02seancorfieldI'd suggest spec'ing ship
a bit differently, perhaps using s/alt
over the four arities, each spec'd with no optional parameters.#2020-05-1222:03seancorfieldOr at least across the first three and leave just :departure-time
as s/?
#2020-05-1222:03Amir EldorThanks a lot! However I don't fully understand how this happens:
> dest-planet-id
Ā could be omitted and theĀ `speed`Ā andĀ `departure-time`Ā could still be provided.#2020-05-1222:05Amir EldorI'm sorry I have to go now as it's getting late. I hope it's ok if I ping you tomorrow in this thread if I still have some trouble. Thanks!#2020-05-1222:09seancorfieldYou have src dest? speed? time? -- each of those three are optional, so each could be omitted while the others are passed.#2020-05-1222:09seancorfieldso src speed is "valid", as is src time or src dest time or src speed time.#2020-05-1222:09seancorfieldThat's what your fdef
says.#2020-05-1222:10seancorfieldSo when Spec sees (ship src-planet-id src-planet-id
) it's going to try src speed
, src time
, and src dest
in some random order.#2020-05-1222:11seancorfieldSince your speed spec just tries to compare the value >
it will throw an exception if passed a non-number: which a UUID is.#2020-05-1222:11seancorfieldBecause Spec encounters an exception, it won't try the other options -- it just propagates the exception.#2020-05-1222:15seancorfield(! 629)-> clj -A:test -Sdeps '{:deps {clj-time {:mvn/version "RELEASE"}}}'
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s] '[clojure.spec.test.alpha :as st])
nil
user=> (import '(org.joda.time DateTime) '(java.util UUID))
java.util.UUID
user=> (defonce default-ship-speed 20)
#'user/default-ship-speed
user=> (defn random-id [] (UUID/randomUUID))
#'user/random-id
user=> (s/def ::id uuid?)
:user/id
user=> (s/def ::src-planet-id uuid?)
:user/src-planet-id
user=> (s/def ::dest-planet-id uuid?)
:user/dest-planet-id
user=> (s/def ::speed #(> % 0))
:user/speed
user=> (s/def ::departure-time #(or (nil? %) (instance? DateTime %)))
:user/departure-time
user=> (s/def ::resources #(>= % 0))
:user/resources
user=> (s/def ::ship (s/keys :req-un [::id ::src-planet-id ::dest-planet-id ::speed ::departure-time ::resources]))
:user/ship
user=> (defrecord Ship [id src-planet-id dest-planet-id speed departure-time resources])
user.Ship
user=> (s/fdef ship
:args (s/alt :arity1 (s/cat :src-planet-id ::src-planet-id)
:arity2 (s/cat :src-planet-id ::src-planet-id :dest-planet-id ::dest-planet-id)
:arityN (s/cat :src-planet-id ::src-planet-id :dest-planet-id ::dest-planet-id :ship-speed ::speed :departure-time (s/? ::departure-time)))
:ret ::ship)
user/ship
user=> (defn ship
([src-planet-id]
(ship src-planet-id src-planet-id))
([src-planet-id dest-planet-id]
(ship src-planet-id dest-planet-id default-ship-speed))
([src-planet-id dest-planet-id ship-speed]
(ship src-planet-id dest-planet-id ship-speed nil))
([src-planet-id dest-planet-id ship-speed departure-time]
(->Ship (random-id) src-planet-id dest-planet-id (if (nil? ship-speed) default-ship-speed ship-speed) departure-time 0)))
#'user/ship
user=> (st/instrument)
[user/ship]
user=> (ship (random-id))
#user.Ship{:id #uuid "3ad92050-2b9b-499e-b7e3-f933053d7b1c", :src-planet-id #uuid "79cd0367-d78d-4774-9f5f-3a29a8c77851", :dest-planet-id #uuid "79cd0367-d78d-4774-9f5f-3a29a8c77851", :speed 20, :departure-time nil, :resources 0}
user=>
#2020-05-1222:15seancorfield^ That shows the s/alt
structure that would work @U0140AKS332#2020-05-1306:38Amir EldorThanks @seancorfield for the detailed answer and thanks @fmnoise for helping out too!#2020-05-1310:36fmnoisethat was interesting case, thanks @U0140AKS332 and @seancorfield#2020-05-1314:09oskarkv(s/def ::fact
(s/cat :var (s/? ::?symbol)
:type (s/or :simple symbol? :acc ::acc-vec)
:destruct (s/? vector?)
:exprs (s/* list?)))
(s/def ::fact-without-var
(s/cat :type (s/or :simple symbol? :acc ::acc-vec)
:destruct (s/? vector?)
:exprs (s/* list?)))
Is there a way to get rid of the repetition here without introducing more nesting and keywords in the conformed data?#2020-05-1314:12oskarkvThere are always macros of course but I was hoping for something easier.#2020-05-1314:13Alex Miller (Clojure team)doesn't the first one include the second?#2020-05-1314:16oskarkvBut I want a spec for data where a var is forbidden. And one where it's optional.#2020-05-1314:16Alex Miller (Clojure team)ah#2020-05-1314:16oskarkvI'm not sure I understand what you meant.#2020-05-1314:17Alex Miller (Clojure team)so, you can create a new spec that is an s/cat that just contains the common portions, then reference it in both ::fact specs#2020-05-1314:18Alex Miller (Clojure team)(s/def ::fact-without-var
(s/cat :type (s/or :simple symbol? :acc ::acc-vec)
:destruct (s/? vector?)
:exprs (s/* list?)))
(s/def ::fact
(s/cat :var (s/? ::?symbol)
:tail ::fact-without-var))
#2020-05-1314:19Alex Miller (Clojure team)that does introduce nesting in the conformed data so is not everything you want#2020-05-1314:19oskarkvYeah š#2020-05-1314:20Alex Miller (Clojure team)another way would be to restrict the bigger one#2020-05-1314:21Alex Miller (Clojure team)maybe something like
(s/def ::fact
(s/cat :var (s/? ::?symbol)
:type (s/or :simple symbol? :acc ::acc-vec)
:destruct (s/? vector?)
:exprs (s/* list?)))
(s/def ::fact-without-var
(s/& ::fact #(nil? (:var %))))
#2020-05-1314:22Alex Miller (Clojure team)can't say I've done this before, might need to tweak that predicate more#2020-05-1314:23oskarkvHm, gotta read up on &
. Thanks!#2020-05-1314:24Alex Miller (Clojure team)& just lets you and arbitrary predicates into the regex spec#2020-05-1314:24oskarkvAh, I see. Could work. Thanks!#2020-05-1419:08Amir EldorHow do you define a map spec with keys of similar spec? e.g.:
(s/def potato-name string?)
(s/def potato-happiness number?)
;; This is a potato:
{:name "potato name"
:happiness-1 12
:happiness-2 54}
Similarly, I see that (s/keys) requires a fully qualified name for each key and it's strange to me. What if I got two specs with the same key but different specs?#2020-05-1419:49Alex Miller (Clojure team)the idea behind spec is to give attributes global semantic meaning, by using namespaced attributes to differentiate and not do that#2020-05-1518:54Amir EldorSo it's not intended, for example, to use things like ::mygameobject-id on other namespaces of specific 'gameobject' each?
Among the things I'm playing around with is having a ::gameobject-id
defined in some-ns.game
and use that in some-ns.game.dog
and some-ns.game.bat
, each of which has an :id
which is a gameobject-id. Am I doing things in a non-Clojure way?#2020-05-1519:06Alex Miller (Clojure team):: means to auto-resolve the keyword in the context of the current namespace#2020-05-1519:07Alex Miller (Clojure team)so if you're using ::id in different namespaces, you'll get different keywords specific to each namespace, and there is no conflicting name#2020-05-1519:08Amir Eldorand if I want to use the same spec definition for each? just use a common function?#2020-05-1519:08Alex Miller (Clojure team)you can (s/def ::id ::common/gameobject-id)
in each namespace#2020-05-1519:08Alex Miller (Clojure team)so have one common spec and then make namespaced specs that refer to it#2020-05-1519:10Alex Miller (Clojure team)removing the potentially confusing auto-resolving keywords, something like:
(s/def :common/gameobject-id string?)
(s/def :some-ns.game.dog/id :common/gameobject-id)
(s/def :some-ns.game.bit/id :common/gameobject-id)
#2020-05-1519:12Amir EldorI was trying to do just that and it didn't work, and not it works. Great. Maybe I had a funky repl state before :man-shrugging:
Thanks!#2020-05-1519:12Alex Miller (Clojure team)good!#2020-05-1519:14Amir EldorI see that when I do something like
(s/def ::thingy (s/keys :req-un [:common/gameobject-id]))
#2020-05-1519:14Amir EldorThen the thingy gets an :id
property, and not { :common/gameobject-id "whatever" }#2020-05-1519:14Amir EldorWhich is good for me#2020-05-1519:15Amir Eldor(or maybe :gameobject-id
, I am mixing up some namings between this chat and my actual code)#2020-05-1519:16Alex Miller (Clojure team)it doesn't "get" a :gameobject-id property, it describes a map expected to have a :gameobject-id key which matches the :common/gameobject-id spec#2020-05-1519:21Amir EldorI meant that when I conform something to ::thingy, the keyword in the conformed map is { :gameobject-id "something" }
and not { :common/gameobject-id "something" }
. I guess it omits the namespace from the keyword?#2020-05-1519:21Alex Miller (Clojure team):req-un means "required unqualified key"#2020-05-1519:22Alex Miller (Clojure team)(as opposed to :req)#2020-05-1519:22Alex Miller (Clojure team)so more importantly it's validating the incoming map expecting an unqualified key#2020-05-1519:22Alex Miller (Clojure team)conforming just follow that#2020-05-1519:23Amir EldorI seeeee I made a thingy2 now with :req and it wants the fully qualified key#2020-05-1519:23Amir EldorInteresting and cool.#2020-05-1519:40Amir EldorDo I have to require common for using :common/gameobject-id or something?#2020-05-1519:43Amir EldorOh nevermind.#2020-05-1519:52Alex Miller (Clojure team)no#2020-05-1601:02Kyle BrodieWhats the best way to spec numbers as strings so something like "12.53" will be generated?#2020-05-1602:21seancorfield@kyle Take a look at https://github.com/worldsingles/web-specs#2020-05-1602:22seancorfieldIt has specs that accept strings or longs and generate as strings that are parseable as longs. Should give you something to work from.#2020-05-1602:49Kyle Brodie@seancorfield Thank you! That library looks very helpful. I might have to write my own date though because mine are "YYYY-MM-DD". Unless "yyyy-M-d" is the right format for zero padded in java-time#2020-05-1603:09seancorfieldYeah, it'll do zero-padded. And you want yyyy
and d
lowercase for dates.#2020-05-1611:41Kira McLeanHi there.. Iām just getting started with spec, have a somewhat noob question that Iām having trouble finding an answer to: Is it possible to have two different specs for the value of a given un-qualified keyword? I.e. I have a map and I know that the values might have different requirements in different contexts, but I would like to keep re-using the same key. So I have specs like this (`fs` is [datoteka.core :as fs]
):
(s/def ::path fs/path?)
(s/def ::html-path (s/and ::path (partial matches-ext "html")))
(s/def ::content string?)
;; This one is great -- checks my maps like {:path ,,, :content ,,,}
(s/def ::page (s/keys :req-un [::path ::content]))
;; Here is the issue: I want to check a map that also has the keys [:path
;; :content], not one with an html-path key
(s/def ::html-page (s/keys :req-un [:path ::html-path ::content]))
In the ::html-page
spec I want :path
to match the html-path spec but for the key to still be called :path
. Is this possible?#2020-05-1611:51ghadiIf you define two specs, :a.b.c/path
and :x.y.z/path
you can use them to spec unqualified keys in different maps#2020-05-1616:14Kira McLeanMakes perfect sense, thanks.#2020-05-1618:17David PhamAre there good library to use with clojure.spec? š#2020-05-1618:49salam@neo2551 you can find spec libraries (along with other libraries) here: https://www.clojure-toolbox.com/#2020-05-1620:57Kira McLeanCan you write a function spec for a multi-method? If so, any chance anyone can share an example of this somewhere?#2020-05-1621:31favilaYou can spec the method, but afaik not the implementations#2020-05-1621:32favilaCaveat with fspec on mm: instrumentation changes the type to a fn, so future defmethods will fail with a type error#2020-05-1621:41favilaYou need to unstrument before evaluating more defmethods #2020-05-1623:29Kira McLeanInteresting.. good to know. Yeah Iām curious about whether itās possible to spec the implementations of a multimethod. I couldnāt find any examples online, but so far Iāve found with clojure that doesnāt always mean itās not possible.#2020-05-1623:41Alex Miller (Clojure team)You can spec the dispatch method used in the mm if thatās useful#2020-05-1623:42Alex Miller (Clojure team)But mms themselves, no#2020-05-1623:42Kira McLeanThatās something! But good to know, thanks!#2020-05-1900:04telekidIām not sure if this is useful or relevant to your goals, but: https://gist.github.com/telekid/f2e588718dbdfe48306d64e5388bdc15#2020-05-2121:33Kira McLeanAh this is really cool! Never thought of separating out the args and mm to spec separately. Perhaps a little unorthodox or is this normal? But definitely interesting. Thanks!#2020-05-1812:41FranklinHow can I fix this:#2020-05-1812:41FranklinError Error: Require clojure.test.check and clojure.test.check.properties before calling check.
#2020-05-1812:42FranklinI get this when I call (stest/check `filter-list)#2020-05-1812:45Franklinpicard-facepalm never mind, I should have just read the error messsage#2020-05-1813:09mpenet@seancorfield @alexmiller @sgerguri fyi we released our internal fork of spec-coerce https://github.com/exoscale/coax#2020-05-1813:10sgerguriNice, I'll have a look today - appreciate the heads up!#2020-05-1818:07Felipe MarquesHi, there, I'm using spec-tools with a multi-spec, and I get the following error when trying to coerce some data:
TypeError: Cannot read property 'cljs$core$IFn$_invoke$arity$2' of null
at type_priority (/js/cljs-runtime/spec_tools.parse.js:270:22)
at eval (/js/cljs-runtime/cljs.core.js:7726:102)
at eval (/js/cljs-runtime/cljs.core.js:7638:85)
at stableCompareFn (/js/cljs-runtime/goog.array.array.js:640:12)
at Array.sort (<anonymous>)
at Object.goog.array.sort (/js/cljs-runtime/goog.array.array.js:626:7)
at Object.goog.array.stableSort (/js/cljs-runtime/goog.array.array.js:642:14)
at Function.eval [as cljs$core$IFn$_invoke$arity$2] (/js/cljs-runtime/cljs.core.js:7686:12)
at Function.eval [as cljs$core$IFn$_invoke$arity$3] (/js/cljs-runtime/cljs.core.js:7724:23)
at Function.eval [as cljs$core$IFn$_invoke$arity$2] (/js/cljs-runtime/cljs.core.js:7720:26)
Does anyone have a similar problem?#2020-05-1906:01joefromcthi, quick question, given a spec ::my-map
based on req
and opt
how could i get a list of the keys that this spec pertains to?
Should i be parsing s/form
or is there an easier way?#2020-05-1908:56ikitommi@marques.goncalves.fel havenāt seen that. please write an issue to spec-tools and you could check also spec-coerce or coax - many libs doing mostly/exactly the same.#2020-05-1909:26mpenetš
vs š
#2020-05-1909:28mpenetI haven't tested extensively the cljs part personally, tests pass but that's it for now#2020-05-1912:01jrychterThis just bit me while writing :post
conditions, where I wanted to assert that the return value was (among other things) a "status", but did not want to assert anything else: this is at an edge API where data has been transformed into external representations (timestamps to ISO8601 date strings, for example):
(s/def ::a string?)
(s/def ::b string?)
(s/def ::foo (s/keys :req [::a ::b]))
;; Let's define ::bar, which is not concerned with ::b at all:
(s/def ::bar (s/keys :req [::a]))
;; This is expected:
(s/valid? ::foo {::a "a" ::b "b"}) => true
(s/valid? ::foo {::a "a" ::b 1}) => false
;; But even though ::bar is not concerned with ::b, validation fails. I was hoping to only check if something
;; is a valid ::bar, without asserting anything about keys that are not in :req or :opt in ::bar.
(s/valid? ::bar {::a "a" ::b 1}) => false
I expected the last form to return true
, to this day I did not realize that spec is eagerly checking all keys, not just those listed by s/keys
.#2020-05-1921:06EddieGood to know! Thanks for mentioning it.
It seems odd to me that we would associate a value that is not a valid ::b
with the very specific qualified keyword of ::b
in a map. Perhaps this indicates that in the :bar
maps should be using a different key, like :b
.#2020-05-1913:06Alex Miller (Clojure team)all keys are checked by s/keys (even (s/keys)
is a valid and useful spec)#2020-05-1913:07Alex Miller (Clojure team)if you want to limit, you could select-keys on the data first, or validate the attributes specifically#2020-05-1920:40jrychterThat was unexpected, I somehow got used to reading s/valid?
as "tell me if this thing is a valid ::bar". Especially since it takes a parameter ā (s/validate {:data :foo})
is intuitively different from (s/valid? ::foo {:data :foo})
.#2020-05-1920:42jrychterPerhaps I'm using it wrong, but I would find the ability to check just "is this a valid ::bar
?" very useful.#2020-05-1921:09seancorfield@jrychter That makes sense for unqualified keys but part of the idea behind Spec for qualified keys is that they are globally unique (within your program) and therefore if :foo/bar
is present in a hash map, it should conform to the Spec for :foo/bar
if any. s/keys
is more about required/optional anyway (and that's emphasized in Spec 2 via schema
and select
).#2020-05-1921:28Alex Miller (Clojure team)the big idea is that spec gives named attributes global semantics#2020-05-1921:28Alex Miller (Clojure team)if you have global semantics, it's ok (and good) to validate them when you see them#2020-05-1921:29Alex Miller (Clojure team)if that's bad then either the spec is bad (b/c it doesn't fully encompass what that attribute can mean), or your data is bad (b/c it doesn't conform to the global semantics) or your version of "valid" does not match spec's semantics :)#2020-05-2005:12David PhamIs it possible to share spec over wire? Or to serialize them?#2020-05-2005:14seancorfield@neo2551 You can get the code form of a spec with s/form
-- and encode that send it over the wire -- but any predicates are going to depend on the code itself (anything beyond simple core predicates).#2020-05-2005:14seancorfieldI'm not sure how easy it is in Spec 1 to reconstitute a spec from its form tho'. That's something that is easier with Spec 2.#2020-05-2012:49Alex Miller (Clojure team)Just eval the form you get#2020-05-2005:15David PhamGreat! Thanks a lot!#2020-05-2007:24jrychterHmm. I think I understand the "global semantics" goals, but what about the specific case I have now? I think I am falling into answer (3) in Alex's list above. I am preparing API responses and I do want to preserve the same key names, as they will be sent to the API callers somewhere down the way. However, those responses fall outside of my app semantics boundary, and some fields will contain data in a different form (like timestamps). That is OK, but I would still like to verify some aspect of those responses ā whether they are a valid ::status/status
, for example.
I think there is also a conflict with the "open maps" idea that Rich has been talking about. I would like to be able to validate data as being valid in terms of the supplied spec, and whatever else is in the map should not matter. I do understand the global semantics idea, just pointing out that these two might sometimes clash.
I would find the ability to check only "is this a valid ::bar ?" very useful.
Anyway, just wanted to describe the use case, as material for thinking.#2020-05-2018:03seancorfield@jrychter FWIW, Spec 2 has support for checking specs with closed maps as an option (it's a check-time option, not a spec definition time option).#2020-05-2018:04seancorfieldAlso, if it helps: what we do at work is have different (but related) sets of specs for domain, API, and persistence -- since API and persistence are constrained by external systems whereas domain is not.#2020-05-2018:04seancorfieldThen we transform data at the boundaries between those layers as needed.#2020-05-2022:35Kyle BrodieI am trying to spec CSV rows that are parsed into Clojure maps but I'm hitting StackOverflowError in (s/keys :req-un [::kw1 ... ::kw156])
and right now the keywords are all (s/def ::kwN string?)
. I mostly want this for generating examples while developing so I don't have to use spec. It works if I give Java more stack space (I'm giving it -Xss512k
to match Heroku's smallest dynos) but I think I'm using spec wrong#2020-05-2022:42Kyle BrodieI think the overflow is in expanding the and
macro for keys-pred at https://github.com/clojure/spec.alpha/blob/5a4834eeb2967b2eca382efe0afcb3f590bdb3c2/src/main/clojure/clojure/spec/alpha.clj#L466#2020-05-2022:46seancorfield@kyle That's certainly a lot of keys in a hash map to require all be present... I assume they have more meaningful names than kw1
, kw2
, etc?#2020-05-2022:50Kyle BrodieYeah its because the CSVs are denormalized so the columns are always there even if there is no data. I can get example data and make them into Clojure maps for my development. Just thought it would be cool to test out my functions with generated data#2020-05-2022:51Kyle BrodieThe data will be reduced to less keys after a few steps so those I should be able to spec then#2020-05-2022:51seancorfieldIf you wanted random key/value rows with random keywords and random strings, just for testing stuff, you could use
(s/def ::csv (s/map-of keyword? string? :min-count 156 :max-count 156))
#2020-05-2022:52seancorfieldI guess you could make a set of all the known keys and replace keyword?
there with the set.#2020-05-2022:53seancorfield(s/def ::csv (s/map-of #{:kw1 :kw2 ,,, :kw156} string? :min-count 156 :max-count 156))
I think that would work (you could make the set of keys into a separate spec)#2020-05-2023:01Kyle Brodie(s/def ::simple-row (s/map-of (set columns) string? :min-count 156 :max-count 156))
(s/exercise ::simple-row 1)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
class clojure.lang.ExceptionInfo
#2020-05-2023:53seancorfield@kyle Ah, yeah, that's because it is trying to randomly generate elements from the full set for each key but each key in the generated map needs to be distinct š#2020-05-2023:53seancorfieldYou'd have to wrap the key part spec with a generator that produces the full set (in some random order, if you cared).#2020-05-2023:55seancorfieldI suspect it would be easier to wrap the whole s/map-of
spec in a custom generator since all it really needs to do is zip together that set of keys with a bunch of random strings...#2020-05-2100:01Kyle BrodieSo I'd do (s/def (with-gen (my-spec-here) (fn [] (something-using-test-check-functions))))
#2020-05-2100:13seancorfield(s/def ::simple-row (with-gen (s/map-of ,,,) (fn [] (...))))
yup#2020-05-2212:46practicalli-johnI am assuming that if I am using spec keys
macro, I do not also need to check the data is a hash-map. So in the following code, the map?
predicate would be redundant. Can someone confirm?
(spec/def ::customer-details
(spec/and
map?
(spec/keys
:req [::first-name ::last-name ::email-address ::home-address ::social-secuirty-id])))
#2020-05-2212:47minimalyes itās redundant#2020-05-2212:54Alex Miller (Clojure team)s/keys does a map? check#2020-05-2220:08David PhamHello everyone. I really like Clojure.spec, it is really fun to use. I am currently wondering if there was a way to extract the default generator of a specs when using with-gen? The reason I am asking is I wish to provide more specific example than the default generator, but would still like to use the randomness of the default one. I know we can use gen/one-of, or gen/frequencies, but can we define the specific generator directly with with-gen? If not what would be a good/concise alternative to my problem, which is to define a spec with a custom generator which combine the default generator as well?#2020-05-2220:21kenny@neo2551 If I understand you correctly, we do this sort of thing a lot. Typically it's because of having a s/and. Here's an example of what we often have:
(s/def ::base-thing (s/keys :req [::a]))
(s/def ::thing
(s/with-gen (s/and ::base-thing pred1? pred2?) #(gen/fmap (fn [base-thing] ) (s/gen ::base-thing))))
#2020-05-2221:33David Pham@kenny It is more if you define the ::color
spec as keyword?
and provide the #{:blue :red}
as examples? In this case the base gen is concise, but in a more general case, I would need to copy/double the definition of my spec, will I?#2020-05-2221:35David PhamI would still like to retain the ability to generate random keywords, while using my blue and red examples. One solution is gen/one-of, but I donāt see how I can get the random one without copying the base spec again.#2020-05-2221:39kenny(def colors #{:blue :red})
(s/def ::color (s/with-gen keyword? #(gen/one-of [(s/gen colors) (s/gen keyword?)])))
?#2020-05-2221:41David PhamWhat if keyword?
was some really long spec?#2020-05-2221:42kenny(s/def ::color (s/or :predefined #{:blue :red}
:other keyword?))
#2020-05-2221:42kennyI'm not really sure I understand your problem š#2020-05-2307:18David PhamIn your answer with with gen/one-of, you repeated keyword? which was the main predicate. In that case you repeated the main predicate twice, once for defining the validity, once in the definition of the generator. Can we avoid this repetition?#2020-05-2307:19David PhamI think it is a common pattern to give some examples to a generator for documentation, but it reduce the power for testing.#2020-05-2321:13niclasnilssonRecursive specs? Iāve tried all old examples Iāve found, but they donāt seem to work (any longer?). In spec or spec2, are recursive specs possible, or is eager lookup making it impossible? I mean things like this:
(def ex '(+ (+ 1 (* 2 3)) 200))
(defn operator? [x] (#{'+ '- '* '/} x))
(s/def ::operator operator?)
(s/def ::operand int?)
;; I've tried many different ways, but fail to get something like this working
(s/def ::expr
(s/or :operand ::operand
:expr (s/cat :operator ::operator
:left ::expr
:right ::expr))
#2020-05-2413:56Charles Fourdrignier@niclasnilsson
You need to define a reference ::expr before the recursive one.
(s/def ::expr any?)
#2020-05-2506:32niclasnilsson@charles.fourdrignier, well, yes and no. It compiles then, but it still doesnāt work.
The following compiles, but the expected output is not useful, and not whatās expected from a recursive spec. (see output from explain and conform)
(require '[clojure.alpha.spec :as s])
(def expr '(+ (+ 1 (* 2 "three")) 200))
(defn operator? [x] (#{'+ '- '* '/} x))
(s/def ::operator operator?)
(s/def ::operand int?)
(s/def ::expr any?)
(s/def ::expr
(s/or :operand ::operand
:expr (s/cat :operator ::operator
:left ::expr
:right ::expr)))
(s/explain ::expr expr) ;; success, but shouldn't be if recursive specs works.
(s/conform ::expr expr)
;; result
[:expr {:left (+ 1 (* 2 "three")) :operator + :right 200}]
#2020-05-2507:43valeraukoi haven't tried this, but would an alias work? eg.
(s/def ::expression ::expr)
(s/def ::expr
; ...
:left ::expression
:right ::expresssion)))
#2020-05-2508:20Charles FourdrignierWhich Clojure did you use ?
Clojure (from 1.9.0) includes clojure.spec and the namespace is different.
(require '[clojure.spec.alpha :as s])
With this namespace, it works for me (got an invalid for your expr
).
Maybe a bugfix done on clojure repository and not in the clojure/spec.alpha ?#2020-05-2511:10niclasnilsson@UAEH11THP, nope, unforturnately doesnāt work.#2020-05-2511:11niclasnilsson@charles.fourdrignier, this is spec 2, but Iāve tried previously on the spec thatās in the box with 1.10 as well.#2020-05-2511:31Charles FourdrignierSorry, I don't get you were using Spec 2.#2020-05-2511:40niclasnilssonBut it seems to work in spec 1, even without the any?
trick. Very odd?#2020-05-2511:44Charles FourdrignierYes.
I use the any?
trick a couple of months ago in Spec 1.
Pretty sure some error push me to use it...
Very strange.#2020-05-2511:28niclasnilssonHmmā¦ But this actually seems to work in āspec 1ā, (clojure 1.10.1) but not in spec 2.
(require '[clojure.spec.alpha :as s])
(def bad-expr '(+ (+ 1 (* 2 "three")) 200))
(def expr '(+ (+ 1 (* 2 3)) 200))
(defn operator? [x] (#{'+ '- '* '/} x))
(s/def ::operator operator?)
(s/def ::operand int?)
(s/def ::expr
(s/or :operand ::operand
:expr (s/cat :operator ::operator
:left ::expr
:right ::expr)))
(s/valid? ::expr bad-expr) ; false
(s/explain ::expr bad-expr) ;
(s/conform ::expr bad-expr) ; :clojure.spec.alpha/invalid
(s/valid? ::expr expr) ; true
(s/explain ::expr expr) ; success
(s/conform ::expr expr)
#2020-05-2512:45Alex Miller (Clojure team)There are some known issues with eager spec dereferencing in spec 2#2020-05-2513:20niclasnilssonAh, ok. Thanks @alexmiller.#2020-05-2619:37wagjoHi, using latest spec2 alpha, I get an exception, is this a known issue or am I doing something wrong?
(s/def :foo/a string?)
(s/def :foo/b :foo/a)
(s/def :foo/m (s/schema [:foo/a :foo/b]))
(s/valid? :foo/m {:foo/a "s" :foo/b "d"})
Unhandled java.lang.IllegalArgumentException
No implementation of method: :conform* of protocol:
#'clojure.alpha.spec.protocols/Spec found for class:
clojure.lang.Keyword
#2020-05-2619:44Alex Miller (Clojure team)it's a known issue for aliased specs#2020-05-2619:45Alex Miller (Clojure team)in that they don't work right#2020-05-2619:47wagjothanks, I'll use (s/def :foo/b (s/and :foo/a))
as a workaround for now#2020-05-2820:32David PhamSaying you a spec ::a
, you can get back the definition with s/form. Is it possible to extract a generator from the s/form?#2020-05-2820:46Alex Miller (Clojure team)sure, just eval to to get a spec object, then s/gen#2020-05-2820:47Alex Miller (Clojure team)but if you have ::a, you're already there, so not sure what you're driving at#2020-05-2820:48Alex Miller (Clojure team)one big caveat is that that s/form does not currently capture when a generator is overridden with like s/with-gen#2020-05-2904:29David PhamHow do You eval? I actually want the default generator because I override it, but for testing i would also like to test the generic one.#2020-05-2913:14Alex Miller (Clojure team)with the eval
function?#2020-05-2913:15Alex Miller (Clojure team)(s/gen (eval (s/form ::a)))
#2020-05-2919:30jacklombardHello, I want to introduce clojure spec to validate API endpoints. The parsed response naturally does not have namespaced keywords. How do I go about validating the data? Should I simply use unqualified keys? Where should I keep my spec?#2020-05-2919:31jacklombardI know it is a very common question of where to put spec but can't find the right answer#2020-05-2919:34Alex Miller (Clojure team)I answered these in #beginners - in the future, it's best to put questions in just one channel#2020-05-2919:55David PhamIs it not a bad thing to use the eval function?#2020-05-2920:16Alex Miller (Clojure team)it's not inherently bad, it's just a tool#2020-05-2920:17Alex Miller (Clojure team)Clojure evals all of your expressions after all#2020-05-2920:58practicalli-johnI have a simple function and an associated fdef specification, but am not getting an error when calling the function with incorrect arguments. The function should take a map that is a ::customer
specification. Have I misunderstood something?
(defn customer-fullname
"Return customer full name from customer details"
[customer-details]
(str (::first-name customer-details)
"_"
(::last-name customer-details)))
(spec/fdef customer-fullname
:args (spec/cat :customer ::customer)
:ret string?
:fn #(= (:ret %)
(str (::first-name :args) " " (::last-name :args))))
(spec/def ::first-name string?)
(spec/def ::last-name string?)
(spec/def ::email-address
(spec/and string?
#(re-matches #"^[a-zA-Z0-9._%+-]
I would have expected a call to the customer-fullname
to fail when the wrong kind of arguments are passed
(customer-fullname "customer")
#2020-05-2921:01practicalli-johnThe ::customer
spec works with spec/valid?
and spec/assert
when I use them with :pre
and :post
conditions in a function definition, but seem to be missing something when using spec/fdef
.#2020-05-2921:25practicalli-johnAh, I have now learned about instrumenting the fdef
specifications, and now it works, well fails when I expect it too...
(spec-test/instrument `customer-fullname)
#2020-05-2921:31practicalli-johnI am not clear on if it is valuable to use fdef
without instrumenting them. Some discussions suggest is it but I have not really understood why as yet. I am still not that clear on the :fn
aspect of fdef
so will be on the look out for more examples. I still have a lot to understand about spec.#2020-05-2921:34Alex Miller (Clojure team)you can see the specs in (doc customer-fullname)
#2020-05-2921:35seancorfield@jr0cket You can also spec-test/check
them, separate from instrumentation.#2020-05-2921:35Alex Miller (Clojure team)^^#2020-05-2921:35Alex Miller (Clojure team)those are the main benefits#2020-05-2921:36seancorfieldIn particular, instrumentation checks :args
passed in are correct (and ignores :ret
and :fn
). Check is generative and passes in conforming random arguments (per the :args
spec) and then checks :ret
and :fn
are satisfied.#2020-05-2921:38practicalli-johnI can see the specs in the docs, yes that is very useful (had forgotten that in my frustration to get the code to fail). Will try out spec-test/check
in the morning, sound promising. Thank you both.#2020-05-3111:07plexusI'm trying to figure out how to prevent my recursive spec-based generators from StackOverflowing. From what I can tell the checks that are built in based on *recursion-limit*
are moot as soon as you use with-gen
.#2020-06-0713:02Vincent CantinThatās really a coincidence, because I am currently working days and days on overcoming this problem.#2020-06-0713:02Vincent CantinYou might want to give Minimallist a try.#2020-06-0713:04Vincent Cantin@U07FP7QJ0 let me know if you want to try the pre-release repo.#2020-06-0810:27plexuswhat's your sales pitch? why should I consider Minimalist over Malli (my preferred solution nowadays), and does it solve the problem I linked to?#2020-05-3111:09plexusI currently have two places where I'm using with-gen
that cause recursion, https://github.com/lambdaisland/regal/blob/master/src/lambdaisland/regal/spec_alpha.cljc#L44-L47#2020-05-3111:09plexushttps://github.com/lambdaisland/regal/blob/master/src/lambdaisland/regal/spec_alpha.cljc#L85-L95#2020-05-3111:11plexusThe second one I could get away without the custom generator, it'll just do some extra work until it finds a valid value, but the first one is annoying. The spec boils down to (s/and (s/cat ...) vector?)
, but (s/cat ...)
never generates vectors so when generating this tries 100 times and gives up#2020-05-3111:11plexusmaybe there's a better way to write that? can I use regex specs but still convey that I really only want to match/generate vectors?#2020-05-3114:48Alex Miller (Clojure team)Not easily (this is something weāve added in spec 2)#2020-05-3115:41claytonHello!
I've recently done some reading on clojure spec and have added it to some side projects of mine, but I'm a little lost on how/when to use instrumentation (or orchestra). I've spec'd multiple functions in my project, but realize those are useless (aside from readability) without running some sort of instrumentation. Are there any resources explaining how to integrate instrumentation in a clojure project? I don't want to permanently leave a call to instrument inside my clojure files, and only want them to really be run during development phase. Aside from just running instrumentation in the REPL during the dev process, are there any best practices for instrumenting in my project? Thanks š#2020-05-3116:48Alex Miller (Clojure team)most useful in dev at the repl#2020-05-3116:49Alex Miller (Clojure team)but could also be useful to turn on/off in a fixture around tests#2020-05-3116:55claytonThanks! I've only ever used instrumentation in tests or the repl, but was thinking I was missing something. Back to the specs I go š#2020-05-3118:10seancorfield@UB4EZH42F In the tests for seancorfield/next-jdbc
I turn on instrumentation as part of test setup in several of the test namespaces.#2020-05-3118:11claytonwill check out the repo for examples - thanks!#2020-05-3118:11seancorfieldI talk about how we use Spec at work in https://corfield.org/blog/2019/09/13/using-spec/ if that helps.#2020-06-0101:21Eccentric JA friend shared this with me https://gizmo385.github.io/clojure/programming/2015/08/11/adts-in-clojure.html which ended with a macro for (data Tree = Empty | Leaf value | Node left right)
. I'm trying to write the spec for something like that just for learning purposes. I got to (s/def ::tree (s/or empty?))
before getting a bit stuck š
#2020-06-0101:24Eccentric JMy first question is can you compose specs together like (def ::leaf ...)
(def ::node ...)
(def ::tree (s/or empty? ::leaf :;node))
?#2020-06-0101:30seancorfieldI think you'd need "tagged" data structures and multi-spec for that @jayzawrotny#2020-06-0101:31Eccentric JAh thanks for the pointer!#2020-06-0101:54Eccentric JLooks like someone had started down that path https://github.com/bluesxman/adt-spec/blob/master/src/adt_spec/core.clj#2020-06-0102:32Eccentric JFound from https://gist.github.com/Olical/f2b934873a49c0638ca673ab764a0131
(s/def ::element (s/or :string string?
:element (s/cat :name keyword?
:attrs (s/? (s/map-of keyword? any?))
:children (s/* ::element))))
#2020-06-0102:33Eccentric JThat's really close I think!#2020-06-0102:44Eccentric J(s/def ::tree (s/or :empty empty?
:node (s/tuple (s/? ::tree) (s/? ::tree))
:leaf number?))
#2020-06-0102:45Eccentric JIt's getting [] => :empty, but failing on anything else#2020-06-0102:56Eccentric J(ns spec-intro.core
(:require
[clojure.spec.alpha :as s]))
(s/def ::tree (s/or
:empty (s/and coll? empty?)
:node (s/cat :left (s/? ::tree)
:right (s/? ::tree))
:leaf number?))
(def tree [0 [1 [2 3]]])
[(s/conform ::tree tree)
(s/conform ::tree [1 0])
(s/conform ::tree [])
(s/conform ::tree 1)]
#2020-06-0102:57Eccentric JGot it!#2020-06-0114:55plexusregarding my question from yesterday re. infinite recursion in generators, I managed to work around it with size
/`resize`, basically making sure that nested generators get increasingly smaller size parameters, and always emitting terminal tokens at small sizes https://github.com/lambdaisland/regal/blob/master/src/lambdaisland/regal/spec_alpha.cljc#L21-L33#2020-06-0114:56plexusmaybe that's useful to someone š#2020-06-0308:20MorongĆaI want to use phrase
to produce human friendly errors messages. Has anyone used this before? What are your comments?#2020-06-0402:20nmkipI have a map that has this shape {"account" {"active" true "balance" 30}}
, what's the best way to spec the "account"
key?
I've tried these and I'm using the first one.
(s/def ::account-key (s/with-gen (s/and string?
#(= "account" %))
#(gen/return "account")))
(s/def ::account-key (s/with-gen (s/and string?
#{"account"})
#(gen/return "account")))
#2020-06-0402:20seancorfieldYou can use #{"account"}
to spec a known set of string values.#2020-06-0402:21seancorfield(and that will also generate correctly)#2020-06-0402:21nmkipjust #{"account"}
that makes sense#2020-06-0402:21seancorfield(s/def ::account-key #{"account"})
#2020-06-0402:21nmkipyes#2020-06-0402:22nmkipgreat, thanks! š I'll use that instead, much shorter#2020-06-0402:23nmkipI was being redundant here:
(s/def ::account-key (s/with-gen (s/and string?
#{"account"})
#(gen/return "account")))
#2020-06-0402:24seancorfieldInventive... but certainly "overkill" š#2020-06-0402:24nmkipindeed#2020-06-0413:05dev.4openIDLearner.
Maybe a dumb question. I have built up a detailed spec using s/def s/valid? s/conform. It works fine. Following best practices all my keywords use :: (e.g. ::timestamp). However, my data covered JSON to map, all keywords are single colon (e.g. :timestamps)
When I construct the data with two colons the valid? and conform has no problem whereas the mapped data with single colon fails.
I was understanding the :: indicates the keyword in this namesapace so having :: in my spec vs. the single : in may data should not matter.
Any clarifications on this? š¤
<script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0413:05dev.4openIDLearner.
Maybe a dumb question. I have built up a detailed spec using s/def s/valid? s/conform. It works fine. Following best practices all my keywords use :: (e.g. ::timestamp). However, my data covered JSON to map, all keywords are single colon (e.g. :timestamps)
When I construct the data with two colons the valid? and conform has no problem whereas the mapped data with single colon fails.
I was understanding the :: indicates the keyword in this namesapace so having :: in my spec vs. the single : in may data should not matter.
Any clarifications on this? š¤
<script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0413:12aisamuThe conversion from JSON will probably give you unqualified keywords, since no such concept exists there.
You have to either qualify your keywords on a separate post-processing step, or make a spec for the payload that takes unqualified keys (e.g. :req-un
instead of :req
)#2020-06-0413:23dev.4openID@U3UFFB420 Thanks for the explanation. But..... why is the :timestamp not found in the namespace or is it global? If global than how do I relate it to my incoming data - as it were. Because the spec is not of use right now. Obviously I do not see the connection#2020-06-0413:24dev.4openIDSo that is the purpose of req-un then?
Does that make the :keyword global?#2020-06-0413:30franquitoMaybe you are mixing concepts. Keywords are just keywords, they can be qualified (`:test/timestamp`) or unqualified (`:timestamp`). ::
It's a shortcut to write a qualified keyword with the same namespace as the current file.#2020-06-0413:31aisamuKeywords can be qualified or unqualified.
Unqualified can be thought of as "global", but it's better to stick with "lacking a namespace".
When you type in ::timestamp
, Clojure uses the current namespace and creates a qualified keyword.
req-un
means that the spec will match unqualified keys to the qualified keys, comparing just the name
(i.e. discarding the namespace
)#2020-06-0413:32dev.4openIDSo in a more complex map structure, where I have several collections do I have to req-un for all definitions or only at the top level and it is inferred?#2020-06-0413:33dev.4openID'Cos I req-un all my defs and now it does not work#2020-06-0413:42aisamuYes, you'd have to req-un all of them.
Try breaking it into smaller pieces to find where it's failing#2020-06-0413:42dev.4openIDhere is the gist <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0413:42dev.4openIDIt is detailled#2020-06-0413:44aisamuThe spec for ::loca
seems a bit off#2020-06-0413:45dev.4openIDseems to work.... what do you mean?#2020-06-0413:46dev.4openIDIf it is all :req for keywords exampl2 works fine and conforms. So I am confused here?#2020-06-0413:46aisamuUnless this is spec2, I don't think this is a valid definition: (s/def ::loca (s/keys :req-un {::addr [::country]}))
#2020-06-0413:46aisamuIIRC, req & req-un take a vector of keys, not a map#2020-06-0413:47aisamuMight have worked by accident, or because you had a correct definition earlier that was stored in the registry#2020-06-0413:48dev.4openIDwould (s/def ::loca :req {::addr [::country]}) do it?
no it does not#2020-06-0413:49dev.4openIDSurely the country spec forces a s/keys?#2020-06-0413:52aisamuI've never seen that syntax (feeding a map to :req
). Does it work on a simple case?#2020-06-0413:53dev.4openID2 rows above and it works -> (s/def ::addr (s/keys :req [::country]))
(s/conform ::addr {::country "USA"}) ;; => #:myproj.trial.spec{:country "USA"}#2020-06-0413:53dev.4openIDIt is effectively a map of map, no?#2020-06-0413:55dev.4openIDand :loca is the map of the above. But if I do not have s/keys it seems now not to work#2020-06-0413:56aisamuThese two are not the same:
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un {::addr [::country]}))
You're feeding a vector to the first req, but a map to the second#2020-06-0413:57dev.4openIDAgreed but I tried ::loca {:: addr {::country and it will not accept it#2020-06-0414:06dev.4openID@U1UQEM078 BTW what do you mean as spec2?
I am using clojure.spec.alpha latest version#2020-06-0415:00dev.4openID@U3UFFB420 I agree with you, how does one relate the one to the other? I changed my spec to have :reg-un at the highest level and it fails
spec is at <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0415:06franquitoTry first working with simpler examples, try to create a spec for a map with unqualified keys#2020-06-0415:07dev.4openIDIf you look at the gist that Is what I am doing? No?#2020-06-0415:10dev.4openIDWhen all the keys are NOT reg-un then all my test cases pass#2020-06-0415:10franquitoWell, https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L19 you need to change it to (s/def ::loca (s/keys :req-un [::addr]))
#2020-06-0415:10dev.4openIDand so my spec seems OK ??#2020-06-0415:11dev.4openIDOK how? I know it is indicating a map but if I do not use s/keys it fails#2020-06-0415:11dev.4openIDhttps://app.slack.com/team/UEGT2541JĀ Ā [2:57 PM]
Agreed but I tried ::loca {:: addr {::countryĀ Ā and it will not accept#2020-06-0415:12dev.4openIDEven though I initially figured it wasa map of map of map#2020-06-0415:14dev.4openIDOh oh!!!! I see it now!#2020-06-0415:15franquitoI see several places you use :req-un
instead of :req
, which is the correct way for conforming/validating these examples#2020-06-0415:16franquitoAs I said, try creating a simpler spec for a non nested map and then try validating maps#2020-06-0415:17dev.4openIDOk. But how do I use the spec against the data map (ex JSON) which is :keyword and not ::timeline (IYKWIM)#2020-06-0415:20franquitoHere's an example for a qualified map
(ns myproj.trial.spec)
(s/def ::country string?)
(s/def ::street string?)
(s/def ::addr (s/keys :req [::country ::street]))
(s/valid? ::addr {::country "USA"
::street "A street"})
;; => true
(s/valid? ::addr {:country "USA"
:street "A street"})
;; => false
(s/valid? ::addr {:myproj.trial.spec/country "USA"
:myproj.trial.spec/street "A street"})
;; => true
#2020-06-0415:23dev.4openIDyes, without changing the above, how do you apply it to a map ex JSON) {:addr {:country "USA :street "A street"}}#2020-06-0415:23franquitoAs aisamu said before, if you are working with JSON you loose the namespace of the keywords, so you probably want to use :req-un
in your specs.
(ns myproj.trial.spec)
(s/def ::country string?)
(s/def ::street string?)
(s/def ::addr (s/keys :req-un [::country ::street]))
(s/valid? ::addr {:country "USA"
:street "A street"})
;; => true
#2020-06-0415:24dev.4openIDI was advise to use :req-un to allow for unqualif namesapces#2020-06-0415:25franquitoRight, but then when you use valid?
or conform
you also have to use unqualified keywords for the map https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L69-L81#2020-06-0415:40dev.4openID(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr ::country ]))
(s/conform ::country "USA") ;; => "USA"
(s/conform ::addr {:country "USA"}) ;; => {:country "USA"}
(s/conform ::loca {:addr {:country "USA"}}) ;; => #:myproj.trial.spec{:addr #:myproj.trial.spec{:country "USA"}}
(s/explain-str ::loca {:addr {:country "USA"}})
;; => "{:addr {:country \"USA\"}} - failed: (contains? % trial.spec/addr) spec: :trial.spec/loca\n{:addr {:country \"USA\"}} - failed: (contains? % trail.spec/country) spec: :parceltrax.tracking.spec/loca\n"
??why this??#2020-06-0415:47dev.4openIDI updated the gist <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script> If you look at line 16
When I look sat it it is a map of map#2020-06-0415:53franquitoThis line is wrong https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L19#2020-06-0415:53franquitoTry conforming this
(s/conform ::loca {:addr {:country "USA"}
:country "OTHER"})
#2020-06-0415:55dev.4openIDYes, in theory that will work, however the data looks like this
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(def exampl2 {::abc [{::loca {::addr {::country "USA"}}
::dets {::some{::timestamp "2020-04-09T15:27:00"
::Url "https://mywebpage.com/answer"#2020-06-0415:57dev.4openIDLook at line 1 of the gist#2020-06-0415:57dev.4openIDso loca is a map of map of map#2020-06-0415:58dev.4openIDThe trouble I a,mm having is the compounded map that does not seems to play nicely#2020-06-0416:06dev.4openIDOK - it does conform to the definition Thx. However it is not as per line 1 - which is how the data is meant to be. So extracting from line 1: :loca {:addr {:country "USA"}}
#2020-06-0416:09franquitoYes, you need to modify ::loca
so it looks like this (s/def ::loca (s/keys :req-un [::addr]))
#2020-06-0416:09dev.4openID@U3UFFB420 OK !!!! I think I have it ! I must take out the ::country in the s/def as the ::country is already defined (s/def ::loca (s/keys :req-un [::addr]))#2020-06-0416:09dev.4openIDI think that is what you were trying to get me to see#2020-06-0416:10franquitoYes, exactly! š#2020-06-0416:10dev.4openIDAppreciated#2020-06-0416:26dev.4openID@U3UFFB420 Thanks for your patience and help. I got it now!! my complete script now works! Kudos for you š#2020-06-0413:12franquitoThe difference is that ::
expands to the current namespace. So if you're working in a ns called my.test
then ::timestamp
expands to :my.test/timestamp
. The spec fails with :timestamp
because it's also expecting the namespace.#2020-06-0419:30dev-hartmannHi fellow clojurians, is anyone aware of a way to parse a swagger json to valid specs that can be used to generate test data?#2020-06-0910:40elimisteveThat's a great idea and something I'm interested in, too#2020-06-0911:04dev.4openIDThis is what I did to learn spec. From this point on one can gen data. What you see is a map converted from a JSON structure. Now this is not a "take a swagger spec and convert" but is the basis of how to clojure spec based on a swagger that I did.
(def exampl {:abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr]))
(s/conform ::country "USA") ;; => "USA"
(s/conform ::addr {:country "USA"}) ;; => {:country "USA"}
(s/conform ::loca {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/valid? ::loca {:addr {:country "USA"}}) ;; => true
(s/check-asserts?)
(s/check-asserts true) ;; by default false!!
(s/assert ::loca {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/assert ::loca {:addr {:countryx "SA"}}) ;; note the exception error BUT look at the repl
(s/def ::timestamp (and not nil?
string?)) ;; this needs work for T-time issue
(s/def ::Url string?) ;; need to validate URLs - technique
(s/def ::name string?)
(s/def ::prod (s/keys :req-un [::name]))
(s/def ::some (s/keys :req-un [::timestamp ::Url ::prod]))
(s/def ::dets (s/keys :req-un [::some]))
(s/conform ::timestamp "2020-04-09T15:27:00")
(s/conform ::Url "")
(s/conform ::name "this product")
(s/conform ::prod {:name "this product"})
(s/conform ::some {:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}})
(s/conform ::dets {:some {:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}})
(s/def ::totalNumber (and pos? int?))
(s/def ::pieceID (and (s/coll-of string?)
vector?))
(s/conform ::totalNumber 399) ;; => 399
(s/conform ::pieceID ["ABC1" "DEF2"]) ;; => ["ABC1" "DEF2"]
(s/def ::abc (s/coll-of (s/keys :req-un [::loca ::dets ::totalNumber ::pieceID])))
(s/conform ::abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}])
;; now ex is an map of a single instance)
ex
;; => {:abc [{:loca {:addr {:country "USA"}}, :dets {:some {:timestamp "2020-04-09T15:27:00", :Url "", :prod {:name "this product"}}}, :totalNumber 399, :pieceID ["ABC1" "DEF2" "GHI3"]}]}
(s/def ::an_instance (s/keys :req-un [::abc]))
(s/conform ::an_instance {:abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(s/conform ::an_instance exampl)
(s/valid? ::an_instance exampl)
(s/explain-str ::an_instance exampl)
Also look at https://www.youtube.com/watch?v=f2hNQdS2VxQ to watch a conversion - the nearest I have found.
Good luck with this š#2020-06-0914:19dev-hartmannhey @UEGT2541J thanks for joining in. actually the challenge is not to create a clojure map from the json and then writing the spec. but parsing the json into specs . the difference being that I don't want to write them manually. but you are absolutely right, the chain is: json -> clojure-map -> transform clojure map entries to spec entries and compose them to upper level spec. then we can use the gen functions to create test data#2020-06-0914:21dev-hartmannif we have a working parser for that, we can just call the endpoint on webservices, slurp the swagger json and create specs for data models#2020-06-0914:22dev-hartmannwhich is a handy little tool š#2020-06-0914:26dev-hartmannI'm not an expert for specs, but searching a swagger yaml/ json for the definitions#2020-06-0914:27dev-hartmannand then using the type information there and maping those and the field names to specs sounds doable š#2020-06-0914:27dev-hartmanndefinitions:
Order:
type: "object"
properties:
id:
type: "integer"
format: "int64"
petId:
type: "integer"
format: "int64"
quantity:
type: "integer"
format: "int32"
shipDate:
type: "string"
format: "date-time"
status:
type: "string"
description: "Order Status"
enum:
- "placed"
- "approved"
- "delivered"
complete:
type: "boolean"
default: false
xml:
name: "Order"
#2020-06-0914:27dev-hartmanni like structured data š#2020-06-0914:28dev-hartmannI'll work on that a bit an will put a link to a gist here if I can create something useful#2020-06-0917:44flyboarder@U0J8XN37Y im really interested in this, we just rolled out swagger to our APIās both internally and externally, I would love to be able to have our apps handle specs for us#2020-06-0917:46dev-hartmannwill keep you posted too.#2020-06-0917:47dev-hartmannI think it's easier than I thought.#2020-06-0917:47dev-hartmann.. I mean until I hit the brick wall š#2020-06-0917:55dev-hartmannat least the basics. translating the conditions on properties can take some time š#2020-06-0920:29dev.4openIDOK, similar to what you are stsing then: look st the library serene - It autoigenerates the specs for REST and graphql APIs.#2020-06-0920:30dev.4openIDI'll shut up now š#2020-06-0922:05dev-hartmannno, please don't. never heard of that library and it looks like they are doing the same thing, but for a different format. š#2020-06-0514:09Felipe Marques@ikitommi
Hi, a few days ago you mentioned some other libs to perform spec coercion, but I forgot to take notes on its names to check it later.
Could you share the names again?
Also, I'm having some trouble with coercing some data using spec-tools. I'm not sure if I got the logic right.
Do you have any resource explaining how the coercion works in spec-tools?
Thanks very much!#2020-06-0514:41ikitommiHi @marques.goncalves.fel! The slack history (& my message) is here: https://clojurians-log.clojureverse.org/clojure-spec/2020-05-19. I believe all spec-coercion libs are doing it in the same way, : given a spec, it's form is parsed and for all different specs (`or`, and
, keys
, integer?
, any?
etc.) a transforming function is selected and applied to the value. Corcion is recursive and best effort. With spec-tools:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])
(s/def ::spec (s/nilable
(s/nilable
(s/map-of
keyword?
(s/or :keys (s/keys :req-un [::c1])
:ks (s/coll-of (s/and int?) :into #{}))))))
(def value {"keys" {:c1 "1" ::c2 "kikka"}
"keys2" {:c1 true}
"ints" [1 "1" "invalid" "3"]})
(st/coerce ::spec value st/string-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{1 "invalid" 3}}
(st/coerce ::spec value st/json-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{"3" 1 "invalid" "1"}}
#2020-06-0514:41ikitommiHi @marques.goncalves.fel! The slack history (& my message) is here: https://clojurians-log.clojureverse.org/clojure-spec/2020-05-19. I believe all spec-coercion libs are doing it in the same way, : given a spec, it's form is parsed and for all different specs (`or`, and
, keys
, integer?
, any?
etc.) a transforming function is selected and applied to the value. Corcion is recursive and best effort. With spec-tools:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])
(s/def ::spec (s/nilable
(s/nilable
(s/map-of
keyword?
(s/or :keys (s/keys :req-un [::c1])
:ks (s/coll-of (s/and int?) :into #{}))))))
(def value {"keys" {:c1 "1" ::c2 "kikka"}
"keys2" {:c1 true}
"ints" [1 "1" "invalid" "3"]})
(st/coerce ::spec value st/string-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{1 "invalid" 3}}
(st/coerce ::spec value st/json-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{"3" 1 "invalid" "1"}}
#2020-06-0514:43ikitommiwrote few years back: https://www.metosin.fi/blog/spec-transformers/#2020-06-0514:52Felipe MarquesThanks for the links and explanation. Sorry for not finding the history. I looked only in slack! š
#2020-06-0520:53dev.4openIDLearner here. Dumb Question maybe, maybe a mindset
I have a map (ex JSON) in it exists the following:
{:status {:timestamp "2020-04-09T15:27:00"
:status "abc"}
Yes, I suppose I could change the data so that the 1st :status has a different name but that is not very elegant. If the structure remains the same, I assume you cannot redefine the :status again within the spec
Is there a way int spec (some technique) that deals with this type of anomaly? I have a data set exactly like that. Any thoughts?#2020-06-0521:08Alex Miller (Clojure team)s/keys with :req-un will match the short name of the provided spec but can use different qualified specs#2020-06-0521:10Alex Miller (Clojure team)(s/def :inner/status string?)
(s/def :inner/timestamp string?)
(s/def :outer/status (s/keys :req-un [:inner/status :inner/timestamp]))
(s/def :outer/map (s/keys :req-un [:outer/status]))
#2020-06-0521:26dev.4openID(s/def :inner/status string?)
(s/def :inner/timestamp string?)
(s/def :outer/status (s/keys :req-un [:inner/timestamp :inner/status]))
(s/def :outer/map (s/keys :req-un [:inner/status]))
(s/valid? :outer/map {:status {:timestamp "2020-09"
:status "abc" }})
Hi, @alexmillerseems to fail against the data. Perhaps I miss it?
(s/explain-str :outer/map {:status {:timestamp "2020-09"
:status "abc" }})
;; => "{:timestamp \"2020-09\", :status \"abc\"} - failed: string? in: [:status] at: [:status] spec: :inner/status\n"#2020-06-0521:47seancorfield@dev.4openid That works for me:
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def :inner/status string?)
:inner/status
user=> (s/def :inner/timestamp string?)
:inner/timestamp
user=> (s/def :outer/status (s/keys :req-un [:inner/status :inner/timestamp]))
:outer/status
user=> (s/def :outer/map (s/keys :req-un [:outer/status]))
:outer/map
(s/valid? :outer/map {:status {:timestamp "2020-09"
:status "abc" }})
true
(s/explain-str :outer/map {:status {:timestamp "2020-09"
:status "abc" }})
"Success!\n"
user=>
(you can use triple-backticks around your code to format it and make it easier to read)#2020-06-0521:59dev.4openIDHi @alexmiller and @seancorfield - thanks for the help - I am wrong: my fat fingers and compiling versions of the trial and error - I screwed up! šš - along day!#2020-06-0607:05practicalli-johnQuestion about spec keys. It seems I can use either qualified keys with :req
or unqualified keys with :req-un
. I cannot use qualified keys with :req-un
. So if I have keys that could be qualified or unqualified, would I have req
and req-un
sections in the keys
(spec/def ::customer-details
(spec/keys
:req [::first-name ::last-name ::email-address ::residential-address ::social-security-id]
:req-un [::first-name ::last-name ::email-address ::residential-address ::social-security-id]))
Or I assume I should just choose to use one or the other.
I could just add the #::
auto-resolve macro to hash-maps with unqualified keys...#2020-06-0607:14practicalli-johnAdding a specific namepace in front of the hash-map, #:practicalli.bank-account-spec
seems to work too and feels like a better approach
(spec/explain :practicalli.bank-account-spec/customer-details
#:practicalli.bank-account-spec
{:first-name "Jenny"
:last-name "Jetpack"
:email-address "
#2020-06-0607:26AronI haven't really had time to watch all the videos, is there one where the organization of the namespaces (especially in relation with models, specs, entities)? I am coming from js where circular dependencies are allowed in certain circumstances and I find it a bit too much work to create new files to create new namespaces (note, it's not the problem that I have to create files, I love lots of tiny files, I just don't like the now additional work that means I have to constantly rewrite all my files if I want to reorganize just a couple.)#2020-06-0607:28practicalli-john@ashnur I will be covering a bit of organisation in the broadcast today in about 30 minutes https://youtu.be/jUgm4zh-vF4 - I create a src, test and spec namespace and have a design-journal namespace to show my working š
It still fairly basic, so you dont need to watch the other videos#2020-06-0607:29Aronthanks, I will be watching#2020-06-0611:36dev.4openIDThere seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated.
So you s/def uses :: and can compose other s/defs with ::
However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere.
At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough".
I see similar being asked above#2020-06-0611:36dev.4openIDThere seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated.
So you s/def uses :: and can compose other s/defs with ::
However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere.
At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough".
I see similar being asked above#2020-06-0614:00Alex Miller (Clojure team)First, spec know nothing about :: - this is just a Clojure shorthand to fully resolve a keyword within the current namespace at read time. Spec will just see the full kw as if you had typed :user/myspec.
s/keys will always validate all fully qualified keys in the map with their associated registered specs. It will check that the map has all of the specified :req keys (match fully qualified) or :req-un keys (match only the unqualified name). And specially for :req-un it will verify the values validate against the spec for the qualified key.#2020-06-0720:59dev.4openID@U064X3EF3 Hi Alex
(s/def ::timestamp string?) ;; => :parceltrax.tracking.spec/timestamp
(s/def ::status string?) ;; => :parceltrax.tracking.spec/status
(s/def :o/status (s/keys :req-un [::timestamp ::status])) ;; => :o/status
(s/valid? :o/status {:timestamp "2020-09"
:status "abc" }) ;; => true
(s/explain-str :o/status {:timestamp "2020-09"
:status "abc" }) ;; => "Success!\n"
(s/def ::mapx (s/keys :req-un [:o/status])) ;; => :parceltrax.tracking.spec/mapx
(s/valid? ::mapx {:status {:timestamp "2020-09"
:status "abc" }}) ;; => true
(s/explain-str ::mapx {:status {:timestamp "2020-09"
:status "abc" }}) ;; => "Success!\n"
I accept is works and maybe I'm being a little thick here: The solution depends of the "temp" creation of a namespace as a workaround. In this case a :o/status in order to differentiate from the actual :req-un namespace of :status.
My perception is this in not very elegant or consistent with the spec model. As is encouraged by the clojure team, developers are to use :: more often. In the simple example provided - the code is not "portable" as such.
I recognize it may seem may seem nitpicking - perhaps I have missed the understanding or this is just accept "it works this way"
In the real workplace there are many occasions of "recursive dup fields in JSON data.
Could you provide a comment or two on this?
Much appreciate your efforts#2020-06-0721:29Alex Miller (Clojure team)Iām not trying to encourage more use of :: and itās typically not what you need (but itās very convenient for examples)#2020-06-0721:30Alex Miller (Clojure team)Personally I most often use full namespaces for data specs#2020-06-0809:32dev.4openIDHi, I was not stating you were the advocate - the spec literature does - just to be clear
Any thought about my point of "making up kw names" in conjunction with the point of recursive/nested JSON map structures that employ a keword such as my example :status - is there no recursive strategy for example? I don't know thus I ask.#2020-06-0809:32dev.4openIDThanks for your patience#2020-06-0613:18tianshuIs it expected if :a/b
is not defined in this case?
(s/def ::c
(s/keys :req [:a/b]))
(s/valid? ::c {:a/b {}})
;; => true
#2020-06-0613:18tianshuIs it expected if :a/b
is not defined in this case?
(s/def ::c
(s/keys :req [:a/b]))
(s/valid? ::c {:a/b {}})
;; => true
#2020-06-0613:43dev.4openIDNo. Imagine they are defined
It's not about a/b nut when :req or :req-un is appropriate#2020-06-0613:52Alex Miller (Clojure team)I would say yes, this is expected. :req means itās required (and itās there). If there are specs for any keys, they must be followed.#2020-06-0703:29tianshuFor the case (s/def ::a ::b)
, ::b
must be declared before use, but in this case it is not. Should it be unified? (I mean always define before use or always check at runtime)#2020-06-0713:33Alex Miller (Clojure team)Yes, in general resolution should be delayed for all specs and itās not entirely true right now. Not going to make any updates on this in spec 1 though, will work on it in spec 2#2020-06-0613:34hadilsI am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!#2020-06-0613:34hadilsI am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!#2020-06-0613:45dev.4openIDI have just done this and it is OK for code and spec to be in different files
I assume you have converted the JSON to a structure. Now define the spec "structure" accordingly
Apply s/conform ::spec "JSON structure" - note the spec has :: and your JSON map has : for all elements - in spec make sure all :req are :req-un and it will work#2020-06-0613:50dev.4openIDin my case the file my namespace is ns: myproj.trail.clj and myproj.trial.spec.clj#2020-06-0613:52dev.4openIDLesson learnt model the API JSON map example in spec file fisrst and all becomes more clear. I started from leaves to trunk and not trunk down as seemingly many do#2020-06-0613:53hadilsThanks! @UEGT2541J#2020-06-0613:54hadilsWhat about for multiple JSON outputs in the same spec file? How do I do that?#2020-06-0613:57hadilsThat's the real problem I'm having.#2020-06-0613:57dev.4openIDIf you break down the JSON maps into components and composed elements that are reusable. You then compose complete "trees" for all the responses for all the API calls. There are commonalities in the APIs returns and it goes faster in the compositions as you do more. i.e do 1 at a time. I went with the most complex one (in order to lean spec) and it just became easier to gen the rest.#2020-06-0613:58hadilsHow do you qualify your spec keys?#2020-06-0614:00dev.4openIDIt becopmes more and more obvious. Just build the leaves and branches slowly - use s/valid? and s/explain-str against sample data - extracted from the JSON map eventually build more complex branches and tada! you will compose the trunk. It seem tedious at first (as I was leaning) but it become faster#2020-06-0614:03hadilsI am hung up on the qualified keys like ::foo instead of using :node/foo. The latter seems more appropriate for my needs. How should I approach this?#2020-06-0614:04dev.4openIDlook at the toy example I made first here <script src="https://gist.github.com/dev4openid/5c9320fb8ef2f8f5383836f379ff507e.js"></script>
It is tedious at first BUT it gave me insights on how to do things
I will leave it up for a while#2020-06-0614:05dev.4openIDin the spec file use :: format for your definitions - saves typing#2020-06-0614:05hadilsThanks!#2020-06-0614:09dev.4openIDnote the :reg-un !!#2020-06-0614:25hadilsYour Gist does not show up properly on my browser.#2020-06-0614:30dev.4openID(def exampl {:abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr]))
(s/conform ::country "USA") ;; => "USA"
(s/conform ::addr {:country "USA"}) ;; => {:country "USA"}
(s/conform ::loca {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/valid? ::loca {:addr {:country "USA"}}) ;; => true
(s/check-asserts?)
(s/check-asserts true) ;; by default false!!
(s/assert ::loca {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/assert ::loca {:addr {:countryx "SA"}}) ;; note the exception error BUT look at the repl
(s/def ::timestamp (and not nil?
string?)) ;; this needs work for T-time issue
(s/def ::Url string?) ;; need to validate URLs - technique
(s/def ::name string?)
(s/def ::prod (s/keys :req-un [::name]))
(s/def ::some (s/keys :req-un [::timestamp ::Url ::prod]))
(s/def ::dets (s/keys :req-un [::some]))
(s/conform ::timestamp "2020-04-09T15:27:00")
(s/conform ::Url "")
(s/conform ::name "this product")
(s/conform ::prod {:name "this product"})
(s/conform ::some {:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}})
(s/conform ::dets {:some {:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}})
(s/def ::totalNumber (and pos? int?))
(s/def ::pieceID (and (s/coll-of string?)
vector?))
(s/conform ::totalNumber 399) ;; => 399
(s/conform ::pieceID ["ABC1" "DEF2"]) ;; => ["ABC1" "DEF2"]
(s/def ::abc (s/coll-of (s/keys :req-un [::loca ::dets ::totalNumber ::pieceID])))
(s/conform ::abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}])
;; now ex is an map of a single instance)
ex
;; => {:abc [{:loca {:addr {:country "USA"}}, :dets {:some {:timestamp "2020-04-09T15:27:00", :Url "", :prod {:name "this product"}}}, :totalNumber 399, :pieceID ["ABC1" "DEF2" "GHI3"]}]}
(s/def ::an_instance (s/keys :req-un [::abc]))
(s/conform ::an_instance {:abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(s/conform ::an_instance exampl)
(s/valid? ::an_instance exampl)
(s/explain-str ::an_instance exampl)
#2020-06-0614:31dev.4openIDstrange#2020-06-0614:35hadilsThanks a lot @UEGT2541J!!!!#2020-06-0616:22hadils@UEGT2541J what do you do when you are trying to spec 2 different JSON outputs with same names -- some of the fields are identical and some are not, but have the same name?#2020-06-0616:23hadilsDo I need to split them out into separate files?#2020-06-0716:45dev.4openID@UGNMGFJG3 Do you mean 2 fields in the JSON structure, where 1 is subordinate to another? {:name {:person "abc" :name}}
or do you mean recurring pasterns that are in a map?#2020-06-0716:45dev.4openIDFor the former:
(s/def :inner/status string?) ;; => :inner/status
(s/def :inner/timestamp string?) ;; => :inner/timestamp
(s/def :outer/status (s/keys :req-un [:inner/timestamp :inner/status])) ;; => :outer/status
(s/def :outer/map (s/keys :req-un [:outer/status])) ;; => :outer/map
(s/valid? :outer/map {:status {:timestamp "2020-09"
:status "abc" }}) ;; => true
(s/explain-str :outer/map {:status {:timestamp "2020-09"
:status "abc" }}) ;; => "Success!\n"#2020-06-0801:53practicalli-johnAfter quite bit of experimentation, I refactored my specs, code and tests for a bank account and learnt a bit about qualified keys and auto-resolve macro. I have specs that I can use with my clojure.test unit tests
I updated the journal to show how I would create the specs, unit tests and code now https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj
And the specs are in their own namespace, using a Clojure Common extension, .cljc
(I believe all the specs are host neutral)
https://github.com/practicalli/leveraging-spec/blob/master/test/practicalli/bank_account_spec.cljc#2020-06-0801:53practicalli-johnAfter quite bit of experimentation, I refactored my specs, code and tests for a bank account and learnt a bit about qualified keys and auto-resolve macro. I have specs that I can use with my clojure.test unit tests
I updated the journal to show how I would create the specs, unit tests and code now https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj
And the specs are in their own namespace, using a Clojure Common extension, .cljc
(I believe all the specs are host neutral)
https://github.com/practicalli/leveraging-spec/blob/master/test/practicalli/bank_account_spec.cljc#2020-06-0803:01seancorfield@U05254DQM Is this a deliberate bug to fall out of testing later? https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj#L56#2020-06-0803:10seancorfield:args
should be a sequence spec, using s/cat
: https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj#L422#2020-06-0803:11seancorfieldDo you understand why this doesn't work? https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj#L337-L344#2020-06-0803:13seancorfields/or
requires labels for the alternatives: (s/or :qualified (s/keys :req [...]) :unqualified (s/keys :req-un [...]))
#2020-06-0807:10practicalli-johnOh yes, labels. At 2.30am I tend to forget things :white_frowning_face:
I find fdef a bit opaque, some more reading and experimenting to do.
Thanks for spotting the bug in last-name, not intentional.
Thanks for the review#2020-06-0813:53practicalli-johnI got the spec/or
expression to wrap the :req
and :req-un
versions working.
I have my fdef :args validating after instrumenting.
Still working on testing the :ret
in the fdef...#2020-06-0816:46seancorfieldRight, :args
is for instrument
-- checking that your code is calling the function correctly -- and :ret
/`:fn` are for generative testing, to make sure your function behaves correctly.#2020-06-0816:47seancorfieldWith instrument
, :ret
and :fn
are ignored. With check
, :args
is used to generate conforming arguments and then the function is called and the :ret
and :fn
specs are checked against the return value and against the arguments/return value respectively.#2020-06-0816:48seancorfieldA lot of people don't feel the docs are clear enough about :args
/`instrument` and :ret
/`:fn`/`check`#2020-06-0817:50practicalli-johnIām comfortable with instrument for now. I donāt seem to have something right with check yet, but will look again tomorrow. Thanks.#2020-06-0915:28dev-hartmannhey folks, excuse my noobiness, but is it possible to return a s/def from a function?#2020-06-0915:28dev-hartmannI'm trying to build them dynamically from a parsed json which is working okish to the point where I try to return them#2020-06-0915:35Alex Miller (Clojure team)s/def is a function which I don't think you mean. do you mean a spec form or a spec object?#2020-06-0917:21dev-hartmannAh, yes. Sry, thatās what I mean. I want to create them dynamically, save their names to an atom and then do an s/def#2020-06-0918:09Alex Miller (Clojure team)one way you could do this is to directly manipulate the registry (as s/def's impl does). in spec 2 we've pulled out a non macro function s/register for this purpose.#2020-06-0921:23dev.4openID(defn len [n] (and (< (count n) 6)
string?))
(s/def ::piece-IDs (and vector?
(s/coll-of len into [])))
(s/valid? ::piece-IDs ["92257" "12" "01234"] );; => true
(s/conform ::piece-IDs ["92257" "12" "01234"]) ;; => ["92257" "12" "01234"]
(s/explain-str ::piece-IDs ["92257" "12" "01234"]) ;; => "Success!\n"
In the above code the s/def is valid, however it depends on the defn provided - seems untidy and not best practice
Can this not be changed to incorporate the count condition soemwhat as below?
(obvs. the coud does not work as the count does not apply to the strings)
(s/def ::piece-IDs (and vector?
(s/coll-of #(and (< count 0)
string?) into [])))
Any ideas to eliminate the defn in the above case?#2020-06-0921:46Alex Miller (Clojure team)you don't need a custom predicate for that at all#2020-06-0921:46vlaaad(s/valid? (s/coll-of string? :kind vector? :max-count 5) ["1" "2" "3" "4" "5" "6"])
#2020-06-0921:46Alex Miller (Clojure team)coll-of
has a :max-count
modifier#2020-06-0921:48Alex Miller (Clojure team)also note that in spec 2, there is now a catv
too that would slim this down to just (s/catv string? :max-count 5)
#2020-06-0922:24dev.4openIDHi, @vlaaad and @alexmiller
(s/def ::pieceIds (s/coll-of string? :kind vector? :max-count 5))
(s/valid? ::pieceIds ["JD014600007821392257" "12" "01234567890123456789" "777777777" "66666666666666666666666666666" "77"]) ;; => false it is counting 6 strings in vector
(s/conform ::pieceIds ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]) ;; => ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]
(s/explain-str ::pieceIds ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]) ;; => "Success!\n" it is not applying the string length limit of 5 chars
produces the count how many string there are; whereas the code I presented determines whether every string length is less than 6 (this being the challenge I find myself in)
Subtle difference to what I suggested. Any ideas?
BTW Alex: I am using leiningen so I am not sure how to exactly refer spec 2; i.e. I do not use an edn file per se. Perhaps some guidance on that would be helpful - It refers to git etc. so I am unsure#2020-06-0922:49seancorfield@dev.4openid Do you want a :min-count
as well as a :max-count
?#2020-06-0922:50seancorfieldOh, you want the strings themselves to be checked for length, not the vector?#2020-06-0922:50dev.4openIDYes#2020-06-0922:51seancorfieldSo you want (s/def ::short-string (s/and string? #(>= 5 (count %))))
and then (s/coll-of ::short-string ...)
#2020-06-0922:52seancorfield:min-count
and :max-count
apply to s/coll-of
, not to things inside the collection.#2020-06-0922:53dev.4openIDWell, it looks like I had the general idea right š but not the implementation. I will try now#2020-06-0923:05dev.4openID@seancorfield
No it must be missing something
(s/def ::short-string (and #(>= 5 (count %))
string?))
(s/def ::piece-IDs (s/coll-of ::short-string into []))
(s/valid? ::piece-IDs ["92257" "12" "012344444"] );; => true NOT
(s/conform ::piece-IDs ["926257" "12" "01234"]) ;; => ["92257" "12" "01234"]
(s/explain-str ::piece-IDs ["9992257" "12" "01234"]) ;; => "Success!\n"
#2020-06-0923:07seancorfieldRead what I suggested for ::short-string
and compare it to what you wrote.#2020-06-0923:09seancorfield(I just edited mine BTW)#2020-06-0923:09seancorfield(and #(..) string?)
is truthy#2020-06-0923:10seancorfield(s/and #(..) string?)
is a spec#2020-06-0923:11seancorfieldAlso, it's safer to add the type predicate (e.g., string?
) first before applying count
otherwise this will blow up (s/valid? ::piece-IDs [42])
because it will try to call (count 42)
before testing it is a string.#2020-06-0923:14dev.4openIDYep, I missed it š now it works and I have to discipline myself on the s/. Bit of improvement work!! @seancorfield thanks for the help.
Will use string (type checks first) Thx#2020-06-0923:15seancorfield(somewhat ironically, my incorrect version -- (and string? #(>= 5 (count %)))
-- worked by accident because it evaluated to just #(>= 5 (count %))
because (and truthy x)
=> x
for any x
#2020-06-0923:16seancorfieldI only realized my version was broken when I tried the 42
example and mine blew up, even tho' it had correctly rejected your test example with "012344444"
#2020-06-0923:29dev.4openIDI am learning this language ad enjoying it. this is after the last serious coding I did 30 years ago. It is a steep curve, it feels like doing "maths" all over again - strict discipline and be aware of the subtilties . There are so many variations and tweaks I sometimes get lost! Too easy too make mistakes. Worth learning as it is pretty powerful. Thx for your help#2020-06-0923:40seancorfield@dev.4openid The REPL is your friend here. If you try out every single function as you write it, you might have caught the problem with your len
function sooner:
user=> (defn len [n] (and (< (count n) 6)
string?))
#'user/len
user=> (len "123")
#object[clojure.core$string_QMARK___5410 0x3bc735b3 "
#2020-06-0923:41seancorfieldand:
user=> (len 42)
Execution error (UnsupportedOperationException) at user/len (REPL:1).
count not supported on this type: Long
user=>
#2020-06-1014:49adamI am trying to validate a server-side form with spec (so Clojure not ClojureScript) and I am a little lost about where to start to formulate human readable error messages.
Is https://github.com/alexanderkiel/phrase the de-facto library for that?#2020-06-1014:49adamI am trying to validate a server-side form with spec (so Clojure not ClojureScript) and I am a little lost about where to start to formulate human readable error messages.
Is https://github.com/alexanderkiel/phrase the de-facto library for that?#2020-06-1114:10aviIāve always used expound for that#2020-06-1109:17jacklombardDo you all unit test specs?#2020-06-1109:17jacklombardDo you all unit test specs?#2020-06-1113:10Jivago AlvesNever unit tested specs. But I'm using it as part of the test. I'd test it if I was going to release it as a library.#2020-06-1113:32jacklombardfor complex specs that i use to validate data, would like to know if the spec is right#2020-06-1114:11aviThatāsā¦ interesting! Because I think I already think of specs as, more or less, akin to tests themselves.#2020-06-1114:12aviI do of course sometimes find that a spec Iāve written turns out to not quite capture what I had in mind, or a requirement, etcā¦ but Iāve done the same with ātraditional unit testsā (i.e. example tests) ā¦#2020-06-1116:08Jivago AlvesI use spec to help me to parse some data but I'm testing the function that does that. The fact I'm using a spec is just a implementation detail. At least, that was most of my use cases.#2020-06-1116:46aviAh, I see, yeah. That makes sense.#2020-06-1116:46aviIn that case, yes, I would probably write some example tests.#2020-06-1118:28seancorfieldMy approach to developing complex Specs is to have a (comment ,,,)
form under the Specs with expressions in it that I use to test validation, conformance, and generation.#2020-06-1118:28seancorfieldI also make sure to "test" each individual part of a Spec (so I can catch generation problems early -- debugging generation problems in a large Spec is no fun!).#2020-06-1115:53mishacan I express "this key is optional, but if is in map - that key has to be there too, " as s/keys out of the box? with some combination of :opt :req and or
like
(s/keys :req [(or :a/foo :a/bar)])
or something?
Or should I s/and
it?:
(s/and
(s/keys :opt [:a/foo :a/bar])
(fn [m]
(let [present (partial contains? m)
missing (complement present)
ks [:a/foo :a/bar]]
(or
(every? present ks)
(every? missing ks)))))
#2020-06-1115:55Alex Miller (Clojure team)s/and'ing a predicate is prob best#2020-06-1115:56mishais it s/and
in spec2 for the same use case?#2020-06-1115:57mishathank you#2020-06-1115:59Alex Miller (Clojure team)yeah, I would do the same in spec 2#2020-06-1115:59Alex Miller (Clojure team)or rather I would probably make 2 different s/selects for the same s/schema#2020-06-1116:00Alex Miller (Clojure team)s/keys is probably going away in spec 2#2020-06-1117:58dev.4openID@alexmiller Over what time-frame are you anticipating spec 2 to come out?#2020-06-1117:58dev.4openID@alexmiller Over what time-frame are you anticipating spec 2 to come out?#2020-06-1118:38Alex Miller (Clojure team)Iām deep in something else right now, but hoping to pop the stack back to that soon. Not sure, prob mostly gated in rework of function specs which rich has been hammocking on#2020-06-1117:59dev.4openIDI am assuming it will be spec.alpha2 right?#2020-06-1117:59dev.4openIDI am assuming it will be spec.alpha2 right?#2020-06-1118:39Alex Miller (Clojure team)Iām hoping it will just be clojure.spec#2020-06-1118:25seancorfield@dev.4openid I'm not Alex but my understanding is that Spec 2 will eventually become just clojure.spec
when it is ready to come out of alpha -- and there's no timeframe for it yet, based on what I've seen/heard, since a lot of things are still being designed/revised.#2020-06-1118:25seancorfieldThe repo is here https://github.com/clojure/spec-alpha2#2020-06-1118:28seancorfieldFor a while, I kept a branch of our codebase current against that repo but it involved quite a few changes (and it had to keep changing as Spec 2 changed). I stopped tracking it back in ... September/October I think? Our approach going forward -- once Spec comes out of alpha -- will be to use the new Spec for new code we write and slowly, over time, migrate our Spec 1 code to "Spec 2" as needed. Spec 1 will stay available as-is "forever" so folks can stay on the old version if they don't want to migrate.#2020-06-1118:35dev.4openIDThx#2020-06-1121:06plinshow can I create a generator from an arbitrary function?
hereās a silly example but It would be something like (s/def ::name (s/spec string? :gen (constantly "Margaret")))
not sure if possible but id like to combine it with a lib like https://github.com/paraseba/faker to generate test data#2020-06-1121:29Alex Miller (Clojure team)s/with-gen
#2020-06-1121:29Alex Miller (Clojure team)s/with-gen
#2020-06-1203:14practicalli-johnHow can I specify the number of tests run when using clojure.spec.test.alpha/check
? By default its running 1000 tests for a check against 1 spec (the playing cards example from https://clojure.org/guides/spec#_a_game_of_cards ) and takes around 80 seconds to complete.
The docs mention :num-tests
within clojure.spec.test.check/opts
but either I have the syntax wrong or missing something
;; runs 1000 tests
(spec-test/check `deal-cards
{:num-tests 1})
;; java.lang.RuntimeException
;; Invalid token: ::clojure.spec-test-check/opts
(spec-test/check `deal-cards
{::clojure.spec-test-check/opts {:num-tests 1}})
Apart from Clojure 1.10.1, the project includes the dependency :extra-deps {org.clojure/test.check {:mvn/version "1.0.0"}}
Requiring clojure.spec.test.check
generates an error when the namespace is evaluated
java.io.FileNotFoundException
Could not locate clojure/spec/test/check__init.class,
clojure/spec/test/check.clj or clojure/spec/test/check.cljc on classpath.
Project code is at https://github.com/practicalli/spec-generative-testing if it helps...#2020-06-1204:06seancorfield@jr0cket There's no such namespace: https://github.com/practicalli/spec-generative-testing/blob/prime/src/practicalli/spec_generative_testing.clj#L6#2020-06-1204:07seancorfieldThe option should be a qualified keyword -- but that doesn't mean a namespace exists.#2020-06-1204:09seancorfield:clojure.spec.test.check/opts
#2020-06-1204:11seancorfieldIf you want to use ::stc
you can introduce an alias:
user=> (alias 'stc (create-ns 'clojure.spec.test.check))
nil
user=> ::stc/opts
:clojure.spec.test.check/opts
user=>
#2020-06-1204:12seancorfieldYour ::clojure.spec-test-check/opts
is going to fail because ::
will try to auto-resolve clojure.spec-test-check
which is not an alias.#2020-06-1204:12seancorfieldYour ::clojure.spec-test-check/opts
is going to fail because ::
will try to auto-resolve clojure.spec-test-check
which is not an alias.#2020-06-1205:08David PhamSpec2 seems so cool. Increased programmability is such a good advantage. I have been using custom macros to generate spec, and it was a bit odd.#2020-06-1207:18AronSorry if this question has an easy answer discoverable that I missed, didn't do a deep dive. I see that https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#select is available in alpha2 but not in alpha. Obviously lot of people use spec already, but which version?#2020-06-1207:22mpenet"spec1" (spec.alpha)#2020-06-1207:23mpenetit's not really clear what will/wont change in spec2 and there are also a few bugs#2020-06-1207:31AronThanks!
is there something similar to spec/select already existing, perhaps written by someone else, published under a different name?#2020-06-1215:42seancorfield@ashnur Spec 2 will eventually become the official clojure.spec
. It's just not ready for use yet as it is still being actively designed and changed. Spec 1 will remain available for everyone already using it. I don't think anyone has tried to copy Spec 2 -- because it is not yet complete (and why would anyone try to recreate an official part of Clojure when Rich himself hasn't fully figured out parts of the design?).#2020-06-1216:03AronI am just curious if I want to use select or something that does similar stuff to select, what are my best options currently.#2020-06-1216:11seancorfield@ashnur If you're just building toy stuff or learning/experimenting, you could use Spec 2. It's just not ready for production use.#2020-06-1216:13seancorfieldWe were tracking it at work, with a branch of our (95k lines) codebase, and I really like the changes in Spec 2 -- above and beyond the schema
/`select` stuff -- but there's been a lot of churn in Spec 2 and Alex has said that Rich will likely overhaul s/fdef
completely before it is released (and it may drop s/keys
completely as well), so we stopped tracking it months ago.#2020-06-1216:13seancorfieldWe're just going to wait for it to be "fully baked" at this point.#2020-06-1216:14seancorfieldOnce Alex signifies that it is stable and just needs testing to help iron out the bugs, we'll pick it up again.#2020-06-1216:14Alex Miller (Clojure team)I will signify that by making a release :)#2020-06-1216:22Aronso, if I understand that correctly, there is nothing else that targets the same problem domain to be used in production in the interim?#2020-06-1216:26seancorfieldclojure.spec.alpha
targets that domain and can be used in production. Spec 2 is the "next generation" of that and will be the preferred solution when it becomes ready.#2020-06-1216:27seancorfieldSpec 2 is definitely "better" than Spec 1 -- because it's designed to incorporate lessons learned from the first version. But Spec 1 definitely has value in production.#2020-06-1216:36AronI like what s/select does and would like to use something like it in production, even if it's not necessarily exactly the kind of s/select that is planned by Rich, since as I understand it, there are still months, maybe years until that version will be ready.#2020-06-1216:47Alex Miller (Clojure team)god I hope it's not years :)#2020-06-1217:27mishahow can not
be expressed in spec? to maximize built-in generators reuse and composability (really just ability to wrap any spec in "it" without looking at spec/form I am wrapping)#2020-06-1217:28misha(before you look at me funny, I am writing a translator from json-schema to clojure-spec, particularly https://json-schema.org/understanding-json-schema/reference/combining.html#not)#2020-06-1217:33mishaanything better than this?
(do
(s/def ::foo string?)
(s/def ::bar (s/with-gen
(complement (partial s/valid? ::foo))
#(s/gen any?)))
(s/exercise ::bar))
#2020-06-1217:33Alex Miller (Clojure team)not really#2020-06-1217:33Alex Miller (Clojure team)not is weird and I would generally avoid doing it :)#2020-06-1217:35mishaat this time, I am trying to generate spec as close to schema as possible, as code you than paste into file, and then might chose to change#2020-06-1217:37mishaAlex, is there an (out the box) way to conform
unqualified map and get qualified conformed map back?#2020-06-1217:40misha(do
(s/def :my/foo string?)
(s/def ::map (s/keys :req-un [:my/foo]))
(s/magic-conform ::map {:foo "x"}) #_=> {:my/foo "x"})
#2020-06-1218:21seancorfieldI'm not Alex @misha but I can't think of any easy way to do that. You'd probably have to derive the (qualified) keys from the s/form
of the Spec, and then zipmap
with a version of those keys that had been mapped to unqualified keys, and then use clojure.set/rename-keys
on your validated data.#2020-06-1218:22seancorfield(but that won't work with nested data structures/specs or anything more complex than just s/keys
)#2020-06-1218:25Alex Miller (Clojure team)You could unform#2020-06-1218:25Alex Miller (Clojure team)I guess you still wouldnāt get unqual#2020-06-1218:26Alex Miller (Clojure team)So Iāll go with no :)#2020-06-1218:57mishaI thought about just including both :req and :req-un sets of keys, but it does not solve "I have a map from example page, show me the specs it uses", and screws up the generators, which you probably want to generate either entirely qualified or entirely unqualified deep tree.#2020-06-1219:03seancorfieldIf we were starting again from scratch with Spec available, and using next.jdbc
instead of clojure.java.jdbc
, I think we would only have unqualified keys at the boundary of our system: either as API input or user input (forms, URLs), and at outgoing boundaries for JSON-based systems. So our use of :req-un
/`:opt-un` would be a lot smaller, and we'd explicitly transform validated input into a domain model that always used qualified keys. Interacting with JDBC via next.jdbc
means you can use qualified keys going out to the DB and you would get qualified keys coming in from the DB as well, automatically.#2020-06-1219:04Joshua SuskaloI'm playing around with custom generators, since there's a type which it seems like spec is having a hard time generating for some tests.
(s/def ::value pos-int?)
(s/def ::name keyword?)
(s/def ::symbol (s/keys :req [::value ::name]))
(s/def ::symbols (s/coll-of ::symbol :kind set?))
(s/def ::rows pos-int?)
(s/def ::columns (s/and pos-int?
#(>= % 3)))
(def machine-gen
(gen/let [machine (gen/fmap
(fn [[cols rows]]
{::rows rows ::columns cols})
(gen/tuple (gen/fmap (partial + 3) gen/nat)
(gen/fmap inc gen/nat)))
symbols (gen/vector-distinct (s/gen ::symbol)
{:min-elements (inc (::rows machine))})]
(assoc machine ::symbols symbols)))
(s/def ::machine (s/with-gen
(s/and (s/keys :req [::symbols ::rows ::columns])
#(> (count (::symbols %)) (::rows %)))
(constantly machine-gen)))
The problem is that whenever I try to sample the machine-gen, it works fine, but if I try to sample the result of (s/gen ::machine)
it always says that a such-that isn't met after 100 tries.
What would be the cause of this?#2020-06-1219:04Joshua SuskaloI'm playing around with custom generators, since there's a type which it seems like spec is having a hard time generating for some tests.
(s/def ::value pos-int?)
(s/def ::name keyword?)
(s/def ::symbol (s/keys :req [::value ::name]))
(s/def ::symbols (s/coll-of ::symbol :kind set?))
(s/def ::rows pos-int?)
(s/def ::columns (s/and pos-int?
#(>= % 3)))
(def machine-gen
(gen/let [machine (gen/fmap
(fn [[cols rows]]
{::rows rows ::columns cols})
(gen/tuple (gen/fmap (partial + 3) gen/nat)
(gen/fmap inc gen/nat)))
symbols (gen/vector-distinct (s/gen ::symbol)
{:min-elements (inc (::rows machine))})]
(assoc machine ::symbols symbols)))
(s/def ::machine (s/with-gen
(s/and (s/keys :req [::symbols ::rows ::columns])
#(> (count (::symbols %)) (::rows %)))
(constantly machine-gen)))
The problem is that whenever I try to sample the machine-gen, it works fine, but if I try to sample the result of (s/gen ::machine)
it always says that a such-that isn't met after 100 tries.
What would be the cause of this?#2020-06-1219:22mishaSean, my initial motivation is exploration, specifically of https://vega.github.io/
So I want to generate spec from schema (which is 9999km long), then take an example json, and with magic-qualify-conform see, which spec is that, and then navigate through keywords and specs in my IDE, instead trying to find things in huge json schema: https://vega.github.io/schema/vega-lite/v4.json or https://vega.github.io/schema/vega/v5.json
This, and, the usual spec goods: exercise, etc.#2020-06-1219:23mishaso it seems I'd have to come up with "qualiform" too.#2020-06-1219:34seancorfieldYeah, I can definitely see the utility of this and it would be nice as an option in s/conform
.#2020-06-1219:35seancorfieldI think it's interesting that Spec 2 takes a different approach, where you can specify unqualified keys inline in a hash map spec or else qualified keys in a schema
, to be select
'ed#2020-06-1220:26Joshua SuskaloSearch is being very unhelpful for this problem, seems like few people run into it. @seancorfield would you happen to know of anything I could do to debug this issue or to alter the generator so that it'll work?#2020-06-1220:30seancorfield@suskeyhose what is gen/
in your code above? I gather it's not clojure.spec.gen.alpha
...?#2020-06-1220:32Joshua SuskaloIt's clojure.test.check.generators#2020-06-1220:35Joshua SuskaloMy understanding was that clojure.spec.gen.alpha was just a namespace that re-exposed some of the test.check vars.#2020-06-1220:37Joshua SuskaloWhich seems to be true looking at the source.#2020-06-1220:41seancorfieldHahaha... OK, it took me a while... What is ::symbols
? What does it generate?#2020-06-1220:41seancorfieldAnd then in machine-gen
, what type is symbols
?#2020-06-1220:42Joshua Suskalosymbols is just a set of ::symbol, which are just maps with ::name and ::value#2020-06-1220:43seancorfield::symbols
is a set. symbols
in machine-gen
is a vector.#2020-06-1220:43Joshua SuskaloOh boy, of course that's it#2020-06-1220:44seancorfieldThat was a nice Friday afternoon debugging diversion -- thank you! š#2020-06-1220:44Joshua SuskaloThanks for helping me out! I feel so dumb when I just get my types misaligned like that š#2020-06-1220:46seancorfieldNo worries. I couldn't see it either. And I was simplifying the ::machine
spec trying to figure out what the problem was... I was quite bewildered by it!#2020-06-1422:35mishais there a sensible way to specify min/max items count for s/?
s/*
other than function predicate, like?:
(s/and ::my-cat-spec #(< 5 (count (s/unform ::my-cat-spec %))))
#2020-06-1422:38sgepigon@misha If youāre using regex specs, you probably want s/&
instead of s/and
#2020-06-1422:39sgepigonso (s/& ::my-cat-spec #(< 5 (count %)))
#2020-06-1422:42mishayou are probably right about s/&, but I still will have to s/unform#2020-06-1422:44mishaactually, since I apply custom pred to the top spec, it does not seem to matter whether I use s/and
or s/&
#2020-06-1422:46mishathe reason I hope there is another way, is because I'd like to avoid unform#2020-06-1422:54sgepigonHmm I guess Iām still unsure why you need s/unform
. Perhaps thereās something about ::my-cat-spec
Iām missing:
$ clj
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::points (s/+ int?))
:user/points
user=> (s/conform (s/& ::points #(< 5 (count %))) [0 1 2 3 4 5])
[0 1 2 3 4 5]
#2020-06-1422:56misha(s/def :foo/bar (s/cat :1 (s/+ (s/or :a int?))))
(s/explain
(s/& :foo/bar #(->> % (s/unform :foo/bar) count (= 3)))
[1 2 3])
;Success!
=> nil
(s/explain
(s/& :foo/bar #(->> % count (= 3)))
[1 2 3])
;{:1 [[:a 1] [:a 2] [:a 3]]} - failed: (->> % count (= 3))
=> nil
#2020-06-1423:00mishain your example conformed int it just int, so unforming is identity. but if you have some branchy spec (s/or, s/alt, s/cat) - it will change the shape of the data, so in custom predicates for s/and and s/& you either have to unform, or validate conformed value (but this requires you knowing exactly what kind of spec is as the first arg to s/and
s/&
)#2020-06-1423:02mishafor the homogeneous collections I can use s/coll
and it has :max-count and :min-count options. I was hoping s/cat has something similar#2020-06-1423:03sgepigonI see. Thanks for providing an example.#2020-06-1423:04sgepigonI believe in spec2 this is ānon-flowing s/and
ā#2020-06-1423:04mishayeah, should have mentioned: need this for spec1 opieop#2020-06-1423:04sgepigonYou can modify the latter one to go into the map and count from there if you really want to avoid s/unform
#2020-06-1423:04sgepigon(s/explain
(s/& :foo/bar #(->> % :1 count (= 3)))
[1 2 3])
#2020-06-1423:05sgepigonbecause you end up putting all the conformed values under the :1
key.#2020-06-1423:05mishawhat about
(s/cat :1 (s/+ (s/or :a int?))) :2 string?)
kappa#2020-06-1423:07mishathe thing is: this is for translating json-schema to clojure spec, so all specs are dynamic, and I don't do this manually. So I need to find the most sane lowest common denominator#2020-06-1423:07mishathe thing is: this is for translating json-schema to clojure spec, so all specs are dynamic, and I don't do this manually. So I need to find the most sane lowest common denominator#2020-06-1506:55ikitommiinteresting. any code to share?#2020-06-1508:29Aronsecond that. I will have to do something similar soonish, and I am not even sure where I will begin : )#2020-06-1509:34mishawithin this week, I hope#2020-06-1423:18sgepigon(defn conformed-count
[conformed]
(->> conformed
vals
(reduce (fn [acc x]
(if (coll? x)
(+ acc (count x))
(inc acc)))
0)))
(s/conform
(s/& :foo/bar #(= 4 (conformed-count %)))
[1 2 3 "blah"])
#2020-06-1423:18sgepigon(defn conformed-count
[conformed]
(->> conformed
vals
(reduce (fn [acc x]
(if (coll? x)
(+ acc (count x))
(inc acc)))
0)))
(s/conform
(s/& :foo/bar #(= 4 (conformed-count %)))
[1 2 3 "blah"])
#2020-06-1423:22mishabut what if it is just int?
instead of s/cat
? troll#2020-06-1423:22sgepigonlol yeah#2020-06-1423:24sgepigonFWIW hereās the non-flowing s/and
I mentioned: https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new#2020-06-1423:33mishathanks!#2020-06-1423:19sgepigonYeah I donāt think thereās a great solution for this now in spec1.#2020-06-1423:20sgepigons/unform
might be your best bet.#2020-06-1423:22mishaso far, it seems very much like it, yes#2020-06-1707:08tianshuI'm trying to use s/coll-of ... :kind map?
and s/or
(s/def ::k-str (s/tuple keyword? string?))
(s/def ::k-num (s/tuple keyword? number?))
(s/def ::kv (s/or
:k-str ::k-str
:k-num ::k-num))
(s/conform (s/coll-of ::kv :kind map?) {:a 1 :b "2"})
;; => {:a [:a 1], :b [:b "2"]}
I have no idea what the conform result should be, but this one looks not right#2020-06-1707:28vlaaad@doglooksgood what about map-of?
sandbox=> (s/conform (s/map-of keyword? (s/or ::string string? ::number number?)) {:a 1 :b "2"})
{:a [:sandbox/number 1], :b [:sandbox/string "2"]}
#2020-06-1708:23tianshu@vlaaad Thanks for the advice!
But I want care about both key and value.
You remind me, since there's a map-of
, so coll-of
is only designed to work with sequences?#2020-06-1708:29vlaaad@doglooksgood figured it out after reading coll-of
docstring:
(s/conform (s/coll-of (s/or
:kw->num (s/tuple keyword? number?)
:kw->str (s/tuple keyword? string?))
:kind map?
:into [])
{:a 1 :b "2" :c 1})
=> [[:kw->num [:a 1]] [:kw->str [:b "2"]] [:kw->num [:c 1]]]
#2020-06-1708:30vlaaadby default it conforms to coll of the same type as input coll. :into
overrides this#2020-06-1708:41tianshu@vlaaad thanks! don't know there's a into
option.#2020-06-1708:41tianshuthis is what I'm looking for#2020-06-1712:14borkdudeRe: god I hope it's not years š (https://clojurians.slack.com/archives/C1B1BB2Q3/p1591980460308000)
Reminded me of:
> We're hoping to have something releasable by Conj or the end of this year
Source: https://youtu.be/KeZNRypKVa4?t=1201
Approaching one year away from that talk soon š#2020-06-1712:59Alex Miller (Clojure team)"soon" is still many months :)#2020-06-2008:16Vincent Cantinwhat are the most difficult parts?
design or implementation?#2020-06-1713:00Alex Miller (Clojure team)I didn't say which Conj#2020-06-1713:08borkdudelol š#2020-06-1713:48AronA school administrator in a speech said once (and I heard it with my own ears), "in 2000, by 9 o'clock, we will finish the new school building!"#2020-06-1803:36jumar@scott.silver asked this in #cider: https://clojurians.slack.com/archives/C0617A8PQ/p1592435925498900
I'm also curious what's your approach with stest/instrument
and evaluating a clojure buffer since it must be a problem with other editors too.
In short, after you evaluate a ns buffer all the instrumentation is gone and you need to call instrument again manually which is cumbersome and easy to forget.
Developers then commit spec errors which are only found much later, etc.#2020-06-1803:56seancorfieldIf I'm using instrument
with dev/test, I have a call to instrument
as part of each test namespace. See https://github.com/seancorfield/next-jdbc/blob/develop/test/next/jdbc_test.clj#L21 (and similar lines in every test ns in next.jdbc
).#2020-06-1803:58seancorfieldYou could make it part of you fixtures if you wanted to ensure it was added for every test you ran via the REPL from your editor but I think this is enough to ensure a full test run uses specs.#2020-06-1804:29jumarInteresting, but I'm more worried about real "src" namespaces, rather than tests... do you have instrument calls there as well?#2020-06-1804:37seancorfieldNo. Why would you?#2020-06-1804:37seancorfieldYou want instrumentation in place for testing, not production.#2020-06-1817:57Scott SilverI agree that you donāt want it in production, but itās very useful in dev, not just in test, yeah?#2020-06-1818:03seancorfieldI run tests during dev so... š#2020-06-1818:05seancorfield(also, I only spec API boundaries in general, or functions that have some unusual aspects -- we have over 600 specs in our code but only 30-some function specs)#2020-06-1818:06Scott Silverdo you run tests manually while developing? or do you use something like lein-test-refresh
to run the test suite when you make changes?#2020-06-1818:09seancorfieldI have hot keys bound to run: a) all tests in the current namespace b) all tests in a test namespace that corresponds to the current namespace c) the test under the cursor#2020-06-1818:10seancorfieldSo it is manual but very easy. I don't use lein
at all and I don't like watcher-based tasks or any sort of reload/refresh workflow.#2020-06-1805:38jumarWe enable instrumentation for all fdef-s in dev mode (when running in REPL) to catch issues early - it's not enabled in production#2020-06-1813:11practicalli-johnI am assuming the following ways to define a spec are equivalent ? The seem to provide the same results with confirm/valid? etc.
(def suit? #{:clubs :diamonds :hearts :spades})
(spec/def ::suit #{:clubs :diamonds :hearts :spades})
Is it overkill to define a spec for a set as it already acts as predicate function? Or does it make little or no difference?#2020-06-1813:29Alex Miller (Clojure team)either is fine. s/form probably works better with sets#2020-06-1913:50David PhamI have a tricky bug: it says that I have an undefined variable in cljs.alpha.spec at line 479 at the gen* method. However, I never referece cljs.spec.alpha (I use clojure.spec.alpha).#2020-06-1913:50David PhamAnyone encountered something similar?#2020-06-1913:52David PhamThis behavior only manifest itself when I am suing advanced compilation#2020-06-1913:59David PhamOkay, solve my problem by putting my specs namespace into the main module.#2020-06-2001:29practicalli-johnAny suggestions on how to create a custom generator for this specification for an email address (or an alternative specification for an email address that can be used - I could just use string until I get chance to think about it properly...)
(def email-regex #"^[a-zA-Z0-9._%+-]
When I use clojure.spec.gen.alpha/generate
on the returned generator from (clojure.spec.alpha/gen ::email-type)
I get an exception (my other specs will all generate data, just not this. So I am assuming I need a custom generator (it will be my first)
The error I get from generate is
Unhandled clojure.lang.ExceptionInfo
Couldn't satisfy such-that predicate after 100 tries.
{:pred #function[clojure.spec.alpha/gensub/fn--1876],
:gen {:gen #function[clojure.test.check.generators/such-that/fn--6372]},
:max-tries 100}
#2020-06-2513:23djtango@jr0cket sorry to be that guy John but https://stackoverflow.com/questions/201323/how-to-validate-an-email-address-using-a-regular-expression#2020-06-2513:23djtangothough if you're just doing this for fun, then w/e#2020-06-2513:44practicalli-johnEverything I do is for fun, unless someone is paying me a 6 figure sum to do otherwise (although I did still have some fun working at the bank).#2020-06-2814:02djtangoš#2020-06-2002:03gfrederickscheck the test.chuck (no typo) library#2020-06-2002:06seancorfieldYeah, a big +1 for test.chuck
-- we use the regex generator a lot!#2020-06-2008:44borkdude@jr0cket Additionally, lambdaisland/regal also has generators for regexes#2020-06-2215:29buttergunsIs this the right place to ask about spec-tools
? If so: I'm trying out Swagger2 generation. I'd love to make use of https://swagger.io/specification/v2/#referenceObject to avoid duplicating the same definitions all over my json doc. Does spec-tools
support this?#2020-06-2218:07ikitommi@mattmorten currently, no.. I think it has been in ring-swagger/plumatic schema for years, just not ported into spec-tools. PR welcome#2020-06-2218:08buttergunsThanks for the response @U055NJ5CC, I'll look into it#2020-06-2423:12misha#2020-06-2508:36ikitommiHow complete are you aiming this to be? Could it generate something else than spec syntax? There is work ongoing in making JSON Schema -> #malli and would like to see this for #schema too#2020-06-2508:38mishaI am doing this just to get familiar with vega
enough to implement few charts for my projects troll#2020-06-2508:39mishaso for now I am going to finish spec1alpha, and then (but don't know when) - spec2#2020-06-2508:44mishaI need to finish few combinator use cases, such as oneOf
, and see what can be done for string
spec (all the regexes, etc.), then I'll package it up as a lib. The main purpose would be to generate spec forms, and it will be up to user what to do with them: paste into file, etc. Now, I am not aiming for it to be fast, just maintainable.#2020-06-2508:49mishaI am not very familiar with plumatic-schema, and only have seen few slides of mali, but I think it would be even easier to translate to those, given: that would be second/third implementation, and those support nested inline anonymous declarations.#2020-06-2516:16denikanyone have an idea why this returns invalid?
(s/conform (s/cat
:kvs (s/and (s/* any?) (s/conformer identity)))
[:foo :bar])
=> :clojure.spec.alpha/invalid
#2020-06-2521:00borkdude@denik
(s/conform (s/cat
:kvs (s/& (s/* any?) (s/conformer identity)))
[:foo :bar])
#2020-06-2708:40Eccentric JDoes anyone have or know of any examples of using spec with reagent views?#2020-07-0615:08Joe LaneHey @jayzawrotny, I'm a little late to the party, but I found this blog post to be very apropos https://juxt.pro/blog/cljs-apps#2020-07-0615:10Eccentric JThanks @U0CJ19XAM #2020-06-2709:18borkdude@jayzawrotny I do have one example from a commercial app I've worked on for the last few years:
(s/fdef dre.components.widget/widget
:args
(s/and
(s/cat
:opts
(s/keys
:req-un [::title ;; title in header
::content ;; Contents to show in widget. It must be a Hiccup
;; vector. Widget does not support varying arguments in
;; content, they are for initialization only.
]
:opt-un [::id ;; unique identifier for widget (page wide, not app
;; wide)
::icon ;; icon to show in header
::widget-class ;; CSS class
::collapsable? ;; If it can be collapsed
::init-collapsed? ;; initialize the widget in a collapsed
;; state?
::help ;; help contents which are shown in modal
::controls ;; Arbitrary component rendered in the
;; header. What it does is up to you.
::selections ;; coll of dropdown/tabs
::dropdown ;; convenience option, singular version of
;; selections with
;; {:type :dropdown, :id :dropdown} by default
::tabs ;; convenience option, singular version of selections
;; with {:type :tabs, :id :tabs}
::loading? ;; Whether or not to display a loading indicator
::locked? ;; Whether or not content modifications are possible
::menu ;; menu
::on-will-unmount]))
(fn [{:keys [opts]}]
(if-let [unexpected
(seq (apply dissoc opts expected-keys))]
(do
(warn "Unexpected options" unexpected)
false)
true))))
#2020-06-2709:20borkdude@jayzawrotny This is about the only Reagent component I've spec-ed because it has so many options#2020-06-2709:22borkdudeNot sure if it's useful, but this is the app: http://covid19.doctorevidence.com/ (this is a free preview of it)#2020-06-2718:29Eccentric JThanks @borkdude this really helps!#2020-06-2815:52sveriWould it make sense to incorporate alpha 2 into a new project? All the documentation I can find right now is the differences to alpha 1.
Is it enough to read the docs for alpha 1 and the differences page?#2020-06-2815:53Alex Miller (Clojure team)No, you should not use it yet#2020-06-2815:53Alex Miller (Clojure team)Unless you enjoy bugs and breaking changes #2020-06-2815:55sveriOk, good to know, thank you @alexmiller#2020-06-2815:57Alex Miller (Clojure team)There are docs at https://clojure.github.io/spec-alpha2/ btw#2020-06-2816:00sveriHonestly, when looking at a library I always look at a high level doc first instead of api documentation. On the other hand, that api doc is very exhaustive š#2020-06-2816:00sveriSo thanks for the pointer#2020-07-0113:16mishaharold
:1
=> :1
(keyword "foo" "1")
=> :foo/1
:foo/1
Syntax error reading source at (REPL:1:1).
Invalid token: :foo/1
#2020-07-0113:17Alex Miller (Clojure team)there's a lot of tedious history here but keywords with numbers as the name part are technically not valid in the reader#2020-07-0113:18Alex Miller (Clojure team)due to a long-standing bug in the regex for keywords, things like :1
are actually read however and we've decided not to make them invalid#2020-07-0113:18mishabackward compatibility is tough#2020-07-0113:19Alex Miller (Clojure team)and just generally, there are a lot of keywords you can make programatically that are not print/read roundtrippable (https://clojure.org/guides/faq#unreadable_keywords)#2020-07-0113:21mishathis is I am aware of, yes.
trying to choose generated keyword format, suitable for all of: item spec (:foo/i1) cat/or branch spec for that item (:i1), conformed value walking (idx) etc.#2020-07-0113:25misha@alexmiller is it possible to get quoted form of lambda verbatim as edn? e.g. #(+ % 2)
so it could be printed out exactly like this #(+ % 2)
?#2020-07-0113:25misha'#(+ % 2)
=> (fn* [p1__7217#] (+ p1__7217# 2))
#2020-07-0113:25Alex Miller (Clojure team)no?#2020-07-0113:25mishaok :(#2020-07-0113:26Alex Miller (Clojure team)if you're worried about the gensyms, you can transform to (fn [%] (+ % 2))
- spec does this for forms#2020-07-0113:28Alex Miller (Clojure team)(as an aside, there is a ticket and some work on better controlling gensym construction - this is often a tricky case when trying to symbolically compare macroexpanded code chunks)#2020-07-0113:29mishaI am generating lambda forms for generated specs, and often fn
one is way too long, so I might as well just defn
it:
(defn <=10? [x] (<= x 10))
(defn >=5? [x] (>= x 5))
(s/def :user/root (s/and number? >=5? <=10?))
#2020-07-0113:30mishawhich is, arguably, better, than
(s/def :user/root (s/and number? (fn [x] (>= x 5)) (fn [x] (<= x 10))))
#2020-07-0123:55Jan KYou could also do #(<= 5 % 10)
#2020-07-0208:36mishanot gonna:
ā¢ complicates lib code
ā¢ less granular spec errors
ā¢ less similar to source json-schema, so when(if) you gonna debug things ā you will have to recalculate another transformation in your mind #2020-07-0219:03vemvhttps://clojuredocs.org/clojure.spec.alpha/int-in#2020-07-0315:39misha> range from start (inclusive) to end (exclusive).
- only for ints
- (ex)Inclusiveness is baked in
- requires both limits
- specifically range spec was not the point, but readability of inline functions was#2020-07-0113:32mishabut then, there are things like ratios, decimals, etc opieop#2020-07-0320:27mishafwiw https://github.com/akovantsev/json-schema-to-clojure-spec
~80% core functionality complete#2020-07-0420:01adamHow do I refer a spec defined in another namespace? The usual :refer
throws an exception#2020-07-0420:03Alex Miller (Clojure team)you don't need to refer it, just use the fq name#2020-07-0420:03Alex Miller (Clojure team)you do need to load the namespace defining the spec (w/ require)#2020-07-0420:09adamGot it, thanks. Ringās wrap-reload middleware got me confused because it is picking it up even without referring the namespace IF I make some changes to my spec file... it suddenly becomes available everywhere.#2020-07-0420:10Alex Miller (Clojure team)specs are loaded into a global registry#2020-07-0606:01tianshuI want to use s/multi-spec
to validate a map that has a few different structures. And I'm using s/conform
to test if the map is satisfy the spec, but I can't find a way to figure out what branch it is matched since the conform on a multi-spec just simply return a map without more information.
how can I know which branch it matched, like in s/or
.#2020-07-0612:36Alex Miller (Clojure team)I assume you're not just checking a keyword in the map, it's something more complicated#2020-07-0612:38Alex Miller (Clojure team)I don't think you're going to automatically get this, but you could add a conformer to the different cases to "tag" the map with a branch identifier#2020-07-0614:34tianshu@alexmiller I end up call that dispatch function again to get the branch identifier.#2020-07-0807:21rivalHi everyone š
to get my head around spec, I decided to just pick the FIX protocol and spec it. The result is my first Clojure lib https://github.com/rvalentini/fix.core. However the way I used spec felt a little bit "forced" here and there. At some point, I had the feeling that I was wrapping my validation logic into a thin spec wrapper definition for the sake of using spec, without much benefits. I couldn't find a clean/nice way how to build the validation logic as composition of individual specs.
To illustrate what I mean: in
https://github.com/rvalentini/fix.core/blob/master/src/fix/spec/primitives_spec.clj
it felt for me like an "intended" usage of spec, where the individual spec definitions compose quite nicely. But here
https://github.com/rvalentini/fix.core/blob/master/src/fix/spec/message_spec.clj
it felt more like I misused spec, since the spec definition is just like a facade.
Any advice on how I could improve this is much appreciated!#2020-07-0815:05misha@riccardovalentini1854 Spec's strengths are granular errors, parsing, generators. Since you basically use/provide neither of those, and just give "yes/no" answer to "valid?" ā I think, such validation would be "lighter" with instaparse, or just giant regex.
What can be alternatively useful, though, is a speced (with generators) edn DSL + an encode/decode functions to/from that edn data structure to FIX string messages. (pretty much what I'm doing for graphviz dot files, right now)#2020-07-0907:04rival@U051HUZLD Thanks a lot for your feedback!
As a next step I planned to improve the error reporting for the client, so that in case the validation fails, the client receives some message indicating which part of the FIX message is invalid. I could make use of spec's granular error reporting there I think. I will also have a look at custom generators and think about how I could apply them to the internal edn representation of the parsed FIX messages. A nice encoder to produce FIX messages also sounds like a good idea! :+1:#2020-07-0815:13mishauser then can explore "WTH is FIX message, anyway?" with s/exercise, and assemble messages as familiar maps/vectors with core functions, validate, get granular errors, etc, and then convert them into message strings.#2020-07-1022:36walterlHi everyone. I'm still pretty new to spec, and can't figure out what's wrong with this simple spec with a generator fn:
(s/def :offset-date-time
(s/with-gen (partial instance? OffsetDateTime)
#(OffsetDateTime/now)))
(s/gen :offset-date-time)
fails with
Execution error (AssertionError) at clojure.test.check.generators/such-that (generators.cljc:346).
Assert failed: Second arg to such-that must be a generator
(generator? gen)
Any ideas?#2020-07-1022:37gfredericksThat's not a generator#2020-07-1022:37gfredericksGenerators are special objects from spec's generators namespace#2020-07-1022:38walterlAh!#2020-07-1022:38walterlThanks, @gfredericks#2020-07-1217:11cobyHey all, I want to check my intuition about something I'm trying to model. I need to design a small API for taking calendar availabilities (start/end pairs of date-times) and returning available appointment windows (also as start/end pairs, and also taking existing appointment times into account). Spec provides inst-in
which is very nice, but I need to do a bunch of date comparisons, for which it's much nicer (IMHO) to use clojure.java-time
. So would the recommended approach be to convert all datetimes to/from java-time at the edges of the API, and take/return inst
s?#2020-07-1219:24Alex Miller (Clojure team)inst is backed by a protocol so you donāt actually need to convert, just ensure the type youāre using extends Inst #2020-07-1512:27tikotusOnce in a while I'm toying with Clojure, currently I'm trying to learn how to benefit from specs. I just re-watched Hickey's "Maybe Not" talk to support some thoughts running through my head. I ended up wondering about a thing in the proposed schema solution:
In the selection we give ::user [::first ::last] etc.
Now what's the point of ::user? Why not just {::first ::last}? Just for communication? I'm not sure of the value it adds to communication.
One thought I had was that it's there to tell us that we might be using other stuff from the ::user too, like ::addr, we just don't require it. But isn't this then just a briefer, less informative way of saying {:req [::first ::last] :opt [::addr]}? Now we're just one step away from the old spec :keys. Where did ::user go, what was the value of having a schema?
Anyways, are there any news on the next take on spec that I'm missing? Couldn't find anything...#2020-07-1518:44jaihindhreddywith schema you can describe trees and not just one-level. For example, we can say, address is optional, but when you do give me an address, I need the zip code and the street. Before schema and select, this kind of a thing would result in an ad-hoc spec to be written, and we end up with a parochiality problem.#2020-07-1522:03tikotusI still have two questions:
1. What's the use of ::user at the root in this case?
2. Why is schema needed for a nested select? I don't need a schema for ::address to say that if ::address exists in the map, I require it to contain ::zip and ::street.#2020-07-1612:16jaihindhreddyLet's say we're dealing with a scenario where we're dealing with a seller and a buyer, and we need the address of both, then ::seller
and ::buyer
would be useful. With just ::user
, it's not very useful, but if we don't have the ::user
in there, that means it's assumed that the ::address
is that of the user implicitly, making it context-sensitive. With ::user
, we're making it clear that it is user information.#2020-07-1513:07Alex Miller (Clojure team)work has been ongoing on the next release at https://github.com/clojure/spec-alpha2#2020-07-1513:08Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select is current state#2020-07-1513:08Alex Miller (Clojure team)and other differences at https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2020-07-1521:00mikebCool. What's best way to begin trying out spec2, or is that discouraged until it's further along?#2020-07-1521:05seancorfield@mikeb I'd say it is discouraged at this point. We were tracking it closely at work for several months last year but it's still very fluid and very buggy and there's clearly still some "hammock time" needed. After talking to Alex in September/October about it, we decided to wait until he says it is "baked" before we try it again.#2020-07-1521:06Alex Miller (Clojure team)instructions are in the readme if you want to try it as a git dep#2020-07-1521:06seancorfieldI think it's going to be a big improvement over Spec 1, based on our foray into it last year, but be prepared for it to change/break quite a bit as it continues to evolve š#2020-07-1616:12kennyAre there some cases where the generator override passed to st/check
will not actually be the generator used? I seem to be hitting a case like this. I do see this note in the docstring for s/gen
"Note that parent generator (in the spec or overrides map) will supersede those of any subtrees." I'm not sure if this is relevant here. It's also not totally clear what "parent generator" means in this case.#2020-07-1616:14Alex Miller (Clojure team)I believe there are some cases where this doesn't work. the note is talking about recursive cases I think#2020-07-1616:15Alex Miller (Clojure team)iirc one common issue in this area is with aliased specs#2020-07-1616:15kennyAliased specs?#2020-07-1616:16kennyi.e., (s/def ::parent ::child) ?#2020-07-1616:16ghadiyeah#2020-07-1616:17kennyYep, this is definitely an aliased spec#2020-07-1616:17kennyGuessing this fix will be to use the alias.#2020-07-1616:18kennyNot exactly a fit but I think it'll work in my case.#2020-07-1616:19kennyfwiw, my usage here is really a workaround to selecting which (deeply nested) keys are required for a test š#2020-07-1710:58Charlie Briggshello, weāve recently started using specs and have a few questions regarding how to load them correctly
say we have a namespace identity-provider
and wish to have specs for identity-providers in a s_eparate file_, how would we go about doing that? Weāre looking at putting the actual specs into a separate namespace as otherwise weāre having cycling import issues
if we put our specs in a separate file/namespace (say specs.core
, and named them as :core.identity-provider/id
, :core/identity-provider
would we need to then require :refer :all
in consumer namespaces to ensure theyāre loaded?
Would it be more sensible to have a separate file for each entity specs, e.g. specs.identity-providers
? How would the ārootā object for the identity-provider
look in that case (rather than using the name :core/identity-provider
)#2020-07-1715:41seancorfieldYou don't need to refer
specs. Just requiring the namespace they're in will cause them all to be registered.#2020-07-1715:43seancorfieldIt's common to put data-describing specs in a separate namespace from code. As long as you require that spec namespace somewhere (anywhere, early in your execution path), those specs will then become globally available.#2020-07-2008:31Charlie Briggsthanks for your response, so would it be common practise to load all specs in the server entrypoint, or a user.clj
dev-namespaced file for use in the repl, rather than loading them in the code which is using them?#2020-07-1715:43seancorfield@charliebriggs ^#2020-07-1715:52Daniel StephensI don't know if this is at all good practice, but we have a ns per 'entity' for our specs to keep things readable, specs in one namespace can refer to others outside of their namespace by using fully qualified keywords (as normal), and then we have one ns which just requires them all.
Those are all package together as one dependency, then anything that wants to use them just imports the dependency and requires that one namespace.#2020-07-1915:38mishaBy āreadableā you mean ::
?#2020-07-2008:22Daniel StephensAssuming this is related to my response above I just meant readable as in 'more easily read (by humans)', I prefer reading a file with 10 specs in all relating to the same entity than trying to find what I want in a file of hundreds.#2020-07-2106:49tikotusIs utilising specs in linters technically possible, or is it too slow? Is there any tool working on this?#2020-07-2211:25denis_krivosheevHello everyone. I want to write some tests for my library but I really donāt want to write example based tests. I thought spec can be an instrument for property based testing, but couldnāt find a way to transform spec (or generator) into a test. Can someone please help me with that?#2020-07-2212:43Alex Miller (Clojure team)With spec, the main way you do it is by writing function specs and running ācheckā#2020-07-2212:45Alex Miller (Clojure team)Spec generators are also test.check generators so you can use test check property testing too#2020-07-2218:43denis_krivosheevOh this should work. Thank you very much #2020-07-2218:47Alex Miller (Clojure team)in fact, I find using (s/gen a-spec) to often be easier than using test.check to create generators directly#2020-07-2212:46Alex Miller (Clojure team)https://clojure.org/guides/spec covers using check #2020-07-2213:56borkdude@jahvenni https://github.com/arohner/spectrum is one attempt at this. I'm not sure if it's actively being worked on.#2020-07-2306:12tikotusThanks @borkdude, that's what I was looking for. But doesn't seem very active.
Is it just me or wouldn't such a tool be simply amazing? It would make transitioning from statically typed languages feel a lot safer.#2020-07-2314:46seancorfield@jahvenni I think its very misleading to think of anything Spec-related as some sort of analogy to static typing...#2020-07-2314:53borkdudeYes. spec is a runtime validation/conformation/generation library. And considering how complex core.typed gets, I don't think getting leverage out of specs at compile time is trivial. Using a tool for something it's not designed for in general is ... hard.#2020-07-2314:55borkdudeI do think you can get <some> leverage out of it. E.g. a tool could inspect specs at runtime, parse out the trivial stuff like, is the first argument an int?
? Then something like https://github.com/borkdude/clj-kondo/blob/master/doc/types.md can be generated and you will get <some> static analysis benefits for clj-kondo, for example.#2020-07-2314:57borkdudeMalli / @ikitommi has demoed a similar approach at ClojureD.#2020-07-2316:27tikotusYeah, figuring out the trivial stuff is what I had in mind mostly. I can see many trivial cases where a linter utilising specs could easily provide value. Things like order of parameters, names of keyword arguments or tricky get-in's are mentally really heavy for someone who is used to have these things given by autofill. To me these sound like trivial things to check in most cases if spec is provided. It would be a much softer landing to a dynamic language.#2020-07-2316:32borkdude@jahvenni The approached I outlined is open for anyone to implement or experiment with š#2020-07-2316:32borkdudeThis comment specifically: https://clojurians.slack.com/archives/C1B1BB2Q3/p1595516128028300#2020-07-3117:15johanatanHave you considered this: https://github.com/arohner/spectrum ?#2020-07-3117:28borkdudeThat was mentioned as part of that conversation#2020-07-3117:30johanatanOh, sorry. Didnāt scroll back far enough I guess. #2020-08-0612:49arohnerspectrum is no where near working, and Iām not sure whether I will have the time to finish it#2020-07-2318:12tikotusYeah, thanks! Thatās exactly the kind of info I was looking for :)#2020-07-2422:07seancorfieldI don't know how many folks here follow http://ask.clojure.org but it would be nice for someone to go and answer this https://ask.clojure.org/index.php/9486/how-to-consume-the-chan-return-by-go-block-in-cljs#2020-07-2423:47kenny(s/valid?
(s/keys)
report-data)
=> false
(s/explain-data
(s/keys)
report-data)
=> nil
(s/valid? (s/keys) (apply hash-map report-data))
=> true
(type report-data)
=> clojure.lang.PersistentArrayMap
#2020-07-2423:47kennyAny idea what's going on there?#2020-07-2423:56seancorfield@kenny I can't repro with Spec 1. Is that Spec 1 or Spec 2 you're using?#2020-07-2423:56kennySpec 1#2020-07-2423:56seancorfielduser=> (require '[clojure.spec.alpha :as s])
nil
user=> (def report-data {:a 1 :b 2})
#'user/report-data
user=> (type report-data)
clojure.lang.PersistentArrayMap
user=> (s/valid? (s/keys) report-data)
true
user=> (s/explain-data
(s/keys)
report-data)
nil
user=> (s/valid? (s/keys) (apply hash-map report-data))
true
user=>
#2020-07-2423:57seancorfieldWhat sort of keys are in your report-data
map?#2020-07-2423:58kennyKeywords#2020-07-2423:59kennyDid you mean something else?#2020-07-2500:01kennyI can't create a repro locally either. It's something about report-data itself.#2020-07-2711:11Aronwhere should I look for docs/guide for how to test sideeffects on the frontend with specs?#2020-07-2715:03vemvdo you want to test that side-effects happen?
or to avoid them, so that things are more cleanly testable?#2020-07-2715:05Arontest them#2020-07-2715:05Aronsadly, no one pays me for code that is pure š#2020-07-2715:08vemvwhat do you use for frontend?
with re-frame you should have effect segregation for free#2020-07-2715:08Aronthe state is handled by react hooks locally in react components so it's not easy to test state transitions. There are specific situations that I want to avoid, so I was thinking I could run the app with generated data and test that specific situations don't occur. Like there is always a button to click or always a feedback message is shown#2020-07-2715:09Aronthe time I started this project re-frame didn't support hooks, which were required by my boss and colleagues#2020-07-2715:09Aronso I use helix#2020-07-2715:19vemvnot sure if I'd run Spec's generative testing (specifically, spec.test.alpha/check
) with side-effectful functions. IME it's the kind of thing that is painful/slow to implement, and later to debug
It might be cleaner to generate data with Spec, and pass said data to a vanilla integration test (whether a purely frontend-bound one, or a full-stack one aided by Webdriver)#2020-07-2715:21vemvFor completeness, with re-frame I think it's feasible to craft a useful generative test that asserts that interactions always yield a valid application state
e.g. click X + click Y, in any order, generate events a, b, c all of which are valid and form a valid new app state
All this side-effect free (thanks to the indirection bought by events)#2020-07-2715:46AronThe first suggestion with the integration test makes sense, it's what direction I was going, but I wasn't sure if there is no other way.#2020-07-2715:46AronThe second part with re-frame I don't understand, or maybe I understand just don't see how it is relevant?#2020-07-2720:05robertfwI do stateful generative testing but not on the frontend, so I can't help with specifics there. But I can confirm what @U45T93RA6 says - it is a bit painful and slow to implement. I've gone through a few iterations and am finally at a spot where I think we are taming the beast, but it's taken some work getting there and we have some work to go to get it running in a more reasonable amount of time.#2020-07-2720:10vemv> The second part with re-frame I don't understand, or maybe I understand just don't see how it is relevant?
it's relevant because it attempts to show how one could write a generative, integration-ish side-effect-free test using a specific framework.
Don't know helix
myself so yeah it may have limited utility :)#2020-07-2720:25AronI still don't understand, are you saying I should use a different library? Or framework, although I do not use frameworks so I hesitate to use the word framework.#2020-07-2721:01vemvno, I'm stating a technique that may be translated to other (but not all) libraries/frameworks#2020-07-2721:02AronWouldn't that mean that I would have to start over and write the whole thing in a different style?#2020-07-2721:02Aronso are you saying this for consideration for future projects?#2020-07-2721:08vemvIDK, I don't know helix so I don't know how practical the described techique could be for you. In the end it's just food for thought :)#2020-07-3010:24arohnerAre there any libraries for parsing swagger/openAPI into specs? I see https://github.com/metosin/spec-tools, but that appears to only do generation, not parsing#2020-07-3010:36misha@arohner this might be useful to you: https://github.com/akovantsev/json-schema-to-clojure-spec#2020-07-3010:37arohnerthanks#2020-08-0602:10Oliver GeorgeI keep wanting ::x/blah to expand to :current.namespace$x/blah
when there's no x alias. It'd open up "private namespaces" for spec use.#2020-08-0602:10Oliver GeorgeMy use case is CLJS function arg specs. I have many fns in a ns. They expect different things from a :db key passed in as context. (re-frame handlers)#2020-08-0602:13Oliver GeorgeWith the proposed change I could do things like this
(ns app.events (:require [clojure.spec.alpha :as s]))
(defn event1 [{:keys [db]}] ...)
(defn event2 [{:keys [db]}] ...)
(s/fdef event1 :args (s/cat :ctx (s/keys :req-un [::event1/db]))
(s/def ::event1/db (s/keys :req-un [::ddb])))
(s/fdef event2 :args (s/cat :ctx (s/keys :req-un [::event2/db]))
(s/def ::event2/db (s/keys :req-un [::form])))
#2020-08-0602:18Oliver GeorgePerhaps a macro would do the job so could add to replace keyword namespaces in a form
(rename-keyword-ns {*ns* :app.events$event1)
(s/fdef event1 (s/cat :ctx (s/keys :req-un [::db]))
(s/def ::db (s/keys :req-un [::ddb]))))
#2020-08-0602:45seancorfieldI suspect Spec 2 (a.k.a. what will eventually become just clojure.spec
š ) will help you here because you can define a schema
will all the possible keys with their specs, all in one place -- and then you can select
the specific keys that are required for each individual handler.#2020-08-0604:29Oliver GeorgeGood point. If/when that lands it'll be an improvement.#2020-08-0604:29Oliver GeorgeI guess it'd look something like this...
(ns app.events (:require [clojure.spec2 :as s]))
(defn event1 [{:keys [db]}] ...)
(defn event2 [{:keys [db]}] ...)
(s/def ::ctx (s/schema {:db ::db}))
(s/def ::db (s/schema {:ddb ::ddb :form ::form}))
(s/fdef event1 :args (s/cat :ctx (s/select ::ctx [:db {:db [:ddb]}]))
(s/fdef event2 :args (s/cat :ctx (s/select ::ctx [:db {:db [:form]}]))
#2020-08-0604:31Oliver George(Guessing at the unqualified key bits)#2020-08-0617:50johanatanhi, does clojure.spec.test.alpha/check
run its test cases in sequence or in parallel ?#2020-08-0618:00Alex Miller (Clojure team)parallel via pmap#2020-08-0618:00johanatanany way to tell it to do sequential? because my code under test is already parallel and is kicking off a bunch of promises#2020-08-0618:01johanatanand the combination of the two is overwhelming the browser i'm afraid#2020-08-0618:10Alex Miller (Clojure team)not currently#2020-08-0618:14johanatanok, no worries. i can just limit the num-tests to a small amount for now#2020-08-0922:19johanatanhi, i have a spec'd function that is taking as input a collection of data where the individual elements have attached metadata via a with-gen
for their spec. however, because stest/check
conforms these inputs to their constituent components (specified via s/cat
) the metadata which was attached to the outer element is lost. what is the recommended way around this? perhaps a way to override "conform" or otherwise propagate the metadata? something else?#2020-08-0922:21johanatani could manually attach the metadata rather than to the outer value to one of its constituents but this doesn't feel like the proper data modeling for the actual situation. i could also add an extra component to the s/alt
but that isn't desirable as it would expose "test-only" data to the code under test (and also generally violate one's tastes with respect to the actual modeling).#2020-08-0922:25seancorfield@johanatan Are you saying that stest/check
is passing conformed data into the function under test? That doesn't seem right since the conformed shape won't necessarily match the input shape...#2020-08-0922:25johanatanno, it's passing it into the :fn
verifier#2020-08-0922:26johanatansorry wasn't clear. stest/check is passing the generated data to the actual function#2020-08-0922:26johanatanbut that data is slightly deformed via conform to the actual :fn
checker#2020-08-0922:26seancorfieldOh, right. So you're trying to rely on input metadata in the :fn
verifier?#2020-08-0922:26johanatan(per the labeling)#2020-08-0922:26johanatanyea#2020-08-0922:27seancorfieldI wouldn't expect metadata to be preserved through that. It doesn't seem like something you should be relying on in :fn
.#2020-08-0922:27johanatanit would be very useful for testing purposes#2020-08-0922:27johanatanthe generators can then inform the tests what the expected behavior should be#2020-08-0922:27johanatanI can't imagine any better way to accomplish that in fact#2020-08-0922:28johanatanand as long as the generators don't use s/alt
s/cat
and the like, this is entirely possible#2020-08-0922:28johanatan(i've done it previously)#2020-08-0922:29johanatani have to step out now so will respond back later this evening. thx#2020-08-0922:31seancorfieldCould you use s/conformer
in the spec to thread the metadata into the conformed result I wonder?#2020-08-1002:44johanatan@seancorfield i'm not sure how to force s/check
to use my conformer
but yes that would seem to work if it were possible to do so.#2020-08-1002:45johanatani wonder if there is a "conformer registry" i could write into ?#2020-08-1002:48seancorfieldNo, I mean directly in your Spec. So it always runs when a Spec is conformed.#2020-08-1002:54johanatansuppose i have a spec defined like so:
(s/def ::an-entity
(s/with-gen
::a-base-entity
#(gen/let [...] (with-meta a-base-entity-conforming-structure {:some :metadata}))))
are you suggesting to use s/and
to weave in a conformer like so:
(s/and (s/conformer ...) ::a-base-entity)
??
or something else?#2020-08-1002:55johanatan@seancorfield ^#2020-08-1002:55johanatanthe base entity in this case looks like this:
(s/def ::re-frame-event
(s/and vector?
(s/cat :event-name qualified-keyword?
:params (s/* any?))))
#2020-08-1002:56johanatanactually i can also just share the derived event:
(s/def ::callback-func-event
(s/with-gen
::re-frame-event
#(gen/let [params (s/gen (s/coll-of any?))
triggers-failure? (gen/frequency [[1 (gen/return true)] [9 (gen/return false)]])
event-name (s/gen qualified-keyword?)]
(with-meta
(concatv [event-name] params)
{:triggers-failure? triggers-failure?}))))
#2020-08-1002:57seancorfieldPut s/conformer
in the second/last slot of s/and
#2020-08-1002:58seancorfieldThat allows you to modify the (conformed) value as part of the conform process without affecting the validation.#2020-08-1002:59johanatanah, ok#2020-08-1002:59seancorfield(s/and ... (s/conformer #(with-meta % {:what :ever})))
-- untested but that's what I had in mind.#2020-08-1003:01johanatanhmm, ok. let me try that#2020-08-1003:02johanatanthe problem with that is that the meta depends on a generated value#2020-08-1003:02johanatanand i need to somehow "attach" that value to the generated structure#2020-08-1003:02johanatanwhile allowing it to survive the first "conform" in the s/and#2020-08-1003:03johanatani.e., the following doesn't work because ::re-frame-event has already stripped away the meta and destructured into labeled data#2020-08-1003:03johanatan(s/def ::callback-func-event
(s/with-gen
(s/and ::re-frame-event (s/conformer identity))
#(gen/let [params (s/gen (s/coll-of any?))
triggers-failure? (gen/frequency [[1 (gen/return true)] [9 (gen/return false)]])
event-name (s/gen qualified-keyword?)]
(with-meta
(concatv [event-name] params)
{:triggers-failure? triggers-failure?}))))
#2020-08-1003:04johanatani could "cheat" and stuff it into the last "param" and allow my conform to strip it out of there and put it in the meta#2020-08-1003:04johanatanbut that kind of violates a sense of purity š#2020-08-1003:05johanatanand would also only work for some specific subset of cases; i.e., we're lucky in the sense that params is an s/*
of s/any
which could accommodate the data#2020-08-1003:06johanatanperhaps i just make the single conformer the "spec" and forget about using s/and ?#2020-08-1003:07johanatanthen do the base conform within that conform#2020-08-1003:07seancorfieldTrue, you could have a completely custom "predicate" that conforms as a spec...#2020-08-1003:07johanatanyea, i think i'll go that route. probably the path of least resistance at this point#2020-08-1003:13johanatanthis works:
(defn metadata-preserving [spec]
(s/conformer
(fn [v]
(let [m (meta v)
res (s/conform spec v)]
(if (= res ::s/invalid)
res
(with-meta res m))))))
#2020-08-1003:13johanatanused like so:
(s/def ::a-spec
(with-gen
(metadata-preserving ::a-base-spec)
#(with-meta ( ... generation here ) {:some :meta})))
#2020-08-1003:15johanatanthanks!#2020-08-1004:10seancorfieldNice!#2020-08-1305:16ikitommia common question for spec-tools has been how to create custom specs that coerce correctly and emit valid JSON Schema (for web-stuff). I though of pasting a sample here too for adding support for ZonedDataTime
#2020-08-1305:17ikitommi#2020-08-1305:23ikitommiwould be happy to throw away the custom wrappers for adding meta-data for specs in favour of core spec supporting that.#2020-08-1313:12respatializedcan I re-use a matched value within the body of spec/cat
for further pattern matching later on by referring to its key? I'm trying something like:
(def lookup {:a 1 :b 2 :c 3})
(s/cat :key keyword?
:contents (spec/* (spec/or :string string? :matched (get lookup :key))))
Is this possible?#2020-08-1313:13Alex Miller (Clojure team)no#2020-08-1313:14Alex Miller (Clojure team)you can s/& at the top level and check constraints with any arbitrary predicate inside the container#2020-08-1313:18respatializedok, thanks! I think the space of possible keys in the lookup
map I'd be relying on is small enough to fully enumerate using individual specs, so I think I'll just do that instead (probably what I should have been doing from the start, but I wanted to see if I could save myself some typing š)#2020-08-1317:35johanatanis it possible to define a spec for a function in a namespace that one doesn't control?#2020-08-1317:35Alex Miller (Clojure team)yes#2020-08-1317:36johanatanhow?#2020-08-1317:36Alex Miller (Clojure team)the same way?#2020-08-1317:36johanatanso like
(s/fdef the-ns/the-func ...) ?
#2020-08-1317:36Alex Miller (Clojure team)yep#2020-08-1317:37johanatanah cool. thx!#2020-08-1317:37Alex Miller (Clojure team)fdef always takes fq symbols and those are just keys in the registry#2020-08-1318:15johanatan:+1:#2020-08-1322:19apbleonardAnyone know whether spec2 is informed by SHACL? https://youtu.be/apG5K3zc4V0#2020-08-1908:08rickmoynihanFYI there is also a #rdf channel on here, it has infrequent but good discussions on it when they occur#2020-08-1908:11apbleonardThanks. Good to know :)#2020-08-1322:30Alex Miller (Clojure team)while I find it's usually a bad bet to say that Rich is not aware of something, as far as I know he is not aware of that. certainly a lot of the design work precedes that talk#2020-08-1322:38Alex Miller (Clojure team)from a quick skim of the video, both Rich and are I well-versed in RDF and OWL and that inspired many things in the design of both Clojure and Datomic (this is covered in some more detail in the History of Clojure paper https://clojure.org/about/history). I'm not sure that SHACL has anything particularly novel in it and it also makes a closed world assumption that's actually intentionally absent from spec to a large degree#2020-08-1322:39Alex Miller (Clojure team)I actually started using Clojure at the semweb startup Revelytix about 10 years ago and wrote a commercial federated SPARQL engine there#2020-08-1322:40Alex Miller (Clojure team)all in Clojure, sadly, no code survives from the company dying :(#2020-08-1418:58apbleonardVery cool you started Clojure in semweb :) I should have asked whether SHACL informed spec2/select as IIUC its all about defining different shapes of data (including closed world for sure) that can be drawn from the same (open world) OWL ontology for different specific use cases - a bit like how select will (I think) fix a shape suitable for e.g. a given function, whereas specs focus on the unchanging characteristics of attribute values in all use cases? (Hope I haven't mangled spec2 there!) I don't think it's novel - I mean it's a web standard ... Thought the specification of a data shape using SHACL as a graph itself was cool.#2020-08-1419:01apbleonardSHACL actually stands for "Shapes Constraint Language" which I hadn't realised :)#2020-08-1811:57rickmoynihan@apbleonard: Iāve noted this similarity before too, but I think the similarity comes more from the property oriented starting point inherent in RDF and spec. i.e. spec and SHACL are similar because they both come from the same starting point, that properties/keywords are first class and more important primitives than the entities on which they exist.
@alexmiller As I understand it SHACL is also open by default; though you can close shapes with sh:closed true
. Itās true though that I think one of the big usecases for SHACL is in being able to close the world; in particular setting the min/max cardinality of a shape to 1, though I think the excitement around SHACL is more that it helps a long-standing area of frustration in RDF, which is that in RDF essentially every property/join can be many->many. Which has historically made it somewhat harder to validate and build user interfaces against it.
In this regard datomic/clojure is I think arguably less open-world, but more pragmatic, in that it closes the world on some properties by allowing essentially cardinality of 1. Sure, OWL does let you sort of do this; by falling back on the non-unique-names assumption, but relying on reasoning everywhere can be impractical.#2020-08-1914:24bhaim123Hi, Would appreciate some help š
What am I doing wrong?
I have a spec for an empty map:
`
(s/def ::empty_map (s/and empty? map?))
and then I build:
`
(s/def ::name string?)
(s/def ::someone1 string?)
(s/def ::someone2 string?)
(s/def ::someone_big (s/or
::empty_map
(s/keys :req-un [::someone1 ::someone2])))
(s/def ::final (s/keys :req-un [::name ::someone_big]))
The idea here that I would be able to pass:
{:name "abc" :someone_big {}}
And it wold pass, since it is an empty map.
But what I get is an error on the required params:
(s/explain ::final {:name "abc" :someone_big {}})
{} - failed: (contains? % :someone1) in: [:someone_big] at: [:someone_big :<NS>empty_map] spec: :<NS>/someone_big
{} - failed: (contains? % :someone2) in: [:someone_big] at: [:someone_big :<NS>/empty_map] spec: :<NS>/someone_big
Any help would be appreciated š#2020-08-1914:48sgepigon@bhaim123 You need to label each branch of the s/or
e.g.
(s/def ::someone_big (s/or :empty-map ::empty_map
:someone (s/keys :req-un [::someone1 ::someone2])))
user=> (s/conform ::final {:name "abc" :someone_big {}})
{:name "abc", :someone_big [:empty-map {}]}
#2020-08-1915:14bhaim123Thanks, but I donāt want to send the keyword empty map, the someone_big should be either an empty map. or a map of 2 keys#2020-08-1915:25sgepigonIām not sure I follow. The spec does capture that itās either an empty map or a map of two keys. Youāre not āsendingā any keywords, youāre labelling alternatives. Both (s/conform ::final :empty-map)
and (s/conform ::final :someone)
are invalid for this spec.#2020-08-1915:26sgepigonThe original ::someone_big
spec you posted was a s/or
with 1 branch: it could only be a map with two keys and it was labelled ::empty_map
.
If you reversed the order it might be clearer why itās wrong:
(s/def ::someone_big (s/or
(s/keys :req-un [::someone1 ::someone2])
::empty_map))
Unexpected error (AssertionError) macroexpanding s/or at (REPL:1:22).
Assert failed: spec/or expects k1 p1 k2 p2..., where ks are keywords
(c/and (even? (count key-pred-forms)) (every? keyword? keys))
#2020-08-1915:27sgepigonNotice the path from the s/explain
you posted:
at: [:someone_big :<NS>empty_map]
#2020-08-1918:33bhaim123Thank you very much @U56NS7GMQ#2020-08-1915:37Alex Miller (Clojure team)as an aside, (s/def ::empty_map (s/and map? empty?))
would be better than what you have as it should gen when the other would not#2020-08-2019:01johanatanHi, I'm running into a crash in some substrate of my testing environment. When calling stest/check
on a particular function, if I increase the number of tests beyond a point or increase the size of the collections involved, i get the following (unfortunately not very illuminating output):
> clojure -A:test
2020-08-20 11:45:17.836:INFO::main: Logging initialized @5575ms to org.eclipse.jetty.util.log.StdErrLog
[Figwheel] Validating figwheel-main.edn
[Figwheel] figwheel-main.edn is valid \(ć)/
[Figwheel] Compiling build test to "target/public/cljs-out/test-main.js"
[Figwheel] Successfully compiled build test to "target/public/cljs-out/test-main.js" in 5.052 seconds.
Launching Javascript environment with script: "./scripts/launch_headless.sh"
Environment output being logged to: target/public/cljs-out/test/js-environment.log
#error {:message "ClojureScript test run failed", :data {:type :end-run-tests, :fail 1, :error 0, :pass 18, :test 4}}
Error: ClojureScript test run failed
at new cljs$core$ExceptionInfo ()
at Function.cljs$core$IFn$_invoke$arity$3 ()
at Function.cljs$core$IFn$_invoke$arity$2 ()
at cljs$core$ex_info ()
at Function.eval [as cljs$core$IFn$_invoke$arity$variadic] (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:64:25)
at redacted$test_runner$_main (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:18:33)
at eval (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:1:26)
at figwheel$repl$eval_javascript_STAR__STAR_ ()
at
at Object.G__12816__2 ()
Execution error (ExceptionInfo) at cljs.repl/evaluate-form (repl.cljc:578).
#error {:message "ClojureScript test run failed", :data {:type :end-run-tests, :fail 1, :error 0, :pass 18, :test 4}}
Full report at:
/var/folders/jz/920rhb8h8xj864001s7_grh80000gn/T/clojure-8321569605138574991.edn
Neither the full report nor the js environment log contains anything of interest. Just stack traces in figwheel main.
(s/def ::any-coll
(s/with-gen
(s/coll-of any?)
#(s/gen (s/coll-of any? :max-count 5))))
(s/fdef concatv
:args (s/cat :x ::any-coll :rest (s/* ::any-coll))
:fn #(let [r (:ret %1)
x (-> %1 :args :x)
rest (-> %1 :args :rest)]
(println (count r) (count x) (count rest))
(= r (apply (partial concat x) rest)))
:ret (s/coll-of any? :kind vector?))
(defn concatv
"Strict version of concat."
[x & rest]
(case (count rest)
0 x
1 (into x (first rest))
(into x (apply concatv rest))))
I can currently execute fine with num tests of 20 but when increasing to 100 or more, the crash occurs.#2020-08-2019:02johanatanThe crash only seems to occur under headless Chrome; when using figwheel-extra-main/auto-testing in the actual browser, it doesn't occur.#2020-08-2019:04johanatanUnfortunately I don't get access to the "seed" when the problem occurs and my printlns do not get executed either so actually diagnosing is difficult.#2020-08-2019:22seancorfield@johanatan At a guess, since it seems both environment-specific and size-specific, I wonder if your (apply (partial concat x) rest)
in the :fn
spec is causing a stack overflow due to a buildup of lazy concat
calls?#2020-08-2019:23johanatanoooh! that's possible. in fact that was the issue that spurred the creation of concatv
to begin with#2020-08-2019:23johanatanwould (mapcat identity colls)
be a better impl ?#2020-08-2019:24seancorfieldI think I would approach testing this via test.check
and properties/generators, rather than trying to use fdef
with :fn
.#2020-08-2019:25seancorfieldAfter all, your :fn
is pretty much re-implementing the function being tested, but using lazy concat
instead.#2020-08-2019:25johanatanthis does use test.check and generators though.
yes, but that's pretty much how all :fn
s end up#2020-08-2019:26seancorfieldYou're misunderstanding my point I think.#2020-08-2019:26seancorfieldYou're basically testing a vector-based concat
implementation against concat
itself -- so your test is going to run into the same problems as concat
.#2020-08-2019:27johanatanah, true. that's why i mentioned (mapcat identity ...)
though. so, basically what we need are "two strict implementations of concat". one to test the other#2020-08-2019:27seancorfieldWhat you should be testing are properties of the result, e.g., count of the result is sum of count of each argument.#2020-08-2019:27seancorfieldSet of values of result is union of set of values in each argument.#2020-08-2019:28seancorfield(I'm currently working through Eric Normand's three courses on Property-Based testing so this is top of my mind right now)#2020-08-2019:29seancorfieldIn the beginner course, he has a specific example of testing some properties of concat
.#2020-08-2019:29johanatanwell i agree that testing the actual sets is better. that supercedes the individual properties#2020-08-2019:30johanatani think there is a difference of philosophy here. if you test for actual equality then you don't need to test for the properties#2020-08-2019:30johanatanthis is what i typically do with my :fn
tests#2020-08-2019:30seancorfieldThe key is to not re-implement the function under test though.#2020-08-2019:30johanatani see the other sort of property-based tests as strictly less powerful / more limited / less robust#2020-08-2019:30johanatanno, re implementation is fine#2020-08-2019:31seancorfieldClearly it isn't š That's what is causing this problem.#2020-08-2019:31johanatanas long as the implementation is in a completely different fashion#2020-08-2019:31johanatanwell laziness is (likely) what's causing this problem#2020-08-2019:31seancorfieldAnd also, you may end up with bugs in your function under test because you accidentally replicated them in your :fn
re-implementation of it.#2020-08-2019:31johanatanso if we have two completely separate strict implementations that both produce the same result, then we're good#2020-08-2019:32johanatansure, but i don't do that š or i typically keep iterating until i eliminate those#2020-08-2019:32seancorfieldYou would probably change your position on this if you took Eric's course.#2020-08-2019:33johanatani find a lot of his material to be a bit more "entry level" than fits my needs. (just in general, having seen a few of his other videos. there are definitely philosophical differences here and his audience is, like you said, beginner-esque)#2020-08-2019:34johanatani've done property-based testing for a while now. it's not like my thoughts on this just formed recently.#2020-08-2019:35johanatani do agree that there are times where checking properties is fine: the canonical example being checking the correctness of an NP-complete problem's solution#2020-08-2019:35johanatanbut a lot of problems in practice do not have "easy checks for correctness"#2020-08-2019:36johanatanbut yea i'll give it a listen#2020-08-2019:37johanatanstill yet, and we're assuming that laziness is the problem here (which it very well could be), there would seem to be a "bug" in the way the failure is presented if you will. i.e., there is literally no trace of helpful information in this case in the actual output š#2020-08-2019:48johanatanand actually just realized mapcat
won't help here since it is a thin wrapper around apply concat
#2020-08-2019:50johanatanalso, btw, nothing about :fn
says that you must do a full reimplementation. you can still do "property" checks in there. of course with the downside being perhaps less readable output upon a subexpression failure. i'm willing to make that tradeoff though for the streamlined / inline specification#2020-08-2019:54johanatani think what i'll do in this case since a second "strict" version of concat seems elusive is a simple pairwise traversal comparing elements along the way#2020-08-2020:20seancorfield> nothing aboutĀ :fnĀ says that you must do a full reimplementation. you can still do "property" checks in there
I would say property checks are better than a re-implementation.#2020-08-2020:22seancorfield> i find a lot of his material to be a bit more "entry level" than fits my needs
I've been doing Clojure in production for a decade and I'm still picking up new ideas from even his "entry level" courses -- but I agree that his style is aimed at beginners in many courses. He has three PBT courses: beginner, intermediate, and advanced -- so I figured since I have a subscription, I might as well watch all of them (I run them at 1.5x speed).#2020-08-2020:22johanatancool, sounds good. i'll give it a look. thanks for your help on this!#2020-08-2020:27seancorfieldIt sounds like fdef
is going to get substantially reworked in Spec 2, based on what Alex has been saying about that. But it's still all in "design mode" right now.#2020-08-2021:01johanatanhm, in what way? btw, which properties would you test here if you were going the "property route" ?#2020-08-2021:01johanataneverything i'm trying to do is hitting the same problem / crash#2020-08-2021:01johanatanlike sum of counts, sum of hashes etc#2020-08-2021:02johanatane.g., this crashes:
(= (count r) (apply (partial + (count x)) (mapv count rest)))
#2020-08-2021:05johanatanperhaps the s/* is causing this to blow up really large. i've read somewhere that it can happen and that it's hard to tweak as "recursion depth" is not all that precise of a control on it#2020-08-2021:05seancorfield> (`fdef`) hm, in what way?
Alex hasn't said... just that it's going to be substantially reworked.#2020-08-2021:15Alex Miller (Clojure team)Rich is working on it, there have been many ideas explored, not sure where it's going to end up#2020-08-2103:37johanatanbtw i think i found the problem
(apply concatv rest)
can blow up the stack / is not in tail position.#2020-08-2103:40seancorfieldOh, interesting... Does that introduce laziness somehow?#2020-08-2103:40johanatan(at least that is "one" problem)#2020-08-2103:40johanatanwell, it's a recursive call#2020-08-2103:41seancorfieldOH! Yeah, that sounds so obvious once you say it out loud! š#2020-08-2103:42johanatanlet me explain. sorry, i noticed that even if I remove the :fn I still get the crash with the original impl of concatv. but with this loop/recur one (inspired by the Stuart Sierra blog on concat
) that crash won't occur:
(defn concatv
"Strict version of concat."
[head & tail]
(loop [seqs (cons head tail) acc []]
(if (empty? seqs)
acc
(recur (rest seqs) (into acc (first seqs))))))
#2020-08-2103:43seancorfieldSeems weird to have head & tail
when you only cons
them together and never use them again.#2020-08-2103:44johanatanoh, could just be [& args]
?#2020-08-2103:44johanatangood point#2020-08-2103:44seancorfieldYou might want to consider whether (concatv)
should work and what it should produce (I'd say yes and []
respectively).#2020-08-2103:45johanatani'm just trying to match behavior of the original concat
(which seems to agree w/ you) š
cljs.user=> (concat)
()
#2020-08-2103:45johanatanand yea that concatv
does similarly:
cljs.user=> (concatv)
[]
#2020-08-2103:46seancorfieldI just tried it in Clojure and got an error#2020-08-2103:48seancorfieldOh, it works after changing to & args
and (loop [seqs args ...
#2020-08-2103:48johanatanhmm, mine was from lumo#2020-08-2103:48johanatanyea, sorry#2020-08-2103:48johanatandidn't paste in here#2020-08-2103:48johanatan(defn concatv
"Strict version of concat."
[& args]
(loop [seqs args acc []]
(if (empty? seqs)
acc
(recur (rest seqs) (into acc (first seqs))))))
#2020-08-2116:45petterikDid you consider (reduce into [] args)
?#2020-08-2120:51johanatanNope, does it work?#2020-08-2120:51johanatanAlso, how much memory will it use? the loop/recur one is tail optimized so should use a constant amount of memory#2020-08-2103:48seancorfieldThe version with [head & tail]
requires 1+ arguments š#2020-08-2103:49johanatanhaha, yea 2+ or 0+ makes more sense than 1+#2020-08-2104:01johanatanhere is a "final" version that works (tested up to 75 iterations at a time):
(s/def ::any-coll
(s/with-gen
(s/coll-of any?)
#(s/gen (s/coll-of any? :max-count 125))))
(s/fdef concatv
:args (s/cat :args (s/* ::any-coll))
:fn #(let [r (:ret %1)
args (-> %1 :args :args)
sumhash (fn [c] (apply + (mapv hash c)))]
(and
(= (count r) (apply + (mapv count args)))
(= (sumhash r) (apply + (mapv sumhash args)))))
:ret (s/coll-of any? :kind vector?))
(defn concatv
"Strict version of concat."
[& args]
(loop [seqs args acc []]
(if (empty? seqs)
acc
(recur (rest seqs) (into acc (first seqs))))))
#2020-08-2104:06johanatanso yea i think i was kinda confounded earlier by the fact that there was a potential stack overflow in the code under test. so even when i got the test code "right" the code under test could screw me#2020-08-2104:07johanatanbut it was somewhat non-deterministic on both sides. bit of an entangled heisenbug#2020-08-2104:07johanatanš#2020-08-2104:08johanatantwo entangled heisenbugs rather#2020-08-2104:08seancorfieldComplected, even š#2020-08-2104:08johanatanhaha exactly#2020-08-2119:04johanatansmall update: so ... there turned out to be 3 problems. my hunch about s/*
was also correct. some of the crashes were emanating from it as well.
here's a "more final" version (at least until I get another random, intermittent failure some time in the future):
(defn arbitrarily-partition [coll freq]
(let [signals (take (count coll) (cycle (cons true (repeat freq false))))
shuffled (shuffle signals)
zipped (map vector shuffled coll)
partitioned (partition-by first zipped)]
(for [c partitioned]
(map second c))))
(s/def ::coll-of-colls
(s/coll-of (s/coll-of any?)))
(s/def ::distinct-coll-of-colls
(s/with-gen
::coll-of-colls
#(gen/let [elems (gen/set (s/gen any?) {:max-elements 150})
freq (s/gen (s/int-in 1 10))]
(arbitrarily-partition elems freq))))
(s/fdef concatenate
:args (s/cat :args ::distinct-coll-of-colls)
:fn #(let [r (:ret %1)
args (-> %1 :args :args)
sumhash (fn [c] (apply + (mapv hash c)))]
(and
(= (count r) (apply + (mapv count args)))
(= (sumhash r) (apply + (mapv sumhash args)))))
:ret (s/coll-of any? :kind vector?))
(defn- concatenate
"Strict version of concat."
[args]
(loop [seqs args acc []]
(if (empty? seqs)
acc
(recur (rest seqs) (into acc (first seqs))))))
(defn concatv
"Strict version of concat."
[& args]
(concatenate args))
#2020-08-2308:23mishaI did not read entire discussion, so it might be irrelevant, but:
there is a :distinct
option in s/coll-of
(s/coll-of int? :distinct true)
#2020-08-2308:25mishaor is it supposed to be "distinct for all 1st and 2nd level items"?#2020-08-2419:40Lennart BuitSo spec (1) has a list of predicates it can create generators for, so (s/gen int?)
works. I kinda forgot, is it possible to extend this on the predicate level, e.g. can I register a predicate so that this works (s/gen my-predicate?)
?#2020-08-2419:46seancorfield@lennart.buit You'd have to write your own generator (and then supply it when you define the spec based on your predicate).#2020-08-2419:47Lennart BuitYah right, so I can only define a spec with s/def
and attach a generator that way#2020-08-2419:47Lennart BuitI canāt extend like this list: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L146#2020-08-2419:47seancorfieldhttps://clojure.org/guides/spec#_custom_generators#2020-08-2419:53Lennart BuitYeah right, reason Iām asking, we have quite a lot of specs that use a (non-spec aware) predicate, e.g. (s/def ::my-spec my-predicate?)
. So I was kinda hoping I wouldnāt have to do like:
(s/def ::my-predicate-spec (s/with-gen my-predicate? (fn [] ...))
(s/def ::my-spec ::my-predicate-spec)
But what Iām finding in the source, I appear to be out of luck#2020-08-2419:55Alex Miller (Clojure team)yeah, this is not currently an open set#2020-08-2419:57Alex Miller (Clojure team)I think the issue is that if you could extend it, then predicates may gen in one place but not in another if you didn't extend it the same way (somewhat reminiscent of reader macros).#2020-08-2420:01Lennart BuitRight ā because you have this global idea of what generators exists for what predicates and that could clash, or not be loaded or ā¦
Yeah I can see that as causing pains#2020-08-2505:11stuartrexkingIs it possible to add a req-un to an existing s/def to create a new s/def. I have a request map with a number of keys, and a response map with all the keys in the request, plus an additional id key. Is there a nice way to do this?#2020-08-2505:13seancorfield@stuartrexking you can use s/merge
to merge two s/keys
specs together.#2020-08-2505:14seancorfield(s/merge ::my-spec (s/keys :req-un [::some-new-key]))
#2020-08-2505:15stuartrexking@seancorfield Thank you.#2020-08-2510:54jacklombardIs there a way to get the results of s/valid? and s/expalin-str in one call?#2020-08-2510:55sogaiuyou don't mean like by using juxt
or something right?#2020-08-2510:56jacklombardNope, want spec itself to return it#2020-08-2510:56jacklombardso that I can destructure say the validity and the error in the same call#2020-08-2511:01jacklombardI am using spec for api validation where I check whether the body or the params are valid, if not I return 400 with error message as s/explain-str against the same spec. Of course the error message is not humanized, but we can deal with it. Donāt want spec to run its validation twice since payloads can be huge#2020-08-2512:10Alex Miller (Clojure team)You could run s/explain-data, check that, then run the function to produce the string from the data#2020-08-2608:43AronWhat's the proper way to generate random emails for testing?#2020-08-2608:46dharriganIf you're using spec you could use a generator#2020-08-2608:46dharrigan
shows one such approach#2020-08-2608:47dharrigan
another#2020-08-2608:51AronYes, I am trying to learn spec to use it : ) Would be a stretch that I am using it now : )#2020-08-2608:53AronI see, so these are valid emails but they only generate a relatively small subset of possible email formats#2020-08-2613:49Joe Lane@ashnur Check out https://github.com/miner/strgen , ignoring that it still uses clojure.spec
in it's examples, I think everything else should work.#2020-08-2613:50AronI was just about to try test.chuck#2020-08-2614:48plexusor (shameless plug): Regal https://lambdaisland.github.io/regal-playground/#2020-08-2717:53AronI tried out regal, and I like it, but I am not entirely sure I understand what is happening.I tried out regal, and I like it, but I am not entirely sure I understand what is happening.#2020-08-2615:14Aronthat link is broken š#2020-08-2615:15AronI am having an extremely frustrating experience while trying out test chuck. Even though the namespace is included, the example code doesn't work, methods like string-from-regexp are nil. I am not even sure in this situation where should I go to ask for help, probably #beginners ?#2020-08-2615:30Alex Miller (Clojure team)here's probably fine - can you share a snippet of what you're doing?#2020-08-2615:31Alex Miller (Clojure team)https://lambdaisland.github.io/land-of-regal/ looks to be the right link#2020-08-2615:37Alex Miller (Clojure team)% clj -Sdeps '{:deps {com.gfredericks/test.chuck {:mvn/version "0.2.10"}}}'
Clojure 1.10.1
user=> (require '[com.gfredericks.test.chuck.generators :as gen'])
nil
user=> (require '[clojure.test.check.generators :as gen])
nil
user=> (gen/sample (gen'/string-from-regex #"([ā-ā„]{3}|B(A|OO)M)*"))
("" "" "BAMBOOM" "āāāāā¢ā“āā”ā" "ā¹āāāā āBOOM" "BAMBAMā„āā¬āāā" "" "BOOM" "ā¤āŗāµāāā»BAMāØā„āāā„āBOOMBOOMāā
ā" "āŖāāā āāBOOM")
#2020-08-2615:47Aronso this only works with java I am guessing#2020-08-2615:48Aronhttps://gist.github.com/ashnur/3f5b7fd20d80a3831fe5a1a4c45bb55a#2020-08-2615:53AronProbably because java regex is not the same as js regex. But then I am probably better off with one of the other options that were mentioned.#2020-08-2615:56Alex Miller (Clojure team)it should work with both#2020-08-2615:57Alex Miller (Clojure team)did you (require '[clojure.test.check.generators :as gen]) ?#2020-08-2615:57Alex Miller (Clojure team)test.chuck builds on top of test.check, so you need to include both of the generator namespaces#2020-08-2615:58Aronhmm#2020-08-2615:58AronI did, but this gives me an idea#2020-08-2616:00Alex Miller (Clojure team)it seems from those errors that you have issues with both the gen and gen' namespaces#2020-08-2616:12hadilsI need help with multi-spec
:
(def a #{1 2 3})
(def b #{4 5 6})
(s/def ::type #{:foo :bar})
;;; This is pseudocode
(if (= ::type :foo)
; I want ::new-sym to be defined as a
else (= ::type :bar)
; I want ::new-sym to be defined as b
; ::type and ::new-sym appear in the same map
I am pretty sure this is done with multi-spec
but I am stumped as to how to make it work...#2020-08-2616:23gfrederickstest.chuck's regex generator is jvm-only#2020-08-2616:36Alex Miller (Clojure team)ah, sorry#2020-08-2616:38Alex Miller (Clojure team)@hadilsabbagh18 s/multi-spec is for the case where you can look at the data and return different specs based on the data#2020-08-2616:38Alex Miller (Clojure team)what is your actual data?#2020-08-2616:40Alex Miller (Clojure team)in general, spec is designed primarily for the case where attributes always have the same spec so you are probably going against that grain a bit.#2020-08-2616:41Alex Miller (Clojure team)if the attribute is in a map, you could create (s/and (s/keys :req-un [::new-sym]) arbitrary-condition))
to tighten a more general spec#2020-08-2616:41Alex Miller (Clojure team)and then use an s/multi-spec to choose which of those applies#2020-08-2616:50hadilsThank you @alexmiller! Here is the actual snippet:
(s/def :budget-item/main-category #{"Household" "Personal" "Savings" "Investments" "Other"})
(defn budget-categories
[main-category]
(into #{}
(mapv first
(d/q '[:find ?cat :in $ ?main :where
[?e :expense/budget-category ?main]
[?e :expense/categories ?cat]] (db) main-category))))
(defmulti ordered-categories :budget-item/main-category)
(defmethod ordered-categories "Household" [_]
(budget-categories "Household"))
(defmethod ordered-categories "Personal" [_]
(budget-categories "Personal"))
(defmethod ordered-categories "Savings" [_]
(budget-categories "Savings"))
(defmethod ordered-categories "Investments" [_]
(budget-categories "Investments"))
(defmethod ordered-categories "Other" [_]
(budget-categories "Other"))
(s/def :budget-item/ordered-categories (s/multi-spec ordered-categories ,,,))
I want to make :budget-item/ordered-categories
dependent on what :budget-item/main-category
is set to. They will be in a map together...#2020-08-2617:12Alex Miller (Clojure team)yeah, I don't think you should do this in a spec#2020-08-2617:13Alex Miller (Clojure team)or just make it a valid-category?
predicate#2020-08-2617:15Alex Miller (Clojure team)just spec ordered-categories as a string?
or keyword?
or whatever, then s/and the whole map predicate with a custom predicate that validates that extracts ordered-categories and main-category, and validates that one is valid according to the other#2020-08-2617:15hadilsOk, I'll do that. Thanks a lot for your help @alexmiller! I am really grateful.#2020-08-2617:16Alex Miller (Clojure team)dependent stuff is hard to do well - it's easiest to just specify that constraint at the level that contains all of the data#2020-08-2710:43AronSo, about this email generation thing. I know I am asking for a lot š. But this is clojurescript, so my expectations are high because of the hype. However, the task to put together a generator for this http://emailregex.com/ in either of the suggested DSLs seem at least as daunting as just doing it from scratch, using built-ins.#2020-08-2710:46AronBasically, I am blocked because it's such a huge task, I don't even want to start, I can write all the necessary tests by not generating emails at all. Some of the hand written email address generators that I have seen suggested or used in libraries are positively naive compared to what is out in the wild. : )#2020-08-2711:17Lennart BuitThe pragmatic approach to validating emails: Check they contain an @
, and send an email to confirm š.#2020-08-2717:55AronNow, what is the pragmatic approach to generate emails in clojurescript? š#2020-08-2717:56Aron(def em-chr [:not "<" ">" "(" ")" "[" "]" "\\" "." "," ";" ":" "@" :whitespace :newline "\""])
(def em-id-wrd [:cat [:+ em-chr] [:* em-chr]])
(def em-str [:cat '\" :any '\"])
(def em-id [:alt em-id-wrd em-str])
(def em-domain [:cat [:+ [:class [\a \z] [\A \Z] :digit "-" "." ]] [:repeat [:class [\a \z] [\A \Z]] 2 63]])
(def email [:cat em-id "@" em-domain])
(prn (gen/generate (regal-gen/gen email)))
#2020-08-2717:57Aronthis is regal, but I am not sure if it will work#2020-08-2718:15vlaaadWhat is the problem you are solving @ashnur?#2020-08-2718:22AronWhich one? š#2020-08-2718:22AronI am writing automated integration tests.#2020-08-2718:59vlaaadfor system that validates emails? š#2020-08-2719:10AronI am not sure why validation is coming up. I didn't speak about validation at all.#2020-08-2719:12vlaaadSorry, too snarky... You need to decide what properties of strings representing emails matter for your tests involving emails. Nothing really matters, it's just stored? Use generator #"
, who cares. Test needs to successfully send an email, but still has to generate different emails? Use my.email+<random-stuff-here>@gmail.com
.#2020-08-2719:18vlaaadbtw your domain generator implies 2-nd level domains like {:tag :mailto:mefoo.commefoo.com, :attrs nil, :content nil}
, but email hosts might be IP addresses (e.g. {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "016c64413033322f3033322f3033322f31"}, :content ("[emailĀ protected]")}
). If your program runs in a corporate network, sysadmins might configure it to have internal resources as 1st-level domain (e.g. just {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "d9b4bc99bab6aba9"}, :content ("[emailĀ protected]")}
).#2020-08-2723:56pandais there a really easy way to make human name generators with spec?#2020-08-2801:53Alex Miller (Clojure team)A set is a valid spec that gens. So override the generator to gen from a known set of names#2020-08-2800:02pandaalso is there a way to get the generators for different items in a nested map to be consistent? i.e. for instance if i had a map like:
(s/def ::begin-date inst?)
(s/def ::end-date inst?)
(s/def ::dates (s/keys :req-un [::begin-date ::end-date]))
how can i get begin-date to be less than end-date in the generated output? sorry if this is documented somewhere š#2020-08-2800:02panda^ appreciate the help in advance š š#2020-08-2800:29seancorfield@lpanda2014 You can wrap the s/keys
part with s/and
and add a predicate to check that #(some-date-lib/after? (::begin-date %) (::end-date %))
#2020-08-2800:45pandathanks! for some reason that didnāt work though š had to change it to ints since i dont have a datetime library in my repl but the below code gave me a null ptr when i went to generate. is my syntax off?
(s/def ::begin-date int?)
(s/def ::end-date int?)
(s/def ::dates (s/and
(s/keys :req-un [::begin-date ::end-date])
#(> (::end-date %) (::begin-date %))))
(gen/generate (s/gen ::dates)) ;; broken
#2020-08-2800:50pandaactually nvm figured it out - the second predicate canāt be :: š thanks for your help!#2020-08-2801:06seancorfieldhttps://github.com/stackoverflow/date-clj is a simple library for manipulating dates, that has before/after comparisons. We use that at work.#2020-08-2800:30seancorfield(most of the date utility libraries have a simple function to check if one date is after another -- don't know what or if you're using)#2020-08-2800:31kennyor built in to Date: #(.after (::end-date %) (::begin-date %))
#2020-08-2800:35kennyNeeds a custom gen too. Something like (gen/fmap #(zipmap [:begin-date :end-date] (sort %)) (s/gen (s/tuple inst? inst?)))
#2020-08-2800:41pandathanks! this worked like a charm
(s/def ::dates (s/with-gen
(s/keys :req-un [::begin-date ::end-date])
#(gen/fmap (fn [d] (zipmap [:begin-date :end-date] (sort d))) (s/gen (s/tuple inst? inst?)))))
#2020-08-2800:42kennyYou'll still (probably) want the predicate added with s/and#2020-08-2800:46pandahm i couldnāt get the s/and
part to work actually#2020-08-2800:46kenny(s/and ::dates #(.after (::end-date %) (::begin-date %)))#2020-08-2800:50pandathanks !#2020-08-2801:08seancorfieldAh, good point. I couldn't remember whether java.util.Date
had comparison built-in (and was too lazy to check the Java API docs š )#2020-08-2919:06Lennart BuitHow would you go about specāing a (http) patch like api using nil
as a sentinel for retraction. Letās say you have an entity that has an optional attribute, Iād hate to say that this attribute is s/nilable
in its global definition just because there is a patch endpoint using nil
to signal retraction#2020-08-2919:06Lennart BuitHow would you go about specāing a (http) patch like api using nil
as a sentinel for retraction. Letās say you have an entity that has an optional attribute, Iād hate to say that this attribute is s/nilable
in its global definition just because there is a patch endpoint using nil
to signal retraction#2020-08-2919:08Lennart BuitLike ā if I send you this attribute, it will either not be there, or it will have a non-nil value.#2020-08-2919:19Lennart BuitI donāt know, it feels asymmetrical: Iāll always promise you either no value at all, or some value satisfying a spec, but you can provide me no value, a value satisfying a spec or nil, but only in this specific case. I guess it doesnāt help me that I can have only one definition for a namespaced keyword (here).#2020-08-2919:56seancorfieldIsn't this just a non-required key in a spec?#2020-08-2919:56seancorfieldIt's either present (and conforms) or it is not present.#2020-08-2920:19Lennart BuitYes, outgoing I would agree: Iāll (= server) never send you a nil
for a key I have no value for and instead omit it. The problem is in how you (= client) tell me that you donāt want this value anymore: If you omit it in your (partial) update request, do you mean to retract it, or to retain it?
So some rest endpoints allow you to patch
an entity, but with nil
as value, meaning āletās get rid of this value for that attributeā. So now we have a bit of asymmetry: Iāll promise you to never send a nil
value, but you can send me a nil
to indicate retraction on this patch
endpoint.#2020-08-2920:20Lennart BuitI donāt know how to express, in spec, that I as a server make a stronger guarantee than you as a client have to when patch
āing entities. If that makes any sense š.#2020-08-2920:34Lennart Buit(Or more specifically, I donāt know how to do that when my keys are namespace qualified. If I were to use s/keys
with :req-un
I could maintain two specs.)#2020-08-2920:41seancorfieldThat's just two different specs (perhaps with reuse on the common stuff). A spec for the result of a call (where the key is optional but spec'd to be non-nil). A spec for the input value (where the key is nilable).#2020-08-2920:43seancorfieldIf it's an API spec, it's going to be for unqualified keys, surely? Since it will be a wire spec, e.g., JSON.#2020-08-2920:49Lennart BuitRight, we may have extended this nilability (or the spec, for that matter) too far into our system. We have a JSON handler that accepts this nil and is spec checked, that is unqualified, but then we have an update method shared between this JSON endpoint and other places that is also checked, but has qualified keys. Thats where friction occurs: the keys are qualified, but their semantics differ between what you supply and what you get.#2020-08-2920:55seancorfieldWell, if you have an internal (qualified) name for that attribute, either it should be optional and non-`nil`, or it should be nilable -- in all cases. And if that's not possible, then the two semantics should have different names.#2020-08-2921:04Lennart BuitSorry for the kinda fuzzy descriptionā¦ Right so in the latter of your options you say to extend the notion of valid values for this attribute to be a superset of all values it can take in all contexts. That means that consumers I could have promised no nil
s (because they consume the āoutgoingā format), have to consider nil
s because thats what I could promises them in my spec, right? I find that kinda sad.#2020-08-2921:07seancorfieldI'm saying use different specs as needed and transform the data to match as you cross boundaries.#2020-08-2921:13seancorfieldAnother option is to chose a specific, unique, representation for a retraction (and, again, map from the inbound retraction to that representation).#2020-08-2921:15seancorfield(we had exactly this situation and we tried to blur the lines with nilable and optionality and it was a mess so we mapped it to a different representation altogether)#2020-08-2921:15Lennart BuitRight that makes sense#2020-08-2921:19Lennart BuitYeah, I think we are going wrong in a similar way: have a single namespace qualified keyword for an attribute in all its contexts, but using it in different ways. Iāll put this in the hammock, thank you for the input, as always š!#2020-08-3010:17borkdudeI'm running a poll about including clojure.spec inĀ babashkaĀ here:
https://twitter.com/borkdude/status/1300428421252747266#2020-08-3014:36borkdudeWill a next Clojure include spec2 and drop spec1?#2020-08-3014:36Alex Miller (Clojure team)tbd#2020-08-3019:56Aronno one knows anything, it's so exciting! š#2020-08-3101:16hadilshttps://clojurians.slack.com/archives/C03S1KBA2/p1598836570469900#2020-08-3102:26rapskalianThe fact that youāre destructuring the key doesnāt change the fact that budget-item-type
simply accepts a map. Assuming you have a spec for :transaction/amount
, your args spec would be something like
(s/cat (s/keys :req-un [:transaction/amount]))
#2020-08-3102:27seancorfield@U6GFE9HS7 That's the answer they got in the other channel.#2020-08-3102:27seancorfieldPlease don't encourage people to cross-post questions here.#2020-08-3102:27rapskalianOops, didnāt see that. Noted @seancorfield #2020-08-3102:23seancorfield@hadilsabbagh18 It's a good idea to not cross-post questions here.#2020-08-3102:23seancorfieldYou've already gotten a response in another channel on this.#2020-08-3102:26hadils@seancorfield Sorry my bad#2020-08-3113:09Petrus TheronHow to constrain the size of two dependent coll-of specs, A & B so that (<= (count (concat A B)) 40)
? AFAICT, I need to use g/bind to constrain the :max-count
of B's sample based on the length of A, but I'm having some trouble getting this to work in Spec2.
@alexmiller you mentioned you are considering basic logic resolution. It would be awesome if it was possible to do something like:
(let [a (s/coll-of digit? :max-count 30)
b (s/coll-of digit? :max-count (- 40 (s/count a))]
(g/fmap (fn [a b] (apply str a ":" b) [(s/gen a) (s/gen b)]))
#2020-09-0204:30johanatanI always use gen/let
for this sort of thing. You can use it just like a normal let except with generators on the rhs and generated values bound to the lhs. if you need a regular value you can use a gen/return
to effectively raise them to that level such that both regular values and generated values can be bound in the same gen/let
.#2020-09-0204:31johanatanin this particular case, you could generate an int with int-in
1 40 and then compute a second int-in
40 - the first. then use the two ints as exact size parameters to your collection generators.#2020-09-0204:16johanatandoes anyone see why the following wouldn't work? I'm getting an error: sym
not defined on the stest/check
line.
(defn filtered-check [sym opts]
(let [filter (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
matches? (fn [s] (or (= filter "all")
(clojure.string/includes? (name s) filter)))]
(when (matches? sym)
(stest/check sym opts))))
#2020-09-0204:25johanatanthis is the exact error:
Unable to resolve symbol: sym in this context
114 (defn filtered-check [sym opts]
115 (let [filter (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
116 matches? (fn [s] (or (= filter "all")
117 (clojure.string/includes? (name s) filter)))]
118 (when (matches? sym)
119 (stest/check sym opts))))
^---
#2020-09-0204:25johanatanit seems that stest/check may be a macro (I got "can't take value of macro" previously) but in the spec.alpha code it is defined as a function:
https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L373#2020-09-0204:39seancorfieldPerhaps it is a macro in ClojureScript?#2020-09-0204:40seancorfieldYup: https://cljs.github.io/api/cljs.spec.test.alpha/check it's a function in Clojure and a macro in ClojureScript.#2020-09-0204:43seancorfield@johanatan You might have to ask in #clojurescript -- that seems an unfortunate and gratuitous difference in the implementations.#2020-09-0212:54borkdudeI've also ran into this a couple of times#2020-09-0213:00borkdudeI think the root of this lies in dynaload which is a macro in CLJS because requires are only possible at compile time (when not using a REPL).
This implies that library code around check yields another macro. E.g. https://github.com/borkdude/respeced/blob/f5ff67aa78f588e7bad2a1b86dd1a646d3fdab3d/src/respeced/test.cljc#L81 and https://github.com/borkdude/respeced/blob/f5ff67aa78f588e7bad2a1b86dd1a646d3fdab3d/src/respeced/impl.cljc#L71.#2020-09-0216:54johanatanI suppose my best course of action is to succumb to macro contagiousness and make this a macro ?#2020-09-0217:44borkdudemacros beget macros yeah#2020-09-0219:03johanatanš#2020-09-0221:17johanatanthis is what i ended up coming up with (for anyone following along):
(defmacro filtered-check [sym opts]
`(let [fltr# (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
check# (fn [res#]
(cljs.test/is (= true (-> res# :clojure.spec.test.check/ret :result))
(goog.string/format "spec check failure:\r\n%s" (with-out-str (cljs.pprint/pprint res#)))))]
(when (or (= fltr# "all") (clojure.string/includes? (name ~sym) fltr#))
(let [check-res# (clojure.spec.test.alpha/check ~sym ~opts)]
(clojure.spec.test.alpha/summarize-results check-res#)
(if (nil? check-res#)
(cljs.test/is false "stest/check result was nil. did you pass it any valid symbols?")
(doall (map check# check-res#)))))))
you can use it with figwheel main to filter your stest/check calls:
location/figwheel-extra-main/auto-testing?filter=the_filter
and with your calls to the macro such as:
(macros/filtered-check ns/sym {:clojure.spec.test.check/opts {:num-tests N})
#2020-09-0221:18johanatan[massive time saver once you start to accumulate many of these]#2020-09-0301:59kennytiltonIs clojure.spec
a timid retreat from dynamism? In part bowing before the sacred cow of testing?
"tests are essential to quality" --https://clojure.org/about/spec
It seems like Clojure ventured bravely forth from the suffocating safety of Java's strong static typing, caught one gust of untyped wind, and scurried back to the shelter of Java's skirts.
Or was this just a pragmatic choice to make Clojure more palatable in tall buildings?#2020-09-0302:00kennytiltonNot trolling, just trying to decide whether to buy into the burden of spec
.#2020-09-0302:08seancorfieldYou certainly sound like you're trolling @hiskennyness but I know enough of your posts to know that's just kind of your "teasing" style š#2020-09-0302:09seancorfieldWe use Spec very heavily at work, in production code, to do data validation. It is not any sort of type system so, no, it is not any sort of "scurrying back to Java's skirts" š#2020-09-0302:10seancorfieldHave a read of https://corfield.org/blog/2019/09/13/using-spec/ and see the many ways we're using Spec at work -- none of which are in any way as a substitute for types.#2020-09-0302:13seancorfieldWe do find Spec to be very helpful in testing (and to some extent in development). We use specs to generate test data, to help support property-based testing, to valid test results...#2020-09-0302:13kennytiltonThx! I'll check it out. But reading that intro I kept hearing "OK, we should have objects with defined properties; maps are too loosey-goosey". Lemme see what HisCorfieldNess is up to...#2020-09-0302:17seancorfield(I'm going off to make dinner but I'm certainly happy to discuss Spec in great depth when I get back -- or tomorrow š )#2020-09-0302:38kennytiltonOK, nice, short and sweet. Actually readable! But we likely disagree on testing, so no progress can be made there. As for runtime parameter checking, spec could be a handy way to code up what folks do anyway, but then that could be trivial little DSL I knock off in an hour then evolve easily over time. spec
seems to be a handful at times, with people asking for help expressing different things. Sounds like a classic case of a good idea run amok. As for code generation from spec, now you are scaring me. š We have macros for DSLs, and I write the macro that parses a tree of symbols whose structure can be as user-friendly as I like? Think "CL loop" š. Why parse a spec
definition, whose structure I cannot completely control? Even if I can make the latter work, it now feels as if I am in places coding my app in spec
so it can be generated, when I already have a fine language to work with (Clojure!). Maybe this is why people get stuck using spec
, because they ineluctably get drawn into more and more intense usage trying to program with it.
So how widely used is spec
?#2020-09-0303:53seancorfieldBack from dinner now @hiskennyness... I'd say Spec is pretty widely used. Lots of libraries provide optional Specs in a separate namespace so folks on Clojure 1.8 or earlier can avoid it and anyone on Clojure 1.9 or later can opt in if they wish.#2020-09-0303:55seancorfieldAs for generating code from Specs, that's about having one "source of truth" and it might as well be the Spec (esp. since mostly it's macros in Spec 1 and they're hard to deal with programmatically -- that will change in Spec 2).#2020-09-0303:55seancorfieldAnd we generate code from Specs we write.#2020-09-0303:58seancorfieldSpec has a lot of different uses. Some people do try to use it like a type system (bad idea, IMO, and that's not what it is designed for). Given your skepticism, I can only assume you've not watched any of the (many) talks about Spec over the years? Spec has been available for about four, four and a half years, since 1.9.0-alpha1 in May 2016.#2020-09-0312:18kennytiltonI'll google up some vids. Yes, well aware of spec for a while. Just ran into it at work for the first time, so it is time to get to know it. Thx for the feedback!#2020-09-0316:47seancorfieldFeel free to ping with any Qs if you want š#2020-09-0309:33borkdude> Lots of libraries provide optional Specs in a separate namespace
I think this is good practice also for programs written for > 1.9#2020-09-0718:06Ho0manHi everyone,
Can I write a multi-spec for the component/config
segment of this data based on the component/type
which is not part of component/config
's corresponding value? :
{:component/type [:instance-manager :v-2.0.0]
;;-----------------------------------------;;
:component/config {,,,
}
;;-----------------------------------------;;
:component/deps {}}
(I'm using spec1)
Thanks a lot#2020-09-0718:07Ho0manthe component/config
part has to be extensible ... so different components of the system across different namespaces define their own internal config map#2020-09-0718:31borkdudeIs it possible to inspect an evaluated regex spec?
(def x int?)
(s/fdef foo :args (s/cat :x int? :y x))
(prn (::s/op (s/cat :x int?))) ;; :clojure.spec.alpa/pcat
(prn (::s/op (:args (s/get-spec 'user/foo)))) ;; nil
Use case: I'd like to know what the x
resolved to. Just inspecting the form won't tell me that.#2020-09-0718:52borkdudeI hoped I could get more into the structure of the regex, but it seems to be an opaque thing#2020-09-0718:54borkdudeOtherwise formulated: it is possible to get back to the int?
function once it's wrapped inside this reified spec object?
(s/spec int?)
#2020-09-0720:10borkdudeMaybe I'm looking for datafy on specs#2020-09-0720:28tekacs@borkdude (s/form (s/spec int?)) => 'clojure.core/int?
#2020-09-0720:29tekacsis I think what youāre looking for#2020-09-0720:35borkdude@tekacs That's already better!
(clojure.spec.alpha/cat :x clojure.core/int? :y user/x)
#2020-09-0720:36tekacs@borkdude
(let [{:keys [x]} (rest (s/form (:args (s/get-spec 'user/foo))))] x)
=> clojure.core/int?
#2020-09-0720:37tekacsdefinitely not ideal, but that works?#2020-09-0720:37borkdudeI kinda wish that I could have user/x
resolved to clojure.core/int?
as well, since this information is available in the spec itself#2020-09-0720:38tekacsright āĀ though you wouldnāt usually use def to define a sub-spec#2020-09-0720:38tekacsyouād usually have done:
(s/def ::x int?)
(s/fdef foo :args (s/cat :x int? :y ::x))
#2020-09-0720:38borkdudeyeah, I guess using this for linter stuff is alright, but then I might as well just inspect the raw sexpr directly#2020-09-1816:46souenzzosee also
https://github.com/wilkerlucio/spec-inspec/blob/master/test/com/wsscode/spec_inspec_test.clj#L10#2020-09-1817:35borkdudeThanks. Added link to https://github.com/clj-kondo/inspector/issues/4#2020-09-0720:39borkdudeyes, but in that case you still get :user/x
#2020-09-0720:39tekacsright, but
(s/def ::x int?)
=> :user/x
(s/form (s/spec ::x))
=> clojure.core/int?
#2020-09-0720:39tekacsif you pass those through s/spec it resolves them#2020-09-0720:40borkduderight, you can recursively call that on qualified keywords#2020-09-0720:40tekacsright, precisely#2020-09-0720:40borkdudecool, thanks!#2020-09-0720:40tekacsindeed you can safely pass most things through (s/form (s/spec ā¦)) I believe#2020-09-0720:40borkdude@tekacs purpose of this exercise: https://gist.github.com/borkdude/c0987707ed0a1de0ab3a4c27c3affb03#2020-09-0720:41tekacsIāve since moved to malli but Iām excited for this! š#2020-09-0720:41tekacsI saw that you posted in #malli too š#2020-09-0720:42tekacsI also forked aave for malli to add CLJS support and some additional features I havenāt seen elsewhere (https://github.com/tekacs/aave), so I should probably pay attention to the clj-kondo type spec š#2020-09-0720:43borkdude@tekacs That's cool, you can use the same kind of trick indeed#2020-09-0720:52borkdude@tekacs#2020-09-0720:56kennyThis is awesome. Support for s/keys would be a huge productivity win. We have many functions that take large maps as params and it's so easy to forget or misspell one.#2020-09-0720:56kennyThank you for working on this @borkdude š#2020-09-0721:24borkdude:)#2020-09-0721:25kennyOMG#2020-09-0721:26kennySo freaking cool. An optional toggle for something like https://github.com/bhauman/spell-spec would epic (I could see some people not liking it).#2020-09-0721:27kennyI wonder how many places in our code base we have a :req key and aren't actually passing it but it happens to magically work in prod š
#2020-09-0721:28borkdudeNote that clj-kondo can only reliably detect this right now in places where literal maps are passed#2020-09-0721:28borkdudethere are some attempts around assoc et al, but not fully worked out#2020-09-0721:31borkdudebut I guess something is better than nothing :)#2020-09-0721:31kennyAh, right - makes sense.
And totally!#2020-09-0721:32kennyWonder how far you could go with it... e.g.,
(defn do-stuff
[x])
(s/fdef do-stuff
:args (s/cat :x int?)
:ret (s/keys :req [::a]))
(defn do-moar-stuff
[m]
)
(s/fdef do-moar-stuff
:args (s/cat :m (s/keys :req [::a ::b]))
:ret int?)
(-> 1
(do-stuff)
(do-moar-stuff))
Technically seems like you could infer that do-moar-stuff will (probably) not get passed the right stuff, assuming the specs are correct.#2020-09-0721:35borkdudedo-stuff can still required a map which key ::b
, although it's not required, so I don't see anything wrong here#2020-09-0721:35borkdudewhat we could however do is, when do-stuff has ret int? and do-moar-stuff expects a map, give an error#2020-09-0721:36borkdudeclj-kondo already has support for this, it just needs to be mapped from spec to the right format.#2020-09-0721:36kennyThe specs for do-stuff and do-moar stuff don't align and they should. By happenstance, do-stuff could always return ::b but that's just luck.#2020-09-0721:37kennyA future refactoring to do-stuff could result in ::b getting removed which will cause downstream do-moar-stuff to break.#2020-09-0721:40borkdudethe clj-kondo "type system" will only complain if it's sure something is wrong, so in this case it would not complain I think#2020-09-0721:41kennyI think that particular scenario would fall under the warning category.#2020-09-0721:41borkdudetype-wise, but maybe it could have a spec linter that gave a warning about this#2020-09-0721:42borkdudenot sure if I want to go very deep into s/keys, since spec2 will have different ways of doing#2020-09-0721:42borkdudebut basic type warnings as you type are already a free win#2020-09-0720:54borkdude(code updated here: https://gist.github.com/borkdude/c0987707ed0a1de0ab3a4c27c3affb03)#2020-09-0806:25ikitommi@borkdude if thatās of any value, there is one decent spec form parser in spec-tools. Doesnāt cover regexs, but lot of other dirty details. There is also a comprehensive regression & progression test suite. e.g. testing that the form bugs are not fixed, together with link to the JIRA issue. Feel free to c&p any parts that you find useful.#2020-09-0806:25ikitommithe parser: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/parse.cljc#2020-09-0806:25ikitommispec visiting tests: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/visitor_all_test.cljc#2020-09-0806:31borkdudeInteresting, Iāll have a look#2020-09-0919:40borkdudePushed the experiment to a repo: https://github.com/clj-kondo/inspector
Contributions welcome.#2020-09-0919:49borkdudeHmm, I realize that this is just valid Clojure syntax:
(defn foo [^:int x ^:keyword y ^:string z]
[x y z])
(prn (foo 1 2 3))
Could be a way to integrate clojure spec with existing defns without the need for writing another macro.#2020-09-0919:49borkdudeNon-namespaced keyword: refers to a predicate, namespaced keyword: refers to a spec#2020-09-0919:51borkdudee.g.:
(s/def ::s string?)
(defn foo [^:int x ^:keyword y ^:ss z]
[x y z])
#2020-09-0919:52Alex Miller (Clojure team)we're not going to do that, but there are some other ideas percolating#2020-09-0919:55borkdudeUnambiguous:
user=> (alias 'c 'clojure.core)
nil
user=> ::c/int?
:clojure.core/int?
user=> (defn foo [^::c/int? x])
#2020-09-0919:56borkdudeBit ugly, but one of those things that could have been deemed: yeah, Rich intended it like this all along, it was all from a design originating back to 2002 ;)#2020-09-0919:58Alex Miller (Clojure team)except it's the opposite of what Rich intends :)#2020-09-0919:58borkdudetell me more#2020-09-0919:59Alex Miller (Clojure team)this just plays to the familiarity from static typing langs, which is just not what Rich is trying to do with spec#2020-09-0920:01borkdudeThis would not be limited to simple annotations. You could do:
user=> (s/def ::foo (s/keys :req-un [::x]))
:user/foo
user=> (defn f [^::foo m])
so I don't get the static typing remark.#2020-09-0920:04Alex Miller (Clojure team)it looks like static typed function signatures, but it's not#2020-09-0920:05borkdudeso?#2020-09-0920:05Alex Miller (Clojure team)so making things that are different look the same causes confusion#2020-09-0920:07Alex Miller (Clojure team)there are other considerations as well - compilation, evaluation, reuse across arities, combination with other features that don't exist yet, etc#2020-09-0920:07Alex Miller (Clojure team)Rich is looking at all that and considering a wide variety of options including the one above#2020-09-0920:07borkdudeI bet#2020-09-0920:08Alex Miller (Clojure team)I just think this one is probably unlikely based on what I've heard#2020-09-1417:42tvalerioHi! I have this spec in a namespace and a test so I can validate it:
(ns develop.rates)
(defn- valid-rate-intervals?
[rate]
false)
(s/def ::rates (valid-rate-intervals?))
(ns develop.rates-test
(:use clojure.test)
(:require
[clojure.spec.alpha :as s]
[develop.rates :refer :all])
(deftest should-test-a-rate
(testing "Test a rate"
(let [result (s/explain-data :develop.rates/rates {:rate "test"})]
(prn "result: " result))))
Why canāt I use the spec like ::rates
instead of the example in the code? If I try yo use it, I get an unable to find spec
error.
Also, when I watch the result of explain-data
I can see that it begins with:
#:clojure.spec.alpha{:problems...
And not like:
#:rates{:problems...
As I saw in the examples.
Iām trying to verify which function got error to return a proper error message but Iām getting a hard time to do so. Trying Spec for the first time. How could I do that?#2020-09-1509:28anthony-galea@tvalerio See the following under the Literals
section at https://clojure.org/reference/reader
> A keyword that begins with two colons is auto-resolved in the current namespace to a qualified keyword
So when you use ::rates
inside rates.clj
itās as if youāve written :develop.rates/rates
but when you do the same inside rates-test.clj
you get develop.rates-test/rates
and not :develop.rates/rates
#2020-09-1512:01tvalerioOk, I understood. Thanks @U446AB17F! =D#2020-09-1517:35kennyIt seems the conformed result of the pred passed to s/coll-of is getting disregarded. Is this behavior of expected?
(s/conform
(s/and (s/tuple #{"a"} boolean?)
(s/conformer (fn [[_ v]] [:a v])))
["a" true])
=> [:a true]
(s/conform
(s/coll-of
(s/and (s/tuple #{"a"} boolean?)
(s/conformer (fn [[_ v]] [:a v])))
:kind map?)
{"a" true})
=> {"a" true}
#2020-09-1517:59seancorfield(the API URL was incorrect)#2020-09-1518:05Alex Miller (Clojure team)@kenny I think because you are using :kind map?
you're into the map-specific conforming logic which (by default) does not conform keys#2020-09-1518:07Alex Miller (Clojure team)You can use :conform-keys to change that default:
(s/conform
(s/coll-of
(s/and (s/tuple #{"a"} boolean?)
(s/conformer (fn [[_ v]] [:a v])))
:kind map?
:conform-keys true)
{"a" true})
{:a true}
#2020-09-1518:37seancorfield(that's a more useful URL, sorry for the noise)#2020-09-1520:25kennyGreat, that works perfect. Thank you!#2020-09-1617:47kennyIs there a way to not include a key when using s/coll-of + kind map? + conform-keys true? For example,
(s/conform
(s/coll-of
(s/and (s/tuple #{"a"} boolean?)
(s/conformer (fn [[_ v]] nil)))
:kind map?
:conform-keys true)
{"a" true})
=> {nil nil}
That behavior seems correct. Curious if there's a way to indicate to not include the key though. Perhaps only way is to s/and in a conformer that removes nil keys?#2020-09-1617:52Alex Miller (Clojure team)no way to do that#2020-09-1617:53Alex Miller (Clojure team)in general, use of conformers to do arbitrary transforms is considered an anti-pattern (conformer really exists for building new spec types primarily)#2020-09-1617:55kennyYep - definitely in a gray area. Thanks.#2020-09-2119:27johanatanhi, is there any way to override the comparator used for :distinct
calculations for s/coll-of
?#2020-09-2119:27johanatanor is the best bet just to use s/and
with a custom predicate ?#2020-09-2119:40Alex Miller (Clojure team)no way to do that currently#2020-09-2119:41Alex Miller (Clojure team)but s/and custom pred would work#2020-09-2119:41johanatan:+1:#2020-09-2120:46adamfreyis there a way to do something like :gen-max
on a regex s/cat
spec?#2020-09-2121:10Alex Miller (Clojure team)not currently#2020-09-2121:11Alex Miller (Clojure team)you can s/& a predicate to only accept smaller colls and that will have the same effect, but it's still producing the larger coll and that at least has memory implications (and may prevent such-that from succeeding at all)#2020-09-2415:53Jeff EvansFrom the guide: https://clojure.org/guides/spec#_custom_generators
Iām trying to figure out whether itās possible to refer to kw-gen
(which was just defined in the previous section) here, but I canāt quite figure out the right syntax. For instance, this doesnāt work:
(s/def ::kws (s/with-gen (s/and keyword? #(= (namespace %) "my.domain")) #(kw-gen)))
#2020-09-2416:38Alex Miller (Clojure team)user=> (def kw-gen (s/gen #{:my.domain/name :my.domain/occupation :my.domain/id}))
#'user/kw-gen
user=> (s/def ::kws (s/with-gen (s/and keyword? #(= (namespace %) "my.domain")) (fn [] kw-gen)))
:user/kws
user=> (gen/sample (s/gen ::kws))
(:my.domain/name :my.domain/id :my.domain/name :my.domain/name :my.domain/name :my.domain/occupation :my.domain/occupation :my.domain/occupation :my.domain/id :my.domain/occupation)
#2020-09-2416:39Alex Miller (Clojure team)kw-gen here is a generator so you need to wrap it in a no-arg function that returns it (not invokes it)#2020-09-2417:30Jeff Evansthanks! I misunderstood something fundamental, since I thought #(kw-gen)
was equivalent to (fn [] kw-gen)
#2020-09-2417:34Alex Miller (Clojure team)the former is the same as (fn [] (kw-gen))
#2020-09-2417:35Alex Miller (Clojure team)I guess you could also do (constantly kw-gen)
#2020-09-2602:39johanatanhas anyone given much thought to (for the purposes of using multi-spec liberally across many namespaces) dynamically constructing "qualified keywords" at reader / compile time ?#2020-09-2602:39johanatan(in clojurescript)#2020-09-2602:40johanatani've attempted both a plain macro and a tagged literal reader#2020-09-2602:40johanatanthis is the macro:
(defmacro ns-grouped-keyword [group keyword]
`(keyword (clojure.string/join "/" [~(str *ns*) ~group]) ~keyword))
#2020-09-2602:40johanatanand this is the reader:
(defn read-ns-grouped-keyword [s]
`(let [[group# kwd#] (clojure.string/split ~s #"/")]
(keyword (clojure.string/join "/" [~(str *ns*) group#]) kwd#)))
#2020-09-2602:41johanataneven the latter apparently only gets expanded and not eval'd at reader time#2020-09-2602:42johanatansuch that:
(s/keys :opt-un [#my/reader "blah/blaz"])
will not work#2020-09-2602:43johanatanwhere as I had expected it to get eval'd to: (s/keys :opt-un [:the-current-namespace/blah/blaz])
before keys
expansion time#2020-09-2615:24uditI am trying to use clojure.spec for validating the inputs to my API handlers. Whatās the idiomatic pattern for usage here?
I was thinking of creating a generic fn which takes in spec and the data to be validated. This fn can throw an exception when the validation fails. This fn would be called as the very first thing in the handlers, thus if the data is not valid the exception will act like an early exit. But this is using exceptions for control flow.
The other approach I can think of is wrapping all my handlers with an if-else block that do this validation. This is fine, except it sort of makes my handlers less readable.
Whatās a clean approach that people have discovered for validating (and coercing) data using spec.#2020-09-2615:45Jeff EvansCanāt you have a single wrapper fn
that basically does
(let [res (s/conform input)]
(if (:s/invalid? res)
(build-error-reponse)
(real-handler input) ; real-handler could be an input to this fn
)
)
Then the real-handler
doesnāt have to worry about whether the input is valid, and your outermost layer (i.e. the one that first receives the request) can call this instead of real-handler
directly. And if you want more details out of build-error-response
you could use explain-str
or explain-data
instead of conform
. Or perhaps Iām misunderstanding the situation.#2020-09-2616:45uditThis works @jeffrey.wayne.evans! Thanks!
Although I would like to have different spec for different handlers, but I presume that can be passed in as a parameter to this fn
a well, right?#2020-09-2616:54Jeff EvansSure! You would still have to somehow organize the specs per handler. Not sure if there is an established pattern for that but you could always have a keyword based map for them (handler function name as key?)#2020-09-2616:56ikitommireitit has a spec-module, you can declare request & response specs to endpoints, and there is a set of predefined middleware / interceptors to use them for validation, also coercion. Example here: https://github.com/metosin/reitit/blob/master/examples/ring-spec-swagger/src/example/server.clj#2020-09-2813:11arohnerWhen using stest/instrument
in tests, how do you clean up after a test? e.g.
`
(deftest foo
(stest/instrument foo {:stub
foo})#2020-09-2813:11Alex Miller (Clojure team)unstrument#2020-09-2813:11arohnerright, but you donāt know what the previous state was#2020-09-2813:12arohnerwas it instrumented, or stub+ instrumented?#2020-09-2813:13Alex Miller (Clojure team)Sorry, not understanding the question#2020-09-2813:14arohnerWe have some code that is always instrumented. During a test, if I call (instrument
foo {:stub foo})
, that overwrites the existing instrument with a stub+instrument. Now if I call unstrument
at the end, the next test will run with foo
un instrumented#2020-09-2813:16Alex Miller (Clojure team)Instrument remembers the old var and unstrument replaces it iirc#2020-09-2821:11borkdude@arohner I have some code to deal with that: https://github.com/borkdude/respeced#with-instrumentation#2020-09-2908:39Eugenhi, is there some tooling to generate spec code from XSD schema files ?
I would like to validate with clojurescript (if possible) some OFX XML I generate#2020-09-2914:16fadrianI tried both of the XSD parsers I found using Google search (which looks like they came from the same root). Neither of them seemed to work. clj-xml will parse an XSD, but moving on from the parsed XML to a spec seems to still be a work-in-progress.#2020-09-2916:13mpenetI have a case where I generate specs, and sometimes I have fwd declarations in spec alises: (s/def ::foo ::bar)
where ::bar is not declared/speced yet, so it blows up of course. A dirty way to do that is to do (s/and ::bar)
as spec for instance but I am sure there's a better way (short of building a dep graph and specing in order)?#2020-09-2916:15Alex Miller (Clojure team)Forward references are actually intended to be ok in specs - the spec alias case (s/def ::foo ::bar) is one known exception to that (there is a ticket for this)#2020-09-2916:15Alex Miller (Clojure team)not sure if you've tried (s/spec ::bar) - that might also work#2020-09-2916:15Alex Miller (Clojure team)would be slightly less dirty if so#2020-09-2916:16mpenetI tried s/spec, doesn't work#2020-09-2916:16Alex Miller (Clojure team)well, can't say I have a better alternative for you then#2020-09-2916:16mpenet"unable to resolve spec ::bar "#2020-09-2916:17mpenetalright, thanks for the info. If it's a bug I'll take the s/and trick as acceptable for now#2020-09-2916:19mpenetI cannot find the jira for it, but I am not a good at jira'ing#2020-09-2916:52Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2067#2020-09-2916:53Alex Miller (Clojure team)or https://ask.clojure.org/index.php/3321/s-def-a-b-throws-unable-to-resolve-error-if-b-is-not-defined?show=3321#q3321 if you want to vote#2020-09-2917:05mpenetThanks!#2020-10-0209:08Petrus TheronWhere do I file Spec2 bug reports?
(let [charset #{\a \b \c}] (s/+ charset))
throws :
Execution error (IllegalArgumentException) at clojure.alpha.spec/resolve-spec (spec.clj:219).
Symbolic spec must be fully-qualified: charset
However moving the set to a separate namespace or passing it in directly works, e.g.: (s/+ #{\a \b \c})
.#2020-10-0212:20Alex Miller (Clojure team)This is expected behavior - there are some changes due to symbolic specs#2020-10-0212:22Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#symbolic-specs#2020-10-0215:10AronHow to have a generator that generates a constant string?#2020-10-0215:16Alex Miller (Clojure team)assuming you're talking spec 1, you can pull a generator from a set of one thing#2020-10-0215:17Alex Miller (Clojure team)(gen/sample (s/gen #{"a string"}))
#2020-10-0215:23Aronthanks. Yeah, I was told not to use spec2 in production by someone#2020-10-0507:58Jim NewtonI'm not a spec user, but I have a few questions about the API. Is it true that spec designates certain symbols as patterns? How can I know whether a given symbol designates a spec pattern? As I understand, given such a symbol, I can call (s/valid? pattern-designator some-value)
to determine whether a given value conforms to the pattern. Are such pattern designators always available globally? or can the be defined lexically and thus be invisible to other applications?#2020-10-0508:01Jim NewtonIs s/valid?
side-effect free assuming the given pattern is side-effect free?#2020-10-0508:02Jim NewtonUnder which situations might s/valid?
throw an exception? Do I need to wrap calls in try/catch
?#2020-10-0513:05Alex Miller (Clojure team)By āpatternā, I assume you mean āsymbolic specā. Specs have both a spec form (comprised of keyword names, list forms, sets, functions, etc) and a compiled instance form. Spec forms refer to macros that expand to code that emits spec instances. The details of this are quite a bit different in spec 1 and 2. But in both cases the spec forms are available globally as they are defined by macros. s/valid? does not have side effects of its own. s/valid? itself wonāt throw but if it includes a predicate, that might throw on a bad input#2020-10-0513:17Jim NewtonIs there a predicate I can use to know whether a given form is a valid "symbolic spec" ?#2020-10-0513:21Jim NewtonAm I correct that s/valid?
will throw an error if you give it an invalid symbolic spec?#2020-10-0513:43Alex Miller (Clojure team)I think s/spec will either give you a spec instance or an error#2020-10-0513:43Alex Miller (Clojure team)s/valid? should do the same#2020-10-0513:44Jim Newtonthat's reasonable, yes. So It could be nice if my application could ask spec whether the given spec instance is valid or not, before putting it in a place where at some time in the future when s/valid?
is eventually called, spec will throw an error.#2020-10-0513:45Jim NewtonOH, am I correct in assuming that s/valid?
checks whether the given data matches the spec? i.e., it does not validate the spec, but rather validates the data...#2020-10-0514:52Alex Miller (Clojure team)yes#2020-10-0515:32Jim Newtonso which function should I use to detect whether a candidate given to my by the function who called me, is a valid symbolic spec?#2020-10-0515:39Alex Miller (Clojure team)spec 2 is a bit better in this area (but is not ready for use yet)#2020-10-0515:41vlaaad#(contains? (clojure.spec.alpha/registry) %)
#2020-10-0515:49Alex Miller (Clojure team)that's not what he's asking for afaict#2020-10-0515:59Jim NewtonWithout knowing spec, it is hard to know really what I'm asking for. But Id like to know whether what I have in variable x
is something I can pass as the first argument of s/valid?
.#2020-10-0515:59Jim NewtonI suppose my first implementation doesn't to need to verify its values, just trust the caller, and let spec sort it out.#2020-10-0516:02Jim Newtonlooks like everything in that registry is either symbol or a keyword. (distinct( map (fn [x] (or (symbol? x) (keyword? x))) (keys (clojure.spec.alpha/registry))))
returns (true)
#2020-10-0516:02borkdudesymbols are used for fdefs#2020-10-0516:59Alex Miller (Clojure team)s/valid? takes spec names, spec forms, and spec objects - there is no method that validates all of those#2020-10-0621:34Michael Stokleyis there a good way to spec out just some of the arguments in s/fdef
's :args
? like :args (s/cat :first-param (constantly true) :second-param int?)
?#2020-10-0621:35ghadiany?
#2020-10-0621:35Michael Stokleythanks!#2020-10-1223:27johnjelinekis there a way to make a spec validate a reference to another key's value? let's say you have a (s/coll double?)
and you want to make a key where the predicate is something like (s/and double? #(= % (apply + ,,,))
where the ,,,
is that (s/coll double?)
#2020-10-1223:28johnjelinekmaybe your schema is like (s/schema [::double-list ::sum])
#2020-10-1223:28johnjelinekI'm prolly not doing a great job of explaining this, does it vaguely make sense?#2020-10-1223:28Alex Miller (Clojure team)Can you give an example of valid data?#2020-10-1223:29Alex Miller (Clojure team)The example you gave should basically work so Iām trying to make sure I understand what you mean #2020-10-1223:29Alex Miller (Clojure team)Are you talking about in a map?#2020-10-1223:30johnjelinekya#2020-10-1223:30Alex Miller (Clojure team)You can s/and a predicate that validates whatever you want#2020-10-1223:31johnjelinekok, so let's say concretely, I've got something like: {::prices '(1 2 3 4) ::total 10}
and so total
's predicate has to match the collection of prices (except doubles instead of ints)#2020-10-1223:33johnjelinekso, how would a (s/def ::total)
's predicates (apply +
on ::prices
?#2020-10-1223:37Alex Miller (Clojure team)(s/def ::m (s/and (s/keys ...) (fn [{::keys [total prices]}] (= total (reduce + prices))))
#2020-10-1223:38Alex Miller (Clojure team)Just spec ::total as an int? or whatever#2020-10-1223:38johnjelinekoh!#2020-10-1223:38johnjelinekok great!#2020-10-1223:38Alex Miller (Clojure team)The attribute by itself does not have this constraint#2020-10-1223:39Alex Miller (Clojure team)It only has that constraint when combined with prices in a map#2020-10-1223:40johnjelinekI'm not sure I grok that yet#2020-10-1223:40Alex Miller (Clojure team)The total=prices is a property of the map, not of the total#2020-10-1223:42johnjelinekah, right#2020-10-1223:50johnjelinek(s/def ::total int?)
(s/def ::prices (s/coll-of int?))
(s/def ::m (s/and (s/keys :req [::total ::prices])
(fn [{::keys [total prices]}] (= total (reduce + prices)))))
(s/valid? ::m {::prices '(1 2 3 4) ::total 10})
#2020-10-1223:50johnjelinekthis spits out an java.lang.IndexOutOfBoundsException
#2020-10-1223:54johnjelinek(let [{::keys [total prices]} {::prices '(1 2 3 4) ::total 10}]
(= total (reduce + prices))) ;=> true
((fn [{::keys [total prices]}]
(= total (reduce + prices))) {::prices '(1 2 3 4) ::total 10}) ;=> true
(s/def ::m (fn [{::keys [total prices]}]
(= total (reduce + prices)))) ;=> java.lang.IndexOutOfBoundsException
(s/def ::m (s/and (s/keys :req [::total ::prices])
(fn [{::keys [total prices]}]
(= total (reduce + prices))))) ;=> :scratch/m
(s/valid? ::m {::prices '(1 2 3 4) ::total 10}) ;=> java.lang.IndexOutOfBoundsException
#2020-10-1300:19johnjelinekoh, hmm:
; (#:scratch{:keys [scratch/total prices]}) - failed: Extra input at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
#:scratch{:keys [scratch/total prices]} - failed: vector? at: [:fn-tail :arity-n :params] spec: :clojure.core.specs.alpha/param-list
#2020-10-1303:05Michael Stokleyis there an fdef
that works for multi-methods?#2020-10-1303:49Alex Miller (Clojure team)no, but you can pull the dispatch function out and spec that#2020-10-1314:21Michael Stokleythank you!#2020-10-1303:13johnjelinekya, dunno what was up with that destructuring, but this works:
(s/def ::m (s/and (s/keys :req [::total ::prices])
#(= (::total %) (reduce + (::prices %)))))
#2020-10-1303:33johnjelinekah-ha! I think I got it! thx @alexmiller!#2020-10-1303:35johnjelinekI did something like this:
(s/def ::m (s/and ::total=prices (s/schema [::prices ::total])))
#2020-10-1303:50Alex Miller (Clojure team)just as a heads up, I do not consider spec 2 to be ready for use and the api may change before release#2020-10-1321:11vlaaadI have the following spec:
(s/def ::grid
(-> (s/keys :opt-un [::grid])
(s/coll-of :kind vector? :min-count 1)
(s/coll-of :kind vector? :min-count 1)))
How do I do generative testing with this kind of spec? any attempt to generate an example of ::grid
makes my OS super laggy until I kill the JVM process...#2020-10-1321:37Alex Miller (Clojure team)that's not a valid spec? what are you expecting -> to do there?#2020-10-1405:14vlaaadOh, sorry, I added threading in the message for readability, actual spec doesn't have it#2020-10-1321:39Alex Miller (Clojure team)I guess maybe it would work in the spec 1 macrology, but I find that very confusing to parse (and it won't in spec 2 I think)#2020-10-1321:39Alex Miller (Clojure team)in general, setting :gen-max is useful for coll specs though to bound nested coll size#2020-10-1405:33vlaaadThanks, I'll try that#2020-10-1407:04vlaaadIt did help, thank you!#2020-10-1322:27kennyWe often build our own spec "types" by doing something like this.
(defmacro my-custom-map
[kpred vpred]
(let [form `(s/map-of ~kpred ~vpred)]
`(s/with-gen ~form #(gen/fmap identity (s/gen ~form)))))
(s/def ::k string?)
(s/def ::v string?)
(s/def ::my-custom-map (my-custom-map ::k ::v))
This, however, does not play nice with :gen overrides.
(gen/generate (s/gen ::my-custom-map {::k #(s/gen #{"a" "b"})}))
=>
{"3" "b4tq9FeW6Gt",
"8h" "o47F763P0v5B63sA52c4xu1FN",
...}
Is there a better way of doing this that works with :gen overrides?#2020-10-1322:35kenny#2020-10-1323:09kennyThis seems to be a common pattern: https://stackoverflow.com/questions/43230546/a-clojure-spec-that-matches-and-generates-an-ordered-vector-of-variable-length#2020-10-1323:15kennyIt's like with-gen
needs to take an argument with :gen overrides or implicitly use a dynamic variable.#2020-10-1323:41kennyCan't figure out any way around this other than to override the top level spec (in my example ::my-custom-map
). That can be quite painful with complex specs.#2020-10-1416:07kennyš¢#2020-10-1416:07kennyhttps://clojure.atlassian.net/browse/CLJ-2095#2020-10-1402:56johnjelinekquestion regarding preference: do you add your specs to an existing namespace of a domain model or do you have a separate spec namespace to share your specs around?#2020-10-1403:07seancorfield@johnjelinek Here's how we organize the different types of specs we write https://corfield.org/blog/2019/09/13/using-spec/#2020-10-1415:49Ivan Fedorovso, how safe is it to use s/conform
as a regex matcher?
As in https://juxt.pro/blog/parsing-with-clojure-spec
(s/def ::contentline
(s/cat
:name ::iana-token
:params (s/*
(s/cat
:semi #{\;}
:param-name ::iana-token
:equals #{\=}
:param-value ::iana-token))
:colon #{\:}
:value ::iana-token))
(s/conform
::contentline
(seq "DTSTART;TZID=US-EAST:20180116T140000"))
=>
{:name [\D \T \S \T \A \R \T],
:params
[{:semi \;,
:param-name [\T \Z \I \D],
:equals \=,
:param-value [\U \S \- \E \A \S \T]}],
:colon \:,
:value [\2 \0 \1 \8 \0 \1 \1 \6 \T \1 \4 \0 \0 \0 \0]}
(apply str (get-in ā¦ [:params 0 :param-value]))
=> "US-EAST"
#2020-10-1416:02uwospec isn't designed for string parsing -- I've seen people recommend instaparse:
(alex miller on the topic) https://www.reddit.com/r/Clojure/comments/fx5ko1/parser_libraries/fmttmzy/?utm_source=amp&utm_medium=#2020-10-1416:04Ivan FedorovThanks @U09QBCNBY!#2020-10-1416:10Ivan FedorovI think Iām not that interested in performance here, more interested in how stable the implementation will be.#2020-10-1415:51Ivan FedorovThinking about using it to parse recurrence rule from RFC5545
https://icalendar.org/iCalendar-RFC-5545/3-3-10-recurrence-rule.html#2020-10-1417:14Alex Miller (Clojure team)we do not recommend using regex specs to parse strings#2020-10-1505:35Ivan Fedorovack, thanks!#2020-10-1810:16mike_ananev@alexmiller do you have any plans to release spec2? The winter is coming, and I'll write to Santa my wishes: new Clojure release and Spec2 release in new year. š#2020-10-1812:46Alex Miller (Clojure team)Sorry, no plans either way#2020-10-1817:39thomIs there some list of milestones that need to be achieved before we ever see a non alpha Spec? Anything the community should be helping with?#2020-10-1818:02Alex Miller (Clojure team)no, it is mostly design work that Rich is working through at the moment#2020-10-1818:22vlaaadI'm curious what's missing in current spec vision that requires more design work...#2020-10-1818:36Alex Miller (Clojure team)redoing function specs#2020-10-1818:36Alex Miller (Clojure team)there are other things on the list but that's the big one atm#2020-10-2417:16wcalderipehey, is there a way to reuse the same spec for different map keys without having to redefine them over and over?
(s/def ::address (some-pred-to-check-ethereum-addr))
(s/def ::tx (s/keys :req-un [:sender ::address ;; apply address spec to :sender
:recipient ::address])) ;; apply address spec to :recipient
#2020-10-2417:38Alex Miller (Clojure team)In short, no#2020-10-2418:36borkdude@wcalderipe you can do this:
(def spec (s/....)
(s/def ::a spec)
(s/def ::b spec)
#2020-10-2418:37borkdudeNote sure if that also works in spec2 - I completely lost track of that#2020-10-2418:41Alex Miller (Clojure team)No, it wonāt like that but you could set it quoted and make it work via s/register#2020-10-2418:42Alex Miller (Clojure team)But I donāt think that works the way itās requested with req-un in either case#2020-10-2418:47wcalderipethanks @alexmiller and @borkdude#2020-10-2418:47borkdudeI think select might work better here right in spec2?#2020-10-2418:49Alex Miller (Clojure team)Yes, but will still have repetition#2020-10-2418:48wcalderipes/keys
is the best way to check the shape of a map?#2020-10-2418:49Alex Miller (Clojure team)For stuff like above, yes#2020-10-2620:28waffletowerIs there a way to limit recursion depth for clojure.test.check.generators/recursive-gen
?
(require '[clojure.test.check.generators :as g])
(binding [s/*recursion-limit* 1]
(g/generate (g/recursive-gen #(g/vector % 1 2) g/boolean)))
#2020-10-2620:28waffletowerSeeing results like:
[[[false]] [[true] [true]]]
#2020-10-2620:30gfredericksNo. It's your example result a problem?#2020-10-2620:30waffletowerYes, this is a contrived example, trying to limit stack overflows with a recursive spec#2020-10-2620:31gfredericksI think recursive specs are generated in a different way and have their own depth control.
Not 100% sure#2020-10-2620:33waffletowerWrapping the collections within my component specs using recursive-gen
has made stack overflows statistically less likely than not using that wrapping. So I thought they might be helpful if they could be further controlled#2020-10-2620:35waffletowerAre there other depth controls beside s/*recursion-limi*t
?#2020-10-2620:40Alex Miller (Clojure team)no, that's it - there are a few places where they are not being properly checked I think (b/c the gen code wrongly doesn't go through the place it's checked)#2020-10-2620:40Alex Miller (Clojure team)that's just a suspicion right now but seems like I've seen that before#2020-10-2620:41Alex Miller (Clojure team)and that's only for spec generators, won't affect directly constructed test.check generators#2020-10-2810:04borkdudeGrep Clojure code using clojure.spec: https://gist.github.com/borkdude/a391146ad81a06c28fb97ccdc1f64d44#2020-10-2810:04borkdudeGrep Clojure code using clojure.spec: https://gist.github.com/borkdude/a391146ad81a06c28fb97ccdc1f64d44#2020-10-2816:49borkdudeAlright, let's ship it: https://github.com/borkdude/grasp#2020-10-3020:54teodorluGood evening!
I've written specs to validate a small math language with addition and multiplication. Now I want to generate values! But I'm hitting Stack Overflow. I assume that's because my specs/generators are mutually recursive.
Any tips for writing mutually recursive spec generators that don't blow the stack?
Code attached.#2020-10-3022:35teodorluIdea: write generators as functions rather than directly on specs. Argument: recursion depth. Call generator functions from the spec.#2020-10-3022:51teodorluFollowed through on a generator namespace with recursion limit. That fixed my StackOverflow.
Sample for add and multiply:
(defn add [depth]
(gen/fmap to-list
(gen/tuple (gen/return '+) (expression (dec depth)) (expression (dec depth)))))
(defn expression [depth]
(if-not (pos? depth)
(number)
(gen/one-of [(number)
(multiply depth)
(add depth)])))
#2020-10-3113:48teodorluNote: Tree depth can also be controlled probabilistically. Make it more probable to take the non-recursive path than the recursive path. Example from the docs:
(gen/frequency [[9 gen/small-integer] [1 (gen/return nil)]])
The probability of the non-recursive path would have to counterbalance the fanout for the recursive paths.#2020-10-3114:23teodorluFrom reading test.check source, I see that many generators accept a size parameter. I suspect that I could have used that instead of inventing my own depth.#2020-10-3022:52borkdudeGrasp now supports finding keywords using clojure specs while also preserving location metadata!
https://github.com/borkdude/grasp#finding-keywords
This should come in handy for finding re-frame events and subscriptions in CLJS apps.
I also made a #grasp channel#2020-10-3114:02borkdudeThe ::invalid
keyword occurs 67 times in the spec repo:
$ ./grasp ~/git/spec.alpha/src -w -e "(fn [k] (= :clojure.spec.alpha/invalid (unwrap k)))" | wc -l
67
#2020-10-3114:26teodorluAnd you're finding both fully qualified :clojure.spec.alpha/invalid
and shorthand ::invalid
/ ::s/invalid
uses? Nice!
I'm curious about use-cases for grasp in tooling for refactoring.#2020-10-3115:01borkdudeyes#2020-10-3116:55kirill.salykinHi
please advice, how I can write a spec for :fileds
(s/explain-data ::fields [{:name "aaa" :type "bbbb"} {:name 1 :type "cccc"}])
so it checks that fields/name is unique - fails on every non unique entry and the spec also contains path of the non unique row
Thanks#2020-10-3118:59borkdude@kirill.salykin The uniqueness of the name is a property of the collection, not of one item
user=> (s/def ::unique-names (fn [xs] (= (map :name xs) (distinct (map :name xs)))))
:user/unique-names
user=> (s/explain ::unique-names [{:name :foo} {:name :bar}])
Success!
nil
user=> (s/explain ::unique-names [{:name :foo} {:name :foo}])
[{:name :foo} {:name :foo}] - failed: (fn [xs] (= (map :name xs) (distinct (map :name xs)))) spec: :user/unique-names
#2020-10-3119:00kirill.salykinI understand and agree
thought there is a dark magic to get indecies of non-unique elements, but apparently no
thanks for the answer#2020-10-3119:01borkdudeThere might be, but I don't know if it's pretty#2020-10-3119:03kirill.salykinmaybe there is a way shape spec output in a way as it reports per failed element (eg with s/problems, path (`:in []`) and predicate?)#2020-10-3119:10borkdude@kirill.salykin Maybe something like this:
user=> (s/def ::unique-names (s/and #(group-by :name %) (s/every-kv any? #(= 1 (count %)))))
:user/unique-names
user=> (s/explain ::unique-names [{:name :foo} {:name :foo}])
Execution error (UnsupportedOperationException) at user/fn (REPL:1).
nth not supported on this type: PersistentArrayMap
but I'm getting an exception#2020-10-3119:11kirill.salykininterestingā¦
so (s/and )
behaves like ->?#2020-10-3119:11Dmytro Buninno afaik#2020-10-3119:16borkdudeIt needed s/conformer around the first spec#2020-10-3119:15borkdude@kirill.salykin Ah, I needed s/conformer
:
user=> (s/def ::unique-names (s/and (s/conformer #(group-by :name %)) (s/every-kv any? #(= 1 (count %)))))
:user/unique-names
user=> (s/explain ::unique-names [{:name :foo} {:name :foo}])
[{:name :foo} {:name :foo}] - failed: (= 1 (count %)) in: [:foo 1] at: [1] spec: :user/unique-names
#2020-10-3119:18Dmytro Bunintil, thanks š#2020-10-3119:16kirill.salykinyou are magician! thank you so much!#2020-10-3120:23borkdudeI've made binary versions of grasp available in the #grasp channel now. Clojure spec on the command line, yeah :)#2020-11-0312:39borkdudeIs there any existing work combining clojure-spec conform + core.match or datalog? Like: I want ?x
and ?y
from (s/conform ...)
? I could see this being useful for grasp#2020-11-0312:54delaguardotake a look at mine matchete https://github.com/xapix-io/matchete if meander is not a good fit#2020-11-0312:58borkdudethanks. can you describe the difference between meander and yours?#2020-11-0313:01delaguardoā¢ no macro
ā¢ less power -> cleaner pattern syntax (this is just my opinion, btw)
ā¢ much smaller codebase -> much easier to read through and explore#2020-11-0313:02borkdudeš#2020-11-0314:34borkdude@U04V4KLKC How do I express this meander usage in your lib?
(m/find (first conformed) {:clauses {:clause {:sym !interface} :clauses [{:sym !interface} ...]}} !interface)
#2020-11-0314:35borkdudeThe output should be something like:
[clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]
#2020-11-0314:35borkdudeAnd the input:
{:reify reify, :clauses {:clause {:sym clojure.lang.IDeref, :lists [(deref [_] (deref-future fut))]}, :clauses [{:sym clojure.lang.IBlockingDeref, :lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]} {:sym clojure.lang.IPending, :lists [(isRealized [_] (.isDone fut))]} {:sym java.util.concurrent.Future, :lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}}
#2020-11-0314:52borkdudeHere's the full example:
https://github.com/borkdude/grasp#meander#2020-11-0315:49delaguardoalmost the same for that particular case
(require '[matchete.core :as mc])
(def data
'{:reify reify
:clauses {:clause {:sym clojure.lang.IDeref
:lists [(deref [_] (deref-future fut))]}
:clauses [{:sym clojure.lang.IBlockingDeref
:lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]}
{:sym clojure.lang.IPending
:lists [(isRealized [_] (.isDone fut))]}
{:sym java.util.concurrent.Future
:lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}})
(def pattern
{:clauses
{:clause {:sym '!interface}
:clauses (mc/scan {:sym '!interface})}})
(mc/matches pattern data)
#2020-11-0315:57borkdudeThis returns:
({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref]} {!interface [clojure.lang.IDeref clojure.lang.IPending]} {!interface [clojure.lang.IDeref java.util.concurrent.Future]})
#2020-11-0315:57borkdudeIs there a way to get a list of only the symbols like with meander?#2020-11-0316:00delaguardosorry, I was too fast copypasting )
instead of mc/scan please use mc/each#2020-11-0316:00delaguardoscan is for inspecting items in collection individually (in separate decision branches)
each is for walking over a collection#2020-11-0316:01delaguardo(def pattern
{:clauses
{:clause {:sym '!interface}
:clauses (mc/each {:sym '!interface})}})
this pattern should work#2020-11-0316:01borkdudeI get:
No such var: mc/each
#2020-11-0316:04delaguardoyou are using experimental 2.0.0 version
check out 1.2.0#2020-11-0316:05borkdudeno, I was using the version that is suggested in the README
$ clj -Sdeps '{:deps {io.xapix/matchete {:mvn/version "1.1.0"}}}'
:)#2020-11-0316:05delaguardoargh, cljdoc badge got updated for master branch as well,#2020-11-0316:06delaguardothen this is my bad, I should update it there#2020-11-0316:06delaguardosorry#2020-11-0316:07borkdudeno problem!#2020-11-0316:55borkdudeExcellent, it works now!
user=> (def conformed (map #(s/conform ::reify %) matches))
#'user/conformed
(def pattern
{:clauses
{:clause {:sym '!interface}
:clauses (mc/each {:sym '!interface})}})
#'user/pattern
user=> (mc/matches pattern (first conformed))
({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]})
#2020-11-0317:05borkdude@U04V4KLKC
https://github.com/borkdude/grasp#matchete#2020-11-0317:18borkdudeNice, it seems your lib also works with babashka:
$ bb -cp "$(clojure -Spath)" -e "(require '[matchete.core :as mc]) (mc/matches '{:x ?x} {:x 1})"
({?x 1})
#2020-11-0317:50delaguardoš#2020-11-0317:51delaguardoanother pusher to polish documentation š#2020-11-0312:41borkdudeMaybe meander is a good match for this#2020-11-0315:27Jim NewtonI'm starting looking at clojure.spec.alpha
can someone explain the different between the s/valid?
function and the s/spec?
function? Their docstrings look pretty similar.#2020-11-0315:27borkdude@jimka.issy spec?
is more like an instance check, valid?
validates a value against a spec#2020-11-0315:28borkdudein fact, it is an instance check :)#2020-11-0315:33Jim NewtonAs I understand the normal use case is the user associates certain tags, normally starting with :: with some DSL object which tells how to check validity? Does spec? check whether the symbol has been associated with a spec object, or does it check whether this is such an object?#2020-11-0315:34Jim NewtonHow can I know whether a symbol such as ::xyzzy designates a spec or not?#2020-11-0315:34borkdudeuser=> (s/def ::foo int?)
:user/foo
user=> (s/spec? ::foo)
nil
user=> (s/spec? (s/get-spec ::foo))
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "
#2020-11-0315:35borkdudeSo get-spec
returns the spec for a keyword or a symbol (in case of fdef)#2020-11-0315:38borkdude@jimka.issy There is also s/spec
:
user=> (s/spec ::foox)
Execution error at user/eval162 (REPL:1).
Unable to resolve spec: :user/foox
user=> (s/spec ::foo)
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "
#2020-11-0315:38borkdudefwiw, I don't use these functions a lot, except maybe when I'm writing tools around spec. Not in normal daily spec usage#2020-11-0315:39borkdudes/def
, s/valid?
and s/conform
are probably the most common ones I use#2020-11-0316:01Jim NewtonAfter something like the following:
(s/def ::big-even (s/and int? even? #(> % 1000)))
can I call some function on ::big-even
to get back (s/and int? even? #(> % 1000)
?#2020-11-0316:06borkdudealmost:
user=> (s/def ::big-even (s/and int? even? #(> % 1000)))
:user/big-even
user=> (s/form ::big-even)
(clojure.spec.alpha/and clojure.core/int? clojure.core/even? (clojure.core/fn [%] (clojure.core/> % 1000)))
#2020-11-0316:09Alex Miller (Clojure team)s/describe returns a shorter form, useful for printing (but not fully resolved so not too useful as data)#2020-11-0316:10Jim Newtonseems like s/form
does what I want. But it's not certain.#2020-11-0316:11Alex Miller (Clojure team)research question - if you are using specs with aliases, do you tend to use one alias, a few, or many in any given namespace?#2020-11-0316:12borkdudewhat do you mean by an alias?#2020-11-0316:12borkdudelike (alias 'foo 'bar)
and then ::bar/spec
?#2020-11-0316:13borkdude@alexmiller I think grasp could be used to find this out#2020-11-0316:13Lennart Buitone reason for me to use many aliases would be if many keys in a s/keys
require a same generator. For one, we have a present string spec, with corresponding generator that we alias a lot#2020-11-0316:17Lennart Buitah nevermind then, I interpreted this as an alias:
(s/def ::my-spec ...)
(s/def ::key-2 ::my-spec)
(s/def ::key-1 ::my-spec)
(s/def ::map (s/keys :req [::key-2 ::key-2]))
#2020-11-0316:14borkdudehmm, never mind, grasp returns the fully qualified keyword, so you can't see if it was aliased or not#2020-11-0316:15Alex Miller (Clojure team)@borkdude yes, that's what I mean#2020-11-0316:16Alex Miller (Clojure team)https://grep.app/search?q=%28alias%20%27&filter[lang][0]=Clojure gets me some idea at least#2020-11-0316:17Alex Miller (Clojure team)but really wondering what people are doing in private code bases#2020-11-0316:17borkdudewe usually don't use alias for this, since a lot of our specs are in CLJS. We create namespaces on disk for this#2020-11-0316:21borkdude@alexmiller typically we have this in our backend, using on disk namespaces:
(s/def ::system
(s/merge ::dict/system
::adis/system
::annotator/system
(s/keys :req [::database/db ::cache/cache])))
#2020-11-0317:23Jim NewtonMore spec questions. When I define ::big-even as in https://clojure.org/guides/spec.
(s/def ::big-even (s/and int? even? #(> % 1000)))
I'm guessing that the function #(> % 1000)
is a closure in its current lexical context. Right? So later when I call s/form
I don't get back that object, but some new text, which doesn't know the original lexical context. Right?
(s/form ::big-even)
==> (clojure.spec.alpha/and
clojure.core/int?
clojure.core/even?
(clojure.core/fn [%] (clojure.core/> % 1000)))
#2020-11-0317:26borkdude@jimka.issy The text #(...)
is not representable as an s-expression, it's something built into the reader which expands to (fn [...] ...)
#2020-11-0317:27borkdudeWhat simply happens is that s/def
is a macro which captures the input and by that time the s-expression has already expanded to the fn
form#2020-11-0317:36Alex Miller (Clojure team)the spec function will capture the lexically scoped values at the point of definition (but the form will not - it will have the names)#2020-11-0317:36Alex Miller (Clojure team)user=> (let [x 100] (s/def ::foo (s/and int? #(= % x))))
:user/foo
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false
user=> (s/form ::foo)
(clojure.spec.alpha/and clojure.core/int? (clojure.core/fn [%] (clojure.core/= % x)))
#2020-11-0317:36Alex Miller (Clojure team)^^ form is broken there#2020-11-0317:42Alex Miller (Clojure team)this is all somewhat more consistent in spec 2 (namely, this will not work at all as the divide between symbolic specs and spec objects is clearer) but there is built-in support for creating parameterized specs (like the one above) if needed#2020-11-0408:51Jim Newton@alexmiller, thanks for the explanation. however, i'd like to get the actual function object which spec would use to validate with. For example given something like (s/and int? #= % x))
, I can use int?
to get the int? function, but I can't use the s-expression form of the closure to recuperate the closure. However, I'm pretty sure spec has the closure somewhere, otherwise s/valid?
would not work.#2020-11-0408:58Jim NewtonWhat I'm trying to do is figure out which spec incantations can be expressed in a simple type system which I have implemented, called genus
, normally abbreviated gns/
.
For example, a naĆÆve first attempt is able to translation ::big-even
as follows
clojure-rte.rte-core> (require '[clojure.spec.alpha :as s])
nil
clojure-rte.rte-core> (s/def ::big-even (s/and int? even? #(> % 1000)))
:clojure-rte.rte-core/big-even
clojure-rte.rte-core> (s/get-spec ::big-even)
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x61966631 "
The important result being the next step:
clojure-rte.rte-core> (gns/canonicalize-type '(spec ::big-even))
(and
(or Long Integer Short Byte)
(satisfies clojure.core/even?)
(spec (clojure.core/fn [%] (clojure.core/> % 1000))))
So genus can unwind the spec into types and predicates, if the predicates are symbolic. But what's left (clojure.core/fn [%] (clojure.core/> % 1000))
is not something which is useable.#2020-11-0409:00Jim NewtonI could try to write a translator/compiler for expressions such as (clojure.core/fn [%] (clojure.core/> % 1000))
which try to convert them back to a closure, in the case that they don't have any free variables. But that work seems redundant since spec already has the closure in its internal data structure.#2020-11-0409:09Jim NewtonI just realized something interesting. If I eval the list (clojure.core/fn [%] (clojure.core/> % 1000))
then I get a function object which seems to have the same semantics as the original function, in case there are no free variables.
clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000))) 12)
false
clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000))) 100000)
true
clojure-rte.rte-core>
Perhaps that is good enough for my proof-of-concept.#2020-11-0409:10Jim NewtonAnd if there's a free variable, eval
throws an exception
clojure-rte.rte-core> (eval '(clojure.core/fn [%] (clojure.core/> x 1000)))
Syntax error compiling at (clojure-rte:localhost:51477(clj)*:45:51).
Unable to resolve symbol: x in this context
#2020-11-0414:11Alex Miller (Clojure team)Well eval is literally the function that takes a form and returns a function#2020-11-0317:42Alex Miller (Clojure team)user=> (s/defop narrow-int [x] (s/and int? #(= % x)))
#'user/narrow-int
user=> (s/def ::foo (narrow-int 100))
:user/foo
user=> (s/form ::foo)
(user/narrow-int 100)
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false
#2020-11-0410:05Jim NewtonYet another question about spec.
This time about the semantics of the sequence designators such as s/alt
, s/+
, s/*
etc.
an element such as (s/+ even?)
means (if I understand correctly, a sequence of 1 or more objects for which the even?
predicate is true.
but I can substitute a spec name for even?
and say (s/+ ::big-even)
and that will be a sequence of 1 or more objects each of which validates the ::big-even
spec.
However, what does (s/+ (s/alt :x even? :y prime?))
mean. I would guess that it means a sequence of 1 or more elements each of which either satisfy even?
or satisfy prime?
But if I define
(s/def ::even-or-prime (s/alt :x even? :y prime?))
what does this mean? (s/+ ::even-or-prime)
It is a sequence of 1 or more sequences (each of which contains elements which are either even or prime), or is it a sequence of 1 or more integers each of which is either even or prime?#2020-11-0410:06Jim NewtonI'm pretty sure spec has a way to designate both, as they are both possible things the user might want to express. right?#2020-11-0410:07borkdude@jimka.issy s/alt
is for regexes, use s/or
for normal predicates#2020-11-0410:08borkdude(s/+ (s/alt ....))
means a sequence of one or more alternatives#2020-11-0410:09borkdudeit doesn't mean each, this can be expressed with s/every
or s/coll
#2020-11-0410:11Jim NewtonSorry, my question too convoluted. Let me ask simpler.
If ::foo
is a spec defined somewhere using s/def
, what does (s/+ ::foo)
mean? Does it mean a sequence of 1 or more objects which each validate ::foo
?#2020-11-0410:11borkdudeyes#2020-11-0410:12Jim NewtonSo it is not referntially transparent?#2020-11-0410:13Jim NewtonI.e. substituting the expression for ::foo in place, changes the meaning?#2020-11-0410:13borkdudeI don't see why that would be the case?#2020-11-0410:17Jim Newtonsuppose
(s/def ::foo (s/alt :x even? :y prime?))
Now
(s/+ (s/alt :x even? :y prime?))
matches a sequence of 1 or more integers each of which are either even or prime, right?
but (s/+ ::foo)
matches a sequence of 1 or more objects which each satisfy :foo
, i.e., a sequence of sequences each of which contain even or prime integers.#2020-11-0410:20borkdude@jimka.issy Please give a working example. E.g.:
user=> (s/def ::foo (s/or :x even? :y neg?))
:user/foo
user=> (s/conform (s/+ ::foo) [2 4 -11])
[[:x 2] [:x 4] [:y -11]]
user=> (s/conform (s/+ (s/or :x even? :y neg?)) [2 4 -11])
[[:x 2] [:x 4] [:y -11]]
#2020-11-0410:23borkdudeA spec is only applied to one thing, not to all elements of a collection, unless you use s/every
or s/coll-of
.#2020-11-0410:24borkdudeuser=> (s/def ::nums-or-strings (s/or :nums (s/every number?) :strings (s/every string?)))
:user/nums-or-strings
user=> (s/conform ::nums-or-strings [1 2 3])
[:nums [1 2 3]]
user=> (s/conform ::nums-or-strings ["foo" "bar"])
[:strings ["foo" "bar"]]
#2020-11-0410:24Jim Newtonyes, but you answer avoids my question. I'm not asking how to write a good spec. I'm asking about the semantics.#2020-11-0410:25borkdudeAnd I asked you about a counter-example of referential transparency. A working one without imaginary functions like prime?
which does not exist in core.#2020-11-0410:25Jim NewtonOk here is a very simple one.#2020-11-0410:26vlaaadI mean you donāt really need prime?
source to know its semantics#2020-11-0410:26Jim Newton(s/+ ::foo)
matches a sequence for which all the elements are either even or negative.#2020-11-0410:26vlaaadlooks right#2020-11-0410:27borkdudeYour words are ambiguous to me. Do you mean: every element is a ..., or on a case by case basis?#2020-11-0410:27Jim Newtonlets stick with integers and sequences thereof.#2020-11-0410:28Jim NewtonNow define
(s/def ::foo2 (s/+ ::foo))
#2020-11-0410:29Jim Newtonwhat does (s/+ ::foo2)
mean. It does not mean a sequence of 1 or more objects which of which match ::foo2
#2020-11-0410:30borkdudeWhat does ::foo
mean in your code here?#2020-11-0410:30borkdudesome kind of integer that's either this or that?#2020-11-0410:30Jim Newton(s/def ::foo (s/or :x even? :y neg?))
#2020-11-0410:31borkdudethen ::foo2 means multiple even or negs, but they can be mixed. it's not all or nothing#2020-11-0410:31vlaaadI would. say (s/+ ::foo2)
is a seq of 1 or more objects each of which is is ::foo2
, why would you say it isnāt that?#2020-11-0410:37Jim Newton(s/def ::foo (s/or :x even? :y neg?))
(s/def ::foo2 (s/+ ::foo))
#2020-11-0410:38Jim NewtonNow ::foo matches even and negative integers#2020-11-0410:38Jim Newton(s/+ ::foo)
matches a non-empty sequence of even and negative integers#2020-11-0410:39vlaaadsounds right#2020-11-0410:39Jim Newtonbut (s/+ ::foo2)
does not match a non-empty sequence of objects which match ::foo2
#2020-11-0410:40vlaaadah, I think thatās because of how regex ops nest#2020-11-0410:40Jim Newton#2020-11-0410:41Jim Newtonyess!!!!!!!!#2020-11-0410:41Jim Newtonthis was my original question (s/+ x)
does not match a non empty sequence of things which match x#2020-11-0410:41borkdude@jimka.issy If you want to do this, you need to use s/spec
around the ::foo2
#2020-11-0410:42borkdudeThis has to do with regex nesting indeed, not with referential transparency. Read the spec guide, which explains this#2020-11-0410:42borkdude> When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context.#2020-11-0410:43vlaaad(s/valid? (s/+ (s/spec ::foo2)) [[2 -3 4] [2 -3 4] [2 -3 4]]) => true
#2020-11-0410:43Jim NewtonI have read down https://clojure.org/guides/spec#_sequences.#2020-11-0410:44Jim NewtonDid I miss the paragraph about when regex ops are combined.#2020-11-0410:44borkdudeI assume you know how Ctrl-f works ;)#2020-11-0410:44Jim NewtonBesides, I'm confused about what "regex" operator means.#2020-11-0410:44borkduderegex operators are things like s/+
, s/?
#2020-11-0410:44Jim Newtonsometimes it means string regexps. and some times it means sequence operations.#2020-11-0410:45borkdudein the context of spec, regex means a spec which describes a sequence of things#2020-11-0410:46borkdudeThe idea of spec is based on this paper:
http://matt.might.net/papers/might2011derivatives.pdf where that terminology comes from#2020-11-0410:47Jim Newtonso does this mean that if a sequence operator is directly within another sequence operator it has this special modal/merging meaning. otherwise it has its normal meaning?#2020-11-0410:47borkdude(or maybe not that one, but the paper where that paper is based on)#2020-11-0410:48borkdude> When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context.#2020-11-0410:48borkdudes/alt
returns a regex op, s/or
is for combining predicates#2020-11-0410:48borkdudeso s/or
is not a regex op.#2020-11-0410:49Jim Newtonhave you seen this paper? https://www.lrde.epita.fr/dload/papers/newton.18.meta.pdf#2020-11-0410:50borkdudeI haven't#2020-11-0410:51Jim Newtonand this one, where I introduced the derivate for implementing something quite similar to spec in common lisp https://www.lrde.epita.fr/dload/papers/newton.16.els.pdf#2020-11-0410:52Jim Newtonthe system I devised (in 2016) and after, is called RTE (regular type expressions).#2020-11-0410:53borkdudecool!#2020-11-0410:53Jim NewtonWhat i'm trying to do now is figure how similar or different it is from spec at a theoretical leve.#2020-11-0410:53Jim NewtonSo I'm trying to see if I can "compile" a spec into an RTE.#2020-11-0410:54Jim Newtonif so, then an RTE can me validated in linear time with no backtracking.#2020-11-0410:54Jim Newtonas I understand it, a spec takes exponential time worst case, so users avoid worst cases.#2020-11-0410:55Jim Newtonhowever, since I have found no academic papers on spec, and I am not an expert of spec, I'm just guessing. Maybe I'm completely wrong about my supposition#2020-11-0410:55borkdude@jimka.issy I'm interested in that for clj-kondo as well. Clj-kondo has a tiny type system in which I try to avoid backtracking of type checking arguments. Currently by just not allowing sophisticated constructs :)#2020-11-0410:55borkdude(https://github.com/borkdude/clj-kondo/blob/master/doc/types.md)#2020-11-0410:55Jim Newtoninteresting*#2020-11-0410:56Jim NewtonI submitted a paper recently to the Dynamic Languages Symposium where I introduced a Simple Type System for use in dynamic languages. The paper was rejected. One of the reasons for rejection was the revewers didn't see practical applications.#2020-11-0410:57Jim Newtona simple type system allows you to reason about types (where type is defined as any set of values) . Furthermore intersections, unions, and complements of types are also types.#2020-11-0410:58Jim NewtonIf your type logic can conclude that a type is empty, then you've usually found a bug in the user's code.#2020-11-0410:58borkdudeIt sounds like it could be very useful for clojure. Are you aware of the work done in core.typed?#2020-11-0410:58Jim NewtonThats Ambrose?#2020-11-0410:58borkdudeyes#2020-11-0410:59Jim NewtonHe's looked at my work and is interested in helping out. but everyone is busy.#2020-11-0410:59Jim NewtonI'd also love to have a student researcher interested in working with clojure.#2020-11-0410:59borkdudeIt looks like your work could help improve clj-kondo's type system since linear checking is necessary for performance#2020-11-0411:00Jim Newtonwhat's clj-kondo written in? is it written in clojure?#2020-11-0411:00borkdudeyes#2020-11-0411:01Jim Newtonare you more of a hacker, or more of an academic?#2020-11-0411:01Jim Newtonor more of an engineer? I don't mean that in any way insulting. sorry if it sounds as such....#2020-11-0411:01borkdudehaha. I do have a CS master degree, but not a Phd in type systems ;)#2020-11-0411:02borkdudeclj-kondo focuses on usefulness though, whereas something like core.typed is maybe more an academic project#2020-11-0411:02borkdudeI don't care about publishing papers, I want the tool to help me#2020-11-0411:04Jim NewtonI'm not sure why my next best step should be.
For the moment I have an implementation of genus, (the simple type system in clojure) and RTE (regular type expressions) which allows genus to represent sets of sequences which match regular expressions in terms of types. Genus claims to be extensible.
For the moment my experimental task is see if I can indeed extend genus (in user space) to incorporate the spec
type. I.e., given a spec, the type will be the set of all objects which validate the spec.#2020-11-0411:05Jim Newtonspec types will be treated as opaque types such as arbitrary predicate. except in the case where I can compile a spec into more fundamental types in which case the system can reason about them.#2020-11-0411:05borkdudeAnother tool I made recently also uses clojure.spec for searching code:
https://github.com/borkdude/grasp
This could potentially made faster
Another project which tries to use spec at compile time:
https://github.com/arohner/spectrum
It's experimental in nature.#2020-11-0411:06Jim Newtonfor example it will be able to decide whether the set of objects satisfying spec1 and also spec2 is empty. or decide whether two given specs are equivelent.#2020-11-0411:07Jim Newtonin order for RTE to represent the regular type expression as a deterministic finite automaton, it needs to be able to determine whether two given types are disjoint. This is currently not possible with spec in the most general case, but could be possible in many particular cases.#2020-11-0411:08Jim NewtonAs I understand spec's internal data structure uses non-deterministic finite automata, thus the exponential behavior in the worst case.#2020-11-0411:09Jim NewtonANYWAY, maybe now you understand a bit better the reason for my strange beginner questions about the semantics of spec ???#2020-11-0411:09borkdudeyes, thanks for explaining!#2020-11-0411:10borkdudeThe authors of spec usually emphasise that spec is not a replacement for a type system, but a runtime validation system.#2020-11-0411:10Jim Newtonin my opinion there is no difference#2020-11-0411:10borkdudeWhat might be of interest, I have a collection of specs for core functions here: https://github.com/borkdude/speculative#2020-11-0411:11borkdudeAs in, spec could be expressed as a case of dependent typing?#2020-11-0411:12Jim Newtonwhat is speculative? I didn't immediately understand by reading the intro.#2020-11-0411:13borkdude> a collection of specs for core functions#2020-11-0411:13Jim NewtonFor me a type is a set of values. And a type system allows programmers to designate and reason about certain types.#2020-11-0411:13Jim Newtonthat's what spec does.#2020-11-0411:14borkdudee.g. a spec for clojure.core/cons
:
https://github.com/borkdude/speculative/blob/4e773794a4065a84bdadd997516e52c76ab51b1f/src/speculative/core.cljc#L17#2020-11-0411:14borkdudeyes, if you define it like that, it makes sense. but this set of values can often not be known at compile time, which makes the difference between static compile time checking#2020-11-0411:15Jim Newtonwhat is s/fdef
?#2020-11-0411:16borkdudeclojure.spec supports defining specs for checking function arguments and return values using fdef
#2020-11-0411:17Jim Newtonyes many times you can indeed know a lot about a type at compile time. in those cases, compilers like the SBCL common lisp compiler can create more efficient code, or tell the user about errors or unreachable code.#2020-11-0411:18Jim Newtonso an fdef decribes what a valid function call site looks like?#2020-11-0411:18borkdudeyes#2020-11-0411:19Jim NewtonI can see how the predicate based assumption of spec, could be a bottle neck for a code analyzer like kondo.#2020-11-0411:20borkdudeI initially used spec in clj-kondo, but I found the performance not good enough#2020-11-0411:20Jim Newtonone think that I've noticed is that many many many of these predicates can be rewritten as a simple type check. And I have code which automates this.#2020-11-0411:21Jim Newtonahhh, so you have abandoned that piste now?#2020-11-0411:22borkdudethe format now uses something more explicit: you provide the seq of types per arity. so you don't have to do any backtracking to to match a seq based on some s/alt expression.
which makes more sense to me considering the structure of multi-arity functions.#2020-11-0411:22borkdudeand a seq of types is usually a fixed number of things or a fixed number of thing + a variable amount of things of the same type#2020-11-0411:22borkdudeI think this matches 99% of cases how people call functions in clojure#2020-11-0411:23Jim Newtonfor example
(gns/canonicalize-type '(satisfies int?))
==> (or Long Integer Short Byte)
(or ...) is a union type. when intersected and unioned with other types, it can be arranged so that redundant checks are eliminated.#2020-11-0411:23borkdudeclj-kondo also supports union types, it's just #{:int :string}
#2020-11-0411:24borkdudeit also tries to infer returns types and threads types through let expressions#2020-11-0411:24borkdudeExample:#2020-11-0411:25Jim Newtonso it has type inferencing?#2020-11-0411:25borkdudeIt has inferencing yes, but limited to the things it knows, else it's any? and it won't complain#2020-11-0411:26Jim NewtonI wrote something pretty similar some years ago for SKILL++ which was a proprietary lisp used by the company I worked for for many years.#2020-11-0411:28Jim Newtonin my system, called "loathing" it would see that foo
returns type string (because of the definition), and the call site inc expects an integer. So the type at that point is (and integer string) which is the empty-type.#2020-11-0411:29borkdudeclj-kondo does something similar. but it just has a simple graph of things that are reachable or not: you can never go from string to int, so this is an error#2020-11-0411:30borkdudebut you can go from any to int, or any to string, so this is not an error#2020-11-0411:30Jim Newtonhowever, if inc had input type (or integer string), then the intersection would be (and string (or string integer)) = string != empty-set#2020-11-0411:30borkdudeyep, same thing in kondo#2020-11-0411:31borkdudee.g.: this will not give a warning:
(defn foo [x]
(if (int? x)
x
(str x)))
(inc (foo 1))
#2020-11-0411:32Jim NewtonYou mentioned that you are not interested in publications. But I need to make publications. It's part of my job.#2020-11-0411:32borkdudeWell, I meant that this is not the goal of clj-kondo.#2020-11-0411:32Jim NewtonI need to get several publications out of this work.#2020-11-0411:32borkdudeIt's not the product of academic research#2020-11-0411:32borkdudeI have nothing against making publications#2020-11-0411:33borkdudeI do have my name on one publication ;)#2020-11-0411:33Jim Newtonand I think there are several interesting results to publish. But i'm a very poor publicist. I don't do well at convincing people of the interest or novelty of my work.#2020-11-0411:34borkdudeIf you could get practical benefit out of your work by e.g. improving clj-kondo or a similar tool, I'm sure it will convince people#2020-11-0411:35borkdude@jimka.issy I'm sharing this for fun, but it's also related:
https://borkdude.github.io/re-find.web/
Find functions based on example in and outputs#2020-11-0411:35borkdudeThis uses the speculative specs to search for matches#2020-11-0411:37Jim Newtonthe clojure community is friendly for the most part. although I do see some snide comments now and then.#2020-11-0411:37Jim NewtonSo it is usally pretty easy to get questions answered.#2020-11-0411:37Jim NewtonI would love to have some spec examples for test cases.#2020-11-0411:37borkdudeI would say the clojure community is better in that respect than some others#2020-11-0411:38Jim NewtonCould I solicit you to give me some spec examples, simple and complex. some which are trivial, and some which are very slow to validate?#2020-11-0411:38Jim NewtonFor the moment I'm just worried about sequences, not functions or maps or data structures. My code is not advanced enough to handle those.#2020-11-0411:39Jim NewtonI want to try to compile these to RTE and see if I can come up with some cases where RTE is faster.#2020-11-0411:39Jim Newtonif RTE is never faster, then i'm in trouble#2020-11-0411:39borkdudeYou could take a look at speculative, which has more than hundred specs#2020-11-0411:40borkdudeI don't have a particular example of a slow spec, but I guess you can fabricate one based one which does backtracking#2020-11-0411:41Jim Newtonwhen analyzing code statically, i suppose you are usually looking at nested sequences, not really at maps and such.#2020-11-0411:41borkdudeThis tool: https://github.com/borkdude/grasp searches through code. It applies the spec at every sub-tree of an s-expression#2020-11-0411:42borkdudeSo performance improvements would be really noticeable there#2020-11-0411:42borkdudeI have to go now. speak to you later#2020-11-0411:45Jim Newtonok, thanks. talk later#2020-11-0411:45Jim NewtonBTW i filed an issue for kondo, was it helpful?#2020-11-0411:48borkdudeYes, thank you. I'm not sure if clj-kondo will be able to support this, as it sees one file as a standalone unit and in-ns
doesn't really match with this model. So either using in-ns
with :refer
on the original namespace requires you to help clj-kondo with some annotations or config, or clj-kondo has to make non-trivial changes.#2020-11-0411:48borkdudeI will think about this for a while#2020-11-0411:49Jim Newtongocha#2020-11-0413:35Jim NewtonI don't think s/& is a regular expression operation. It doesn't seem regular. as far as I can tell, it requires the implementation to remember the sequence being tested. a regular expression demands that every decision be based only one the current object of the sequence, and a state. whereas only a finite number of states are allowed. In this case if there are N states, you cannot test a sequence of length N+1.#2020-11-0410:31Jim NewtonWait. I think I'm confused. Let me work out a more consistent example. And come back to you in 5 minutes. OK?#2020-11-0410:31borkdudeok#2020-11-0410:35Jim Newtontaking this conversation to a thread. I hope everyone follows.#2020-11-0411:47Jim Newtonanother spec question. Are all spec definitions global? If I'm writing test cases, I'd love to introduce local definitions that don't pollute the global space.#2020-11-0411:54borkdude@jimka.issy Yes, spec has one global registry (compare spec keywords to RDF uris for example). Malli (#malli) which aims to be an alternative schema library has local registries as well.#2020-11-0411:55Jim NewtonI suppose for test cases I can use ::xyzzy to keep the symbols in my local namespace.#2020-11-0411:59Jim NewtonDo I understand correctly that there is no intersection equivalent of s/alt
for regex semantics. I.e., s/or
is to s/alt
as s/and
is to what?#2020-11-0411:59borkdudes/&
#2020-11-0412:05Jim Newtonahh, it's not included in https://clojure.org/guides/spec#_sequences, at least not in the table with the others#2020-11-0412:08Jim Newtonsyntactically does s/&
work like s/alt
and s/cat
in that I need to tag the components? or like s/and
where no tags are used?#2020-11-0412:09borkdude@jimka.issy In the guide, search for:
> Spec also defines one additional regex operator, &, which takes a regex operator and constrains it with one or more additional predicates.
to see an example#2020-11-0412:19Jim Newtonahh so s/&
is not really analogous to s/alt
it is yet a third syntax and semantic.#2020-11-0412:19borkdudeI thought you were asking analogous to s/and
.
> I.e.,Ā s/orĀ is toĀ s/altĀ asĀ s/andĀ is to what?#2020-11-0412:25borkdudes/alt
and s/&
are supposed to be used to build regexes, s/and
and s/or
are just general ways of combining predicates#2020-11-0412:26borkdudebut sometimes they do the same thing#2020-11-0412:26Jim NewtonIt looks (from the documentation) that s/&
is not analgous to s/and
in the same way that s/alt
is analogous to s/or
. if they were analogous I'd expect s/&
to work like this.
(s/* (s/alt :x (s/cat :a neg? :b even?) :y (s/cat :c odd? :d pos?)))
(s/* (s/& (s/cat :a neg? :b even?) (s/cat :c odd? :d pos)))
but instead it works like this
(s/* (s/& (s/cat :a neg? :b even?) #(even? (count %))))
#2020-11-0412:27borkdudeSee the difference here:
user=> (s/conform (s/alt :foo int? :bar string?) [1])
[:foo 1]
user=> (s/conform (s/or :foo int? :bar string?) [1])
:clojure.spec.alpha/invalid
user=> (s/conform (s/or :foo int? :bar string?) 1)
[:foo 1]
#2020-11-0412:29Jim Newtonyes s/or matches an object, and s/alt matches a sequence of objects#2020-11-0412:35Jim NewtonSo why doesn't this return true?
(s/valid? (s/* (s/& (s/cat :a neg? :b even?)
(s/cat :c odd? :d pos?)))
[-3 4 -5 2])
it is both a sequence of neg even occurring 0 or more times, and also a sequence of odd pos occuring 0 or more times#2020-11-0412:36Jim NewtonIf I replace s/& with s/alt (and insert the required keys), it works as expected.
(s/valid? (s/* (s/alt :x (s/cat :a neg? :b even?)
:y (s/cat :c odd? :d pos?)))
[-3 4 -5 2])
#2020-11-0412:49Jim NewtonIt looks to me (on first and second reading of the specification) that (s/& pattern predicate)
means that the subsequence which matches the pattern must as a sequence match the predicate. So the following
(s/cat :a (s/& (s/* string?) #(even? (count %)))
:b (s/& (s/+ int?) #(odd? (count %))))
matches a sequence which begins with an even number of strings and ends with an odd number of integers.
Is my interpretation correct?
I don't think these are really regular expressions any more. I have to think about it, but my suspicion is that this cannot be solved without using a stack.#2020-11-0414:00Jim NewtonHow can I recognize an object such as the one returned by
(s/* (s/alt :x (s/cat :a neg? :b even?)
:y (s/cat :c odd? :d pos?)))
s/spec?
returns false
. type
returns clojure.lang.PersistentArrayMap
.
s/get-spec
returns nil
.#2020-11-0414:13Alex Miller (Clojure team)isnāt there a regex?
#2020-11-0414:15Jim Newtonso regex?
returns non-nil. is that one I should use?#2020-11-0414:16Alex Miller (Clojure team)Yeah#2020-11-0414:18Jim Newtonseems to be any object for which ::op returns boolean true.#2020-11-0414:19Jim Newtonwhere ::op is :clojure-spec-alpha/op#2020-11-0414:19Jim Newtonfair enough#2020-11-0517:02Jim NewtonI just discovered that the following doesnt work.
(ns some-ns
(:require [clojure.spec.alpha :as s]))
(defn foo []
(= (resolve 's/*) (resolve 'clojure.spec.alpha/*)))
because when foo
gets called by the application, the name space might not have required [clojure.spec.alpha :as s]
Instead, I did the following and it seems to work.
(ns some-ns
(:require [clojure.spec.alpha :as s]))
(defn foo []
(binding [*ns* (find-ns 'some-ns)]
(= (resolve 's/*) (resolve 'clojure.spec.alpha/*))))
#2020-11-0517:02Jim NewtonI just discovered that the following doesnt work.
(ns some-ns
(:require [clojure.spec.alpha :as s]))
(defn foo []
(= (resolve 's/*) (resolve 'clojure.spec.alpha/*)))
because when foo
gets called by the application, the name space might not have required [clojure.spec.alpha :as s]
Instead, I did the following and it seems to work.
(ns some-ns
(:require [clojure.spec.alpha :as s]))
(defn foo []
(binding [*ns* (find-ns 'some-ns)]
(= (resolve 's/*) (resolve 'clojure.spec.alpha/*))))
#2020-11-0517:05andy.fingerhutYou get different results than this in a REPL?
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (resolve 's/*)
#'clojure.spec.alpha/*
user=> (resolve 'clojure.spec.alpha/*)
#'clojure.spec.alpha/*
user=> (= (resolve 's/*) (resolve 'clojure.spec.alpha/*))
true
#2020-11-0517:06andy.fingerhutNote that you have no *
after clojure.spec.alpha/
I do not know if that is a typo in chat, or reflects your actual code.#2020-11-0517:06Jim Newtonyes a copy/paste error. corrected, should be /* everywhere#2020-11-0517:07andy.fingerhutNow I see clojure.spec.alpha
. Did you intend to use clojure.spec.alpha/*
instead?#2020-11-0517:08andy.fingerhutdefn foo
is in the namespace some-ns
? If so, how does some part of your code call foo
without first require'ing some-ns
?#2020-11-0517:09Jim Newtonimportant observation is that the behavior of resolve
depends on the value of *ns*
when foo
is called, not when foo
is defined.#2020-11-0517:09Jim Newtonthe caller of foo has required some-ns, but he might not have aliased closure.spec.alpha to s#2020-11-0517:10andy.fingerhutGot it. Yes, that is true, and resolution of aliases like s
definitely depends upon the namespace in which they are resolved.#2020-11-0517:11Jim Newtoni found out because I ran my tests from the repl where I had required spec and aliased to s. But when I tested from lien test
at the comand line, it ran the tests with the user
namespace, which I never tested in.#2020-11-0517:11andy.fingerhutAlias s
could be not an alias in some namespace, be used for clojure.spec.alpha
in another namespace, and be used for clojure.set
in yet another namespace. Not recommended for developer sanity to use the same alias for different namespaces, but Clojure allows it.#2020-11-0517:12Jim Newtonbesides, when I develop the code, I use some convention. But I don't impose that convention on my customers who might have very different programming styles. or they theoretically might even be calling my code from java or some such#2020-11-0517:12andy.fingerhutThe doc string for resolve
gives a pretty strong hint about this dependence upon the current value of *ns*
#2020-11-0517:14Jim NewtonI think all calls to resolve or suspect within code.. I'm thinking of refactoring every call in my program and replacing with ns-resolve so I'll have to think about which ns to use. Then test from repl and also from lein command line as a general pratice#2020-11-0517:14Jim Newtonit's on my to-do list.#2020-11-0517:15andy.fingerhutUsing ns-resolve
instead of resolve
sounds like a good idea, if you want to force yourself to be explicit about in which namespace the resolving is performed.#2020-11-0517:17Jim Newtonof course nothing prohibits me from using (ns-resolve *ns* foo)
if that's the appropriate one to use.#2020-11-0720:39borkdudeWhat was the take on spec and coercions again - spec isn't meant to do coercions of values (during conformation), rather it only validates or conforms values, right?
Then what is the opinion on a library like this one? https://github.com/exoscale/coax#2020-11-0721:01seancorfieldMy reading of comments by Alex, at least, is that Specs should not do coercions, but libraries that infer coercions from Specs, and do those coercions "outside" Spec are a good way to do stuff.#2020-11-0721:03seancorfieldOur "web specs" library does the coercions internally in the Specs ("bad") but I've been looking at moving to spec-coerce
instead (or maybe coax
) and deprecating our library @borkdude#2020-11-0721:08borkdudeNice, thanks. Coax looks pretty cool#2020-11-0806:10mpenetcoax is used heavily in production, so far so good :) we fixed a number of bugs from the original spec coerce fork and made it more flexible and performant. I am not sure I would still call coax a fork, the source is now totally different.#2020-11-0806:17mpenetspec-coerce outlined some good ideas but it felt a bit like a POC, I am not sure it is/was used seriously.#2020-11-0914:53Jim Newtonhow can I recognize an object such as the one returned from (s/or :1 int? :2 number?)
. I see that s/regex?
, s/get-spec
, and s/spec?
all return nil
. I notice however that the class of the object has (type (s/or :1 int? :2 number?))
in its ancestors?
(ancestors (type (s/or :1 int? :2 number?)))
==> #{clojure.spec.alpha.Spec clojure.lang.IObj clojure.lang.IMeta
java.lang.Object clojure.spec.alpha.Specize}
So I can use the following ugly idiom:
(instance? clojure.spec.alpha.Spec (s/or :1 int? :2 number?))
isn't there a better way?#2020-11-0914:56Jim NewtonNext question, how can I extract int?
and number?
from the object? I see that (keys (s/or :1 int? :2 number?))
throws an Exception.
(keys (s/or :1 int? :2 number?))
Execution error (IllegalArgumentException) at clojure-rte.genus-spec-test/eval17912 (form-init5322407590801644377.clj:30310).
Don't know how to create ISeq from: clojure.spec.alpha$or_spec_impl$reify__2118
I see that (s/describe (s/or :1 int? :2 number?))
returns the list (or :1 int? :2 number?)
. However, to recognize the object as one which has a describe
method.#2020-11-0915:07Alex Miller (Clojure team)it's important to be clear in talking about the two worlds of spec - spec forms and spec objects. (s/or :1 int? :2 number?)
is a spec form, which when evaluated, returns a spec object (something that satisfies the spec protocol). Trying to nail down concrete types for the latter part is going to be a bad time - the important thing is the protocol which is inherently polymorphic.#2020-11-0915:08Jim Newtonso how do I know whether I have an object which satisfies the spec protocol?#2020-11-0915:09Alex Miller (Clojure team)spec 1 does not have great answers for extracting information from either spec objects (which are opaque) or spec forms. One path to this is to write specs for spec forms and then use s/conform to "parse" the forms. A significant stab at this exists in https://clojure.atlassian.net/browse/CLJ-2112 but I think the future direction is really making a more data-centric representation which is one thing we are doing in spec 2 (but that's still very much a work in progress)#2020-11-0915:11Alex Miller (Clojure team)spec?
is the predicate for that#2020-11-0915:11Jim Newtonyes but spec? returns nil.#2020-11-0915:11Alex Miller (Clojure team)for what?#2020-11-0915:12Alex Miller (Clojure team)user=> (s/spec? (s/or :1 int? :2 number?))
#object[clojure.spec.alpha$or_spec_impl$reify__2118 0x51ec2df1 "
#2020-11-0915:13Jim Newtonaaaaaaaaahhhhhhhhh!!!!!! yikes. s/spec?
returns non-nil. That's EXCELLENT I have a local function named gs/spec?
which just asks whether it is a sequence whose first element is spec. spec?
was my first guess. just was shadowed by a local function. Cool. thanks.#2020-11-0915:13Alex Miller (Clojure team)yeah, your predicate is asking a symbolic spec form question#2020-11-0915:13Jim Newtonand I just typed spec? at the REPL.#2020-11-0915:13Jim NewtonREPL issue.#2020-11-0915:17Jim Newtonis there a way to undef
a function. Sometimes I rename a function while debugging, and the old function (of course) is still defined in vm, and if my code accidentally calls it because I didn't find and change all references, that introduces bugs which are hard to find.#2020-11-0915:26borkdude@jimka.issy You can use (s/fdef foo/bar nil)
#2020-11-0915:26borkdude(I assume, I know (s/def ::foo nil)
at least works)#2020-11-0915:29Alex Miller (Clojure team)yes, s/def to nil#2020-11-0915:29Jim Newtonwhat is s/ ?#2020-11-0915:29Alex Miller (Clojure team)clojure.spec.alpha#2020-11-0915:29Jim Newtonyou mean use spec to redefine a function?#2020-11-0915:29Alex Miller (Clojure team)are you asking about function specs or functions?#2020-11-0915:30Alex Miller (Clojure team)I think @borkdude and I assumed you meant specs since we're in the spec channel here#2020-11-0915:30Jim Newtonfuntions. Sorry if I mistyped before?#2020-11-0915:30Jim Newtonsorry, its probably my fault.#2020-11-0915:30Alex Miller (Clojure team)you can use ns-unmap
#2020-11-0915:30Jim Newton(def name nil)
works of course.#2020-11-0915:31Alex Miller (Clojure team)^^ that doesn't unmap, just redefines#2020-11-0915:31Jim Newtonahhh ns-unmap
. thats the cleaner way.#2020-11-0915:31borkdude@jimka.issy
user=> (def x 1)
#'user/x
user=> (ns-unmap *ns* 'x)
nil
user=> x
Syntax error compiling at (REPL:0:0).
Unable to resolve symbol: x in this context
user=> user/x
Syntax error compiling at (REPL:0:0).
No such var: user/x
#2020-11-0915:32borkdudeI recently discovered ns-unmap also works for imported classes. I need to fix a bug in sci which doesn't support that yet :/ ;)#2020-11-0915:32Jim NewtonI have several functions which are memoized. it speeds up my program 1000 fold. But when debugging, it can be a source of errors, because I forget that the function might not really be called.#2020-11-0915:32Alex Miller (Clojure team)the double-edged blade of memoization :)#2020-11-0915:33Jim Newtonindeed. So sometimes I really want to set-the-d*mn-function to undefined#2020-11-0915:33Jim Newtonto make sure it's not a problem of memoization#2020-11-0915:33borkdude@jimka.issy A solution to that problem might be to call the original function foo*
and the memoized one foo
#2020-11-0915:34Jim NewtonHere's the macro I'm using.
(defmacro defn-memoized
[[public-name internal-name] docstring & body]
(assert (string? docstring))
`(let []
(declare ~public-name) ;; so that the internal function can call the public function if necessary
(defn ~internal-name
This allows my to locally rebind the name such as
(binding [my-function (memoize -internal-name)]
...)
which I often do in test suits#2020-11-0915:35borkdudemakes sense#2020-11-0915:35borkdudefor tests you can also use with-redefs
which doesn't require your vars to be dynamic#2020-11-0915:36Jim Newtonnever heard of with-redefs. what's that.#2020-11-0915:36borkdude(doc with-redefs)
#2020-11-0915:38Jim Newtonindeed. so that just saves needing to declare them as dynamic? or does it do something else?#2020-11-0915:38borkdudedynamic bindings aren't visible to all threads, with-redefs
changes the root binding of vars temporarily#2020-11-0915:39Alex Miller (Clojure team)(note that there are issues using with-redefs with concurrency, including future etc)#2020-11-0915:40Jim Newton@U064X3EF3 are these the same issues with any dynamic variable?#2020-11-0915:40borkdudeyeah, when running tests concurrently this may bite you#2020-11-0915:41Jim NewtonAren't dynamic variables thread-local in clojure?#2020-11-0915:41borkdudeyes, but with-redefs doesn't use thread-local, it's just a global mutation that gets restored afterwards#2020-11-0915:42Jim Newtonah ha.#2020-11-0915:42borkdudePersonally I don't have any test suites that run into problems with this, but then again, I hardly use with-redefs at all#2020-11-0915:43borkdudeThere could be a performance penalty to marking all your vars dynamic, but not if they haven't been dynamically re-bound yet since there's a pretty efficient check for that happy path#2020-11-0916:00Alex Miller (Clojure team)we regularly see people run into failing test suites when using with-redefs#2020-11-0916:00Alex Miller (Clojure team)because they don't understand what it does, which is why I mention this#2020-11-0916:01borkdudeš#2020-11-0916:01Jim Newton:+1::skin-tone-2:#2020-11-0915:52Jim Newtonwhy doe s/describe return lists using and
and or
rather than clojure.spec.alpha/and
and clojure.spec.alpha/or
? that makes programmatic usage more difficult.#2020-11-0915:53borkdudes/describe
isn't intended for programmatic usage. use s/form
rather than that#2020-11-0915:54Jim NewtonExcellent. that does the trick. much more program friendly#2020-11-0916:01Alex Miller (Clojure team)s/describe is primarily useful for printing shorter specs for humans (used in doc
for example)#2020-11-0916:01Jim Newtonmakes sense.#2020-11-0920:48amorokhI just gotta ask, is there a way to create a shorthand alias for a namespace to use in fully-qualified keys without actually creating that namespace (using the ns
form)?#2020-11-0920:51borkdude@amorokh
I think this is a pretty common pattern:
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (alias 'foo (create-ns 'foo))
nil
user=> (s/def ::foo/bar int?)
:foo/bar
Rumor has it that core team is working on making this easier#2020-11-0920:53amorokh@borkdude thanks, not sure I want to do that but at least I know about that possibility#2020-11-1016:35Jim NewtonI'm getting an error from spec that I don't understand. Maybe someone can help me.
The following example, which I've based on an example in https://clojure.org/guides/spec, works fine, and valid?
returns true
.
(s/valid? (s/keys :req [::first-name ::last-name ::email]
:opt [::phone])
{::first-name "Bugs"
::last-name "Bunny"
::email "
However, when I've programmatically generated the following list
(let [sss '(clojure.spec.alpha/keys
:req [:clojure-rte.genus-spec-test/first-name
:clojure-rte.genus-spec-test/last-name
:clojure-rte.genus-spec-test/email]
:opt [:clojure-rte.genus-spec-test/phone])]
(s/valid? sss
{::first-name "Bugs"
::last-name "Bunny"
::email "
I get an uninformative exception
Execution error (ClassCastException) at (REPL:1).
null
is there a way to all s/valid?
with a programmatically generated spec?#2020-11-1016:37borkdude@jimka.issy An s-expression is not yet a spec. I would assume most people would write a macro for creating specs programmatically#2020-11-1016:37Alex Miller (Clojure team)Yes, but you need to understand more about how spec works#2020-11-1016:37Alex Miller (Clojure team)valid? works on spec objects#2020-11-1016:38Alex Miller (Clojure team)Spec forms need to be evaluated into spec objects first#2020-11-1016:38Alex Miller (Clojure team)In the case above, you could do that with eval#2020-11-1016:39borkdudeor generate the form with a macro#2020-11-1016:39Alex Miller (Clojure team)Even so, youāre still leaning on eval#2020-11-1016:40borkdudeyep#2020-11-1016:41borkdudeMade some fun macros the other day:
(def kws (map keyword (repeatedly gensym)))
(defmacro cat [& preds]
`(clojure.spec.alpha/cat
This allows you to write (g/cat int? string?)
, when you're not interested in the conformed value#2020-11-1016:43Alex Miller (Clojure team)Spec 2 has a lot more tools for all this stuff#2020-11-1016:43borkdudedefop
right?#2020-11-1016:43Alex Miller (Clojure team)Thatās one#2020-11-1016:43borkdudeany docs I could read yet?#2020-11-1016:43Alex Miller (Clojure team)The wiki pages if you havenāt seen those#2020-11-1016:44Alex Miller (Clojure team)But not everything is in there and itās missing a lot of the internal design #2020-11-1016:44borkdudeI'll just wait, no hurry#2020-11-1016:44Alex Miller (Clojure team)There is now an intermediate map data form now that you can work on directly#2020-11-1016:45Alex Miller (Clojure team)But will almost certainly change#2020-11-1016:45borkdudeWorking with maps is a lot easier than writing macros (since macros beget more macros)#2020-11-1016:46Alex Miller (Clojure team)Some day rich or I will do a talk about the internals - there are several interesting aspects of it#2020-11-1016:47borkdudeLooking forward#2020-11-1016:52Jim Newtonmacros wont help in this case. the origin of my spec is not sitting in an evaluation position.#2020-11-1017:04Alex Miller (Clojure team)thus, eval#2020-11-1017:04Alex Miller (Clojure team)evaluating a spec form gives you a spec object#2020-11-1117:10Jakub HolĆ½ (HolyJak)What is the state of / plan for Spec 2? Or is there I can check for the answers? š#2020-11-1117:23seancorfield@holyjak It's still in the design phase -- Rich is rethinking the whole function spec thing, as far as I can tell from comments Alex has made.#2020-11-1117:23seancorfieldSo it's still very alpha and subject to (a lot of) change.#2020-11-1117:24Jakub HolĆ½ (HolyJak)thank you!#2020-11-1117:24seancorfieldI'm chomping at the bit to use it -- for a while I had a branch of our code at work tracking Spec 2 but it just had too much churn and too many bugs to keep up with it, so I abandoned that branch after several months.#2020-11-1117:24seancorfieldIt definitely has some nice usability improvements in it, and the whole schema
/`select` thing is much better than keys
#2020-11-1118:20Alex Miller (Clojure team)sorry, don't have any concrete plan#2020-11-1118:21Alex Miller (Clojure team)it continues to get slices of attention is about all I can tell you :)#2020-11-1118:42borkdudeWithout encouraging more delay, I must say I kind of envy the patience that Rich has before releasing anything ;)#2020-11-1220:39Ronny LiHi, does anyone have advice for how to read the following error message from plumatic/schema? My understanding is that the arity doesn't conform to spec but where did the error occur? (`(demo/data.clj:77:1)` is simply where I defined the function)
Error refreshing environment: Syntax error macroexpanding clojure.core/defn at (demo/data.clj:77:1).
Call to clojure.core/defn did not conform to spec. clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec. #:clojure.spec.alpha{:problems ({:path [:fn-tail :arity-1 :params], :pred clojure.core/vector?, :val :-, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/params+body :clojure.core.specs.alpha/param-list :clojure.core.specs.alpha/param-list], :in [1]} {:path [:fn-tail :arity-n :bodies], :pred (clojure.core/fn [%] (clojure.core/or (clojure.core/nil? %) (clojure.core/sequential? %))), :val :-, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/params+body :clojure.core.specs.alpha/params+body], :in [1]}), :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x406684fa "
#2020-11-1220:42borkdudeMaybe you could post the function itself#2020-11-1220:53seancorfieldSounds like a defn
that has :-
instead of an argument vector. I'm guessing you used defn
(from core) with Schema syntax instead of Schema's own variant of defn
? @ronny463#2020-11-1220:54Ronny Lithat's exactly what happened, thank you š#2020-11-1220:54seancorfield(based on :pred clojure.core/vector?, :val :-
in the first line of the exception)#2020-11-1316:15borkdudeWhat is the reason spec chose a custom deftype LazyVar
over the built-in delay
in its implementation?#2020-11-1316:17Alex Miller (Clojure team)for which?#2020-11-1316:17borkdudeI'm trying to come up with something that would work in Clojure for this case:
(def input (LazyVar. (fn [] (line-seq (java.io.BufferedReader. (java.io.StringReader. "1\n2\n3\n")))) nil))
such that you don't have to write @input
but just input
so the first time it's derefed by Clojure it starts its realization. Just a naked line-seq
doesn't work, since that starts reading once it's def-ed.#2020-11-1316:17borkdude@alexmiller in the gen namespace#2020-11-1316:19borkdudeI think in Clojure I would have to write a subclass of clojure.lang.Var
to make that work.#2020-11-1316:19Alex Miller (Clojure team)sorry, I don't actually see what you're talking about#2020-11-1316:20borkdudeAnd maybe I should just go with @input
and not do any magic... This lead to the question: why is gen/LazyVar not just a delay but a custom data structure.
Ah right.. sorry, the LazyVar
is something in CLJS, not CLJ. facepalm#2020-11-1316:21borkdudehttps://github.com/clojure/clojurescript/blob/5e88d3383e0f950c4de410d3d6ee11769f3714f4/src/main/cljs/cljs/spec/gen/alpha.cljc#L15#2020-11-1316:22borkdudeI'll ask over there in #clojurescript#2020-11-1316:23Alex Miller (Clojure team)ah#2020-11-1318:39socksyI am confused about the syntax for (or)
and (and)
in s/keys
. My understanding of it was that anything that is passed to or
could be correct, and anything that passed into and
has to be present. Did I misunderstand? My intuition does not work for the following example:
(s/def :foo/foo #{:foo})
(s/def :bar/foo #{:bar})
(s/def ::an-int int?)
(s/def ::baz (s/keys :req-un [(or :foo/foo
(and :bar/foo
::an-int))]))
(s/valid? ::baz {:foo :foo})
;; => false
(s/valid? ::baz {:foo :bar})
;; => true
#2020-11-1318:51socksyok it looks like when you have the same naked keyword (not sure of the right terminology, keyword without the NS) then it will always take the last one defined. That's a huge bummer, since I wanted to be able to spec something like "In this case, do this, in this other case, do this + another piece of data", and the way that was being specced before was by storing everything as a tuple#2020-11-1318:52borkdude@socksy can't you use s/or
for either case?#2020-11-1318:52borkdudemore verbose#2020-11-1318:53seancorfield@socksy You'll need to wrap s/keys
with s/and
and add your rules via a predicate -- or use s/or
around s/keys
as @borkdude suggests.#2020-11-1318:53socksyyes I think it might be possible. I'll go hit my head on it#2020-11-1318:53socksyi am thinking my own predicate will probably be the most readable but let's see#2020-11-1318:54borkdudemaybe spec2 has a better answer to this... although I'm not sure if s/select
can solve this case#2020-11-1318:54seancorfieldAnother option would be a multi-spec I think?#2020-11-1318:57socksy(s/def ::baz (s/or :foo-case (s/keys :req-un [:foo/foo])
:bar-case (s/keys :req-un [:bar/foo ::an-int])))
is not too bad actually#2020-11-1318:59borkdudeNow I wonder how you would use this in spec2. I think you would define ::baz
as the thing with all possible keys and then use s/select
for either case?#2020-11-1319:02borkdude(s/def ::baz (s/schema [:foo/foo :bar/foo ::an-int]))
(s/def ::foo-case (s/select [:foo/foo]))
(s/def ::boo-case (s/select [:bar/foo ::an-int]))
Something like this?#2020-11-1408:25Rob Hanlonhey all, iām trying to write a spec for a map using s/keys
that may contain keys that are namespaced or unnamespaced:
(s/keys :opt [::foo ::bar ::baz] :opt-un [:foo ::bar ::baz])
alright, thatās all well and good! but now iād like to reduce the duplication (in my real-life application, the vectors have ~10 items each)
(def ks [::foo ::bar ::baz])
(s/keys :opt ks :opt-un ks)
this doesnāt work, because ks
is a symbol within the keys
macro. iāve tried unquoting:
(def ks [::foo ::bar ::baz])
(s/keys :opt ~ks :opt-un ~ks)
but this doesnāt work eitherāit fails the namespace-qualification assertion within the keys
macro. any pointers on how to reuse these keys? thanks š#2020-11-1409:01borkdudeThat doesn't work indeed. You will need to write macros to accomplish this because s/keys
itself is a macro#2020-11-1414:53dvingo@robhanlon I came up with this helper for composing keys:
(defmacro def-domain-map
"Return s/keys :req for fields supports passing in symbols for keys"
([spec required]
(let [req (eval required)]
`(s/def ~spec (s/keys :req ~req))))
([spec required opt]
(let [req (eval required)
opt (eval opt)
global-opt (eval global-keys)]
`(s/def ~spec (s/keys :req ~req :opt ~(into opt global-opt))))))
It works from clj and cljs:
(>def :work-log/id fuc/id?)
(>def :work-log/description string?)
(>def :work-log/begin tu/date-time?)
(>def :work-log/end tu/date-time?)
(def required-work-log-keys
[:work-log/id :work-log/description :work-log/begin :work-log/end])
(def optional-work-log-keys [])
(su/def-domain-map ::work-log required-work-log-keys optional-work-log-keys)
#2020-11-1414:55dvingoI wanted the required and optional keys to be reused in different parts of the app without needing to maintain two lists of these keys#2020-11-1415:29Alex Miller (Clojure team)at some point you have to ask yourself - is this easier than just saying the keys twice?#2020-11-1420:04Rob HanlonI think I have my answer hereāIāll just repeat the keys. Thank you š #2020-11-1815:11Dmytro BuninI stumbled upon this behavior, which I found a bit weird.
(s/def :sequence-list/eid number?)
(s/valid?
(s/keys :opt []) ;; ā no reference to :sequence-list/eid
(merge #:entity {:label "foo"}
#:sequence-list {:eid "sample"}))
=> false
does the s/keys
look at global registry in some way?#2020-11-1815:11Alex Miller (Clojure team)yes#2020-11-1815:11Alex Miller (Clojure team)s/keys will check all registered keys in the map#2020-11-1815:12Alex Miller (Clojure team)so even (s/keys)
is a spec that does things#2020-11-1815:13Dmytro Bunininteresting, thanks š#2020-11-1815:13Alex Miller (Clojure team)this is covered if you (doc s/keys)
or look at the spec guide https://clojure.org/guides/spec#2020-11-1815:13Dmytro Bunin>
> In addition, the values of all namespace-qualified keys will be validated
> (and possibly destructured) by any registered specs. Note: there is
> no support for inline value specification, by design.#2020-11-1815:13Alex Miller (Clojure team)^^#2020-11-1815:13Dmytro Buninyeah says so in docstring š#2020-11-1906:52lassemaattaA question concerning multi-spec
: Suppose I have a spec similar to the one given in the clojure spec guide (ie. :event/event
), where different events contain a :event/type
key. How can I generate sample events of a particular type, for example :event/error
events? Or refer to the spec of a particular event somewhere (e.g. in a s/fdef
). What I've tried so far: If I replace the spec from the corresponding defmethod
call into a s/def
definition, I can obtain the generator for it and I get all the fields I expect but the :event/type
key will be a random keyword. Another hack was to generate events from :event/event
and then use s/and
+ predicate to only accepts events which have the correct type, but this "lacks elegance" š#2020-11-1907:01lassemaattaOne might argue that it reflects poor design if one has to refer to particular type like this, but still it might be nice to know if & how this is possible :thinking_face:#2020-11-1914:23Alex Miller (Clojure team)I think something like what you describe is how most people are approaching it right now#2020-11-2019:53neilyioCould anyone give me a hand with this error?
Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required
Ā happens every time I callĀ `(spec/explain ::my-spec my-val)`Ā in ClojureScript. I don't have instrumentation turned on.
I've followed the advice atĀ https://stackoverflow.com/questions/57877004/how-to-fix-clojure-test-check-generators-never-required-when-exercising-a-funcĀ and addedĀ `org.clojure/test.check`Ā to myĀ `deps.edn`Ā dependencies, but it hasn't fixed the problem.#2020-11-2019:55borkdudewhen you see this message, in CLJS you should require the ns yourself.#2020-11-2019:55borkdudebut it surprises me that s/explain needs test.check, that should not be the case I think#2020-11-2019:58borkdude@neil.hansen.31 which version of CLJS are you using?
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "22404d504946574647626f60721012131b"}, :content ("[emailĀ protected]")}
#2020-11-2020:45neilyio@borkdude I'm on 1.10.773. I'm trying it on an empty REPL now and it's working as your example is... I guess something is going on in my Figwheel build that I'll have to figure out...#2020-11-2020:55neilyioActually, this is happening only on a specific spec.#2020-11-2020:55neilyioWill debug.#2020-11-2020:56borkdudeMaybe you are using with-gen in this spec?#2020-11-2021:25kennyCould potentially be an fspec#2020-11-2021:49neilyioYes, it's definitely something to do with fspec. It also seems to act strange in this case:
(spec/fdef foo
:args (spec/fspec :args (spec/cat)))
#2020-11-2021:50kennyFspec uses gen testing during explain. #2020-11-2021:52neilyioWhen I instrument these functions:
(defn foo []
nil)
(defn bar []
nil)
I get a "spec failed... should satisfy ifn?" expound error with this:
(foo bar)
#2020-11-2021:52neilyioI would expect that to Success!
. Am I misunderstanding something?#2020-11-2021:55borkdude@neil.hansen.31 Why are you using fspec
here?#2020-11-2021:56borkdudeI would have expected:
(spec/fdef foo
:args (spec/cat))
#2020-11-2021:57neilyioYup, I'm starting to realize I'm doing something seriously wrong here š#2020-11-2021:59neilyioI made an assumption from the https://clojure.org/guides/spec#_higher_order_functions that fspec
would work for :args
, but it seems like it's only for :ret
. I was hoping that spec could check that the arguments to my function would receive only certain arguments.#2020-11-2022:00borkdudespec can check that. it does when you instrument the function#2020-11-2022:01neilyioOh, also my example for foo was wrong up above, sorry, it should have been something like:
(defn foo [f]
(f))
(defn bar []
nil)
(foo bar)
#2020-11-2022:02borkdudeThen your spec may have been correct, and the error message makes sense. Foo wasn't passed an IFn#2020-11-2022:03borkdudeI think it should be something like:
(spec/fdef foo
:args (spec/cat :f (spec/fspec :args (spec/cat))))
#2020-11-2022:05borkdudeIn Clojure:
user=> (require '[clojure.spec.alpha :as spec])
nil
user=> (spec/fdef foo
#_=> :args (spec/cat :f (spec/fspec :args (spec/cat))))
user/foo
user=> (defn foo [f] (f))
#'user/foo
user=> (stest/instrument 'user/foo)
[user/foo]
user=> (foo identity)
Execution error (FileNotFoundException) at user/eval35926 (form-init3141792093562271177.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
user=> (foo (fn []))
Execution error (FileNotFoundException) at user/eval35926 (form-init3141792093562271177.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
#2020-11-2022:06neilyioOh my goodness, you're right, thank you so much. The :args
always have to bee a tuple don't they.#2020-11-2022:06borkdudea sequential#2020-11-2022:06borkdudebtw, you can also check for a function using fn?
or ifn?
.#2020-11-2022:06neilyioGot it. Funny enough, when I instrument that spec you gave me, I get the error again:
Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required
#2020-11-2022:06borkdude:args (s/cat :f ifn?)
#2020-11-2022:07borkdudeyeah, it seems fspec somehow needs test.check. I've never used fspec personally. It seems to be the same in CLJ:
user=> (foo (fn []))
Execution error (FileNotFoundException) at user/eval35926 (form-init3141792093562271177.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
#2020-11-2022:09neilyioMy issue is that I've installed test.check, I've even required it in my namespace.#2020-11-2022:10neilyioSo not sure what's happening... For now I'll take your advice and check for ifn?
in my :args
instead.#2020-11-2022:10borkdudewhat does your ns form look like?#2020-11-2022:29neilyioHmm it looks like this is a Figwheel-specific problem. I've made a very minimal test file:
(ns cards.core
(:require
[clojure.test.check :as check]
[clojure.spec.test.alpha :as stest]
[clojure.spec.alpha :as spec]))
(spec/fdef foo
:args (spec/cat :f (spec/fspec
:args (spec/cat))))
(defn foo [f] f)
(defn bar [] nil)
(stest/instrument)
(foo bar)
#2020-11-2022:30neilyioEvaluating the final (foo bar)
with clj
returns the function object.#2020-11-2022:31neilyioEvaluating (foo bar)
with clj -m figwheel.main
returns #object[Error Error: Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required]
#2020-11-2022:31neilyioSlightly different error from my original one, but I'd think the problem comes from the same place?#2020-11-2022:32borkdudeYou can try to compile this without figwheel and see if that's it. And then file a bug report with figwheel#2020-11-2022:32neilyioYup, I think it's bug report time.#2020-11-2022:33neilyioThanks for talking me through this @borkdude#2020-11-2022:33borkdude:thumbsup:#2020-11-2022:44neilyiohttps://github.com/bhauman/figwheel-main/issues/272 for all those who are lost in the future.#2020-11-2022:51borkdudeHave you also tried to compile this without Figwheel, but using regular CLJS? That's not clear from this issue#2020-11-2022:55neilyioOh geez, you're right, I get the same error running with clj -m cljs.main
.#2020-11-2022:56neilyioI'm just so used to starting cljs with figwheel. Thanks for following up on that.#2020-11-2022:56neilyioI'll close the Figwheel issue, what do you think I should do with this?#2020-11-2023:00borkdude#clojurescript#2020-11-2023:00borkdudeyou could link to the figwheel issue to explain your issue#2020-11-2023:00neilyioI'll do that, I'll close the figwheel issue when someone advises. Thanks again for everything.#2020-11-2023:05borkdudeI think it would be good to close the figwheel issue and comment that it's not an issue with figwheel, else you might be wasting the maintainer's time.#2020-11-2023:06borkdudeHave you tried requiring cljs.test.check
?#2020-11-2023:06borkdudeinstead of clojure.test.check
#2020-11-2023:07borkdudeneh, that's not it#2020-11-2023:15neilyioClosed it up, posted in #clojurescript. Will buy Bruce Hauman beer someday, will give you full credit for helping me recover from this.#2020-11-2023:15neilyioBeer coming your way too.#2020-11-2023:15borkdudeSounds good ;) Have a nice weekend! š» I'm off, getting late over here in Europe#2020-11-2206:30tianshuIs this the expected behavior?
(s/def ::entry-as (s/tuple #{:as} symbol?))
(s/def ::entry (s/or :the-as ::entry-as))
(s/def ::map (s/coll-of ::entry :kind map?))
(s/conform ::map '{:as e});; => {:as [:as e]}
I was thought it should return something like {:the-as [:as e]}
, because the branch in s/or
is tagged as :the-as
.
spec version 0.2.187
.#2020-11-2207:56Alex Miller (Clojure team)By default, maps are nonconforming on keys, that logic may be modifying what you expect here. You could try adding :conform-keys true
to the coll spec and see if that helps#2020-11-2210:56tianshuYes, set :conform-keys true
will work. Thank you!#2020-11-2215:24mishahttps://akovantsev.github.io/corpus/clojure-slack/clojure-spec
filterable/searchable log for this channel on a single static/offline page
1. double click on any word to filter,
2. cmd+f to highlight matches with browser's search,
3. click any date/time to show filtered out nearby lines, for context
4. recur
data extracted from https://clojurians-log.clojureverse.org/clojure-spec pages#2020-11-2215:33mishalittle teaser:#2020-11-2215:33misha#2020-11-2216:56Alex Miller (Clojure team)@misha can you stop spamming all the channels with this and just put one in announcements or whatever?#2020-11-2220:17mishasure#2020-11-2220:30seancorfield@misha Posting the clojure-slack level link to #news-and-articles would be fine. Not to #announcements since that is for Clojure code projects/libraries. But as Alex said, posting these links individually to channels as you add indexed versions is really annoying as we already have two existing searchable archives of nearly all channels here.#2020-11-2308:28mishaThank you, Sean, if I add more - Iāll announce in news channel, which I didnāt know about. I am sorry to hear 3 specific links are perceived as spam and annoying, and answering the same questions few times a week-is not.
Especially after I actually tried finding things in other āsearchable logā, and decided that week of dev time investment is worth it instead, and is useful to others.#2020-11-2313:32Drew VerleeIs it possible to extract the argument list e.g (s/cat :x int?) from an existing spec?#2020-11-2315:04sgepigonuser=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/form (:args (s/fspec :args (s/cat :x int?))))
(clojure.spec.alpha/cat :x clojure.core/int?)
;; or if you use `s/fdef`:
user=> (s/fdef foo :args (s/cat :x int?))
user/foo
user=> (s/form (:args (s/get-spec `foo)))
(clojure.spec.alpha/cat :x clojure.core/int?)
This also works for :ret
and :fn
as fspecs implement clojure.lang.ILookup
.#2020-11-2313:33Drew Verlee(s/arg foo)
;;=> [{:x int?}]#2020-11-2313:50borkdude@drewverlee I'm not entirely clear on what your code should do, but maybe s/form
is something that's useful for you?#2020-11-2315:04sgepigonuser=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/form (:args (s/fspec :args (s/cat :x int?))))
(clojure.spec.alpha/cat :x clojure.core/int?)
;; or if you use `s/fdef`:
user=> (s/fdef foo :args (s/cat :x int?))
user/foo
user=> (s/form (:args (s/get-spec `foo)))
(clojure.spec.alpha/cat :x clojure.core/int?)
This also works for :ret
and :fn
as fspecs implement clojure.lang.ILookup
.#2020-11-2320:07Drew Verleethanks @borkdude and @sgepigon that answers the question for me.#2020-12-0119:48Kira McLeanWhatās the best to way to list the keys that a map is missing? E.g. with a problem like this for a specced map, what do you guys do to collect those key-name
s? Iām looking at using spec to validate data in a web app so ultimately Iād like to translate the missing keys into a message like ā<key-name> is requiredā but Iām not sure whatās the best way to get a hold of the names of the missing keys.
{:path [],
:pred (clojure.core/fn [%] (clojure.core/contains? % :key-name)),
:val
{},
:via [,,,],
:in []}
#2020-12-0119:56Kira McLeanI was looking through expound to try to see how they do it.. it looks like the core of it is https://github.com/bhb/expound/blob/1c0d78570be3865eab8e69c1b568c4e7acee5bd8/src/expound/printer.cljc#L187-L194, where form
is a :pred
https://github.com/bhb/expound/blob/1c0d78570be3865eab8e69c1b568c4e7acee5bd8/src/expound/alpha.cljc#L317.. but I donāt understand how that :expound.spec/contains-key-pred
works..#2020-12-0119:59Kira McLeanI donāt see how it returns the name of the failed key.. when I try something similar it just returns ::s/invalid
, which seems like what I would expect.#2020-12-0120:31bbrinck@UPGS9BS0L Expound is just parsing the s-expression of the pred. The trick is that expound will take the pred e.g. something like
(clojure.core/fn [%] (clojure.core/contains? % :expound.printer/foo))
And then it will throw away the clojure.core/fn [%]
part with the (nth form 2)
on line 188#2020-12-0120:31bbrinckThen it uses spec to parse the s-expression to pull out the keyword of the spec#2020-12-0120:32bbrinckHereās a little script to show each step. Itās just an example to show how Expound figures out each part of the :pred
(let [first-problem (first (::s/problems (s/explain-data ::some-spec {})))]
{
:pred (:pred first-problem)
:part-of-pred (nth (:pred first-problem) 2)
:match (s/conform :expound.spec/contains-key-pred (nth (:pred first-problem) 2))
}
)
;; This returns:
#_ {:pred (clojure.core/fn [%] (clojure.core/contains? % :expound.printer/foo)),
:part-of-pred (clojure.core/contains? % :expound.printer/foo),
:match [:simple {:contains clojure.core/contains?, :arg %, :kw :expound.printer/foo}]}
#2020-12-0120:33bbrinckDoes that help?#2020-12-0120:45Kira McLeanah cool, yeah that does make sense. Thank you! So is that how youād recommend going about getting a hold of those keys? Is there a simpler way to just get spec to tell me which req
keys are missing?#2020-12-0120:49bbrinck@UPGS9BS0L Unfortunately, as of spec1, thatās the way I know of to get the missing keys. Things may change in spec2, but I donāt know of any specific plans for this.#2020-12-0120:52bbrinckItād be really nice if spec2 included both the missing key and the fully-qualified key in the case of keys included via :req-un
.#2020-12-0120:53bbrinck(right now if you use :req-un
, sometimes Expound will warn <can't find spec for unqualified spec identifier>
because it doesnāt know what spec you meant by the kw :foo
)#2020-12-0120:55bbrinck@UPGS9BS0L This is apparently how Phrase works as well https://cljdoc.org/d/phrase/phrase/0.3-alpha4/doc/examples#required-keys#2020-12-0121:52Kira McLeanThis is really helpful, then. Thanks for all the info!#2020-12-0217:30dogenpunkIs anyone here aware of a spec -> xml printer? That is, a library that takes explain-data and outputs an xml report?#2020-12-0217:36Alex Miller (Clojure team)I'm not aware of such a thing#2020-12-0400:07Sean HIs there a good way to have multiple generators for clojure.spec.test.alpha
& clojure.spec.gen.alpha
on the same spec on a namespaced keyword? In particular, in cases where we know that a spec is restricted in certain contexts, such as Datomic refs (which when sent TO datomic are either a 2-tuple, an int, or a map [component], while when received FROM datomic are an int or a map).
If not, my guess would be the most sensible way to do this is to just override the specās generator with a generator that generates the more restrictive form (e.g. int/map in the above example, or just int for non-component refs) and then do generation some other way (or not at all) for cases where you need the more general form?
Thanks!
EDIT: I think Iāve figured it out. For anyone whoās curious, this can be done by clojure.spec.test.alpha/instrument
options#2020-12-0417:51Thomas MoermanQ: consider following snippet:
(s/valid? (s/keys :req [:nexus.annotation/id]) {:nexus.annotation/id 1
:nexus.annotation/annotation-set {:invalid :map}}) => false
#2020-12-0417:51Thomas Moerman1. Why does spec take the second (invalid) key into account?
2. Is there a way to make spec ignore the second key#2020-12-0417:59Thomas MoermanApparently this is by design:#2020-12-0417:59Thomas Moerman>
> In addition, the values of all namespace-qualified keys will be validated
> (and possibly destructured) by any registered specs. Note: there is
> no support for inline value specification, by design.#2020-12-0418:11Alex Miller (Clojure team)yes, it is by design#2020-12-0418:11Alex Miller (Clojure team)attributes have global meaning#2020-12-0716:41Aleh AtsmanHello!
Is there a way to get list of keys from (s/keys...) ? And if I had few specs merged can I get back a full list of keys?
If not, how do you do it?
Thx#2020-12-0814:12mishahttps://akovantsev.github.io/corpus/clojure-slack/clojure-spec#t1487996119012135#2020-12-0814:25mishahttps://akovantsev.github.io/corpus/clojure-slack/clojure-spec#t1480167464011304#2020-12-0915:20Aleh Atsmanexplain data approach - maybe
but clojure.spec/form - doesn't work for merged spec
thanks for links#2020-12-0915:20Aleh Atsmanboth ways feels more hacky than just abstracting clojure.spec away and managing things by hands#2020-12-0915:33mishaFor merged specs probably need to walk it doing s/form and s/registry. Walking datastructures aināt hacky in a Lisp :op:#2020-12-0915:35mishaThey say spec2 is gonna be easier in this regard, fwiw#2020-12-0717:37Aleh AtsmanWell, I am simple guy and solved my problem with first creating entities as plain maps, later wrapping decision on what fields to use in 2 level multimethods, only from there creating spec, and returning it together with fields š#2020-12-0908:26Ricardo CabralHello all, I hoe you are doing good and safe. I am a newbie in Clojure and I am trying to learn about spec. I am using spec-alpha2 and I am facing an issue trying to generate a contact as you can see in the code below. The error is Couldnāt satisfy such-that predicate after 100 tries. when I try to generate a sample of an email using regex. Is it possible to do like this?
(def email-regex #"^[a-zA-Z0-9._%+-]
#2020-12-0908:34lassemaattaI'm no spec expert, but my understanding is that s/and
uses the first spec (in this case string?
) to generate values and then checks that the value matches the rest of the conditions (in this case the regex). Because string?
generates random strings, it is unlikely that it matches the very specific email regex. After generating several random strings, spec gives up as it cannot create a random string which satisfies the email regex.#2020-12-0908:39lassemaattapossible solutions: a) use a constant set of sample email addresses and use s/with-gen
to pick one of the samples, b) read about test.check and the tools it provides to build custom generators and try to build an email address generator by hand, c) check out libraries like lambdaisland/regal, which provide facilities to create generators for regex#2020-12-0909:01Ricardo CabralThank you @UC506CB5W for your quick answer. I will check that#2020-12-0914:38Alex Miller (Clojure team)You may have some trouble with this as spec2 is a work in progress and with-gen in particular may have some issues. I would recommend using spec 1 for now#2020-12-0917:49Ricardo CabralThank you @U064X3EF3 so far I am not facing any big issue. I am just trying some small experiments to understand how spec2 works and also learn about the idea behind it, and so far it is working as expected#2020-12-0917:49Ricardo Cabral(def email-regex #"^[a-zA-Z0-9._%+-]
#2020-12-1012:58konrad szydloHi, I'm trying to figure out how to implement xor for a map keys. Where I'd like to check if one of the keys is present but not both at the same time.
(s/def ::a int?)
(s/def ::b int?)
(s/def ::c string?)
Where following maps are invalid:
{::c "string"} ;; missing ::a
xor ::b
{::a 1 ::b 2 ::c "string"} ;; both ::a
and ::b present
and the following maps are valid:
{::a 1 ::c "string"}
{::b 2 ::c "string"}#2020-12-1014:12englishs/keys
only supports regular or
: https://stackoverflow.com/a/41901585
but you could combine the s/keys
spec with a custom xor
spec via s/and
, as described in https://stackoverflow.com/a/43374087#2020-12-1014:45konrad szydloThank you. My searching on duck duck go didn't return this answer from SO. That's what I was looking for.
In the docs for s/keys
I saw that or
is supported but not xor
#2020-12-1014:12vlaaad(s/and (s/keys ...) #(= 2 (count (select-keys % [::a ::b ::c]))))
?#2020-12-1622:36kennyCurious if there are any ideas around how spec2's select would allow for requiring deeply nested keys based on some dispatch function? e.g., (if the event is type "a" then a specific set of keys are required, if event type is "b" then ...)#2020-12-1623:47dgb23@kenny if I understand correctly select is there to be used in specific contexts. Not sure if that answers your question.#2020-12-1623:49kennyNo. Essentially I'm curious what the story is wrt deeply nested multi-spec and select since that is a common use case.#2020-12-1700:13Alex Miller (Clojure team)Can you explain?#2020-12-1700:33kennyPerhaps an example is best.
(s/def ::order (s/schema [:order/id
:order/type
:order/a
:order/b
:order/c]))
(s/def :user/orders (s/coll-of ::order))
(s/def ::user (s/schema [:user/orders]))
(s/select ::user [:user/orders {:user/orders [:order/id
:order/type
;; if :order/type is :order.type/a then require :order/a
]}])
The required order keys depend on the order's type.#2020-12-1700:33kennyI only require :order/a when :order/type is :order.type/a.#2020-12-1700:38dgb23So it is a separate schema for order?#2020-12-1700:39kennyWhat do you mean?#2020-12-1700:40dgb23Iām curious (never used spec2) as well. My intuition is that you are trying to solve something with select when it should be a separate schema for order types. But I have to think about it/read a bit more š#2020-12-1700:41kennySorry, I don't understand what you mean by separate schema. Could you clarify?#2020-12-1700:42dgb23using spec/or to distinguish the order types#2020-12-1700:42dgb23and each order type has a separate spec that holds the context of order/a order/b etc#2020-12-1700:42kennyOh. Yes, that is possible here. It results in really bad error messages though (if you have a bad input, every single case of the or is outputted). multi-specs output a really nice error.#2020-12-1700:43kennyIn our case there may be hundreds of different "types". The explain-data is crazy šµ#2020-12-1700:43dgb23i see#2020-12-1700:44dgb23have you tried to parse explain-data to kind of narrow it down? just talking off my ass here#2020-12-1700:46kennyYou can manipulate the output however you normally would š The thing is, multi-specs solve this problem nicely (albeit a bit cumbersome to work with). They just don't work with select, and it seems that there is a good fit for something there.#2020-12-1700:47dgb23ah I see it now!#2020-12-1700:49kennySo I was curious if there were any thoughts on how/if something like this would be a part of spec2. For us, this use case happens everywhere.#2020-12-1700:55dgb23since select is just a spec too, then you would dispatch with a multimethod too right?
so it would be something like (defmulti order-type :order/type...
#2020-12-1700:55dgb23each returning a select
#2020-12-1700:55dgb23that is specifically tailored to query a variant#2020-12-1700:56kennyNot sure I follow how that'd work with the above.#2020-12-1700:56dgb23me neither š#2020-12-1700:56kennyThe main thing is that you always declare required keys "top-level." Pretty sure what you have there wouldn't work :thinking_face:#2020-12-1700:57dgb23got me hooked Iām going to play a bit with spec2 now#2020-12-1700:57dgb23what do you mean with required keys?#2020-12-1700:57dgb23schema keys are not required#2020-12-1700:59kennyNo matter the "level" (read nesting) you're at, you're always able to select which keys are required.#2020-12-1701:53dgb23(s/def ::order (s/schema [::order-id
::order-type
::order-a
::order-b
::order-c]))
(s/def ::orders (s/coll-of ::order))
(defmulti order-type ::order-type)
(defmethod order-type ::order.type-a [_]
(s/select ::order [::order-a]))
(defmethod order-type ::order.type-b [_]
(s/select ::order [::order-b]))
(s/def ::order-typed (s/multi-spec order-type ::order-type))
(s/valid? ::order-typed {::order-type ::order.type-b ::order-b "foo"})
(s/def ::orders-typed (s/coll-of ::order-typed))
(s/valid? ::orders-typed [{::order-type ::order.type-b ::order-b "foo"}])
#2020-12-1701:54kennyYou've changed the problem š#2020-12-1701:54dgb23I get the feeling that I didnāt understand it in the first place#2020-12-1701:54dgb23š#2020-12-1701:55kennyAt the level of the "user" map, I want to select what keys are required in each map under :user/orders.#2020-12-1701:55kennyWithout renaming my :user/orders key.#2020-12-1702:00dgb23i see but those are different specs#2020-12-1702:00kennySame schema, different selected keys.#2020-12-1702:01dgb23the schema doesnāt know about select (just tried it)#2020-12-1702:02dgb23i have to leave it at that (itās late) but was fun so far. maybe someone actually knowledgeable can find a better way#2020-12-1702:03kennyNot sure what you mean exactly š This is what I'd like.
(s/select ::user [:user/orders {:user/orders [:order/id
:order/type
;; if :order/type is :order.type/a then require :order/a
]}])
#2020-12-1702:06dgb23What I mean is that Iām still a beginner with these things. I donāt really see a solution right now other than dispatching the select and conform/validate on the select spec.#2020-12-1702:07kennyOh, I understand. Yes, I was 99% certain spec2 does not cover this. I am interested in thoughts on this particular problem and how spec2 might solve it.#2020-12-1702:08dgb23i mean this sound pretty powerful (from the wiki): https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#creating-specs-programmatically#2020-12-1702:10kennyFor sure! I'm confident something could be built to solve this (likely even in spec1). Curious if this use-case is in scope for spec2 though. It seems like a core problem.#2020-12-1714:46Alex Miller (Clojure team)Two things here - first selects do not currently have the ability to handle maps of colls of maps of ... , only nested maps. That is something we are thinking about though as it is very common. Second, there is not currently any way to indicate any sort of predicate or condition on the nested data like you're talking about but that is something we're thinking about. What you're asking for might be beyond where we end up though, not sure yet.#2020-12-1714:46Alex Miller (Clojure team)select is currently only about nested map structures (what keys are provided at each level)#2020-12-1722:24kenny> there is not currently any way to indicate any sort of predicate or condition on the nested data like you're talking about but that is something we're thinking about.
This sounds exactly like what I'm after š This is such a big issue for us I may need to tackle it internally for now. Would love to know what those thoughts are. Though, I assume they are not public yet?#2020-12-1722:59Alex Miller (Clojure team)sorry, not working on this right now#2020-12-1814:20Shantanu KumarDoes anybody know of a tool/library that can derive spec based validators from OpenAPI schemas? I looked at spec-tools https://github.com/metosin/spec-tools/blob/master/docs/06_openapi.md but it only appears to cover the generation (reverse) part.#2020-12-1915:53misha@kumarshantanu not sure if openapi and json-schema are same, but have a look: https://github.com/akovantsev/json-schema-to-clojure-spec#examples#2020-12-2019:27Shantanu KumarThanks @U051HUZLD, as of v3.0.3 OpenAPI has some overlap with JSON schema https://spec.openapis.org/oas/v3.0.3#properties but does not appear to be fully compatible.#2020-12-1915:55misha*Alpha quality and all that.#2020-12-2118:08Michael Stokley(do
(defn f [x] x)
(s/fdef f :ret int?)
(stest/instrument)
(f "hello"))
i would expect this to throw, but i don't see that happening. what am i missing?#2020-12-2118:14valeraukoare you doing this in the repl? I've had surprising behavior wrapping defn and similar in do/let. Try executing line-by-line?#2020-12-2118:14seancorfieldinstrument
is for argument checking, not return value checking.#2020-12-2118:15seancorfieldinstrument
checks that your function is being called correctly by other code. check
is what you use to generatively test the behavior of the function itself.#2020-12-2118:16seancorfieldSee https://clojure.org/guides/spec#_instrumentation_and_testing @U7EFFJG73#2020-12-2118:37Michael Stokley@U04V70XH6, thank you!#2020-12-2118:37Michael Stokleyi misunderstood instrument
#2020-12-2118:38seancorfieldI think almost everyone does when they first start using Spec.#2020-12-2119:32borkdudeThere is an unofficial spec tool which will check return values as well, called orchestra#2020-12-2119:35seancorfieldI think that library is a bit misguided since it deliberately blurs the lines that Clojure Spec draws by design. Caveat programmer.#2020-12-2119:36borkdudeSeemed worth mentioning, because a lot of people use it exactly for this reason.#2020-12-2120:34seancorfieldNo comment š#2020-12-2120:35borkdudeHehe, well, I don't use it either ;)#2020-12-2214:55mpenetthere are many of these libs, guardrails, orchestra, ghostwheel#2020-12-2214:55mpenetsome use instrument/check, some avoid it (intentionally)#2020-12-2214:56borkdudeI saw in the #announcements channel that guardrails now has an async instrumentation thing, so it doesn't slow down your functions but you will eventually know something is wrong - at least that's what I got from it#2020-12-2214:58mpenetyeah, it's optional. I think the approach with guardrails is by default just to log the failures#2020-12-2214:59mpenetso that's ok to do it that way#2020-12-2214:59mpenetI *reall*y wonder what spec2 will do in that regard, since apparently it's one area where there's some heavy hammocking going on. [edit] By that I don't mean Rich is heavy!#2020-12-2214:59borkdudeInteresting way to solve the slowness that instrumentation brings.#2020-12-2215:07mpenetbut yes, guardrails is the most interesting of the bunch so far imho. One thing is missing is editor support (indentation annotations), it's a bit ugly out of the box#2020-12-2215:11borkdudeyou mean like flycheck integration? that's pretty easy if you have file:line:col: ERROR message kind of output#2020-12-2215:11mpenetno, indentation, if you use something like aggressive-ident on emacs you'll get some ugly defn's with guardrails#2020-12-2215:11mpenet(same with cljfmt)#2020-12-2215:12borkdudeisn't that a general clojure tooling problem then?#2020-12-2215:12mpenetit's easy to solve, at conf level or even lib level#2020-12-2215:12mpenetyou can add annotation to the macros to give hints to editors :style/ident
#2020-12-2215:12mpenetit's not standard, but I think both emacs/cursive support it at least#2020-12-2215:13borkduderight. oh, this tool forces you to use a different defn macro? hmm#2020-12-2215:13mpenetmaybe even cljfmt#2020-12-2215:13mpenetyeah#2020-12-2215:13mpenetwell no, you can also use the annotation version#2020-12-2215:13mpenetbut you also have these def defn macros#2020-12-2215:14mpenetwe use it quite a lot, and like it. Sure there are rough edges but nothing too bad#2020-12-2215:14mpenet(spec1 I mean)#2020-12-2214:52dgb23Not sure if this is a #beginners question:
There is something that bothers me about (my understanding of) spec.
Example: #{āfooā ābarā}
implies string?
.
(s/explain-data #{āfooā ābarā} 42)
(s/explain-data (s/and string? #{āfooā ābarā}) 42)
What bothers/interests me here is that I have to write the second, but my first spec is a subset of string?
.
I know the semantics of the first might very well be heterogenous over time. But at that specific point in time the above sentence is true. It seems useful or interesting to me that a more involved and concrete spec should or could fail on and explain broader invariants if they are not satisfied.
So here is my first question: How feasible would it be to infer a superset of a spec?
The problem I see with this, is that the first spec can be expressed in multiple ways. For example with s/or
or even worse: with reduce
. Maybe the first spec function should have a function spec that checks :args
to be string?
, but that seems clunky.
A better question might be: is this a matter of design? Are these two specs really saying something meaningfully different or is the second strictly superiour?#2020-12-2215:09Alex Miller (Clojure team)"I have to write the second" - why?#2020-12-2216:06dgb23My intuition is that 42 should fail at string?
before it fails at #{"foo" "bar"}
. Or more generally: to explain why an input fails, wouldnāt it be more useful to say that it doesnāt satisfy an implied superset before I explain that it needs to satisfy an arbitrarily concrete predicate?
But from your question I guess this is a matter of designing a spec.#2020-12-2216:16Alex Miller (Clojure team)it's important to shift think a type mindset to a "value set" mindset. specs are predicative descriptors of allowed values. Saying #{"foo" "bar"}
means "these are the only two values allowed" vs string?
which means "satisfies the string? predicate". those are semantically a lot different (I would generally hesitate to use the first one in a function signature as time is long and you'll probably come up with a 3rd value later). you might for example, spec it as string?
but then validate a narrower spec inside the function.#2020-12-2216:18Alex Miller (Clojure team)it is really a matter of how you choose to design the spec for your data and what should imply to a user#2020-12-2216:48dgb23Thank you, this advice is very valuable (sic!), I have to think about it a bit moreā¦
I might be stuck in deduction world. I have this nagging feeling that deduction, where possible, gives you āfreeā stuff. But if I understand correctly this isnāt what spec is about. Itās supposed to be used concretely and specifically. AKA nothing additional should be inferred from a spec.#2020-12-2216:52Alex Miller (Clojure team)yes#2020-12-2216:52Alex Miller (Clojure team)it's about values, not types#2020-12-2216:53Alex Miller (Clojure team)an interesting question though is subsumption - "does this spec subsume (include) all possible values of this other spec?"#2020-12-2216:53Alex Miller (Clojure team)and that is something that we hope to potentially provide in the future#2020-12-2216:54Alex Miller (Clojure team)b/c it lets you ask questions about spec evolution in tandem with data evolution#2020-12-2217:34dgb23So in my case string?
subsumes #{āfooā ābarā}
right?
But youāre not interested in providing that for a specific isolated spec (aka subtyping or similar) but you want be able to ask questions about different specs across a timeline or maybe across different systems.
There seems to be tons of research surrounding the term āsubsumptionā in CS, robotics, AI, Lisp, Prolog and Datalog. And also in OWL https://en.wikipedia.org/wiki/Web_Ontology_Language.
Thankās again š#2021-12-3113:49stephenmhopperHappy new yearās eve, everyone! Iām working on writing some form validation for a web application. In my web app, most of my requests have well-formed request bodies that are known ahead of time. However, some of the requests have fields / properties whose structure is determined by metadata entries in my applicationās database. Is there a good example somewhere of programmatically generating Clojure specs for handling these at runtime?
So far, Iāve found that I can create a spec
with (s/spec int?)
(for example), but then Iām not sure how to use that with (s/keys)
as all of the s/keys
examples Iāve seen require specs to be registered in the global spec registry. Any ideas on how I can do this?
Do I need spec2 for this?
Also, itās worth noting that changes to these metadata DB rows are infrequent enough that I could probably create the specs by reading the DB when the application first starts, but Iād like to avoid doing that if possible.#2021-12-3116:28Alex Miller (Clojure team)you certainly can register the specs dynamically but it requires a bit of care with macros or eval to do so. spec 2 is a little friendlier for this use case but is not ready for real use yet#2021-12-3116:30stephenmhopperOkay, thank you, Alex. I just found https://github.com/metosin/spec-tools, which might fit my use case. TBD#2021-01-0411:20Karol WĆ³jcikHello! Is it possible to disable load of clojure.spec.alpha completely?#2021-01-0411:34hkjelsclojure.spec.alpha
is not part of Clojure, so what do you mean by ādisable loadā?#2021-01-0412:40delaguardoit is not possible.#2021-01-0413:03Karol WĆ³jcik@U0B1SDL67 it seems java11 dynamically loads clojure.spec.alpha which results in error: https://clojurians.slack.com/archives/C03S1KBA2/p1609747607299900#2021-01-0413:11delaguardostrange, clojure.walk is not requiring clojure.spec.alpha#2021-01-0413:13Karol WĆ³jcikFor me the strangest thing is that even after removing clojure.walk dynamic class load requires spec.alpha which results in different error. Never seen it before.#2021-01-0413:18delaguardobut spec.alpha is require clojure.walk ā¦#2021-01-0413:20delaguardoand clojure compiler is using clojure.spec.alpha there
https://github.com/clojure/clojure/blob/05c193c031aa3a825ebb824b4135e36a79693f51/src/jvm/clojure/lang/Compiler.java#L6961#2021-01-0414:37Alex Miller (Clojure team)spec depends on clojure, and clojure depends on spec. so there is actually a cycle here conceptually.#2021-01-0414:55Karol WĆ³jcik@U064X3EF3 tried to set -Dclojure.spec.skip-macros=true
without results.
Here is the example:
https://github.com/ekoontz/holy-lambda/blob/repro-spec/examples/hello-lambda/project.clj
Wondering why Clojure compiler macroexpands on runtime#2021-01-0516:09Karol WĆ³jcikWill try to provide minimal repro during weekend.#2021-01-0414:36Alex Miller (Clojure team)See -Dclojure.spec.skip-macros=true
https://clojure.org/guides/faq#skip_macros for turning off macro checking#2021-01-0523:02bennyis it possible to write specs for maps like you can in schema?
(def User
{::id s/Int
::name s/Str})
vs
(s/def ::id int?)
(s/def ::name string?)
(s/def ::user
(s/keys :req [::id ::name]))
#2021-01-0523:08Alex Miller (Clojure team)that is how you do it in spec#2021-01-0523:13bennythereās no way to do it inline like schema?#2021-01-0523:22Alex Miller (Clojure team)no, there is a philosophical difference in approach here#2021-01-0523:23Alex Miller (Clojure team)spec is trying to build a registry of spec'ed attributes. the attributes are seen as primary, the map only as container.#2021-01-0523:23Alex Miller (Clojure team)spec 2 is developing this further and will have some support for unqualified attributes with inline specs in a schema#2021-01-1222:46Alys Brooksis there any guidance on where to start adding specs to an existing project?#2021-01-1222:53Alys Brooksmy current strategy is basically to add specs to new code like I do tests. my next step will probably be trying to add specs as I happen to touch other code. however, I'm wondering about more proactive approaches.#2021-01-1223:04seancorfield@actuallyalys_slack I tend to focus on only adding specs where I think it will help me understand the code -- to avoid the temptation to treat Spec like a type system.#2021-01-1223:06seancorfieldCommon guidance is: write specs for your data structures but don't over-specify them; write specs for functions at "boundaries" (APIs of modules, for example) but don't get carried away. If you over-specify the code, it will be harder to make changes in the future because you may bake in assumptions that don't really hold as invariants over time.#2021-01-1223:14Alys Brooksgood point. i'm not as worried about over-specification because instrumentation can always be turned off or specs commented out.#2021-01-1223:16Alys Brooksbut there's still a time and mental cost to over-specified specs even so, so that's a good reminder.#2021-01-1223:19Alys Brooksi do think non-boundary specs can be useful because sometimes it's hard to know the exact shape of the data an internal function deals with or tedious to create it for the purpose of REPL experimentation. i guess that falls under "where i think it will help me understand the code"#2021-01-1223:26seancorfieldYup, I recently had to tackle a large part of our code base at work that I hadn't had to work on before and the first thing I did was to spec out some of the data structures, add some function specs, instrument the code and see what broke (and what worked) -- and then evolve the specs until everything worked again and that gave me a good understanding of what the data structures looked like and how the functions transformed them.#2021-01-1316:32mgIt appears that closed spec checking doesn't work on select
in spec2. Is this something deliberate?#2021-01-1316:35Alex Miller (Clojure team)don't remember :) I think it should work? but could easily be buggy#2021-01-1316:37mgToy example:
(spec/def ::a string?)
(spec/def ::s (spec/schema [::a]))
(spec/valid? ::s {::a "hello" ::b "world"} {:closed #{::s}}) ;; => false
(spec/valid? (spec/select ::s [*]) {::a "hello" ::b "world"} {:closed #{::s}}) ;; => true
#2021-01-1316:38mgI read through the implementation of select
and it looks like it doesn't do anything with :closed
#2021-01-1316:42Alex Miller (Clojure team)the things you "close" are schemas#2021-01-1316:43Alex Miller (Clojure team)select has some pending rework to make it build more on top of schema#2021-01-1316:43Alex Miller (Clojure team)when that's done it should start getting that effect#2021-01-1316:44Alex Miller (Clojure team)so without thinking more, this is probably in the category of wip#2021-01-1316:47mgOk, thanks for the clarification! Sounds also like it would not be fruitful for me to submit a patch, also, so I'll wait on it#2021-01-1316:49Alex Miller (Clojure team)nah, it's a non-trivial amount of work and bound into other things#2021-01-1815:17borkdudeIs the clojure.test.check lib an implementation detail of clojure.spec, or can we rely on the coupling to always to remain exist between the two?#2021-01-1815:59Alex Miller (Clojure team)I'd say it's more than an implementation detail, it's just somewhat hidden so that it can be dynamically loaded#2021-01-1816:00Alex Miller (Clojure team)no plans to change anything about that#2021-01-1816:00Alex Miller (Clojure team)spec generators are test.check generators (in a thunk)#2021-01-1911:31vlaaadI was thinking, is there (should there be) a way to describe something in spec as āthis has to be produced from a call to that functionā?
Sort of a bit like (s/fdef some-fn :args ... :ret ::the-thing)
without specifying ::the-thing
anywhere else, but, like, inverted: (s/def ::the-thing (s/ret-of some-fn))
. I donāt think there is a good way to fit this into conform
machinery, but Iām just thinking about the ability to express the API contract in spec (so far expressing it in the docstring is fine š )#2021-01-1911:58borkdude@vlaaad you can do this without spec maybe, just by letting the "factory" function put in some unique property which indicates it was made by that function#2021-01-1911:58borkdudeand you can validate that property using spec or using some other assertion#2021-01-1911:59borkdudeyou can also do this via .impl naming, e.g. create a defrecord in an impl namespace and have one public API function for creating those records#2021-01-1912:04vlaaadI also can use blanket (https://github.com/vlaaad/blanket) to cover the implementation details š To be clear ā Iām not looking for solutions outside of spec, I just thought the problem Iām solving is about describing the API, and spec seems like a fitting tool for this use case. I also heard @richhickey is redesigning spec particularly around fn specs, hence decided to share a use case to consider š#2021-01-1913:55Alex Miller (Clojure team)I donāt understand the use case#2021-01-1913:57Alex Miller (Clojure team)Saying āthis has to be produced from a call to that functionā seems weird#2021-01-1913:57Alex Miller (Clojure team)Data is data#2021-01-1914:05vlaaadthere is evolution over time#2021-01-1914:09vlaaadI want to express the intention ā if next version returns something else (or more realistically ā if the next version has more functions that produce something valid in a other context), it will remain valid in that other context#2021-01-1914:09Alex Miller (Clojure team)then spec the truth - what does the data look like?#2021-01-1914:09Alex Miller (Clojure team)don't couple the spec to code#2021-01-1914:09Alex Miller (Clojure team)this is the whole schema/select idea#2021-01-1914:09Alex Miller (Clojure team)schema is all possible fields that may travel together#2021-01-1914:10Alex Miller (Clojure team)select tells you what fields are selected from the schema at different points#2021-01-1914:11vlaaadbut I want to keep parts of the data implementation details#2021-01-1914:13Alex Miller (Clojure team)then don't spec those parts#2021-01-1914:14vlaaadGood point, although Iām not sure it captures the intention#2021-01-1915:31emccue@vlaaad Wouldn't just speccing the return values in the namespace that effectively declares that structure be enough?#2021-01-1915:31emccueat least from an encapsulation POV?#2021-01-1917:43dgb23Would be speccing it with any?
(and a docstring when we get it) a better idea in this case?#2021-01-1917:46dgb23The user can infer that ::the-thing
is explicitly anything forever.#2021-01-1917:48Alex Miller (Clojure team)why bother?#2021-01-1917:49Alex Miller (Clojure team)if it truly is "implementation details", then you are just erecting scaffolding and barriers in the way of future change#2021-01-1917:52Alex Miller (Clojure team)there needs to be some balance between the agility of Clojure data and the constraints of specs - don't spec everything to death#2021-01-1920:11vlaaadSounds like a disapproving opinion. I want to express a single thing: the shape of this object is implementation detail, and despite this fact, it can be used in that context. This is not a barrier in the way of future change, and definitely not specing everything to death, this is a contract that helps user to see the relationship between parts of API.#2021-01-1920:15vlaaadI like the (s/def ::the-thing any?)
btw, coupled with API fspecs with :ret ::the-thing
I think it reaches the intention#2021-01-2005:35Nolanhi there! have recently run into a scenario where i would like to access specs defined in clojure.core.specs.alpha
from clojurescript. those specs all seem to be in the registry in clojure as soon as you require clojure.spec.alpha
, but don't seem to be included in the final registry-ref
in the cljs spec namespace. i imagine this is intentional, but i'm wondering if there might still be a way to access them somehow. really appreciate any guidance on this. those are some valuable shapes!#2021-01-2314:02ikitommiwhat is the use case or rationale for s/map-of
not conforming the keys by default?
(s/conform
(s/map-of (s/or :s string? :b boolean?)
(s/or :s string? :b boolean?))
{"k" "v"})
; => {"1" [:s "1"]}
(s/conform
(s/map-of (s/or :s string? :b boolean?)
(s/or :s string? :b boolean?)
:conform-keys true)
{"k" "v"})
; => {[:s "k"] [:s "v"]}
#2021-01-2314:14Alex Miller (Clojure team)Usually keys are things that are not changed by conforming (keywords, strings, longs, symbols) and there is cost to conforming things, so defaults to not doing it.#2021-01-2314:46ikitommiok, thanks.#2021-01-2512:33borkdudeis there a possibility that spec1 and spec2 will be bunded in the same lib and logic that was unchanged will map to the same functions (better re-use when fixing bugs, smaller byte code and native-images)?#2021-01-2513:44Alex Miller (Clojure team)Seems highly unlikely as all functions have changed#2021-01-3017:17hadilsIs there any documentation on clojure.spec.alpha.gen (spec2)?#2021-01-3017:40seancorfield@hadilsabbagh18 Spec 2 is not ready for use yet. It's still evolving and it has a number of bugs right now.#2021-01-3017:41hadilsThanks @seancorfield#2021-01-3017:43seancorfield(and the only documentation for it is what you'll find in the repo and the auto-generated API docs: https://clojure.github.io/spec-alpha2/ )#2021-01-3018:23Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2021-01-3018:24Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2021-01-3018:27hadilsThanks @alexmiller -- I have already read those websites. I was asking about gen specifically.#2021-01-3018:30Alex Miller (Clojure team)No changes from spec 1#2021-01-3018:30Alex Miller (Clojure team)So far#2021-01-3018:57hadils@alexmiller -- my IDE indicates differently. I am requiring clojure.alpha.spec.gen
-- e.g. it does not have fmap
#2021-01-3018:59Alex Miller (Clojure team)Thatās doesnāt seem right#2021-01-3019:01Alex Miller (Clojure team)I think your ide is wrong :)#2021-01-3019:02hadilsOk.#2021-01-3019:02Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/blob/9dc3344bfe6d13213bdb15ca2e7680cd8eb439e3/src/main/clojure/clojure/spec_alpha2/gen.clj#L95 #2021-01-3019:03Alex Miller (Clojure team)No changes here from spec 1#2021-01-3019:03Alex Miller (Clojure team)But it is generating those defs as wrappers so a static analyzer wouldnāt see them#2021-01-3019:03Alex Miller (Clojure team)If you load the ns you should see those vars though#2021-01-3019:05hadilsI have a different sha. Let me try yours and see what happens...#2021-01-3019:11hadilsI tried it and it works now. Thanks for all your help.#2021-01-3021:01Greg RynkowskiHi, Iām trying to create a time generator based on given time range. I thought about using s/int-in
but distribution for default generator for this spec is not very useful. E.g.
(gen/sample (s/gen (s/int-in 1 1000000000)))
=> (2 1 2 5 5 6 15 2 3 4)
(the range is from 1 to bilion while all sample results are leaning to beginning of the range)
Are there any built-in generators I could use that can give me better distributed results?#2021-01-3021:27Greg RynkowskiOh, ok, I just realised that I get better results when Iām calling gen/generate
. Weird. š¤·#2021-01-3022:09andy.fingerhutI am not sure if what I am about to describe is what is happening in your case, but note that many spec generators try to generate "small test cases" first, then "larger/more complex test cases" only after those pass. For numbers, that sometimes means generating smaller values first, and only when those test cases pass, larger values. Default generators can be overridden if you don't like how they work.#2021-01-3022:10andy.fingerhutThe idea being if that "small" test cases fail, those are usually easier to debug the root cause and fix it, versus seeing large complex failing test cases.#2021-01-3022:32Greg RynkowskiCould be, I donāt know. I end up creating custom integers generator using MersenneTwister PRNG.#2021-01-3022:36Greg Rynkowski(defn gen-int-in
[{:keys [min max]}]
(gen/fmap
(fn [_] (-> (twist/next-int) (mod (- max min)) (+ min)))
(gen/return :something)))
(defn millis->date-time
[millis]
(-> millis
(Instant/ofEpochMilli)
(.atZone (ZoneId/systemDefault))
.toLocalDateTime))
(defn gen-local-date-time
[[from to]]
(gen/fmap
millis->date-time
(gen-int-in {:min from :max to})))
sample execution:
(gen/sample (gen-int-in {:min 1 :max 1000000000}) 15)
=>
(242825682
150977944
334804692
989355517
482024216
627890233
163007490
227494133
464164671
747363986
762489210
261499033
582274967
50926907
472313391)
(gen/sample (gen-local-date-time (create-future)))
=>
(#object[java.time.LocalDateTime 0x4139290 "2021-02-22T13:25:24.531"]
#object[java.time.LocalDateTime 0x53185984 "2021-01-31T00:17:03.320"]
#object[java.time.LocalDateTime 0xcb5632b "2021-02-23T02:22:41.585"]
#object[java.time.LocalDateTime 0x773ccacd "2031-01-23T16:49:36.048"]
#object[java.time.LocalDateTime 0x6fa351a8 "2021-02-09T10:45:48.003"]
#object[java.time.LocalDateTime 0x7ee6f1a1 "2031-01-29T13:07:57.477"]
#object[java.time.LocalDateTime 0x596d796d "2021-02-20T11:15:51.061"]
#object[java.time.LocalDateTime 0x4896aa2f "2031-01-18T13:37:21.066"]
#object[java.time.LocalDateTime 0x3dcd9c84 "2021-02-23T23:50:04.910"]
#object[java.time.LocalDateTime 0x746f0965 "2021-02-21T22:55:50.011"])
(`create-future` returns pair of timestamps [now, 10-years-from-now])#2021-02-0115:52Greg Rynkowski@U0CMVHBL2 re your answer https://clojurians.slack.com/archives/C1B1BB2Q3/p1612044579020800?thread_ts=1612040474.020100&cid=C1B1BB2Q3: This is exactly the case I was experiencing two days ago. I didnāt understand how generators work. This talk was eye opening: https://www.youtube.com/watch?v=F4VZPxLZUdA.
Creating a custom generator was an overkill, simple gen/scale
was enough for what I needed.#2021-02-0115:53andy.fingerhutCool. Glad you are finding ways to get the behavior you are hoping for.#2021-02-0115:55andy.fingerhutThe "small is simpler and should be generated earlier" makes sense for some numeric inputs, e.g. if your Fibonacci function has bugs for small input values, you probably want to debug those first before figuring out why it fails when given 10,325 as input. But for your use case of date/time ranges, in most applications it seems like the values early in the range are no simpler than any others (I imagine there could be date/time use cases where just-inside and just-outside of acceptable ranges are important corner cases to ensure that you test).#2021-02-0313:59hadilsDoes spec2 work with ClojureScript?#2021-02-0314:06Alex Miller (Clojure team)no#2021-02-0314:06Alex Miller (Clojure team)but it doesn't necessarily work with Clojure either as it's still a wip :)#2021-02-0314:06Alex Miller (Clojure team)CLJS is going to wait till it's "done" to port it#2021-02-0314:09hadilsThanks#2021-02-0512:56Daniel StephensHi, is there a better way to have a completely custom spec generator?
(let [my-custom-generating-fn #(do {:something (rand-int 100) :complicated (rand-int 200)})
g (gen/fmap #(%) (gen/return my-custom-generating-fn))]
(gen/generate g))
all the examples I found start from some primitive like (gen/int)
. Feels like what I was trying is intended to be difficult, is that because it doesn't follow some underlying random seed or something, or am I just missing the key bit#2021-02-0513:40Alex Miller (Clojure team)Correct - generally generators should control randomness, not introduce their own sources of randomness, as that makes it impossible to either shrink or regen from the same seed#2021-02-0513:53Daniel Stephensvery useful, thanks Alex#2021-02-0514:15borkdudeA one time shout out: finding specs by keyword is now a lot easier in clojure-lsp. You can even navigate to the definition of a spec via the keyword.#2021-02-1313:08vlaaadI was playing with built-in specs and discovered there are A LOT of places to embed var metadata in defn:
(defn ^{:b 1} foo
{:c 1}
([])
{:a 1})
:a
, :b
and :c
all end up in var meta!#2021-02-1315:53vemv> I wonder how's that
I guess it's because if there was just one place, people could accidentally place it elsewhere and get no metadata attached#2021-02-1313:09vlaaadI wonder how's that, and why :arity-n
defn variation allows attr map at the end...#2021-02-1316:52Alex Miller (Clojure team)I found that when I specāed it and asked Rich about it. The thought at the time was that metadata might get big (this predated some of the ^ syntax) and it would obscure the main part of the function so you could put it at the end#2021-02-1316:52Alex Miller (Clojure team)Itās rarely used but I did find some places using when testing a spec without it#2021-02-1319:01vlaaadVery interesting, thanks for this explanation!#2021-02-1903:54Zak SinghI have a bit of a messy spec/generators question which Iāve written up here - https://stackoverflow.com/questions/66271188/spec-for-map-with-interdependent-values-between-nested-levels - is such a thing possible?#2021-02-1904:16Alex Miller (Clojure team)Doing this kind of thing is inherently challenging. The general approach to take is: first generate a model by building up a core and extending it with gen/bind, then at the end generate the actual data structure with gen/fmap#2021-02-1904:17Alex Miller (Clojure team)As a simple example, donāt try to generate a square by generating random points. Instead, generate the right and left x, the top and bottom y (thatās your model), then generate the points of the square using fmap at the end#2021-02-1904:20Alex Miller (Clojure team)So Iām yours, maybe first generate a pool of terminals (collection of more constrained nodes, then randomly pick subsets to combine, and then maybe do some massage at the end#2021-02-1904:20Alex Miller (Clojure team)How far you go with this depends how much of the space you want to cover#2021-02-1904:22Zak SinghReformulating the problem like that makes a lot of sense - I essentially need to build up layers from the terminal case. This is really an incredibly powerful tool#2021-02-1904:26Alex Miller (Clojure team)Make sure to lean on s/gen of specs that may not match your public specs - easiest way to make new sub generators#2021-02-1904:33Alex Miller (Clojure team)Like (s/gen (s/tuple ...))
#2021-02-1904:34Alex Miller (Clojure team)Or (s/gen #{:magic :values})
#2021-02-1904:36Zak Singh(def terminal-gen
(gen/bind
(spec/gen (spec/tuple ::terminal-name ::terminal-kind))
(fn [[name kind]]
(gen/hash-map
:name (spec/gen #{name})
:kind (spec/gen #{kind})))))
#2021-02-1904:36Zak Singhlike this?#2021-02-1905:51Zak SinghManaged to get it built! That was some fun code:
(spec/def ::kind #{"NON_NULL" "LIST" "SCALAR" "OBJECT"})
(spec/def ::name (spec/nilable string?))
(spec/def ::ofType (spec/or :terminal nil?
:type ::type))
(spec/def ::terminal-kind #{"SCALAR" "OBJECT"})
(spec/def ::terminal-name string?)
(spec/def ::wrapper-kind #{"NON_NULL" "LIST"})
(def terminal-gen
(gen/bind
(spec/gen (spec/tuple ::terminal-name ::terminal-kind))
(fn [[name kind]]
(gen/hash-map
:name (spec/gen #{name})
:kind (spec/gen #{kind})
:ofType (gen/return nil)))))
(defn build-type
([max-depth] (if (= max-depth 1) terminal-gen
(build-type max-depth 0 terminal-gen)))
([max-depth curr-depth inner-gen]
(if (< curr-depth max-depth)
(recur max-depth
(inc curr-depth)
(gen/bind inner-gen
(fn [inner-gen]
(if (= "NON_NULL" (:kind inner-gen))
(gen/hash-map
:name (gen/return nil)
:kind (spec/gen #{"LIST"}) ; two NON_NULLs cannot be child-parent
:ofType (spec/gen #{inner-gen}))
(gen/hash-map
:name (gen/return nil)
:kind (spec/gen ::wrapper-kind)
:ofType (spec/gen #{inner-gen}))))))
inner-gen)))
(def type-gen
(gen/bind
(spec/gen (spec/int-in 1 5))
build-type))
#2021-02-2115:20borkdudeA video about grasp: a tool which lets you search code using clojure.spec specs#2021-02-2218:06Alex WhittWould anyone like to jump on this discussion thread I created on the subreddit? (I'd prefer to keep it there so it doesn't disappear behind Slack's paywall)
https://www.reddit.com/r/Clojure/comments/lpv8ok/spec_vs_malli/#2021-02-2221:41Alex WhittThe sentiment from the community so far has been leaning towards Malli. I'd love to get some alternative viewpoints in there.#2021-02-2221:50seancorfield@alex.joseph.whitt All I'll say is that I try hard to use "official" Cognitect stuff where it is available and we've been very happy with Spec for several years at work, although we have recently started using exoscale/coax in addition to Spec so that we can tease apart/replace our "coercing web specs" and use a more standard approach (i.e., separate coercion of strings, such as form input data, to stuff like long, bool, date... from the actual specs themselves).#2021-02-2221:54Alex Miller (Clojure team)I think your summary and the comments there are fair#2021-02-2221:57seancorfieldI have not looked at Malli in any depth but we did use Schema for a while (and abandoned it -- actually twice -- because it felt non-idiomatic with its syntax and it was only as good as your testing strategy, without generative testing: we were happy with our switch to Spec in comparison to Schema). We also tried Typed Clojure (again, twice) and abandoned that for different reasons. Spec hits a sweet spot for us -- and we use it in a lot of different ways per https://corfield.org/blog/2019/09/13/using-spec/#2021-02-2221:58Alex Miller (Clojure team)if I could wave a magic wand and have a final design and impl for spec 2 I would, but it's tackling some hard problems (beyond what we tried to tackle in spec 1). I know it feels stalled but work really does continue on it and I have hope that we will pop the stack of other work in progress back to it "soon".#2021-02-2316:06uwoI'm glad y'all are taking your time. It's worth it.#2021-02-2221:58seancorfieldYour comment about Malli embracing coercion would make me want to avoid it -- we went down that path with our own library and it was a mistake, hence our shift to Spec + coax now.#2021-02-2221:59borkdude@seancorfield Can you explain why it was a mistake?#2021-02-2222:00seancorfieldComplecting coercions with validations is just messy and it can hide errors.#2021-02-2222:00seancorfieldOur specs are simpler now, and don't need custom generators as often.#2021-02-2222:01seancorfieldThe coercions are clear and separate. We can test coercing code and validating code separately.#2021-02-2222:02seancorfieldI was one of those arguing in favor of having specs that coerced values for a long time. I was wrong about that.#2021-02-2222:02borkdudeI would love to add spec to babashka btw. But I don't feel certain about the alpha suffix and where it's going next and how long it's going to be. It is a recurring question from users. Since it's part of the current clojure people sometimes expect it to be just there. clojure.test.check is already in there, as a preparation step#2021-02-2222:04borkdudeI would also love to hear some insights from @ikitommi about the coercion philosophy in malli. Maybe it's one of those easy vs simplicity things, where a lot of people really just want the easy?#2021-02-2222:09borkdude@seancorfield Do you use clojure.spec to validate incoming web requests, in an open world way? @dominicm recently pointed out that this can be a security issue. This problem will be solved with spec2 which supports closed.#2021-02-2222:15seancorfieldI disagree that it's a problem with spec. If you have a context where you must only accept a certain subset of keys, that's what select-keys
is for.#2021-02-2222:16seancorfieldWe use spec as the "source of truth" for various things and we have situations where we derive the set of keys from the spec and then either explicitly restrict the data we process to that subset or we simply trim the keys present down to that subset, depending on whether we want to flag unwanted keys or not. But that trimming/checking is independent of how we validate incoming web requests.#2021-02-2222:18borkdudeIf it's not a problem, then why is spec2 introducing closed specs again? :thinking_face:#2021-02-2222:19seancorfieldBecause people whined about it not being in Spec 1 š#2021-02-2222:20seancorfieldYou could certainly do closed spec checking with Spec 1 -- you just have to do it manually.#2021-02-2222:20borkdudeI think what @dominicm was hinting at was that when you allow any spec to be validated in a web request, this may trigger functions you did not expect to be executed, or something, which may lead to DoS, or whatever#2021-02-2222:21seancorfieldThat's a lot of hand-waving š#2021-02-2222:21borkdudeI was asking questions, I did not take a position in this. Just wondering if you had considered it#2021-02-2222:22seancorfieldI think it's a strawman argument to "blame" spec for something like that, frankly.#2021-02-2222:22borkdudeok#2021-02-2222:24seancorfieldIf he has a specific, realistic scenario where just using Spec 1 to validate an incoming web request would cause a DoS, I'd be very interested to see that.#2021-02-2222:29seancorfieldThis is what Dominic said in #yada when asked why Yada does not have integration with Spec: "Because it gives attackers access to all your spec keys. Some of which might be very slow (and not run in production in normal circumstances). This opens you up to DOS attacks, the guidance from Alex on this was to validate data before passing it to spec.
On top of that, you usually want to restrict keys in the web context as you're going to pass them into the database." -- I've covered the latter part (above) and I would expect in real-world code what you put in the database isn't necessarily going to be anything like the set of keys that you get passed in a web request because the persistence model and the API model aren't likely to be a 1:1 match anyway.#2021-02-2222:51colinkahnTried following the link to the yada channel to see the conversion but perhaps it happened awhile ago. Curious what validation was suggested before passing to spec.#2021-02-2223:05seancorfieldI didn't dig far enough into the Zulip archives to see what Alex had actually said about that -- I'm pretty sure Alex didn't literally say folks should "validate data before passing it to spec" but he has talked about doing coercion on input data before passing it to spec (a separate thing).#2021-02-2223:06seancorfieldSlack has a 10,000 message limit on the free plan but most channels are mirrored to Zulip http://clojurians.zulipchat.com which has an unlimited archive and search facility (for the open source plan we're on there).#2021-02-2223:15colinkahnGot it thanks š#2021-02-2222:30robertfwI've had cases where the data I wanted to spec had to enforce a closed nature, namely, working with an HTTP API that throw an error if you gave it data it wasn't expecting. In that case we want a spec that validates that we're not including anything extra. The same would apply for a spec describing incoming data on a system that likewise wants to error out if the user sends unexpected fields (e.g., so users get warnings if they typo a parameter name or send data they think is doing something when it is actually being ignored)#2021-02-2222:31borkdudeI guess you could ask the question: why should web APIs validate differently than functions?#2021-02-2222:31seancorfieldI don't find the first part very convincing: if you have "very slow" specs that don't run as part of production code, why have them in your production code at all? Why not have them in test code (and if they're "very slow" they're not even going to be part of your unit tests).#2021-02-2222:34seancorfieldAs for Robert's case: I covered that above -- if you do want to fail validation on additional/unknown keys, that is possible with Spec 1. It's just not directly built in. I don't see Spec 2's "closed checking" as any sort of admission that Spec 1 was wrong -- I see it much more as a convenience that lets you write a little less code in certain situations. I think most people made a big ol' mountain out of wanting closed specs when it's really just a molehill.#2021-02-2222:34seancorfield(I think this is absolutely one of those cases where people pushed for "easy" rather than "simple")#2021-02-2222:35borkdudeWell, sometimes UX matters#2021-02-2222:36ikitommi@seancorfield just to clarify: malli separates coercion from validation, for the reasons you mentioned. You first "coerce" a value, then validate and explain (and humanize the explanation) if needed.#2021-02-2222:38seancorfieldGood to know -- that wasn't clear to me from @alex.joseph.whittās comment.#2021-02-2223:51Alex WhittWell, I don't know if we're any closer to a decision... so many good points brought up on both sides. I'd love to know if the Cognitect team ever considered implementing spec with a data-driven architecture like Malli's, and if so, what factors led to deciding against it.
For some context, I've been an avid spec user for years, and only recently became aware of Malli. As we're starting a new project, I guess I'm experiencing some FOMO regarding some of Malli's features, but I can't escape thinking that a lot of Spec's design decisions were really on-point and forward-thinking.#2021-02-2300:04Alex Miller (Clojure team)I think Rich would claim that spec forms are data :)#2021-02-2300:05Alex Miller (Clojure team)But spec 2 already has an alternative map-based data format (which is heavily subject to change)#2021-02-2300:09Alex WhittThat thought has occurred to me as well... I think we're conditioned to think that "data" means "EDN," but quickly forget that this is a LISP.#2021-02-2309:45mpenetspec is generally very opinionated, but once you "get" the design choices it's hard to be in disagreement with it. It fits quite well with datomic, "Maximal Graph" & co approach to dealing with data, which imho is spot on (I am not a datomic or pathom user but I agree with the general ideas).
Right now it's also quite bare bones but there are a few libraries that fill the important gaps (coercion, error messages, etc).
Then for me at least spec2 would fix most of the important problems I have with v1 (mostly schema+select, or whatever these will be called), but even without spec1 is very usable. Most of the problems stated about spec 1 imho are very often overblown or just misunderstanding on how to use spec or its goals.
But I get the frustration of malli authors who wanted spec2 now (I do too, but I rather have something carefully designed than a rushed draft) and the amount of work they poor into malli earns respect. Personally I do not agree with some of their design choices and most importantly the fact this will cause yet more fragmentation in that space. I would rather have had all that fire power put in spec libraries. I also had to deal with code bases that invested in spec-tools and broke some teeth in the process, so I prefer to be very conservative when I choose a library to fill that spot now.#2021-02-2309:50mpenetabout closed specs & data for me these are quite minor issues (if at all), I had to generate specs from another dsl on work I did recently and it's not a big deal at all. For more advanced composition I can't remember a case where i was blocked by the current api, in extreme cases you can rely on eval/macros to get to what you want but it's extremely rare in my experience at least. That said if spec2 improves that, great. For closed specs it's the same, writing a strict-keys version of s/keys is quite easy a naive version with poor error reporting is a couple of lines and a more involved one with nice error messages more work but it's under 50 loc#2021-02-2309:53mpenethaving multiple choices might be good, that's could be a sign we have a "healthy" community#2021-02-2310:47borkdudeAbout the data driven nature of malli vs spec:
I think it's fair to say that malli schemas are a better fit if you want to have a programmatic way of creating/changing schemas (in spec you end up with macros for this usually) and when you want to have (de)serialization of your schemas. That's not to say that you can't do this with spec, but the UX of malli is probably better for this kind of use.#2021-02-2317:33seancorfield@mpenet Thank you for being so eloquent about this issue -- like you, we haven't found Spec 1 to be limiting and we've been using it heavily in production code ever since it appeared. And thank you for Coax which is allowing us to retire our coercing "web specs"!
For a while we tracked Spec 2 at work, on a branch, and the data representation and programmatic Spec construction is pretty nice and it seems to address the concerns folks have expressed about the difficulty of building Spec 1 specs.#2021-02-2317:33seancorfield@mpenet Thank you for being so eloquent about this issue -- like you, we haven't found Spec 1 to be limiting and we've been using it heavily in production code ever since it appeared. And thank you for Coax which is allowing us to retire our coercing "web specs"!
For a while we tracked Spec 2 at work, on a branch, and the data representation and programmatic Spec construction is pretty nice and it seems to address the concerns folks have expressed about the difficulty of building Spec 1 specs.#2021-02-2310:52Eamonn SullivanI've been using clojure a lot lately to transform or combine data (usually from one or more REST endpoints) from one form into another. I am using spec the same way I would have used case classes in scala or json schemas -- to validate the input and output (often exactly that, with :pre
and :post
conditions). I'm assuming that I'm staying away from anything controversial and that it will be easy to change these to spec2. But that may be an incorrect assumption. I'm sticking with the "official" one only because I'm so new (I really only use Clojure one or two days a week) that I'm not experienced enough to judge third-party libraries. I'm just trusting the manufacturer here.#2021-02-2310:57borkdude@eamonn.sullivan The clojure.spec.alpha lib will be around probably forever, so even if spec2 gets bundled with core, you will still be able to use the old one#2021-02-2310:58borkdudeThe main difference in spec1 and spec2 is how s/keys
works, this will be replaced with s/select
(this is probably an over-simplification as I haven't been following spec2 for a while - I would love to get some updates!)#2021-02-2313:11Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha is the overview#2021-02-2313:11Alex Miller (Clojure team)s/keys actually works the same for now but will be superseded by s/schema and s/select https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2021-02-2311:32lassemaattaregarding the discussion about the strict separation of coercion & validation: are there any public non-trivial repos/projects showcasing this principle in practice (e.g. coercing and validating http requests/parameters etc)?#2021-02-2317:35seancorfieldI fear we are in another one of those situations where the non-trivial codebases using Spec heavily are all in-house/proprietary:
Clojure source 348 files 87710 total loc,
3532 fns, 896 of which are private,
556 vars, 29 macros, 93 atoms,
849 specs, 33 function specs.
Clojure tests 378 files 23192 total loc,
4 specs, 1 function specs.
That's a quick break down of stats for our codebase at work.#2021-02-2318:29Alex WhittWe decided to go with spec, by the way. We're betting that the number and quality of libraries in spec's orbit will explode when the official release drops, seeing as it will be so much easier to interact with specs programmatically. And regardless of the ecosystem of the future, it has what we need right now.
Malli is doubtless pushing the state of the art forward, but I have no doubt that many people will want to incorporate features and discoveries Metosin makes along the way into a spec-based architecture.#2021-02-2321:52vlaaad(s/form (s/keys* :opt-un [:clojure.core.specs.alpha/exclude :clojure.core.specs.alpha/only :clojure.core.specs.alpha/rename]))
=> (clojure.spec.alpha/&
(clojure.spec.alpha/*
(clojure.spec.alpha/cat
:clojure.spec.alpha/k
clojure.core/keyword?
:clojure.spec.alpha/v
clojure.core/any?))
:clojure.spec.alpha/kvs->map
mspec__2546__auto__)
How do I introspect s/keys*
spec? form says nothing about actual keys...#2021-02-2321:55vlaaadfound it...
(-> (s/keys* :opt-un [:clojure.core.specs.alpha/exclude :clojure.core.specs.alpha/only :clojure.core.specs.alpha/rename]) :ps second s/form)
=> (clojure.spec.alpha/keys
:opt-un
[:clojure.core.specs.alpha/exclude :clojure.core.specs.alpha/only :clojure.core.specs.alpha/rename])
that looks very unreliable though... can I rely on regex ops data shape?#2021-02-2321:57Alex Miller (Clojure team)what you need are specs for spec forms themselves - then you can conform the spec and get a map (you can also modify that map and s/unform it back to a spec form). some work on this in https://clojure.atlassian.net/browse/CLJ-2112 (but likely we'll never actually finish this as spec 2 will make it unnecessary)#2021-02-2322:10vlaaadWell, the form of s/keys* has no useful information at all if you look at the output#2021-02-2322:11vlaaadI.e. there are no mentions of actual expected keys#2021-02-2322:51Alex Miller (Clojure team)yeah, this has to do with the mixture of symbol / object in spec 1 which is almost entirely cleaned up in spec 2#2021-02-2410:50ikitommiabout the slow specs:
(time
(s/valid?
(s/keys)
{:clojure.core.specs.alpha/ns-clauses (repeat 100000 (list :gen-class))}))
; "Elapsed time: 5069.738452 msecs"
; => true
we added auto-closing of api specs to reitit (using spec-tools) to avoid this.#2021-02-2611:17jumarI know that you have to call stest/instrument
again when you change the function definition because instrument
wraps the original function and replaces the var root - when you redefine the fn, it's no longer instrumented.
But I thought that just changing the fdef
definition would be fine (no need to call instrument
again) because the spec is hold in global registry. But any change inside fdef
still requires calling instrument
again - is this because the spec fn wrapper captures the fdef spec value as it was at the instrument
was called?#2021-02-2611:25jumarAfter looking at spec code, I think the "problem" is that it doesn't call get-spec
at the time the function is invoked but rather at the time when instrument
is called and thus it only captures the spec as it was at that time - no later redefinitions are taken into account.#2021-02-2613:33Alex Miller (Clojure team)yep, exactly - this is a performance optimization#2021-02-2613:33Alex Miller (Clojure team)typical tradeoff of perf vs dynamicity :)#2021-02-2618:09seancorfield@jumar What I tend to do in situations like that is put the stest/instrument
call in my test namespace as a top-level form (after ns
but before any tests) so that the tests run with up-to-date instrumentation. If you want to do it in a source file, you could always have something like
(comment
(do
(stest/instrument `my-fn)
(my-fn 123)) ; eval this form with both the instrument call and the my-fn call
,)
#2021-02-2706:14jumarFor us the most common case is working on an app in the REPL and changing functions or specs. Then you forget to call instrument and you either realize pretty late or not at all that your spec is broken or some data arenāt really passed in. We do have a decent test suite but not everything is covered so sometimes itās only another developer who catches the issue#2021-02-2706:33seancorfieldWhen I'm writing Specs, I always have an RCF with s/exercise
calls to test the specs. I also tend to write a stub -test
ns when I write code and then I can run the tests in it with a hot key while I'm in the source file. The key is keeping a short feedback cycle and making sure you can run "tests" or at least "sanity check expressions" easily via hot keys while you are writing code. "forget" isn't an option: write the code so you can't forget (like the do
form I suggested).#2021-02-2716:46uwo@U04V70XH6 Would love to see a youtube video about the spec dimension of your development workflow, if you ever feel like it. I remember enjoying the demonstration of your REBL workflow#2021-02-2717:44seancorfield@U09QBCNBY Noted. I'm not sure when I'll get back to making videos -- I really hate the process (and the medium): I much prefer prose and code to videos.#2021-02-2618:51borkdudeI also have a couple of helper macros here for in/unstrumentation:
https://github.com/borkdude/respeced#with-instrumentation
https://github.com/borkdude/respeced#with-unstrumentation
They ensure the desired state in the body, but will restore the state like it was afterwards#2021-03-0215:45mpenet(defrecord Foo [])
(s/valid? (s/coll-of any?) (Foo.))
boom - Can't create empty: user.Foo
#2021-03-0215:46mpenetseems like a bug#2021-03-0215:46Alex Miller (Clojure team)it is, we have a ticket for it#2021-03-0215:46mpenetyou can avoid it using :into coll?
#2021-03-0215:46mpenetok#2021-03-0215:47Alex Miller (Clojure team)it's one of those things that seems obvious but is way more subtle than it seems in the impl.#2021-03-0215:48Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2481#2021-03-0215:48mpenetrelated too https://clojure.atlassian.net/browse/CLJ-1975#2021-03-0215:48Alex Miller (Clojure team)yeah, it's a dupe of that#2021-03-0216:09mpenetwould setting a default for :into at coll-of level to coll? be bad (I mean in the impl itself)?#2021-03-0216:10mpenetI get the fact that every-impl is shared quite a bit so that can't be baked in at this level#2021-03-0216:11mpenetlike here https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L592 , I guess because "performance"#2021-03-0217:39Alex Miller (Clojure team)very hard to say without a lot of thinking#2021-03-0217:39Alex Miller (Clojure team)and I rewrote half of that stuff in spec 2#2021-03-0504:08tianshuWhat is the stage spec2 working on? If the design is finished, is there anything we can help?#2021-03-0504:23Alex Miller (Clojure team)Rich is working on the design for function specs#2021-03-0616:50teodorluHow is https://clojuredocs.org/clojure.spec.alpha/fdef not good enough?#2021-03-0616:53borkdude@teodorlu I think the spec2 function spec is related to some kind of notation that alludes to defn
#2021-03-0616:54borkdudesomething like the plumatic/schema defn
macro (but very different, because it's not schema)#2021-03-0616:55teodorluAh, OK, so looking into "a single form that both defines a function and specifies its inputs" rather than a "normal defn and something additional"?#2021-03-0616:55borkdudeWe will only know when it comes out#2021-03-0616:56teodorluThanks!#2021-03-0617:06Alex Miller (Clojure team)correct#2021-03-0617:07Alex Miller (Clojure team)it's not just "putting the current function spec in defn", but trying to reach quite a bit beyond that into some new ideas, and it's taking a long hammock to sort through all the tradeoffs there#2021-03-0911:30AronI wish I could be part of this hammock thing, not constantly just sometimes, I feel I could contribute, especially since I have at least 3 projects on hold for years now since I've been waiting spec2#2021-03-1017:13ikitommimore discussion about the slow spec1 specs & external input: https://www.reddit.com/r/Clojure/comments/m0c64g/clojurespec_and_untrusted_input/#2021-03-1114:16mokrQuick question before I resort to rewrite my code:
Is there something inherently wrong with trying to keep code DRY by extracting values directly from a spec like this?
(ns my.specs ;; cljc-file
(:require [clojure.spec.alpha :as s]))
(s/def :select/legal-values #{1 2 3 4})
(ns my.ui ;; cljs-file
(:require [clojure.spec.alpha :as s]
[my.specs]))
(def dropdown-config [{:id :select-legal-values
:tooltip "Select your value"
:values (s/form :select/legal-values)},,,])
This only illustrates the part where I retrieve values from this spec. Data verification is elsewhere. What makes me ask is that Iāve had this working in my Luminus+Shadow-cljs+re-frame app, but as soon as I upgrade shadow-cljs, my project wonāt load in the browser. From e.g. āReferenceError: spec is not definedā I get the impression that the spec might not be ready yet. I donāt think it is shadow-cljs that is to blame as I encountered the same issue when upgrading yesterday as well, but after cleaning and trying a few times it suddenly started working. Maybe I got lucky with a race condition or somethingā¦
Anyone?#2021-03-1114:22Alex Miller (Clojure team)Yeah, this wonāt work in spec#2021-03-1114:23Alex Miller (Clojure team)Oh wait, Iām misreading#2021-03-1114:24Alex Miller (Clojure team)Might be a bug in spec emitting the wrong namespace on the spec form? We have fixed a few things like that#2021-03-1114:26Alex Miller (Clojure team)Iām keying off of āspecā there - not sure if thatās a hard coded word or a ns from data#2021-03-1115:15mokrBased on the error I donāt get the impression that there is a wrong namespace issue, but itās not that informative really.
Itās often a bit hard to reduce the actual code down to a suitable snippet that captures to issue. In this case I would say the gist is: Can a spec from a required namespace be used in a normal def that calls s/form to extract the āvalueā from that spec. Is the spec ready for value extraction at the time of def evaluation?
Even though I had this working Iāll just rewrite it as it seems a bit brittle. For instance IIRC I had to put the set directly inside the s/def, as s/form did not work if the spec used a var containing the set like this:
(def my-set #{1 2 3}
(s/def :foo/bar my-set)
(def boom (s/form :foo/bar)
I guess I might be abusing spec a bit, or relying on functionality that is not suitable for that purpose.#2021-03-1115:21mokrA quick rewrite where both the spec and my config refers to the same vars/defs resolved the issue. Still DRY, a few more lines of code, less ācleverā, but maybe easier to understand and less brittle. š#2021-03-1115:41Alex Miller (Clojure team)some of the fiddliness of this is improved in spec 2#2021-03-1117:25mokrSounds great. I was holding back on spec for a long time waiting for spec 2 š
But, I understand and appreciate the attention to detail that goes into the process.#2021-03-1119:43vlaaadAnyone knows if there is a library to convert json schema to spec definitions?#2021-03-1120:02robertfw@vlaaad I've looked for this previously (and for the reverse transformation) but came up empty. luposlip/json-schema
is the library I used for validation, and it seems to have sprouted schema inference since I last looked through its docs, that may not be useful in its exact form but could have some useful code within#2021-03-1815:35Eamonn SullivanHere's a question that'll probably out me as someone who doesn't understand the concept: Is is possible to spec a "closed" map, one that has some number of required and optional keys, but no other keys?#2021-03-1815:36Eamonn SullivanI'm trying to translate the JSON schema concept of
additionalProperties = false
#2021-03-1815:39Eamonn Sullivanprobably s/map-of #{...}
will try that.#2021-03-1815:44Eamonn Sullivannope#2021-03-1815:44Eamonn SullivanI guess that's a spec2 thing?#2021-03-1815:45borkdudeThis will be supported in spec2, malli already supports it#2021-03-1815:46borkdudeyou could also try to do this using a predicate, (s/and ... (fn [map] check-the-keys))
#2021-03-1815:46Eamonn Sullivanš” Thank you!#2021-03-1815:55Alex Miller (Clojure team)the important part of our perspective in spec 2 is that we think that "closed" should be a property of the check, not a property of the spec#2021-03-1815:56borkdudegood point!#2021-03-1815:57borkdude@U064X3EF3 Are you considering doing some journal updates again in the future? I for one really enjoyed them.#2021-03-1815:58Eamonn SullivanI think I see. So, I have an API that's dumb as a bag of rocks and will spit out any blob of json that includes anything other than, say, five keys. There are many combinations of those that are allowed, but it will throw a hissy fit if a key isn't one of those five.#2021-03-1816:00Eamonn SullivanOnly one of them is required and the rest optional, which all fit nicely into (s/keys :req-un [,,,] :opt-un [,,,])
until I did a typo in one of the opt-uns.#2021-03-1816:01Eamonn Sullivanpasses the spec; fails in life.#2021-03-1816:29vemv> should be a property of theĀ check, not a property of theĀ spec
(FWIW!)
while one can see the value of this, it's also hard to see how this would not conflict with DRY.
If one tried to DRY out n identical calls as a reusable "thing", what would be that thing be?
It seems to me that inherently, the 'thing' would effectively become a spec (whether it's an actual spec, or a de-facto spec: e.g. a defn)#2021-03-1816:31Alex Miller (Clojure team)makes sense to me#2021-03-1817:00Eamonn SullivanActually, this was a bit too easy. Must have missed something?
(s/def ::environments #{:int :test :stage :live})
(s/def ::valid-top-level-keys #{:name :description :values})
(s/def ::name string?)
(s/def ::description string?)
(s/def ::values (s/map-of ::environments string?))
(s/def ::config (s/and
(s/keys :req-un [::name]
:opt-un [::values ::description])
(s/map-of ::valid-top-level-keys any?)))
(s/def ::release-config (s/coll-of ::config))
(comment
(s/valid? ::release-config [{:name "something" :description "some description"}])
(s/valid? ::release-config [{:name "something" :desciption "some description with typo"}])
)
#2021-03-1817:00Eamonn SullivanI was already doing something similar with the :environments spec.#2021-03-1921:48pbrownHey there -- I really like this idea of closing specs at check-time in spec 2. Will there be (or is there already) a way to apply the same idea to sequential specs?
Let's say I'm speccing hiccup as
(defmulti form-of first)
(defmethod form-of :default [_]
(s/cat :tag keyword?
:props (s/? map?)
:children (s/* ::node)))
(s/def ::component (s/multi-spec form-of (fn [val _tag] val)))
(s/def ::node
(s/or :component (s/and vector? ::component)
:list (s/coll-of ::node
:kind (every-pred sequential?
(complement vector?)))
:atom (complement coll?)))
and define a bunch of components like
(defmethod form-of :script [_]
(s/cat :tag #{:script}
:props (s/? map?)
:children (s/* string?)))
Could I later restrict ::component
somehow to reject [:script ...]
as part of the instruction to s/valid?
? Or could my defmethod
s check a dynamic binding, say *illegal-tags*
?#2021-03-2315:01ognen.ivanovskiHi, I was wondering, when one has a repeating spec (e.g. non-empty string, something like a type) in spec2, what is more idiomatic:
a) simply define the spec behind a keyword, e.g.
(s/def ::non-empty-string (s/and- string? (comp not str/blank?))
(s/def :foo/bar ::non-empty-string)
or b) make non-empty-string a spec op, e.g.
(s/defop non-empty-string [] (s/and- string? (comp not str/blank?)))
(s/def :foo/bar (non-empty-string))
#2021-04-0222:21Michael Stokley(s/def :corp/employee
(s/keys :req [:employee/first-name]))
(s/def :employee/first-name string?)
(s/def :favorite/ice-cream #{:chocolate :vanilla})
(e/expound :corp/employee {:employee/first-name "michael"
:favorite/ice-cream :strawberry})
;; -- Spec failed --------------------
;; {:employee/first-name ...,
;; :favorite/ice-cream :strawberry}
;; ^^^^^^^^^^^
;; should be one of: :chocolate, :vanilla
;; -- Relevant specs -------
;; :favorite/ice-cream:
;; #{:chocolate :vanilla}
;; :corp/employee:
;; (clojure.spec.alpha/keys :req [:employee/first-name])
;; -------------------------
;; Detected 1 error
I find this pretty surprising#2021-04-0222:21Michael Stokleygiven that the :corp/employee
spec doesn't say anything about ice cream#2021-04-0222:23seancorfieldThat is by design in Spec ā and the reference docs on http://clojure.org talk about that. I think the rationale may do too.#2021-04-0222:24Michael Stokleythanks Sean - i'll take a look#2021-04-0222:25seancorfieldhttps://clojure.org/guides/spec#_entity_maps says āWhen conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. Weāll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.ā#2021-04-0222:29Michael Stokleyyeah, there it is. checks that every registered key has a conforming value. good to know#2021-04-0518:35dvingoHow do you deal with creating specs for maps whose keys require differing semantics based upon the context in which they are used?
Use case, of two contexts for modeling a 'task':
1. UI input form of a domain entity.
2. DB version of a domain entity.
The description must be a certain length when persisting to the DB:
(s/def :task/id uuid?)
(s/def :task/description (s/and string? #(> (count %) 100)))
In a UI form we allow the description to be empty:
(s/def :task/id uuid?)
(s/def :task/description string?)
(s/def ::task (s/keys ::req [:task/id :task/description]))
These are obviously in conflict, but given the use case they are both valid
in differing contexts. How does one design a solution using spec (alpha) for this use case?#2021-04-0518:38localshredUsually we'll just specify ns specific s/def's, since the context is different for the data. ::task-id
instead of :task/id
#2021-04-0518:39localshredsince the registry is global there's not many other options we could come up with that preserved the exact same namespaced key#2021-04-0518:39localshredyou can use (s/or ...
and tag the different cases, then when conforming verify it's the case you're expecting#2021-04-0518:41localshred(s/def :task/description
(s/or
::db (s/and string? #(> (count %) 100))
::ui string?)
#2021-04-0518:45dvingoI landed on something similar - saying the description could always be empty and then checking the DB requirements in the map spec - which knowingly goes against the design of spec in that the entity map is now aware of the semantics of its keyset#2021-04-0518:56localshredright, it gets a little muddy#2021-04-0518:42localshredthis has a different set of tradeoffs since you're now joining db and UI validation into a single spec in some shared ns... ultimately trade-offs you need to decide on#2021-04-0518:44dvingothanks for the quick reply - so in the ::task-id case your map data would not have :task/id instead it would have ns/task-id? and then at some point you transform to :task/id format before persisting?#2021-04-0518:53localshredcorrect#2021-04-0518:54localshredfor us we've kept the UI specs in ui namespaces, and the db specs correlating to the datomic keys we ultimately store them under#2021-04-0518:55localshredso the translation from front to backend keys occurs at the http layer#2021-04-0520:15dvingoOk thanks for the info, good (err bad) to hear that others have the same problems#2021-04-0520:22em@danvingo What about multi-specs? https://clojure.org/guides/spec#_multi_spec
Separating your task specs into various domains is entirely okay, but if you want to keep the same structure but have different contexts, just reifying that context with a keyword tag (or however else you want to do it, multimethods are open) keeps things fairly simple and open.#2021-04-0520:25dvingoI believe the core of the problem is that I want to use the kw :task/description
and have its spec be different based on some context - I don't think mutli-specs would alleviate that problem#2021-04-0520:45emIt's one of the only solutions for your exact problem statement - keeping everything else about your spec the same, but having it be treated differently depending on context.
You would inject context in the map wherever you needed it, and update that context at the boundary (when submitting from frontend to DB, for example). Injecting is literally an assoc
with an extra keyword, and the multi-spec just checks that context.#2021-04-0521:02dvingo(s/def :task/type keyword?)
(s/def :task/description string?)
(defmulti task-type :task/type)
;; in this "context" I only care that :task/description is a string.
(defmethod task-type :ui/task [_]
(s/keys :req [:task/type :task/description]))
;; in this "context" I want :task/description to be a non-empty string of a certain required length.
(defmethod task-type :db/task [_]
(s/keys :req [:task/type :task/description]))
I don't see how multi-methods make this tractable.
I think the answer above is the point - I'm misusing the tool fundamentally, as it is designed.#2021-04-0521:22emYeah, on further thought you're totally right, my bad. Multi-specs are one of the only dynamic dispatch mechanisms for specs, but only really in the context of maps. My envisioned solution is too clumsy, as you'd need to change :task/description
to be a map, and not a raw value. It'd work, but for every "context-variable" attribute you'd need to do a lot of extra work injecting context, as you couldn't do it at the top level.
Technically, in accordance to the design and intention of spec, you'd want the information determining the specification of an attribute at the level of that attribute, so it makes sense why multi-specs work pretty much only on maps, as they can contain such extra information.#2021-04-0520:29dvingoit seems like spec's answer to this is that you should create two specs under two different names, which: 1. feels like a bad idea - naming things are hard enough already. 2. If I have existing code and want to add specs for it, this requires me to change the names of my existing codebase, also tough decision to justify#2021-04-0520:31Alex Miller (Clojure team)You are indeed at cross purposes to the principles guiding spec #2021-04-0520:32Alex Miller (Clojure team)But this is why spec requires qualified names - the qualifier exists to disambiguate context#2021-04-0520:35Alex Miller (Clojure team)https://clojure.org/about/spec is worth reading for the big picture#2021-04-0520:35dvingoQuestions:
1. What is its purpose then?
2. What is the recommended approach to solve this problem. (I can acknowledge my data design skills are subpar and I should have thought about better names up front, but one of the things I like about clojure is the iterative design it enables when discovering a domain. And well, it's too late to go back now and rewrite that name all over a large system).#2021-04-0608:50Aroni have a similar problem to yours on the frontend, when i want to write a form generator, I have to keep 3 different specs for the same entity
1. the remote state specs that the backend API expects to get when I put or post or update the resource
2. the ui state specs which contains invalid data that was inputted by the user and is fully controlled by the user, but it still needs to be validated and used
3. my local display state specs about the inputted resource that holds stuff like a list of strings to shown in a predictive input while the user is typing their stuff
it's all in the same Entity!
the only solution is to accept that actually no, these are separate entities and require separate specs with separate names.
similarly in your case, if your context is different then you need a new spec#2021-04-0520:36dvingothanks, I've read it a few times. This is a real world problem that I don't have a good answer to at the moment and seems like other people do as well.#2021-04-0520:37Alex Miller (Clojure team)Given that you are at cross purposes with intents, there are no easy answers (well, donāt use spec is easy)#2021-04-0520:38Alex Miller (Clojure team)But given that qualified names have global meaning, you need to spec the union of possible global shapes#2021-04-0520:38dvingoCross purpose how? I think that's what I'm struggling to understand#2021-04-0520:39Alex Miller (Clojure team)Spec wants you to assign a spec with global meaning to qualified names#2021-04-0520:39Alex Miller (Clojure team)That is, specs are not contextual#2021-04-0520:40dvingoI see this makes more sense now#2021-04-0520:41Alex Miller (Clojure team)Unqualified names are assumed to be contextual and can be somewhat handled in s/keys with :req-un and :opt-un with contextual specs#2021-04-0520:41Alex Miller (Clojure team)(to some degree)#2021-04-0520:45dvingoperhaps the wording here should change:
> These maps represent various sets, subsets, intersections and unions of the same keys, and in general ought to have the same semantic for the same key wherever it is used
From "in general ought to" to "will always"#2021-04-0520:45emIt's one of the only solutions for your exact problem statement - keeping everything else about your spec the same, but having it be treated differently depending on context.
You would inject context in the map wherever you needed it, and update that context at the boundary (when submitting from frontend to DB, for example). Injecting is literally an assoc
with an extra keyword, and the multi-spec just checks that context.#2021-04-0520:46dvingocould you provide an example?#2021-04-0521:33dvingoI have come to see the problem with my :task/description
example. task
is not a useful namespace and should be updated to be globally unique.
Perhaps it would be useful to add to the guide some suggestions on use and data design in Clojure - as well as some anti-patterns. I think for small apps (which large apps usually start off as) it is common to use namespaces from the domain, but which are not globally unique, such as task
instead of com.myorg.myapp.task
and then as the app grows you get into problems such as the above ui/db distinction but now you have that too-simple namespace all over your code and probably persisted in storage and you have a tough decision to make on how to refactor it.
The guide actually includes these simple names in some places (`:animal`, :event
) which I think helps encourage the notion that this is a good idea, even though it is directly against the rationale: https://clojure.org/about/spec#_global_namespaced_names_are_more_important
In my specific case I am thinking, if I wish to continue using spec, the design that is aligned with spec's design would be to have:
:com.myorg.myapp.task.ui/description
and :com.myorg.myapp.task.db/description
or similar names.
Thanks for the feedback and discussion. Things are making more sense now.#2021-04-0618:34marciolSeems that this is the correct way of use. If you think about :task/description
and handle it differently depending on the context you can somewhat get in trouble because the context sometimes is not devised clearly.
So when you start to specify directly these keys each in their namespace, you are giving them a clear and unambiguous definition, so that you can even if needed to combine these two keys in the same map (keyset).
The only extra burden is to handle the encoding of data to an external medium when all this meaningful context will be lost, but for it, I'd expect some convention established by the team in charge of such system.#2021-04-0618:50marciolAnd following the accretion paradigm, you can create a new namespace when your system grows, and eventually remove the old namespace, without any conflict or problems with contextualization, etc.#2021-04-0521:38Alex Miller (Clojure team)the guide is intended to teach spec concepts so aims for easier names, but I agree this could probably be better to align with intent. issue welcome at https://github.com/clojure/clojure-site/issues#2021-04-0521:46dvingoagreed - when starting out the simple namespaces are great - but then there's no "advanced usage" or "scaling issues" or similar for perusing when you're in the intermediate stages of spec use.
Thanks - I'll add an issue#2021-04-0618:34marciolSeems that this is the correct way of use. If you think about :task/description
and handle it differently depending on the context you can somewhat get in trouble because the context sometimes is not devised clearly.
So when you start to specify directly these keys each in their namespace, you are giving them a clear and unambiguous definition, so that you can even if needed to combine these two keys in the same map (keyset).
The only extra burden is to handle the encoding of data to an external medium when all this meaningful context will be lost, but for it, I'd expect some convention established by the team in charge of such system.#2021-04-0618:50marciolAnd following the accretion paradigm, you can create a new namespace when your system grows, and eventually remove the old namespace, without any conflict or problems with contextualization, etc.#2021-04-0608:54Aronis "specs are not contextual" the same as "if your spec depends on context, you have more than one specs"?#2021-04-0609:17borkdude@ashnur spec 2 makes selection contextual (e.g. the combination of required keys)#2021-04-0609:18borkdudebut the keys themselves are always globally described#2021-04-0609:23Aronnot allowed to use spec2 in production yet afaik#2021-04-0609:26borkdudetrue, for practical purposes it doesn't really exist yet, but this is one of the biggest issues they wanted to address I believe#2021-04-0609:32Aroni am not sure it's the same issue#2021-04-0609:33Aronthe same logic goes for selections as well, if your selection depends on the context, isn't it true you have more than one selections?#2021-04-0609:35borkdudeyes, so?#2021-04-0609:36Aroni am good š#2021-04-0618:57Alex Miller (Clojure team)you don't even have to remove the old one :)#2021-04-0618:59marciolexactly#2021-04-0619:02marciolIt's a paradigm shift and sometimes takes time to finally get it.#2021-04-0910:25Matti UusitaloBackground:
I want to create a spec which can validate values in different branches of a document in a manner that what I get out of spec/explain-data points to the right value. For example:
{:min 0 :max 10 :values [0 1 2 3 11 2]}
I want the spec to be able to say that because 11 is not between the values given in :min and :max, it is in error.
Iām trying to implement the Spec protocol to add a wrapper for a spec which works like this
(defmacro value-binding [pred bind]
`(let [wrapped# (delay (#'spec/specize ~pred))]
(reify
spec/Specize
(specize* [s#] s#)
(specize* [s# _] s#)
spec/Spec
(conform* [spec x#]
(binding [~bind x#]
(spec/conform* @wrapped# x#)))
(unform* [spec y#]
(binding [~bind y#]
(spec/unform* @wrapped# y#)))
(explain* [spec# path# via# in# x#]
(binding [~bind x#]
(spec/explain*
@wrapped#
path# via# in# x#)))
(gen* [spec overrides# path# rmap#]
(spec/gen* @wrapped# overrides# path# rmap#))
(with-gen* [spec gfn#]
(spec/with-gen* @wrapped# gfn#))
(describe* [spec]
(spec/describe* @wrapped#)))))
The idea is that when the validation passes this spec, the validated value is bound to the given dynamic variable. The wrapped spec can then use the value in that dynamic variable.
I got this working with spec/valid? but not with spec/explain-data, which always returns an empty collection. Any ideas why? I made a test spec which prints if the dynamic variable has been bound to a value or not. Inside conform* it is bound, but inside explain* it is not.#2021-04-0911:01Matti UusitaloSome further infoā¦ I can make a minimal spec in unit tests, but in a larger context this breaks down#2021-04-0911:07ikitommifor that given example:
(s/explain (s/coll-of (s/int-in 0 10)) [0 1 2 3 11 2])
; 11 - failed: (int-in-range? 0 10 %) in: [4]
#2021-04-0911:18Matti UusitaloThe issue here is that values in the collection should be able to be valid or invalid depending on values in a different part of the document. The example I wrote is a minimal example that tries to convey the idea#2021-04-0911:22Matti UusitaloI would use my spec like this
(def ^:dynamic *document*)
(spec/def ::min int?)
(spec/def ::max int?)
(spec/def ::value (fn [v]
; compares that v is between min and max
))
(spec/def ::values (spec/coll-of ::values))
(spec/valid?
(value-binding
(spec/keys :req-un [::min ::max ::values])
*document*)
{:min 0 :max 10 :values [0 1 2 3 4 5 11]})
#2021-04-0911:45ikitommiif spec supported maps with inlined entry definitions, that would be easy to do - you could create a new spec with new ::values
subspec based on the whole document, but now it would require going through the global registry. I believe spec2, plumatic & malli all make this easy to do. there is also spec-tools, with a working(?) dynamic var for stuff like this, but donāt recommend it.#2021-04-0911:47benoitI would usually put the constraint on the map itself since it is a constraint between its elements.#2021-04-0911:48benoit(s/and (s/keys ...)
(fn [{:keys [min max values]}] (every? #(< min % (inc max)) values)))
#2021-04-0911:51benoitBut you want explain-data to return the value ... I see. Good luck with that š#2021-04-0912:00benoitCould you just redefine the spec for every document?#2021-04-0912:00benoit(defn data-spec
[{:keys [min max values]}]
(s/def ::min int?)
(s/def ::max int?)
(s/def ::values (s/coll-of (s/int-in min max)))
(s/def ::data (s/keys :req-un [::min ::max ::values])))
(let [data {:min 0 :max 10 :values [0 1 2 3 11 2]}]
(data-spec data)
(s/explain-data ::data data))
#2021-04-1103:42Matti UusitaloThe most annoying thing is when I make a simple example to see where it breaks down it works fine.#2021-04-1106:13ikitommithe data-spec
is not safe as it mutates the global registry. If you have two documents with different min & max, the last one overrides the first.#2021-04-1106:13ikitommiHow did you make it work @USGKE8RS7?#2021-04-1120:10benoitYes, you have to be careful to not use data-spec
for multiple documents at once. Sometimes it is an acceptable trade-off.#2021-04-1120:35benoitI'm not seeing a way around it if you want to use s/keys
and the global registry. It does not make sense to me to define a spec on the global keyword ::values
that is specific to a given map. The contract that the integers in ::values
must be between :min
and :max
is a property of the map, not the global ::values
keyword.
If you still want to benefit from the s/explain
infrastructure, you can always write a "local spec" like this:
(defn validate-map
[{:keys [min max values] :as data}]
(let [s (s/coll-of (s/int-in min max))]
(when-not (s/valid? s values)
(throw (ex-info "Invalid map."
{:explain (s/explain-data s values)})))))
#2021-04-1204:45Matti UusitaloSo I finally figured out what was the problem. I had to wrap the nested spec/explain* call to a doall, because apparently explain returns a lazy sequence which canāt access the bound value if it is returned out of the binding block before realizing the sequence#2021-04-1204:51Matti Uusitalo@U055NJ5CC i have now
(defmacro value-binding [pred bind]
(let [pf #?(:clj (#'spec/res pred)
:cljs (res &env pred))]
`(let [wrapped# (delay (#'spec/specize ~pred ~pf))]
(reify
spec/Specize
(specize* [s#] s#)
(specize* [s# _] s#)
spec/Spec
(conform* [spec x#]
(binding [~bind x#]
(spec/conform* @wrapped# x#)))
(unform* [spec y#]
(binding [~bind y#]
(spec/unform* @wrapped# y#)))
(explain* [spec# path# via# in# x#]
(binding [~bind x#]
(doall (spec/explain*
@wrapped#
path# via# in# x#))))
(gen* [spec overrides# path# rmap#]
(spec/gen* @wrapped# overrides# path# rmap#))
(with-gen* [spec gfn#]
(spec/with-gen* @wrapped# gfn#))
(describe* [spec]
(spec/describe* @wrapped#))))))
and then for example
(testing "Bound values can be referred to in specs"
(let [test-spec
(sut/value-binding
(fn [v]
(= *test-binding* v))
*test-binding*)]
(is (spec/valid? test-spec 123))))
at some point that breaks down without that doall because of the lazyness & bindings#2021-04-1204:51Matti Uusitalothat cljs stuff is there because clojurescript has a different implementation of clojure.spec.alpha#2021-04-0919:37Michael Stokleygiven that all map keys are validated against the spec registry, what is the significance of the :opt
argument to s/keys
? documentation?#2021-04-0919:53jjttjjI think it adds the generator stuff for the optional key#2021-04-0919:53jjttjj(so it will sometimes be generated)#2021-04-0920:04Alex Miller (Clojure team)also ends up in the doc
output#2021-04-1120:01Braden ShepherdsonI'm trying to spec a large, complicated map for a bunch of game data. there are several possible states the game might be in, and the fields vary depending on which state we're in. I want a spec something like
(s/def ::state-pregame (s/merge ::state-core (s/keys pregame-specific-things) {::state :states/pregame}))
#2021-04-1120:01Braden Shepherdsonwhere the key is that last bit, specifying a particular key and value pair for the pregame state.#2021-04-1120:08Braden Shepherdsonoh, TIL about multi-spec
, which is exactly this.#2021-04-1209:15Carlowhat would be the most idiomatic way to spec a graph structure? I would want something around (`graph` is the loom library):
(s/def ::my-graph
(s/and
graph/directed?
#(not= :cljs.spec.alpha/invalid (s/conform (s/coll-of ::my-node) (graph/nodes %)))))
#2021-04-1209:16Carlobut I find that predicate quite cumbersome to write#2021-04-1209:24Carlo(s/def ::my-graph
(s/and
graph/directed?
#(s/valid? (s/coll-of ::my-node) (graph/nodes %))))
is a bit better. Is there anything better still?#2021-04-1210:01borkdude@meditans not an answer to your question, but in general: instead of = :cljs.spec.alpha/invalid
use (s/invalid? ...)
#2021-04-1210:02borkdudeI think you should be able to leave out s/valid?
and just pass a spec#2021-04-1211:42Carlobut how would I also invoke graph/nodes
?#2021-04-1218:05emWhat does your graph data structure look like? Is it some kind of custom data type, and therefore it requires the graph/nodes
accessor?#2021-04-1218:06em@meditans If it's just a map, which is preferred in most situations, you can just spec a :graph/nodes
key while also getting all the benefits of spec for your other graph metadata#2021-04-1219:10Carlothe data structure is the one in the loom
library. (graph/nodes %)
is the invocation of the function that gives the nodes back#2021-04-1220:22colinkahnOne potential option is to use a conformer in your s/and
to handle the conversion into your specable form:
https://clojuredocs.org/clojure.spec.alpha/conformer
(s/and graph/directed?
(s/conformer graph/nodes)
(s/coll-of ::my-node))
#2021-04-1301:42Michael Stokleyare there ways to programmatically work with specs when the output of s/get-spec
is an object, in some cases? are there ways folks approach this?#2021-04-1301:42Michael Stokleyi'm seeing that i can't build up expressions programmatically out of existing specs#2021-04-1301:43Alex Miller (Clojure team)this is one of the big things addressed in spec 2 (via several different options) but it can be cumbersome to do in spec 1#2021-04-1301:44Alex Miller (Clojure team)in spec 1, one approach is to use macros to emit spec forms#2021-04-1301:47Alex Miller (Clojure team)the other option is to s/form a spec object (the result of s/get-spec) to get back to a form, then manipulate it as a collection, and either s/def or eval it#2021-04-1301:49Michael Stokleywow, ok! thank you!#2021-04-1413:09vlaaadIs there a way to make fn spec that allows fn to throw exceptions? I want to express a contract "this function returns any value and can even throw"#2021-04-1413:29aviIIRC, no.#2021-04-1413:31aviThis has encouraged me to spec out some of my functions to return values that indicate success or failure. Often theyāre :cognitect.anomalies/anomaly
values. Iām pretty happy with this approach. It makes the functions more testable, I think. And by speccing out the error values you get the property testing to extend to error cases as well.#2021-04-1413:33vlaaadYeah, but I want to document that I allow user-provided callback to throw an exception. Itās like a promise āitās okay to failāā¦#2021-04-1413:33aviYeah, makes sense, reasonable. I just think thatās out of scope for spec.#2021-04-1413:35aviIf I were to speculate the thinking behind that call, Iād guess that itās an example of the position that exceptions should be used in truly exceptional cases ā cases that are hard to predict and handle ā and can thus be thrown anywhere, at any time. (Thatās a reductive summary missing much nuance but I hope potentially slightly useful.) Just speculation though, I could easily be mistaken.#2021-04-1413:36aviThis might be a little silly, but in some cases Iāve written functions that catch Exceptions and then return them as values!
e.g.
(s/fdef check-render-result
:args (s/cat :result (s/or :success ::r/success-result
:failure ::r/failure-result)
:path ::fs/file-path-str)
:ret (s/or :success nil?
:failure (partial instance? Exception)))
#2021-04-1414:06Alex Miller (Clojure team)function specs document non-exceptional use#2021-04-1414:06Alex Miller (Clojure team)so they don't include any way to talk about exceptions#2021-04-1414:52vlaaadunderstood#2021-04-1419:26jmromrellI am trying to spec the requests and responses to an API, but am running into issues with namespace collisions.
(ns my-app.validate.endpoint-a)
(s/def :ok-response/status #{200})
(s/def :ok-response/body ...)
(s/def ::ok-response (s/keys :req-un [:ok-response/status
:ok-response/body]))
(s/def :created-response/status #{201})
(s/def :created-response/body ...)
(s/def ::created-response (s/keys :req-un [:created-response/status
:created-response/body]))
(s/def ::response (s/or :ok ::ok-response
:created ::created-response))
The problem comes when I add validation for endpoint-b
which will also have an :ok-response/body
which will be different than that for endpoint-a
.
I am already encoding the type of response (`ok-response` vs. created-response
) in the kw namespace due to s/keys
mandating that the kw name matches the key in the map being spec'd.
I can continue to encode further dimensions (endpoint, method) into the namespace to avoid these collisions, but it quickly gets unwieldy:
(s/def :get-endpoint-a-ok-response/body ...)
Am I overlooking an idiomatic way of handling this problem?
I'd love to see something like
(s/def ::ok-response-body ...)
(s/def ::ok-response (s/named-keys :req {:status #{200}
:body ::ok-response-body}))
#2021-04-1419:27jmromrellNote: s/named-keys
above could also address support for string keys https://clojure.atlassian.net/browse/CLJ-2196#2021-04-1419:40seancorfieldSpec 2 will provide that functionality @jmromrell if Iām understanding correctly what you are asking.#2021-04-1419:44jmromrellThat's great. The inflexibility of s/keys
has been a pain point for me multiple times.#2021-04-1419:40seancorfieldBut it sounds like you might also want to look at multi-spec
?#2021-04-1419:44jmromrellI'll look into it. Thank you!#2021-04-1505:38samI could use some guidance on using clojure.spec to validate input arguments to a function. Hereās some context:
ā¢ I am debating instrumenting with fdef
vs. using :pre
and :post
conditions.
ā¢ The function is internal to the app and the argument values are completely under the control of the programmer (i.e. there is no validation of external data going on).
I was initially leaning toward instrumenting with fdef
because I see this as a check for program correctness rather input validation. It should be safe to turn this check off in production with no change in behavior.
But I want the instrumentation to be turned on by default during development and turned off by default in production. I can imagine ways to achieve that, but the fact that I couldnāt find any mention of a typical pattern for doing that (e.g. my understanding is that assert
is set up to be easily enabled/disabled in a blanket manner) made me think I might be missing something.#2021-04-1505:44samAnother thing I should mention is that I have no intention of ever testing this function with generative tests. Itās not a good fit for generative testing.#2021-04-1510:21vemv> I was initially leaning toward instrumenting withĀ fdefĀ because I see this as a check for program correctness rather input validation.
what's this
?
(I can interpret it in two different ways)#2021-04-1514:37samYeah, thatās fair. this = āthis solution Iām designingā, not this = āinstrumenting with fdefā.#2021-04-1514:52vemvIf ease of turning on/off is the main concern, perhaps I'd go for fdef because it's the default choice, and toggling it globally, and both for inputs and outputs seems relatively easy with https://github.com/jeaye/orchestra
There are reasons beyond toggling ease why some people might prefer assert
to instrumentation. https://github.com/fulcrologic/guardrails or https://github.com/nedap/speced.def (disclaimer: I authored it) and probably a couple more libraries favor it.
There's no fixed truth in the topic, but in absence of other concerns I'd probably 'start small'.#2021-04-1514:59sam@U45T93RA6 Thanks a ton! I need to digest this more and check out orchestra. Off the top of my head, my only concern is that I usually like to be a little wary about adding new dependencies if I donāt have to.#2021-04-1512:38Alex Miller (Clojure team)Are you talking about clojure.core/assert or s/assert? Seems like the latter would be good here#2021-04-1514:48samThanks! Somehow I missed s/assert
, I think youāre right. Maybe s/assert
in a :pre
conditionā¦#2021-04-1514:53Alex Miller (Clojure team)I'd put it in the mainline#2021-04-1514:54Alex Miller (Clojure team)s/assert in mainline code is designed so that it can be compiled completely out of the program via https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/*compile-asserts*#2021-04-1515:00samOkay, thank you! I really appreciate the guidance.#2021-04-1522:22Franco GasperinoHello. Where i have several defined and registered predicates, can i compose them in this state?
;; contrived example
(s/def ::string-starts-with-an-a? (s/and string? #(string/starts-with? % "a")))
(s/def ::not-an-apple? #(not= "apple" %))
(s/def ::starts-with-an-a-but-not-an-apple? (s/and ::string-starts-with-an-a? ::not-an-apple?))
#2021-04-1522:25Franco Gasperinopredicate composing after registration#2021-04-1522:26Franco Gasperinodisregard the above. my forms had not been evaluated, thus the error received was entirely my doing#2021-04-1810:52mokrI got a bit carried away experimenting with specs and I believe I might have hit a dead end unless spec can actually pick apart a string (like I could do with regex capturing groups or Instaparse).
Can spec accomplish something like this contrived example?
(s/def ::title #{"mr" "mrs" "dr" ,,,})
(s/def ::surname #{"green" "brown" "who" ,,,})
(s/def ::title-surname (s/__ :title ::title :surname ::surname))
(s/conform ::title-surname "mrgreen") ;;alt1 => [:title "mr" :surname "green"]
(s/conform ::title-surname "mrsbrown") ;;alt2 => {:title "mrs" :surname "brown"}
#2021-04-1810:56borkdude@mokr The standard answer to this is: although you could, spec isn't designed for parsing strings#2021-04-1810:57mokr@borkdude Thanks, that settles it. I need to backtrack a bit.#2021-04-1821:06joelittlejohnI agree with @borkdude, spec isnāt the right tool for this, however I have done this kind of thing in past for fun:
(s/def ::title
(s/alt :mr (s/cat :m #{\m} :r #{\r})
:mrs (s/cat :m #{\m} :r #{\r} :s #{\s})
:dr (s/cat :d #{\d} :r #{\r})))
(s/def ::surname
(s/alt :green (s/cat :g #{\g} :r #{\r} :e #{\e} :e #{\e} :n #{\n})
:brown (s/cat :b #{\b} :r #{\r} :o #{\o} :w #{\w} :n #{\n})
:who (s/cat :w #{\w} :h #{\h} :o #{\o})))
(s/def ::title-surname
(s/cat :title ::title :surname ::surname))
(s/conform ::title-surname (seq "mrwho"))
;; => {:title [:mr {:m \m, :r \r}], :surname [:who {:w \w, :h \h, :o \o}]}
(s/conform ::title-surname (seq "mrsbrown"))
;; => {:title [:mrs {:m \m, :r \r, :s \s}], :surname [:brown {:b \b, :r \r, :o \o, :w \w, :n \n}]}
#2021-04-1822:43twashingDoes cljs.spec.test.alpha/check
generate bad input? I have a function spec that specifies a map for its input. gen/generate
and gen/sample
work fine.
But calling cljs.spec.test.alpha/check
fails with the error: More than one element found in structure
. And it looks like the spec system is generating incorrect input.
bar spec
(s/def ::check-run
(s/keys
:req-un
[::action
::check_run
::installation
::organization
::repository
::sender]))
foo.cljs
(s/def ::payload :bar/check-run)
(s/def ::check-run-started (s/keys :req-un [::payload]))
(s/fdef check-run->cijob-created
:args (s/cat :arg ::check-run-started))
(defn check-run->cijob-created [arg])
While the function spec only declares A, the spec system is generating B.
;; A
{:payload {:action "", :check_run {:html_url "", }}, ...}
;; B
[({:payload {:action "", :check_run {:html_url "", }}, ...}})]
workbench
(cljs.spec.test.alpha/check
`foo/check-run->cijob-created
{:clojure.spec.test.check/opts {:num-tests 10}})
[{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha50916],
:clojure.spec.test.check/ret
{:shrunk
{:total-nodes-visited 313, :depth 148, :pass? false, :result #object[Error Error: More than one element found in structure: 0], :result-data #:clojure.test.check.properties{:error #object[Error Error: More than one element found in structure: 0]}, :time-shrinking-ms 11299,
:smallest
[({:payload {:action "", :check_run {:html_url "", }}, ...}})]},
:sym foo/check-run->cijob-created,
:failure #object[Error Error: More than one element found in structure: 0]}]
#2021-04-1903:04twashingOk, figured this one out. It was failing due to my use of a specter macro + navigator. Iām not sure how. But somehow this messes up test.check generators.
https://github.com/redplanetlabs/specter/wiki/List-of-Macros#select-one#2021-04-1903:26twashingIām assuming itās some kind of strange interplay thatās unworkable with Clojurescriptās macro system.#2021-04-2016:09cap10morganIs it expected behavior that s/valid?
needs the test.check
library and needs to construct a gen for every (nested) spec it's validating against? seems counterintuitive to me, but maybe I'm missing something.#2021-04-2016:16kennyIt's only necessary when checking fspec
s.#2021-04-2016:17cap10morganI'm getting "Unable to construct gen at: ..." errors when calling (s/valid? ::my-spec my-val)
#2021-04-2016:18kennyI suspect ::my-spec
uses s/fspec
.#2021-04-2016:19cap10morganthe one it's erroring on currently looks like this: (s/def ::log-file #(instance? File %))
#2021-04-2016:26Alex Miller (Clojure team)I agree with kenny, that this seems like what you'd see with s/fspec#2021-04-2016:27Alex Miller (Clojure team)and generally, s/valid? will not require test.check unless you are gen'ing#2021-04-2016:30cap10morganhmm... ok. yeah, commenting out my two fdef
's eliminated it. thanks!#2021-04-2108:18danmAhoy! Is there any way to generate a spec for an anonymous function without using s/fspec
? Or without s/fspec
generating data to test the function with? We've got a pattern where a callback is passed into a function, and we want to validate that the callback conforms to the expected function 'shape'/arity.
As I understand it we can't use s/fdef
, because at the point we get the callback (as an arg to our function) it's not bound to a symbol, which s/fdef
requires as the first arg.
Our callback function may be side effecting/have state (store some of the callback results in an atom for later reference), so if we use s/fspec
then we do get an error if the callback doesn't have the right arity, but our state also gets corrupted with all the junk data that s/fspec
generated to test the function.#2021-04-2112:52Alex Miller (Clojure team)You can just spec it as ifn?#2021-04-2112:53Alex Miller (Clojure team)It wonāt check the āshapeā then #2021-04-2310:29danmYeah, that's what we've ended up doing. Ta š#2021-04-2315:48Alex Whittin spec-alpha-2, will select
s be composable? Assuming the answer is "yes," what does that look like?#2021-04-2316:22Alex Miller (Clojure team)I think the answer will be no, actually, but it depends what you mean. which part are you looking to compose?#2021-04-2316:24AronPersonally, on the frontend the concept of static views that can define a select from the global state of the UI, then combining all of these into a single one for the whole of the UI?#2021-04-2316:29Alex WhittWe have a data structure that is going to accumulate keys as it moves through our system. To model that, it would make sense to have a selection for "phase 1," then a "phase 2" that takes phase 1 and adds a few keys, and a phase 3 that builds on phase 2, and so on. It would be nice to keep that DRY.
(Actually we have a data model DSL that generates our specs, so we could always do the composition at that layer, but I wanted to know what spec-2 was planning on supporting.)#2021-04-2316:32Alex WhittI guess I should've phrased my question as "are selections mergeable"#2021-04-2316:46Alex Miller (Clojure team)I think that's a very interesting and unresolved question#2021-04-2316:48Alex Miller (Clojure team)it depends somewhat on whether you see merge as an operation that produces an intersectional spec or whether it acts as parallel filters through which a value must pass (which is closer to how it's implemented)#2021-04-2316:49Alex WhittWouldn't the first option be better for generation?#2021-04-2316:49Alex Miller (Clojure team)certainly the latter seems doable#2021-04-2316:49borkdudeare the args to select
dynamic, i.e. can you merge those programmatically?#2021-04-2316:49Alex Miller (Clojure team)for generation, this is not that different than s/and in that you need to generate something that "passes" both specs but it's probably more challenging here#2021-04-2316:50AronI remember having serious issues with json-schema based validation because json-schema can be async, that is remotely resolved, and circular#2021-04-2316:50Alex Miller (Clojure team)in spec 2, all of the spec can be passed as data which makes it amenable to merging the specification as data#2021-04-2316:50AronSo merging there was only possible with serious limitations and strict conventions#2021-04-2316:53Alex Miller (Clojure team)there are a couple different ways data-based specs can be created in spec 2, so you actually have several choices - custom ops with s/defop (think spec template with holes), symbolic forms as data, and specs represented as maps. all of those are implemented now (with some caveats that the map form is kind of a hack in a few spots that are likely to change)#2021-04-2317:29Alex WhittI was wondering if selection merging would be a first-class feature, but it sounds like I'll probably want to do the composition at the data model DSL layer. That will generate our specs, schemas, and selections. The implementation will be a lot cleaner with spec 2.#2021-04-2920:37twashingRunning
cljs.spec.test.alpha/instrument
then cljs.spec.test.alpha/check
on fns w/ a namespaced keyword is producing the error: RangeError: Maximum call stack size exceeded
. This seems like a bug in Clojurescript.
https://clojure.atlassian.net/browse/CLJS-3023
https://clojure.atlassian.net/browse/CLJS-2995
But maybe Iām doing something wrong? Iāve tried on these lib pairs.
org.clojure/clojure {:mvn/version "1.10.3"}
org.clojure/clojurescript {:mvn/version "1.10.844"}
org.clojure/clojure {:mvn/version "1.9.0"}
org.clojure/clojurescript {:mvn/version "1.10.339"}
a.cljs
(s/fdef github-deployment->deployment
:args (s/cat :message (s/keys :req [::github.deployment/payload])))
(defn github-deployment->deployment [{payload ::github.deployment/payload :as message}]
)
(cljs.spec.test.alpha/instrument)
b.cljs
(cljs.spec.test.alpha/check
`backend.topology.processor.transform.deployment/github-deployment->deployment
{:clojure.spec.test.check/opts {:num-tests 1}})
[{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha20390], :clojure.test.check/ret {:result #object[RangeError RangeError: Maximum call stack size exceeded]}, :sym backend.topology.processor.transform.deployment/github-deployment->deployment, :failure #object[RangeError RangeError: Maximum call stack size exceeded]}]
#2021-05-0109:08lassemaattaI doubt I can help you solve this, but I assume you've verified that the generator for ::github.deployment/payload
works and produces values without encountering that call stack issue?#2021-05-0320:33twashingNo, thatās a great suggestion. I did check that, and I was generating the spec in question.
I did find the problem though. A given namespace has to handle messages, all keyed by a :payload
entry. When I put a namespace on itā¦ and place it in a file with other specs, it turned out that there was a separate :payload
somewhere in the spec tree.#2021-05-0415:41NoahTheDukei'm looking to write a spec that checks a list of tuples to see if there's 0 or 1 tuple that starts with some value#2021-05-0415:42NoahTheDukei have the spec that verifies and conforms that the shape is correct:
(s/def ::handler-clause (s/cat :name (s/nonconforming
(s/or :keyword keyword?
:class symbol?))
:arglist vector?
:body (s/* any?)))
#2021-05-0415:42NoahTheDukeand then in the fdef
i have (s/* (s/spec ::handler-clause))
for that argument#2021-05-0415:44NoahTheDukeinputs look like [:error [& args] (println args)]
or [:no-error [& args] args]
#2021-05-0415:44borkdude*
means 0 or more, not 0 or 1#2021-05-0415:44Alex Miller (Clojure team)?
is for 0 or 1#2021-05-0415:45NoahTheDukemy apologies, i mean, i'd like to have it check that there is any number of tuples that don't start with :no-error
and 0 or 1 tuples that start with :no-error
#2021-05-0415:49NoahTheDukeso something like
(s/fdef handler-case
:args (s/cat :expr any?
:bindings (s/and (s/? (s/cat :name (s/and #(= :no-error %)
(s/nonconforming
(s/or :keyword keyword?
:class symbol?)))
:arglist vector?
:body (s/* any?)))
(s/* (s/cat :name (s/and #(not= :no-error %)
(s/nonconforming
(s/or :keyword keyword?
:class symbol?)))
:arglist vector?
:body (s/* any?)))
)))
#2021-05-0515:49pinkfrogFor a new project, use spec or spec2? If I go with spec, will it be obsoleted sometime?#2021-05-0515:55emccuehttps://en.wikipedia.org/wiki/Osborne_effect#2021-05-0519:28dvingoI recently got into malli - https://github.com/metosin/malli can highly recommend if you're looking to invest in something - it's also quite simple to generate clojure.spec.alpha or clojure.alpha.spec forms from malli schemas if you ever do want to switch#2021-05-0521:24naomarikI've been using malli since before it had a version and it's made my life so easy#2021-05-2415:30andy.fingerhutIt seems unlikely that spec2 will obsolete spec1 in 2021.#2021-05-2415:31andy.fingerhutspec2 will not cause spec1 to change its behavior.#2021-05-0516:01Alex Miller (Clojure team)spec 2 is not ready for use yet, so spec#2021-05-0516:01Alex Miller (Clojure team)when spec 2 is ready, it will include help for migrating to it#2021-05-0517:53seancorfield@alexmiller Since this seems to be a āFrequently Asked Questionā about Spec 2, maybe you could put a big bold notice on the repo that says āEXPERIMENTAL: DO NOT USE YET!ā or something? š#2021-05-0518:35Alex Miller (Clojure team)It says "This library is a work in progress provided to gather feedback during development." and there are no releases of it so....#2021-05-0518:42emccueyeah but "v2 not ready yet" doesn't alleviate people's concerns about whether they should invest in spec 1#2021-05-0518:43Alex Miller (Clojure team)spec 1 is a thing, feel free to use it#2021-05-0518:44Alex Miller (Clojure team)the spec language between the two is substantially the same. part of releasing spec 2 will be providing a migration path from spec 1#2021-05-0618:38Aronthe word 'alpha' everywhere should give a hint#2021-05-0618:40seancorfieldTo be fair, both Spec 1 and Spec 2 have āalphaā all over them, but Spec 1 is solid and has been in heavy production use for a lot of people for years at this point. But the lack of an artifact to depend on for Spec 2 ā that should be a hint š#2021-05-0618:40seancorfieldmutter, mutter, git deps, mutter Yes, I know š#2021-05-0618:46borkduderemoving the .alpha suffix would break a lot of programs. is this ever going to happen? ;)#2021-05-0618:47borkdudeI could see a sudden swift of everyone moving to 1.11 if that did happen and that might be a good thing. Then we could also starting using the new kwargs stuff.#2021-05-0619:39seancorfield@borkdude My understanding is that Spec 1 will stay around as-is (with .alpha
) and Spec 2 will eventually become the ācoreā non-alpha version when it is ready.#2021-05-0619:39seancorfieldSo folks wonāt need to change their code, unless they want to move to Spec 2 (or start using Spec 2 alongside Spec 1 ā which is probably what weāll do).#2021-05-0619:52Arondamn, so alpha means good after all š#2021-05-0620:41andy.fingerhutKinda depends on who it is calling a thing alpha, and the thing.#2021-05-0619:59seancorfieldWeāve run alpha builds of Clojure itself in production since 2011 š#2021-05-0715:43NoahTheDukeis it possible to take a function that creates a string and use it in/as a generator? converting the function to a generator would be impractical, if not outright impossible#2021-05-0716:05Alex Miller (Clojure team)there is such a thing in test.chuck - generates string that match a regex#2021-05-0716:06Alex Miller (Clojure team)https://github.com/gfredericks/test.chuck#string-from-regex#2021-05-0716:23borkdudeRegal also has generators from regexes: https://github.com/lambdaisland/regal#use-with-specalpha#2021-05-1119:07Nolanapologies if this is documented somewhere, but i was recently downgrading from spec2 to spec1, and was surprised to find that s/keys
in spec1 will validate registry keys that are present in the value, but not present in the spec:
(require '[clojure.spec.alpha :as s)
(s/def ::a int?)
(s/def ::b even?)
(s/def ::c odd?)
(s/def ::ks1
(s/keys :req-un [::a ::b])) ; NOTE: no mention of `::c`
(s/explain ::ks1 {:a 1 :b 2 ::c 4})
; => 4 - failed: odd? in: [:user/c] at: [:user/c] spec: :user/c
#2021-05-1119:10Nolani tried using the latest version of spec1, and it seemed to still work this way. is this the intent? it seems like that runs contrary to the idea of openness that spec promotes. design-wise, it's almost certainly a bad idea to be using qualified keys that dont conform to their spec, but strictly from the spec perspective this seems counterintuitive.#2021-05-1119:11Nolani'm curious if others have run up against this or if it's a rare case in practice#2021-05-1119:19andy.fingerhutI believe this is by design. spec is for defining legal values for particular keys, no matter which maps they might appear in. I think that it is in support of openness of specs -- you can include any key in any map if you want.#2021-05-1119:22Nolani think i agreeāthat once you specify the set of legal values for a qualified key, any instance of that key (in any map) should always have a value that conforms to the specification, no matter where it is in your program. so design-wise, i'm completely with you. where i get hung up is mainly around the presence of the :req
and similar options of the s/keys
form, if the s/keys
op is going to validate every key that exists in the registry regardless of the keys you provide for those options#2021-05-1119:22localshredif true, what purpose does s/keys
provide? Seems like it's supposed to allow you to narrow your focus of key values that you want verified#2021-05-1119:22localshredand if it's also by design, why did spec2 change it?#2021-05-1119:24Nolanthat's why i ask about the intent versus the way its currently workingāit may be a bug in spec2, for example#2021-05-1119:24localshredfair. definitely not saying one way is right over another, just want to understand for sure#2021-05-1119:36Alex Miller (Clojure team)this is doc'ed in the s/keys docstring and should (afaik) be the same in spec 1 / 2#2021-05-1119:37Nolanah, i see now! apologies to have missed that#2021-05-1119:37Alex Miller (Clojure team)there have been some changes around kw spec references in spec 2 and its possible something was broken there#2021-05-1119:38Nolanyep, that makes sense. i just wanted to get a bit more clarity on things given the difference i was seeing. appreciate all the help @andy.fingerhut @alexmiller š#2021-05-1121:32Nolanapologies to dredge this back up, but i'm looking at my program and realizing that i'm redundantly validating values because s/keys
works this way. is there any way to avoid this? is there a recommended way to validate a subset of the keys in a map? it doesn't seem possible with spec1 without using select-keys
or otherwise filtering the map first#2021-05-1121:38seancorfield@nolan If you are using qualified keys, this is documented behavior and is intentional. Qualified keys are supposed to be āglobally uniqueā and therefore the Specs declared for them should also be globally applicable. Thatās by design.#2021-05-1121:39seancorfieldIn Spec 2, select
is about requiredness, theyāll still be validated if they are present.#2021-05-1121:45Nolanabsolutely! i agree completely with the design ethos of global uniqueness of the specification of qualified keys. it's really a matter of difficult-to-detect redundancy in the validation. for example, if i do an expensive validation using s/keys
upfront, then compute with that map, potentially adding keys to it, and later i want validate the new keysā`s/keys` is a deceptively dangerous choice for that use-case. now that i know this is the case, i can certainly rework things to account for the multiple validations, but it seems like it would be a common use-case to simply validate only certain keys in a map. or maybe i'm just not thinking about it the right way!#2021-05-1121:49seancorfieldThe latter, I think.#2021-05-1121:49seancorfieldIf you use unqualified keys, you only get validation of what is in s/keys
ā because theyāre considered to have a ālocal contextā.#2021-05-1121:50Nolani mainly find myself running up against it when trying to avoid unnecessary work. luckily we've got all kinds of scissors and scalpels for chopping these maps up in order to validate subsets, so that's i think what i'll need to do#2021-05-1121:51Nolanthanks for all the help and direction!#2021-05-1121:51seancorfieldWell, the Spec validation is all about the key names. s/keys
is more about requiredness and generation.#2021-05-1121:53Nolangot it. that's helpful. i'm going to approach this through that lens and see what comes out. thanks again @seancorfield!#2021-05-1215:07ElsoI am trying to make the best case for clojure.spec I can, and apart from instrumentation and generators, I am trying to see if there is spec-powered tooling which is worth investigating
thinking of something along these lines:
https://reposhub.com/python/miscellaneous/clj-kondo-inspector.html
in particular, any ways to integrate spec with cursive would be of great interest, as most folks on my job use it
are there things you would suggest to check out?#2021-05-1216:45dvingoI wanted to do data transformations from specs and down that path leads macro hell (see: https://github.com/dvingo/my-clj-utils/blob/master/src/main/dv/spec_util.clj#L36
with this helper I was told to just repeat the keys instead: https://clojurians-log.clojureverse.org/clojure-spec/2020-11-14/1605365585.379500)
After this interaction, I have found much delight in using malli. Being based on data it makes analysis and transformation fairly trivial; additionally it has clj-kondo support already built in:
https://github.com/metosin/malli#clj-kondo
perhaps this clj-kondo transform and the graphviz one, plus the others built in would help make the case for adoption on your team.#2021-05-1216:41seancorfield@d.eltzner012 maybe my blog post from 2019 will help provide some useful insights for you? https://corfield.org/blog/2019/09/13/using-spec/#2021-05-1218:28vlaaadWhy the difference in spec forms of anon fns?
(s/form (s/nilable #(= 1 %)))
=> (clojure.spec.alpha/nilable
(clojure.core/fn [%]
(clojure.core/= 1 %)))
(s/form (s/or :maybe-one (s/nilable #(= 1 %))))
=> (clojure.spec.alpha/or
:maybe-one
(clojure.spec.alpha/nilable
(fn* [p1__11188#]
(clojure.core/= 1 p1__11188#))))
#2021-05-1218:42Alex Miller (Clojure team)no good reason#2021-05-1218:43Alex Miller (Clojure team)probably just not applying the right cleanup function on the latter#2021-05-1514:01danierouxIs there a reason that clojure.test.check.generators/let
was left out of clojure.spec.gen.alpha/lazy-combinators
?#2021-05-1514:45Alex Miller (Clojure team)Itās a macro iirc#2021-05-1514:46Alex Miller (Clojure team)I think thereās actually a ticket about this#2021-05-1514:47Alex Miller (Clojure team)As a macro, you canāt do the same dynaload we do with the other combinators#2021-05-1514:50Alex Miller (Clojure team)https://ask.clojure.org/index.php/4633/let-ported-from-test-check-let-to-clojure-spec-gen#2021-05-1515:37danierouxThank you#2021-05-1516:22danierouxThis is what I could come up with, it feels unwieldly:
(try
(requiring-resolve 'clojure.test.check.generators/let)
(catch Exception _))
(s/def ::prodquint
(s/with-gen
(s/and string? #(re-matches pro-dquint-regex %))
(fn []
(clojure.test.check.generators/let
[pro-quint-hi (s/gen ::proquint)
pro-quint-lo (s/gen ::proquint)]
(str pro-quint-hi "-" pro-quint-lo)))))
#2021-05-1516:33gfredericksare you trying to produce code that still works if test.check isn't available? if so I can't see how your code accomplishes that#2021-05-1516:34gfredericksand if not, then I'd think you could just require let
in the more vanilla fashion and everything would be fine#2021-05-1516:49danierouxThat is what I am trying, and failing, to accomplish. I want the spec to be there with its generator attached to it. Knowing that the generator will only be used int dev/test.#2021-05-1516:51gfredericksif you want the let
call in the same spot as your spec then I think you basically end up needing eval
or something equivalent#2021-05-1516:52gfrederickswell maybe not#2021-05-1516:56gfredericksthis might work
(defmacro tclet
[& args]
(let [e (try
(requiring-resolve 'clojure.test.check.generators/let)
nil
(catch Exception e e))]
(if e
`(throw (Exception. ~(.getMessage e)))
`(clojure.test.check.generators/let
#2021-05-1517:00danierouxThank you @gfredericks!#2021-05-1517:00gfredericksdoes it work does it work?#2021-05-1517:08danierouxIt does!#2021-05-1517:09danieroux(turns out I had test.check on the prod classpath all along, now it breaks and works in expected ways)#2021-05-1816:27donavanIs there a way to do this:
(s/def ::bar #{:one :two :three})
(s/def ::foo
(s/with-gen
(s/keys :req [::bar])
#(gen/fmap
(fn [foo]
(assoc foo ::bar :one))
(s/gen ::foo))))
without having to create temporary specs like this due to the spec lookup loop
(s/def ::bar #{:one :two :three})
(s/def ::foo*
(s/keys :req [::bar]))
(s/def ::foo
(s/with-gen
::foo*
#(gen/fmap
(fn [foo]
(assoc foo ::bar :one))
(s/gen ::foo*))))
#2021-05-1816:33Alex Miller (Clojure team)You can gen recursive specs#2021-05-1816:33Alex Miller (Clojure team)Can you explain why you need with-gen?#2021-05-1816:35donavan::foo
is actually a multi-spec and only the one value of ::bar
is valid for this implementation#2021-05-1816:35donavanSo without tweaking the generator it fails to generate valid examples#2021-05-1816:36Alex Miller (Clojure team)Have you looked at the generator override map?#2021-05-1816:36donavanBy āYou can gen recursive specsā do you mean create the s/def
calls dynamically in a macro?#2021-05-1816:37Alex Miller (Clojure team)No, I mean you can have recursive specs and call gen on them#2021-05-1816:38Alex Miller (Clojure team)I think I did not get what you were trying to do#2021-05-1816:38donavanHaha, so it actually is both ultimately a multi-spec and a recursive one. The recursion āwithinā the generator works fine if I split the spec into the ::foo*
and ::foo
defs#2021-05-1816:39donavanItās just the call to s/gen
within the s/with-gen
that loops expectedly#2021-05-1816:36donavanItās being fed into another library that does the generating so I canāt supply the override map#2021-05-1816:40Alex Miller (Clojure team)Whatās the point of the initial gen if you just override :bar?#2021-05-1816:41Alex Miller (Clojure team)Arenāt you always generating the same map?#2021-05-1816:43Alex Miller (Clojure team)If so, just (s/gen #{ {::bar :one} })#2021-05-1816:43Alex Miller (Clojure team)Or gen/return if that is easier to read #2021-05-1816:43donavanIām not I follow exactly; the ::bar
spec is used in all the multi-specs itās just that some of the types itās values are restricted#2021-05-1816:43Alex Miller (Clojure team)Is this simplified and there are other keys too?#2021-05-1816:44donavanOh, yes sorry this is rather simplified!#2021-05-1816:44Alex Miller (Clojure team)You left out half the problem and half the constraints#2021-05-1816:45Alex Miller (Clojure team)It may not actually be in an inifinite loop - it may just be making very big intermediate colls#2021-05-1816:45donavanThe simple case above loops forever#2021-05-1816:45donavanApologies š I was just trying to ask the simplest question. I could mull it over some moreā¦#2021-05-1816:46Alex Miller (Clojure team)If you have any coll-of specs, make sure you set :gen-max 3 on them#2021-05-1816:46Alex Miller (Clojure team)There are also some spec dynvars controlling recursion, might want to check those too#2021-05-1816:47Alex Miller (Clojure team)All that said, there are some outstanding bugs in this area and itās inherently tricky#2021-05-1817:27donavanScratching a little deeper the loop Iām referring to is within the implementation, as spec is currently implemented you cannot define a generator āin terms of itselfā like I do above for any of the implementations such as map-spec-impl
. Itās deeply baked into the implementation so I suspect it wonāt change even though at first glance that seemed like a natural way to scratch my itch š .
Iāll re-evaluate what Iām trying to do, thanks @alexmiller#2021-05-1818:43Jeff Evansdoes it make sense to call instrument
from a clojure.test
fixture? https://clojuredocs.org/clojure.test/use-fixtures
ex: if we have tests for various namespaces, each of which should really instrument the specs as they run#2021-05-1818:43Alex Miller (Clojure team)yes#2021-05-1818:44Alex Miller (Clojure team)and unstrument at the end :)#2021-05-1819:55borkdude@jeffrey.wayne.evans I've also got an fdef-testing library here: https://github.com/borkdude/respeced#2021-05-1820:41Jeff Evansnice, thanks. will have a look#2021-05-1821:19Jeff Evanshow about this situation? I want b
to be an optional argument to my-fn
, but if provided, then it should match some predicate (like map?
)?
this doesnāt quite work.
(defn my-fn
[a & [b]]
;; do stuff with a and b
)
(s/fdef my-fn
:args (s/cat :a int? :b map?)
:ret map?)
#2021-05-1821:19Jeff Evans(my-fn 1)
Execution error - invalid arguments to user/my-fn at (form-init6149560983531223969.clj:1).
() - failed: Insufficient input at: [:b]
#2021-05-1821:20Alex Miller (Clojure team)you spec'ed it as a function taking 2 args (int and map)#2021-05-1821:21Alex Miller (Clojure team)you can use s/? to spec an optional part of a regex op#2021-05-1821:22Alex Miller (Clojure team)it's a little unclear to me if you meant [b] or b in the my-fn definition#2021-05-1821:23Alex Miller (Clojure team)I guess you mean what you said#2021-05-1821:23Alex Miller (Clojure team)and :args (s/cat :a int? :b (s/? map?))
should work for int and optional map#2021-05-1821:31Jeff Evansawesome. thank you, Alex!#2021-05-1902:11Joshua SuskaloAre there any libraries that have regex ops for lookahead?#2021-05-1902:12Joshua SuskaloI saw seqexp, but I'd prefer if it were as a part of spec#2021-05-1902:25Alex Miller (Clojure team)Can you give an example of where you would want that?#2021-05-1902:33Joshua SuskaloA gross overapplication of spec to a usecase it wasn't designed for: parsing a sequence of tokens output by another process#2021-05-2218:21Adam HelinsHey!
I have been doing a lot of work involving generating recursive data. This is a space where actually, from what I see, neither Spec nor Malli have a working solution. Any trivial example is already too much and generation gets completely out of hand. However I haven't used Spec in a while so I might be mistaken. I'd appreciate your thoughts and comments, especially if you have encountered that kind of problems.
Albeit extremely simple, with a pretty standard size of 30, generating a single instance of this example results commonly in 1000-3000 leaves (nested vectors containing that much booleans):
(s/def ::bool
boolean?)
(s/def ::vec
(s/coll-of ::data))
(s/def ::data
(s/or :bool ::bool
:vec ::vec))
There is an "exponential explosion" so that such definitions to not reflect the size linearly but rather exponentially. I tried explaining it thoroughly here and how test.check
solves it with recursive-gen
(the fact it is a Malli issue is irrelevant): https://github.com/metosin/malli/issues/452
So, surprisingly, it is somewhat impossible to generate truly random Clojure values where collection items can be collections as well. Unless I am of course missing something, so here I am :)#2021-05-2415:45Joshua SuskaloPersonally I recommend writing two generation functions, one that takes a generation depth and generates a full data structure to that depth, and another that takes a generation depth and generates a "narrow" structure to the given depth. This way you can test with the first one to a low depth to see "wide" structure generation, and use a much higher generation depth for the narrow one, using a weighted rng to choose which one to generate when using a with-gen form in the spec.#2021-05-2415:45Joshua Suskalo@adam678#2021-05-2415:48Joshua SuskaloOr as you've noted, recursive-gen already provides a reasonable solution, in that case you can also use that with spec's facilities to wrap something with a custom generator.#2021-05-2416:01Joshua Suskalo(require '[clojure.test.check.generators :as gen])
(s/def ::bool
boolean?)
(def data-gen (gen/recursive-gen gen/vector gen/boolean))
(s/def ::vec
(s/with-gen
(s/coll-of ::data)
(gen/vector data-gen)))
(s/def ::data
(s/with-gen
(s/or :bool ::bool
:vec ::vec)
data-gen))
#2021-05-2416:10Joshua SuskaloFor reference, the clojure spec generators are just a reexport of the functions in clojure.test.check.generators, which means that you can use them interchangeably.#2021-05-2416:24Alex Miller (Clojure team)well, almost interachangeably - most of the spec apis actually take a no-arg function that returns a generator instead of a generator (to allow for dynamic require/invoke)#2021-05-2515:42lassemaattaI remember wondering why e.g. with-gen
accepts a "generator returning function" instead of the generator itself; can you clarify what you mean by that "dynamic require/invoke" functionality? Something to do with requiring additional namespaces within the function but only if the generator is actually invoked (e.g. in generative testing)?#2021-05-2515:44Alex Miller (Clojure team)yes, that#2021-05-2515:45Alex Miller (Clojure team)by putting it in a thunk, we can avoid dynaloading the test.check.generators ns#2021-05-2515:46lassemaattaI see, thank you#2021-05-2416:31Joshua SuskaloOh that's good to know. Thanks for the heads up on that.#2021-05-2520:05Adam HelinsThanks @suskeyhose, so as I expected it is best writing that kind of generation from scratch#2021-05-2520:07Joshua SuskaloYes, and I would say that custom generators aren't something to shy away from either. Most structures can be made with just the base ones, but email addresses or other data that has many constraints upon what is valid should have custom generators.#2021-06-0318:04Braden ShepherdsonI'm struggling with nesting s/cat
calls.
I've got a tuple-like thing, but it's not implemented by a real vector, rather by a deftype. so I originally wrote the spec for it with s/tuple
but it failed to match. I switched to s/cat
with the parts spelled out, and that works.#2021-06-0318:05Braden Shepherdsonbut then in a (s/fdef foo :args (s/cat :x ::tuple-like :y int?))
it fails, since the cat
s end up running together.#2021-06-0318:06Braden ShepherdsonI guess I can always write the ::tuple-like
predicate as an #(instance? TheDeftype %)
but that seems really clunky.#2021-06-0318:51sgepigonTry wrapping it in a s/spec
to prevent them running together: (s/cat :x (s/spec ::tuple-like) :y int?)
. Spec 1 does this with s/spec
, spec 2 will do this with https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nested-regex-contexts.#2021-06-0421:13Frank HenardHow do I do an ::Anything
spec in Clojurescript? In Clojure I have (s/def ::Anything #(instance? Object %))
#2021-06-0421:14borkdude@ballpark any?
#2021-06-0421:15borkdudenote that your ::Anything
spec will not tolerate nil
#2021-06-0421:16Frank HenardThanks, Yeah, I'll have to do a parity match on those#2021-06-0421:17borkdudejust to be clear: the answer in both dialects is any?
.#2021-06-0421:22Frank Henardright, thanks, here's what I changed it to for my cljc file: (s/def ::Anything #(s/and some? any?))
This way I can specify s/nilable
in the specs that reference ::Anything
. Seem like an ok solution to you?#2021-06-0421:24borkdudeit seems a little weird to me. why not just use some?
and any?
directly?#2021-06-0421:25Frank HenardYes, that's it! Something seemed wrong to me too, thanks again!#2021-06-0423:59Joshua Suskalos/and with any? as an item is exactly equivalent to just the rest of the items#2021-06-0716:31Franco GasperinoHaving watched the 2018 talk on monads (Maybe Not) and appreciating the case made for place oriented programming using maps and spec, the question i have - Does this lead to design of most non-pure functions accepting and returning maps (almost) exclusively?#2021-06-0716:45delaguardoWhat makes you think that?#2021-06-0717:31Franco Gasperinocomposing, avoiding imperative style, while still supporting error context#2021-06-0716:48Alex Miller (Clojure team)in Clojure 1.11, you can pass a trailing map to a function that takes kwargs, so kwarg style also accepts map invocation#2021-06-0716:49Alex Miller (Clojure team)positional args are a tradeoff - they make invocation easier by making the arg name implicit#2021-06-0716:49Alex Miller (Clojure team)but that also means they are fragile to positional changes, whereas maps / kwargs are not#2021-06-0721:19emccueIf there wasn't a perf hit for it, I could very much see myself having all args named Objective C method call style#2021-06-0721:20emccuei guess in practice thats all maps#2021-06-0915:37MatthewLispHello
Does somebody happen to have a ready spec for JWT (Json web token) ?#2021-06-0918:48emccuehttps://datatracker.ietf.org/doc/html/rfc7519#2021-06-1017:52mathias_dwhi, i have a big collection of specced data that went into mongodb and lost all the namespace info. I suppose I can transform it all by recursively using the information in (s/describe) of the top-level (big) data structure. But before doing that, I thought I'd ask if something like that already exists. Anything similar? Just to complicate it a little: there are also plenty of non-namespaced and non-specced keys that got in there as well. They shouldnt be given a namespace.#2021-06-1105:55delaguardoMongodb is storing data as json. So you probably need to have some sort of convention how to encode qualified keywords to/from json#2021-06-1114:33mathias_dwThanks for you reply! Yeah, i realize that now, but the problem is that a lot of production data was stored without the namespace info before I noticed, and it's not something i can re-do. Anyway, the approach with using s/describe also doesnt work, because of things like merge etc. So guess I'll have to write custom code for the exact data structure.#2021-06-1117:41vlaaada bit surprising:
(s/valid? (s/map-of keyword? int? :kind vector?) [[:a 1]])
=> true
#2021-06-1117:45Alex Miller (Clojure team)what is that even supposed to mean?#2021-06-1117:46Alex Miller (Clojure team)seems like it's doing exactly what the docstring says to me#2021-06-1117:50vlaaadI don't know what that's supposed to mean, I'm trying to build a tool on top of spec and I have to support what it supports#2021-06-1117:53vlaaadI'll probably ignore this particular thing... I just was trying to understand what kind of data shapes various variations of s/every
support.#2021-06-1117:54borkdudespec needs a spec? ;)#2021-06-1117:55vlaaadless surprising:
(s/valid? (s/every-kv keyword? int?) #{[:a 1]})
=> true
every-kv
docstring states it works on associative collections...#2021-06-1117:56borkdudewhat kind of tool are you building, or is it secret?#2021-06-1117:56vlaaad"spec forms"#2021-06-1117:57borkdudesomething cljfx related?#2021-06-1117:58vlaaadIt's a reveal plugin that builds a UI form for a given spec. Then you can fill the form to get back the data in the shape that satisfies the spec.#2021-06-1118:00vlaaadhttps://twitter.com/v1aaad/status/1390689740308074498?s=19#2021-06-1118:01vlaaad...and the form resembles the data#2021-06-1118:02borkdudeso it's way to visualize a spec form?#2021-06-1118:06vlaaadMore than that. You can bind changes in form to custom code and custom views. For example, if you have spec for vega/vega-lite data format, you can have a UI form for creating vega chart input and a vega chart itself that updates on changes to the form. This allows learning and experimenting with data-driven APIs interactively#2021-06-1118:06Alex Miller (Clojure team)every-kv is really about "seqs to map entries", but we have no way to say that in Clojure (yet)#2021-06-1118:07Alex Miller (Clojure team)Rich and I have been talking about that gap for years, some day we'll figure it out :)#2021-06-1118:08borkdudewhat's the 2-version of triple?#2021-06-1118:08Alex Miller (Clojure team)duple?#2021-06-1118:08vlaaadmap-entry?#2021-06-1118:08Alex Miller (Clojure team)pair is commonly used though :)#2021-06-1118:08borkdudeduple-blob#2021-06-1118:09borkdudeduple-flock#2021-06-1118:09borkdudesolved? ;)#2021-06-1118:09Alex Miller (Clojure team)the "seqs to" part is the hard part :)#2021-06-1118:10vlaaadbtw I think every-kv is fine, I'd expect it to accept any coll#2021-06-1118:10vlaaadbut maybe the docstring shouldn't mention associative...#2021-06-1118:11Alex Miller (Clojure team)I think that may be vestigial from earlier impls#2021-06-1416:42bortexzHi, I know spec2 is still in alpha/experimental phase and buggy. Still, I have been playing around with it to model the domain of an experimental project I have, where I want to test out the capabilities of domain modelling with schema/select. Is it possible to have a multi-spec return a schema, and do select on the multi-spec? It doesnāt seem to work on current git-sha, I wonder if itās a bug or is it intended that this is not possible:
(spec/def :account/id string?)
(spec/def :service/id string?)
(spec/def :service-1/field string?)
(spec/def :service-2/field string?)
(defmulti account-schema :service/id)
(defmethod account-schema :service-1
[_]
(spec/schema [:account/id :service-1/field]))
(defmethod account-schema :service-2
[_]
(spec/schema [:account/id :service-2/field]))
(spec/def :account/account (spec/multi-spec account-schema :service/id))
(gen/generate (spec/gen :account/account))
(spec/valid? (spec/select :account/account [:account/id :service-1/field :service/id])
{:service/id :service-1
:account/id "id#1"
:service-1/field "abcd"})
The generator works correctly, but not the validation, that fails with:
; Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval5667$fn$G (protocols.clj:20).
; No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.alpha.spec.impl$multi_spec_impl$reify__7055
#2021-06-1417:09Alex Miller (Clojure team)undecided stuff in this area about what can act as a schema provider#2021-06-1417:09Alex Miller (Clojure team)if you have a use case where this would be useful, would be helpful to have an https://ask.clojure.org question with request
tag#2021-06-1417:12bortexzThanks Alex, I will open the question with a more detailed use case#2021-06-1521:19bortexz^ Opened the question for this https://ask.clojure.org/index.php/10696/spec2-use-case-for-multi-spec-as-schema-provider#2021-06-1716:57jcdOther than the official docs, does anyone have any recommendations for familiarizing oneself with using spec?#2021-06-1717:29Alex Miller (Clojure team)if the official docs are lacking, would be interested in how so they can be improved#2021-06-1719:47jcdI wouldnāt say they are lackingā¦ but Iām curious if there are talks or more introductory guides people would recommend. The official docs are good, but somewhat intimidating since their scope is comprehensive.#2021-06-1720:00borkdude@livingfossil on the official Clojure site there is a "guide" and a "reference", the guide is usually an introduction and the reference is more encompassing (hope I said that right)#2021-06-1720:09jcdaha! thanks!#2021-06-1720:22jkxyzāAll-encompassingā is the usual phrase meaning ācomprehensiveā but the meaning was clear š¤ #2021-06-1722:13Alex Miller (Clojure team)you could just read random parts of it and then it would be less comprehensive :)#2021-06-1722:13Alex Miller (Clojure team)there are a few talks out there if that would match better to your learning style#2021-06-1722:14Alex Miller (Clojure team)https://www.youtube.com/watch?v=tEWSw8H9KJU#2021-06-1800:43JP Silvahi folks, I'm curious about the current status of https://github.com/clojure/spec-alpha2#2021-06-1800:44seancorfield@jplfdsilva Not production ready. Still being designed. Has bugs.#2021-06-1800:45seancorfieldIn particular, thereās a bunch of design work going on behind the scenes around how fdef
should work/be integrated with defn
etc.#2021-06-1800:45JP Silva@seancorfield thanks for the update#2021-06-1800:45seancorfieldItāll be a big improvement over Spec (1) when it is finalized thoughā¦#2021-06-1800:47JP Silvaagreed. I just finished watching the Maybe Not talk (again) and was wondering when the new select spec op would be available.#2021-06-1800:47seancorfieldāsoonā š#2021-06-1800:48seancorfield(which, in Clojure terms, could be a while!)#2021-06-1800:49JP Silvait's an amazing idea I hope other languages would follow#2021-06-2015:10hkjelsI have a collection where I want an identifier for each entity in that collection to be unique. Can that be specified?#2021-06-2015:10hkjelsI know of distinct
on coll-of
, but that just enures that each entity is unique AFAICS#2021-06-2016:00Alex Miller (Clojure team)If you can write a predicate function, you can make that a spec#2021-06-2016:00Alex Miller (Clojure team)Nothing for this built in#2021-06-2108:09Jakub HolĆ½ (HolyJak)Hi! When was work on Spec 2 started? I guess late 2018? Thank you!#2021-06-2111:48Alex Miller (Clojure team)I donāt know, you can check the repo#2021-06-2119:37Jakub HolĆ½ (HolyJak)I checked the repo but it seems like the older commits are still Spec 1 commits, there is not lear (to me) transition between v1 and v2.
I will just assume that it was around the Maybe Not talk š#2021-06-2113:30borkdudeMaybe Not could be seen as a milestone when Rich declared that spec1 contained some design choices that were not optimal and he wants to address in spec2. This was in November 2018.#2021-06-2113:33borkdudeStill an enjoyable talk btw#2021-06-2218:40Michael Stokleyran into a surprise with s/or
just now#2021-06-2218:40Michael Stokley;; `s/or` destructures the value that the second pred sees
(let [spec (s/and (s/or :even even? :small #(> 10 %))
#(= 5 %))]
(s/explain-str spec 5))
;; => "[:small 5] - failed: (= 5 %)\n"
;; flipped order of arguments to `s/and
(let [spec (s/and #(= 5 %)
(s/or :even even? :small #(> 10 %)))]
(s/explain-str spec 5))
;; => "Success!\n"
#2021-06-2218:41Michael Stokleyis there a way to avoid this?#2021-06-2218:44Michael Stokleyah, i guess it's in the docstring of s/and
#2021-06-2218:44sgepigonI believe itās more of s/and
ās behavior as it āflowsā values through (see https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new) You can try https://clojuredocs.org/clojure.spec.alpha/nonconforming#2021-06-2218:55Alex Miller (Clojure team)spec 2 has a non-flowing s/and variant to cover this#2021-06-2419:12UsmanHi all, How would I go about creating random URLs that begin with https://
using spec? I'm new to clojure and spec so any support/explanations will be much appreciated. :)#2021-06-2419:49seancorfield@usman.jamil I'd probably use a string regex and gfrederick's test.chuck
lib which has a generator for regexes.#2021-06-2419:49Alex Miller (Clojure team)https://github.com/gfredericks/test.chuck#string-from-regex#2021-06-2419:49Alex Miller (Clojure team)and ditto#2021-06-2419:52UsmanThanks. I will have a look at that.#2021-07-0912:19Jim NewtonI am not a spec expert, but my application has a spec interface which accepts specs and deals with them. I have some tests that are failing with randomly generated data. I'd like advise about how to prepare my tests.
(s/def ::test-spec-1 (s/* (s/alt :1 (s/cat :3 neg? :4 even?)
:2 (s/cat :5 odd? :6 pos?))))
I have a spec defined as above, ::test-spec-1
, when this is applied to a sequence containing a floating point number, rather than the spec telling me that it fails to match, instead the even?
function raises an exception
ERROR in (t-canonicalize-spec-type) (core.clj:1391)
Uncaught exception, not in assertion.
expected: nil
actual: java.lang.IllegalArgumentException: Argument must be an integer: 1.0
at clojure.core$even_QMARK_.invokeStatic (core.clj:1391)
#2021-07-0912:20Jim Newtondo I need to write my own functions int-and-odd?
, int-add-pos?
, etc? or is there a more idiomatic way of handling this ?#2021-07-0912:26delaguardoinstead of even?
you can use (every-pred int? even?)
#2021-07-0912:26delaguardosame for odd?
#2021-07-0912:27Jim Newton(s/def ::test-spec-1 (s/* (s/alt :1 (s/cat :3 (every-pred int? neg?) :4 (every-pred int? even?)
:2 (s/cat :5 (every-pred int? odd?) :6 (every-pred int? pos?)))))
#2021-07-0912:27Jim Newtonis this what you mean?#2021-07-0912:28delaguardo(every-pred int? neg?)
this is unnecessary because neg?
is expecting number#2021-07-0912:29delaguardobut in general - yes#2021-07-0912:29Jim Newtonahhh, yes indeed that's correct#2021-07-1213:06Ian Fernandezhow can I select-keys
giving a spec that has required keys?#2021-07-1213:06Ian FernandezI only want to pick the keys in a map that has in this spec#2021-07-1213:15vlaaadAre you trying to write at function that, given a spec, will return a function that, given a map will strip all keys not mentioned in spec?#2021-07-1213:16vlaaadIād suggest looking at s/form
to get a form of a spec that mentions those keys#2021-07-1213:35nbardiukwe are using https://cljdoc.org/d/metosin/spec-tools/0.10.5/api/spec-tools.core#select-spec to drop keys that are not in spec#2021-07-1213:44Ian Fernandez(s/def ::my-one (s/keys :req [::a ::b]))
#2021-07-1213:45Ian Fernandezhow can I make a
::my-two
that has ::my-one
keys and a key ::c
too?#2021-07-1214:20colinkahnSee s/merge
https://clojuredocs.org/clojure.spec.alpha/merge#2021-07-1411:16mathias_dwHi, I'm sure it's been asked before, but googling didn't turn anything up: is there an easy way to associate comments to specs? My usecase is working together with some non-clojure (actually, non-programmer) people on a data definition. I'm used to spec and all the related tooling, but I can't expect the others to really understand everything from code.#2021-07-1412:12mpenetit's not supported out of the box. you need to build your own metadata api for specs right now if you need that.#2021-07-1412:13mpenetit's quite easy to do tho, basically an atom and a few functions that will work against it. We have something like that internally, we can do (-> (s/def ::foo string?) (x/vary-meta! assoc :doc "some docstring"))#2021-07-1412:13mpenets/def returns the spec key, you can take advantage of this. and from there writing x/with-doc is just a small fn#2021-07-1414:01Alex Miller (Clojure team)this is the most requested feature for spec and I expect it will be added in spec 2#2021-07-1922:04dgb23How would I go about writing a spec for a data structure that detects (disallows) cycles?
Context: As a fun learning project Iāve been writing a state machine for a lexer, as a map from states->inputs->[outputs/->states] (that last part is the transition).
Now as is typical for these, some of the transitions are similar on different states (here because of lexemes). Instead of composing the map programmatically I thought I invent an additional behaviour and add a :with
key on the transition, to say that the output is combined with the output of another transition. This way the state machine is fully described as a plain data-structure. :with
merely points to
The terrible thing here is that I introduced the possibility for recursion/infinite loops. So my intuition is that I can write a spec that detects cycles, but I cannot really wrap my head around it. Is this possible?
My context is just an example, maybe the design is wrong, but I would be generally interested in applying spec this way and how one would go about it.#2021-07-1922:08Alex Miller (Clojure team)I think I would probably not write such a spec#2021-07-1922:09Alex Miller (Clojure team)but a spec can be any predicate function#2021-07-1922:09Alex Miller (Clojure team)so if you can write a function (no-cycles? data-structure), then that's a valid spec#2021-07-1922:10dgb23right I was getting stuck somehow in trying to make the spec both check the structure and detect the cycle at the same time#2021-07-1922:10dgb23but there is s/and
ā¦#2021-07-1922:10Alex Miller (Clojure team)(s/and ::structure no-cycles?)#2021-07-1922:10dgb23right ty š#2021-07-1922:11dgb23you think such a spec is a smell?#2021-07-1922:11Alex Miller (Clojure team)I just don't think you're getting any value out of using spec to write that :)#2021-07-1922:12Alex Miller (Clojure team)spec's sweet spot is describing information structures (maps of attributes) or syntactic structures (sequential regex type things). the more generic or abstract you get from that, the less value you get#2021-07-1922:15dgb23I get what youāre saying. But say I used spec during development time with instrumentation I would still get value from this right?#2021-07-1922:19Alex Miller (Clojure team)hard for me to say#2021-07-1922:19Alex Miller (Clojure team)keep in mind that slow specs can really slow down execution. cycle checking sounds slow. :)#2021-07-1922:20Alex Miller (Clojure team)would you be better off preventing a graph with a cycle from being created in the first place?#2021-07-1922:21dgb23I just started reading a book that thematises this. I have to think about this more deeply! ty#2021-07-2221:55vemvlong time no spec. how do I un-nilify a nilable again? spec-tools maybe?#2021-07-2222:16colinkahnWould (s/and ::nilable-spec some?)
be what you want?#2021-07-2222:18vemvyes that would work! I was using some other workaround in the meantime but your snippet is more concise#2021-07-2221:56vemv(spec1)#2021-07-3021:03isakIf you are using cat
, is it possible to somehow spec on all remaining elements, instead of just the next element in the sequence?#2021-07-3021:10seancorfieldWith *
perhaps? (not quite sure what you're asking @isak)#2021-07-3021:12isakI'm trying to do something like this: (s/cat :a ::number :b ::number :rest ::my-spec)
, where ::my-spec
should run on all remaining items in the sequence, not just the very next one. For example for the sequence [1 2 3 4], it should run on [3 4], not just 3.#2021-07-3021:15seancorfieldLike so?
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2])
true
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2 3])
false
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo"])
true
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo" "bar"])
true
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2])
{:a 1, :b 2}
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2 3])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo"])
{:a 1, :b 2, :c ["foo"]}
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo" "bar"])
{:a 1, :b 2, :c ["foo" "bar"]}
#2021-07-3021:16isakYea like that, except I want to be able to run a spec on the whole collection that gets gathered#2021-07-3021:17seancorfieldI don't understand.#2021-07-3021:18seancorfieldDo you mean that you want ::my-spec
to be another s/cat
-based spec?#2021-07-3021:18isakWell with (s/* string?
), it is saying what should be true about the individual elements, but what if I needed something to be true for the whole collection?#2021-07-3021:19seancorfieldUse s/cat
there.#2021-07-3021:19isakOh hmm#2021-07-3021:21seancorfielddev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2])
false
dev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 3])
false
dev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo"])
true
dev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo" "bar"])
true
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 3])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo"])
{:a 1, :b 2, :c {:d "foo"}}
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo" "bar"])
{:a 1, :b 2, :c {:d "foo", :e "bar"}}
#2021-07-3021:22seancorfieldThe sequence regex specs "unroll" or merge into one long sequence, so you can use them to describe subsequences inside another sequence.#2021-07-3021:23seancorfieldI'm showing the Spec forms inline, but that Spec for :c
(or :rest
in your case) could be ::my-spec
.#2021-07-3021:25isakHm, but what if I don't have a condition about the individual elements? For example, if I needed to match a number, a number, then all the remaining elements have to be in a collection that is odd?#2021-07-3021:26isakFor example, [1 2 3 4 5] would pass, but not [1 2 3 4] (contrived example, but gets at what I'm trying to do)
(And I can only do a cat
on the first 2 elements)#2021-07-3021:27isakI would try this but it doesn't really make sense:
(s/def ::odd-collection (s/and seqable? #(odd? (count %))))
(s/conform
(s/cat :a int? :b int? :rest (s/cat ::odd-collection ))
[1 2 3 4 5])
#2021-07-3021:27seancorfield"a collection that is odd" -- what do you mean? Can you express that as a predicate?#2021-07-3021:28isakSorry I meant the count should be odd#2021-07-3021:29isakOk this works:
(s/def ::odd-collection (s/and seqable? #(odd? (count %))))
(s/conform
(s/cat :a int? :b int? :rest (s/& (s/* any?) ::odd-collection))
[1 2 3 4 5 ])
#2021-07-3021:29isakNot sure if there is a better way#2021-07-3021:31seancorfieldThat was what I was about to suggest: s/&
is the "and" of sequence regexes!
dev=> (s/conform (s/cat :a number? :b number? :c (s/& (s/* number?) (comp odd? count))) [1 2 3 4])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/& (s/* number?) (comp odd? count))) [1 2 3])
{:a 1, :b 2, :c [3]}
dev=> (s/conform (s/cat :a number? :b number? :c (s/& (s/* number?) (comp odd? count))) [1 2 3 4 5])
{:a 1, :b 2, :c [3 4 5]}
#2021-07-3021:32isakGotcha, thanks @seancorfield#2021-08-0422:46Franco GasperinoI'm looking for the preferred route on the following..
Requirement:
* Read JSON messages from external streaming source.
* Perform message schema validation.
Search for required keys and data pairs.
* Conform message and filter invalid messages.
Question:
* Spec definitions are centered around keywords, not strings.
* Perform a recursive keywordize-keys call on original string keys?
* Alternative spec definition using strings instead of keywords?#2021-08-0423:26Franco Gasperinoit appears keys -> keywords model is at least optionally suggested in a couple libraries, such as cheshire and clojure/data.json#2021-08-0423:43seancorfieldYou need to be a bit careful about just converting all JSON input to keyword-based hash maps since a malicious user could bombard your server with random JSON with long, unique strings and potentially cause performance/heap problems for you.#2021-08-0423:45seancorfield(that said, I think a lot of people do simply read the JSON as keyword-based hash maps)#2021-08-0423:46seancorfieldWe no longer have the problem of old where keywords were interned and never GC'd so it's not as dangerous as it used to be š#2021-08-0423:48vemvIs there a known recipe for avoiding security pitfalls when using spec at the edges of a production app?
Other than "don't" :)#2021-08-0423:49seancorfieldAt work, our APIs are all behind authentication so we can "trust" the input to some degree and we do read JSON to keyword-based hash maps and then validate it with Spec. You could write a Spec that just validated the top-level keys as strings -- using a set for the valid keys, but if you're accepting values that can also be structured data that will get a bit gnarly.#2021-08-0423:50seancorfieldAs for security pitfalls with Spec being used on arbitrary data, I think you mostly need to ensure that validation can't be sent into a deep CPU hole because your specs allow arbitrary nesting and structure (again, the malicious user and the large payload issue).#2021-08-0423:54vemvsounds like a hard-to-write meta-spec :) I'm open to everything though
it came to mind just now, maybe one create a simple translation layer from spec to malli which is more performant / fit for the use case. Obviously only a subset could be translated
there's the precedent of https://github.com/threatgrid/flanders which is a unified DSL that can spit out Spec and Plumatic Schema alike#2021-08-0500:16dgb23Instead of making parsing more performant (which is certainly a good thing) one can treat payload size and request frequency as separate problems and simply reject stuff that goes above a certain threshold respectively. In common, simple cases itās enough to have sensible heuristics for each of those. At scale one likely needs statistical models. In other words, spec is concerned with analysing stuff thatās already read (parsed), so I would assume you put those kind of checks before it.#2021-08-0500:17emccue@franco.gasperino
(mu/closed-schema [:map ["thing" :string] ["other" [:map ["thing" :int]]]])
#2021-08-0500:18emccueyou can join the malli dark side#2021-08-0500:18dgb23I talked a bit with a googler who was working on systems that detect anomal request patterns on the edges a couple of years ago, was quite fascinating.#2021-08-0500:23dgb23If a stranger keeps loudly knocking on your door with a big hammer, you donāt ask for their credentials, just call the police.#2021-08-0500:25vemvOTOH rolling out a ML-powered infrastructure sec thingy sounds like a 10x harder problem
Still one worth solving though#2021-08-0500:27dgb23I shouldnāt have started to think about web security before going to bedā¦#2021-08-0519:32vlaaad(let [s (s/cat :cat? (s/? (s/cat :int int?)))]
(s/unform s (s/conform s [1])))
=> ((1))
should be (1)
#2021-08-0520:46colinkahnI think itās this issue: https://clojure.atlassian.net/browse/CLJ-2003#2021-08-0520:59Franco Gasperino@seancorfield @vemv @denis.baudinot Thanks for the discussion.#2021-08-1016:07dev-hartmannHey folks, ist there a way to parse a JSON schema to spec at runtime?#2021-08-1219:27souenzzoIf is not this lib, is another metosin lib
https://github.com/metosin/spec-tools#2021-08-1610:41dev-hartmannThx!#2021-08-1519:28vlaaads/* wrapping s/alt conform gives a vector of tuples (expected):
(s/conform (s/* (s/alt :strs (s/+ string?)
:ints (s/+ int?)))
["a" "x" 1 2])
=> [[:strs
["a" "x"]]
[:ints
[1 2]]]
s/+ wrapping s/alt conform gives a vector with tuple and a vector of tuples (unexpected):
(s/conform (s/+ (s/alt :strs (s/+ string?)
:ints (s/+ int?)))
["a" "x" 1 2])
=> [[:strs
["a" "x"]]
[[:ints
[1 2]]]]
#2021-08-1519:43dgb23so weird when you look at:
(s/conform (s/+ (s/alt :strs (s/+ string?)
:ints (s/+ int?)))
["a" "x" 1 2 "b" "c" 4 5])
#2021-08-1520:05vlaaadfixed formatting#2021-08-1521:28Alex Miller (Clojure team)I'm not seeing what's unexpected here, can you describe?#2021-08-1605:02vlaaadSecond element of second result unnecessary wrapped in a vector#2021-08-1521:06vlaaadgiven (s/def ::foo (s/* any?))
, shouldn't (s/regex? ::foo)
also return true?#2021-08-1521:26seancorfieldThat operates on the (internal) spec form I believe, not on spec names.#2021-08-1521:29seancorfieldUse (s/regex? (s/get-spec ::foo))
@vlaaad#2021-08-1521:30seancorfielddev=> (s/def ::bar string?)
:dev/bar
dev=> (s/def ::foo (s/* any?))
:dev/foo
dev=> (s/regex? (s/get-spec ::bar))
nil
dev=> (s/regex? (s/get-spec ::foo))
{:clojure.spec.alpha/op :clojure.spec.alpha/rep, :p2 #object[clojure.core$any_QMARK_ 0x18d396eb "
#2021-08-1605:01vlaaadI know how it operates, I just find it limiting#2021-08-1605:14vlaaadBecause to check if something is a regex I have to do (s/regex? (or (s/get-spec spec) spec)) instead of just (s/regex? spec)#2021-08-1606:39vlaaad(s/form (s/keys* :req-un [::foo]))
=> (clojure.spec.alpha/&
(clojure.spec.alpha/*
(clojure.spec.alpha/cat
:clojure.spec.alpha/k
clojure.core/keyword?
:clojure.spec.alpha/v
clojure.core/any?))
:clojure.spec.alpha/kvs->map
mspec__2546__auto__)
that doesn't look right...#2021-08-1607:01dgb23Youāre looking at an expanded macro.#2021-08-1607:21vlaaads/form for other specs, that are also implemented as macros, return the form used in code#2021-08-1608:41dgb23right!#2021-08-1612:45Alex Miller (Clojure team)This is a known issue thatās addressed in spec 2#2021-08-1723:23richiardiandreaHi there, I have a burning question š
Say I have a collection of maps and I want to spec them. There a lot of maps in this collection but I want to inspect and solve errors one by one.
Is there a way to say to st/instrument
or s/explain
to limit the number of errors outputted for that collection?#2021-08-1817:23mishayou can use s/expalain-data
and trim output errors as you wish:
(-> (s/explain-data (s/coll-of even?) (range 100))
(update ::s/problems #(take 10 %)))
(you probably will need to trim ::s/value as well)
Or, you can validate each map individually, not as a collection:
(->> (range 100)
(keep #(s/explain-data even? %))
(take 3))
#2021-08-2016:29meta-metaHi, how do I write a spec to validate values in a map depending on the key? Say I have a map {:foo 42 :bar [1 2 3]}
I want to validate that if :foo
is defined, it's an int
and if :bar
is defined, it's a coll
.#2021-08-2016:41Russell Mull(s/def ::foo int?)
(s/def ::bar (s/coll-of int?))
(s/def ::mymap (s/keys :req-un [::foo ::bar]))
#2021-08-2016:52meta-metathank you! And now I see it's right there in the first part of the spec guide. I was looking at map-of and every, totally forgetting what I read the other day.#2021-08-2304:15vemvGiven an fdef
, can I get its source code back? Like clojure.repl/source
does with vanilla def
s
(my guess is that nope, but maybe someone wrote a thing?)#2021-08-2310:19reefersleepMaybe create a macro that uses a central registry to keep track of the source of each fdef, as per its fully evaluated name.#2021-08-2310:19reefersleepNot cool enough with macros to write the example, but itād be something like, swap!
ing into your central atom, where you keep a map of fdef-symbol -> unevaluated source, and then of course evaluating your fdef normally afterwards.#2021-08-2310:22reefersleepThis is just an off-hand idea, hope it makes sense#2021-08-2310:31vemvI think a minimalistic macro is a step in the right direction, thanks! Hadn't thought of that much
In that case I wouldn't need an extra atom - I could simply let clojure.repl/source
do its work
e.g. (defmacro def-fdef ..)
+ (def-fdef the-defn the-spec)
The macro would create a predictable symbol, like the-defn-fdef-source
. Then I (source the-defn-fdef-source)
#2021-08-2310:33reefersleephm, yeah, I guess š#2021-08-2310:34reefersleepLooking forward to see how you fare š#2021-08-2311:17Lennart BuitIsnāt s/form
close to what you are looking for?#2021-08-2311:27vemvI'm interested in newline/comment preservation as well#2021-08-2319:05vemvNot for the faint of heart https://github.com/reducecombine/.lein/commit/ca5bccefd672191d5c703047d4e206304cac9453#2021-08-2318:10bortexzCurrently, on spec alpha 2, you can select things that are not in the schema passed. Is this expected behaviour?
(ns user
(:require [clojure.alpha.spec :as s]))
(s/def ::schema-field1 int?)
(s/def ::schema-field2 int?)
(s/def ::not-schema-field int?)
(s/def ::schema (s/schema [::field1 ::field2]))
(s/def ::selected (s/select ::schema [::schema-field1
::schema-field2
::not-schema-field]))
(s/valid? ::selected {::schema-field1 1
::schema-field2 2
::not-schema-field true}) => false
(s/valid? ::selected {::schema-field1 1
::schema-field2 2
::not-schema-field 3}) => true
#2021-08-2318:18Alex Miller (Clojure team)schemas are not exclusive#2021-08-2318:19Alex Miller (Clojure team)so yes#2021-08-2318:24bortexzGreat, thank you! I think itās great to contextually āextendā schema selections to keywords not originally on the schema#2021-08-2318:27bortexzIs there any reason to have a schema in select at all? This:
(s/select ::schema [::schema-field1
::schema-field2
::not-schema-field]))
and this:
(s/select [::schema-field1
::schema-field2
::not-schema-field]))
seem to express the same#2021-08-2318:30Alex Miller (Clojure team)gen#2021-08-2318:30Alex Miller (Clojure team)docs#2021-08-2318:31Alex Miller (Clojure team)closed schema checking#2021-08-2318:32bortexzI see, that makes sense, thanks#2021-08-2410:58vlaaadA bug:
(defn- retag [x v]
(assoc (vec x) 0 v))
(defmulti command first)
(s/form (s/multi-spec command retag))
=> (clojure.spec.alpha/multi-spec current.ns/command #object[current.ns$retag 0x708d0436
#2021-08-2411:08vlaaadMade a report on ask: https://ask.clojure.org/index.php/10969/s-form-s-multi-spec-returns-a-list-with-non-symbolic-objects#2021-08-3015:02Joshua SuskaloWhat do people do about having records conform to specs? Since it's generally good practice to use namespaced keywords with specs, but records don't support namespaced keywords on the enumerated fields, the only ways I've seen is either to make a less extensible spec by not using namespaced keywords, or to just not spec the records and prevent them from crossing interface boundaries.#2021-08-3015:06Alex Miller (Clojure team)I think those are the big ones#2021-08-3015:19Joshua SuskaloFair enough. Sometimes I wish I could just define namespaced record fields, but I can see why that'd be a challenge.#2021-08-3019:05seancorfield@suskeyhose What drives you to use records rather than hash maps?#2021-08-3019:06Joshua SuskaloPerformance. I write game engines in my spare time and things like field access time from entities can make the difference between being able to support hundreds of objects at 60 fps to being able to support thousands, which entirely changes the types of games you can make.#2021-08-3019:07Joshua SuskaloAnd in those sorts of systems, spec brings a lot of value since many of these systems have complex calling contracts that must be upheld and instrumentation and generative testing aid a lot in tracking down errors when you're able to encode those contracts via spec.#2021-08-3019:16seancorfieldAh, interesting niche. Yes, I can see performance being a good driver for that decision.#2021-08-3019:18Joshua SuskaloI'll also say that yes, game engines is very niche for Clojure development, and for someone who wants to make a very performance-sensitive game they'll have better luck with Unity's new systems, or writing something from scratch in Rust or C++, I prefer Clojure because it makes it easy to write my game code, and I don't make highly performance-sensitive games, but at an engine level I'd like to be able to make as few concessions as possible so that the overhead of being in Clojure takes away the least creative freedom.#2021-08-3019:19Joshua SuskaloAnd since I'm an engine developer by education and desire, I'm perfectly willing to take on lots of complexity in the engine if I can make writing something atop it simple and unconstrained.#2021-08-3019:52colinkahnCouldnāt you write a one-way conformer for records to be specād? Like (def ->map (s/conformer #(into {} %)))
, then use something like (s/and #(instance? MyRecord %) ->map (s/keys :req-un [ā¦.]))
#2021-08-3021:19Joshua SuskaloThe problem I have with that is then I have to write code that doesn't follow the spec, but follows an implementation detail for performance. Ideally I'd still be able to do the keyword-based lookup.#2021-08-3110:20jumarAlready asked in #cider but it looks unrelated, at least for one of my projects where it started to fail with lein repl
(maybe a dependency update but I believed it worked fine and don't remember any changes in dependencies).
Caused by: Syntax error macroexpanding clojure.core/defn at (clojure/spec/alpha.clj:85:1).
at clojure.lang.Compiler.checkSpecs(Compiler.java:6972)
at clojure.lang.Compiler.macroexpand1(Compiler.java:6988)
at clojure.lang.Compiler.macroexpand(Compiler.java:7075)
...
at clojure.main$loading__6721__auto____8974.invoke(main.clj:11)
at clojure.main__init.load(Unknown Source)
at clojure.main__init.<clinit>(Unknown Source)
... 55 more
Caused by: java.lang.Exception: #object[clojure.spec.alpha$and_spec_impl$reify__1057 0x459f703f "
What could be a reason for this ?
We use these if it matters:
[expound "0.8.9"]
[orchestra "2021.01.01-1"]
#2021-08-3112:22Alex Miller (Clojure team)Sorry, not sure what that is#2021-08-3112:23Alex Miller (Clojure team)Can you make something reproducible to share?#2021-08-3112:51jumarI'll try to check if it isn't something specific to the dependencies / toolchain (which is likely)#2021-08-3119:37MatthewLispHello š
I'm defining a spec using https://clojure.org/guides/spec#_collections, and i quote:
Tuple - (s/tuple double? double? double?)
Designed for fixed size with known positional "fields"
Conforms to a vector of the values
What if, i have a fixed size collection but with unknown positional "fields", and the collection is NOT homogenous ?#2021-08-3119:48flowthingcat
might work, if I understood correctly?
user=> (require '[clojure.spec.alpha :as spec])
nil
user=> (spec/def ::elements (spec/cat :int int? :string string?))
:user/elements
user=> (spec/valid? ::elements [1 "a" :b])
false
user=> (spec/valid? ::elements [1])
false
user=> (spec/valid? ::elements [1 "a"])
true
#2021-08-3119:48MatthewLispwhat about ["a" 1]
?#2021-08-3119:49MatthewLispThat's what i meant with unknown positional fields#2021-08-3119:49flowthingOh, I see.#2021-08-3119:51flowthingYeah, this is basically what Alex said, but:
user=> (spec/def ::element
(spec/or :int int? :string string? :keyword keyword?))
:user/element
user=> (spec/def ::elements (spec/coll-of ::element :count 3))
:user/elements
user=> (spec/valid? ::elements [1 "a" :b])
true
user=> (spec/valid? ::elements [1])
false
user=> (spec/valid? ::elements [1 "a"])
false
#2021-08-3119:50Alex Miller (Clojure team)what are you expecting to validate about this collection?#2021-08-3119:50Alex Miller (Clojure team)just it's size?#2021-08-3119:51Alex Miller (Clojure team)(s/coll-of any? :count 2)
#2021-08-3120:02MatthewLispNot just the size, an example of the collection:
[{:identity "username"
:value "john"}
{:identity "age"
:value "30"}]
One might think this can be described with s/coll because it appears to be an homogenous collection, however, depending on the context of the application, i have different values on this collection...
The way i'm defining the two entries of the collection is as follows:
;; username
(s/def :username/identity
#{"username"})
(s/def :username/value
"PREDICATE_HERE")
(s/def :entry/username
(s/keys :req-un [:username/identity
:username/value]))
;; age
(s/def :age/identity
#{"age"})
(s/def :age/value
"PREDICATE_HERE")
(s/def :entry/age
(s/keys :req-un [:age/identity
:age/value]))
;; now i'm trying to define the collection using :entry/username and :entry/age
#2021-08-3120:04MatthewLispi've over simplified the keywords names because i'm obfuscating the real names#2021-08-3120:04MatthewLispbut the structure is this#2021-08-3120:06Alex Miller (Clojure team)(s/coll-of (s/or :user :entry/username :age :entry/age) :count 2)
#2021-08-3120:07Alex Miller (Clojure team)there are more options for the content, like wrapping that in s/nonconforming
to avoid the s/or tag, or use s/multi-spec
instead for open extension there, etc#2021-08-3120:12MatthewLispThis approach leads me with a collection having two entries of for example :entry/username as being valid
I'm starting to think something using multi-spec, might work!#2021-08-3120:14MatthewLispWill look into s/nonconforming docs also#2021-08-3120:23Alex Miller (Clojure team)the last is un-doc'ed but it basically conforms, then returns the original value#2021-09-0215:4441ph4_Ph4unHi, I'm having a weird issue where I only call (st/instrument) in my code and aim to instrument only the used functions in my code base#2021-09-0215:4441ph4_Ph4unbut somehow it seems to begin to generate arguments for some of the functions in the codebase#2021-09-0215:4541ph4_Ph4unIf I'm only aiming to instrument my code, I shouldn't need clojure.test.check
and its submodules, right?#2021-09-0215:4641ph4_Ph4unsomehow when I run my program it whines those need to be required.. then when I add them it begins to generate inputs and all hell breaks loose#2021-09-0215:48borkdude@theamazingekko do you by any chance use s/fspec
?#2021-09-0215:4941ph4_Ph4unand yes, I do use s/fspec
#2021-09-0215:4941ph4_Ph4unoh boy do I use it!#2021-09-0215:5141ph4_Ph4unso okay.. I'm starting to get a picture here#2021-09-0215:5241ph4_Ph4unso it is possible that instrumentation may attempt to generate arguments for arguments that are functions in some cases?#2021-09-0215:5241ph4_Ph4unif theres HOFs#2021-09-0215:5541ph4_Ph4unhmm... so if this is a problem I need to maybe simplify those specs.. maybe by using ifn?
instead of full-fledged f/spec
#2021-09-0215:5641ph4_Ph4unare there any other workarounds.. in a way I'd like to keep that stricter form so I could then run generative tests separately later on#2021-09-0215:5741ph4_Ph4uncan I be a total pig and set clojure.spec.alpha/*fspec-iterations*
to 0 š š š#2021-09-0216:00borkdudeyes, I think this is an "interesting" feature of spec#2021-09-0216:0141ph4_Ph4unyeah it seems that just restricting the iterations solved my issues for now#2021-09-0216:0141ph4_Ph4unthanks! I've been stuck with this over a day š#2021-09-0216:0241ph4_Ph4unbasically it was just about finding a right config.. it's very "interesting" indeed#2021-09-0317:26Joshua SuskaloYeah, fspec is mostly useful for pure functions. In general the philosophy that seems to be present in spec is anything stateful and side effecting shouldn't be significantly specced. So some of these cases where you have fspec may be best to just have ifn and then describe the exact function which must be given in detail in the docstring.#2021-09-0317:26Joshua SuskaloGenerally specs and types are no substitute for good docstrings.#2021-09-0616:05mpenet@alexmiller just curious: any progress on spec2 lately?#2021-09-0616:05Alex Miller (Clojure team)no#2021-09-0616:51CarloI'd like to model a binary tree with a map of the form {:payload 0 :left sub1 :right sub2}
. Here's my spec for now:
(s/def ::binary-tree-2
(s/with-gen
(s/keys :req-un [::payload]
:opt-un [::left ::right])
#(gen/recursive-gen
(fn [f] (gen/let [[val l r] (gen/tuple (s/gen int?) f f)]
(merge
{:payload val}
(when l {:left l})
(when r {:right r}))))
(gen/return nil))))
But this doesn't check that ::left
and ::right
have the correct shape. What should I do instead?#2021-09-0617:13CarloI added more predicates with s/and
, but I still wonder if there's a better way#2021-09-0620:57Bobbi TowersI was wondering something similar a while back, and happened to find this: https://deque.blog/2017/05/20/a-clojure-spec-for-generic-binary-trees/ perhaps it might help?#2021-09-0722:27Franco GasperinoI have read that s/keys only supports keywords. I have a large external json message for which i would to perform some top-level schema validation. Rather than performing a keywordize-keys, which may descend past the desired schema and into embedded content, I was hoping to keep the message keys as strings and use validation upon them.
From what i can see, this would be done similar to this:
(def last-name? #(= "last" %))
(def first-name? #(= "first" %))
(def age? #(= "age" %))
(spec/def ::last-name (spec/tuple last-name? string?))
(spec/def ::first-name (spec/tuple first-name? string?))
(spec/def ::age (spec/tuple age? (spec/and pos-int? int?)))
(spec/def
::who
(spec/coll-of
(spec/or :last ::last-name :first ::first-name :age ::age)
:kind map?
:count 3
:distinct true))
(spec/explain ::who {"last" "smith" "first" "bob" "age" 10})
Is this the recommended path when using map keys which are strings, not keywords?#2021-09-0722:37Alex Miller (Clojure team)This approach should work, you can simplify those preds using sets though, like #{āfirstā}
(and at that point I would just inline them into ::first-name
etc#2021-09-0722:39Alex Miller (Clojure team)You also might find it useful to wrap s/nonconforming around the s/or to simplify the conformed structure#2021-09-0722:51Franco Gasperinocan you provide a case where s/conform ::who json-map would return something undesirable, where s/nonconforming would be wanted?#2021-09-0723:08Alex Miller (Clojure team)(spec/or :last ::last-name :first ::first-name :age ::age)
is going to conform to [:last ["last" "smith"]]
#2021-09-0723:09Alex Miller (Clojure team)(spec/nonconforming (spec/or :last ::last-name :first ::first-name :age ::age))
will conform to ["last" "smith"]
#2021-09-0723:10Alex Miller (Clojure team)if you use :into {}
in your ::who spec, you can actually conform back to the original map if you want that#2021-09-0723:10Franco Gasperinogreat to know. appreciated#2021-09-0723:19Franco Gasperinodidn't need the :into. wrapping the s/or in a nonconforming was enough to return a map when calling conform#2021-09-0908:13FranklinI'm trying to run tests for a specedĀ `cljs`Ā function using karma... it seems trickier than I expected
I'm getting an error that looks like this
--------------------------------------------------------------------------------
23 |
24 | (defn ^:export run
25 | [karma]
26 | (-> (stest/enumerate-namespace 'quagga.utils.components.share-modal.events) stest/check)
---------^----------------------------------------------------------------------
Encountered error when macroexpanding cljs.spec.test.alpha/check.
Error in phase :compile-syntax-check
RuntimeException: No such namespace: stest
Here's how I'm running the test; not sure if its' the correct way to do it
(defn ^:export run
[karma]
(-> (stest/enumerate-namespace 'quagga.utils.components.share-modal.events) stest/check)
(karma/run-tests karma
'quagga.utils.components.account-select.events-test
'quagga.components.dataview.table.events-test
'quagga.utils.components.share-modal.events-test))
Any clue as to how I can get it to work?#2021-09-0910:47FranklinI've also imported/required stest
[clojure.spec.test.alpha :as stest]
#2021-09-1008:28Franklinok.. I made a lot of progress.... I now have my tests looking like this
(deftest my-gen-test (is (= [](stest/check `ranged-rand))))
#2021-09-1008:29Franklinand this fails as expected when I call run-tests
from cljs.test
> (run-tests)
Testing cljs.user
FAIL in (my-gen-test2) (repl-input.cljs:1:27)
expected: (= [] (stest/check (quote cljs.user/ranged-rand)))
actual: (not (= [] [{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha38994], :clojure.spec.test.check/ret {:shrunk {:total-nodes-visited 4, :depth 2, :pass? false, :result #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -3, :end 0}, :ret -4}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.test.alpha/args (-3 0), :cljs.spec.test.alpha/val {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.alpha/failure :check-failed}}, :result-data {:clojure.test.check.properties/error #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -3, :end 0}, :ret -4}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.test.alpha/args (-3 0), :cljs.spec.test.alpha/val {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.alpha/failure :check-failed}}}, :time-shrinking-ms 2, :smallest [(-3 0)]}, :failed-after-ms 7, :num-tests 5, :seed 1631262481706, :fail [(-6 -1)], :result #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -6, :end -1}, :ret -9}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.test.alpha/args (-6 -1), :cljs.spec.test.alpha/val {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.alpha/failure :check-failed}}, :result-data {:clojure.test.check.properties/error #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -6, :end -1}, :ret -9}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.test.alpha/args (-6 -1), :cljs.spec.test.alpha/val {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.alpha/failure :check-failed}}}, :failing-size 4, :pass? false}, :sym cljs.user/ranged-rand, :failure #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -3, :end 0}, :ret -4}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.test.alpha/args (-3 0), :cljs.spec.test.alpha/val {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.alpha/failure :check-failed}}}]))
Ran 1 tests containing 1 assertions.
1 failur
#2021-09-1008:30FranklinI wonder if there's an existing cljs.test
function that can check if stest.check
fails and show me log the results#2021-09-1008:35FranklinI also think I can write a custom function... for this one using test/abbrev-result
... but I wonder if there's something there already#2021-09-1013:53colinkahnhttps://github.com/borkdude/respeced has some utilities for this.#2021-09-1018:46uwoThe correct way to spec a singular value is a set correct? Silly contrived example:
(s/conform (s/tuple #{:db/add} int? #{:attr} #{:value}) [:db/add 17592311384347 :attr :value])
#2021-09-1018:55Alex Miller (Clojure team)yes#2021-09-1219:06Eric DahlsengDoes anyone know why spec/check
seems to take forever to cleanup? In the following minimally reproducible example, Clojure takes ~60 seconds to shutdown after "Done" is printed, but exits immediately if the check
call is removed:
(ns test
(:require [clojure.spec.alpha :as spec]
[clojure.spec.test.alpha :as spec-test]))
(spec/def ::number number?)
(spec/fdef plus
:args (spec/or
:nothing (spec/cat)
:numbers (spec/* ::number))
:ret (spec/or
:number ::number))
(defn plus [& values]
(apply + values))
(defn -main [& args]
(spec-test/check `plus)
(println "Done"))
#2021-09-1219:16seancorfield@edahlseng There's probably a thread pool that it is waiting on. Call (shutdown-agents)
in -main
to solve that.#2021-09-1219:17seancorfieldSee the FAQ for Clojure https://clojure.org/guides/faq#agent_shutdown#2021-09-1219:22Eric Dahlseng@seancorfield, that works wonderfully! I wasn't aware of (shutdown-agents)
, thanks for the tip!#2021-09-1219:22seancorfieldThere's lots of good stuff in the FAQ.#2021-09-1219:26Eric DahlsengThere definitely is. I've ended up there through some Google searches, but haven't given it a direct read through otherwise. Looks like I should do that now to preempt future questions!#2021-09-1307:04Ben SlessQuestion regarding conventions and style. For some map spec ::foo
having a few variants, how would you name the specs?
Should keys be named :foo/key
?
For the variants I thought of just using or
(s/def ::foo (s/or ,,,))
To name the variants I can think of ::foo.a
::foo-a
:foo.type/a
Recommendations and ideas very welcome#2021-09-1311:34Alex Miller (Clojure team)Thereās not enough detail here for me to really answer that - how does a variant vary?#2021-09-1311:37Alex Miller (Clojure team)Generally though, I would urge you first to focus on the attributes not on the maps. Name the attributes well and fully describe the set of values that attribute can cover. If you then need multiple sets of attributes, kind of depends how they relate#2021-09-1312:02Ben SlessRight, then to be specific, consider https://clojure.github.io/tools.analyzer.jvm/spec/quickref.html#binding, which I want to spec out unambiguously#2021-09-1800:16vemvif you don't mind my asking, are you building something off t.ana? Would be curious about it (as someone who maintains two derived tools namely Eastwood and refactor-nrepl)#2021-09-1804:43Ben SlessI am exploring the possibilities.
Background, clj-fast, one of my first projects, is just a bunch of heuristic inlinings and optimizations of core clojure. I can keep on working at it for every conceivable use case, but I became interested in solving the general problem. I want to implement some intermediate compiler passes - constant inlining, beta reductions, copy and constant propagation.
If I get this, I can write a compiler from clojure to clojure that cuts down on the biggest performance hits in the language, iteration and dynamism#2021-09-1804:45Ben SlessThis ia clearly not for for development time, but for final artifacts it could be huge#2021-09-1312:03Alex Miller (Clojure team)this is probably a good use case for s/multi-spec#2021-09-1312:03Alex Miller (Clojure team)switching on :op
#2021-09-1312:03Ben SlessBut even before op, zooming in on binding#2021-09-1312:03Ben SlessA very rough spec would be:
(s/def ::binding
(s/keys
:req-un
[:binding/op :binding/form :binding/name :binding/local :binding/arg-id
:binding/variadic? :binding/init :binding/atom :binding/children]))
#2021-09-1312:04Ben Sless(imagine I merged it with ::common
)#2021-09-1312:04Ben SlessI should have split out the :opt-un, too#2021-09-1312:04Ben Slessbut the point is, there are keys which just don't occur some times#2021-09-1312:05Ben Slessfor example, :binding/arg-id
occurs only when :binding/local
is :arg
#2021-09-1312:05Alex Miller (Clojure team)you could do a second level of s/multi-spec on :local
#2021-09-1312:05Ben SlessSo I need to "subtype" bindings then or
between them#2021-09-1312:06Ben Slessis there a reason to prefer multi-spec to or?#2021-09-1312:06Alex Miller (Clojure team)it's designed for open extension, and it doesn't tag the conformed value#2021-09-1312:07Ben Slessokay, and how would you name the specs you select between? would you even name them?#2021-09-1312:08Alex Miller (Clojure team)I think I would name them, and I would name them something related to the ast node type#2021-09-1312:09Alex Miller (Clojure team)in general, I find relying on s/merge and s/multi-spec to "build up" evolves better than trying to s/or#2021-09-1312:10Ben SlessI have no sense of style or habits with spec so I thought it would be better to ask instead of produce something horrendous š#2021-09-1312:11Ben Slesshow would you name the specs you dispatch to? ::arg-binding
, ::binding.arg
, :binding.type/arg
?#2021-09-1312:12Ben Slesssomething else entirely?#2021-09-1312:29Alex Miller (Clojure team).'s should generally be used only in qualifiers, so I'd rule out the middle one#2021-09-1312:30Alex Miller (Clojure team)since these are ast nodes I would probably be trying to include either "ast" and/or "node" in the name of each one#2021-09-1312:31Alex Miller (Clojure team)whether you put that in the qualifier or in the name is a matter of taste#2021-09-1312:31Ben SlessIn that case, I can even start from naming it something like :node/binding
then the multi case could be :node/binding.arg
? :node/arg.binding
?#2021-09-1312:32Ben SlessLike I said previously, I still have no sense of taste when it comes to specs, so I prefer to ask someone with experience#2021-09-1312:32Alex Miller (Clojure team)"node" is not sufficiently unique - you should treat this as a global naming space#2021-09-1312:32Alex Miller (Clojure team):clojure.tools.analyzer.node/binding
#2021-09-1312:32Ben Slessack#2021-09-1312:33Alex Miller (Clojure team)this is admittedly a bit painful to use without an alias, and aliases to namespaces that don't exist don't work, so you either need to do the create-ns
/ alias
dodge or use a real namespace#2021-09-1312:33Ben SlessIf I require the analyzer as ana
I can also go with ::ana/node.binding
#2021-09-1312:33Alex Miller (Clojure team)(this is going to get better in 1.11 with lightweight alias support)#2021-09-1312:34Alex Miller (Clojure team)yes, that would work#2021-09-1312:34Alex Miller (Clojure team)although .'s in names are bad#2021-09-1312:35Ben SlessI should put that on a sticky note#2021-09-1312:35Alex Miller (Clojure team)the problem is they have meaning understood in the compiler, which is: class name for symbols#2021-09-1312:37Alex Miller (Clojure team)the https://clojure.org/reference/reader#_literals spec actually says keywords "cannot contain '.' in the name part, or name classes"#2021-09-1312:35Alex Miller (Clojure team)so I'd either use node-binding
or bite the bullet on moving it into the qualifier#2021-09-1312:36Ben SlessOr cheat and create-ns
for my own convenience, but that seems like bad form#2021-09-1312:38Alex Miller (Clojure team)well, you'd hardly be alone :)#2021-09-1312:38Ben Slesshm :thinking_face:#2021-09-1312:38Alex Miller (Clojure team)and like I said, better solution for this is coming#2021-09-1312:38Ben SlessBut it won't be backwards compatible#2021-09-1312:39Alex Miller (Clojure team)true#2021-09-1312:39Alex Miller (Clojure team)that is important here if it's part of analyzer.jvm#2021-09-1312:39Ben SlessI'll try to be considerate in my implementation#2021-09-1312:40Ben SlessIt would be a tall order for it to be integrated into analyzer.jvm but having it as a companion library would be nice#2021-09-1312:45Ben SlessWhile we're here, I'll also use this opportunity to ask regarding namespace and artifact naming convention (and copyright)
A "correct" name for the spec's ns would probably be clojure.tools.analyzer.jvm.spec
, but can I distribute a library whose root namespace is clojure
? Even if the license allows it, which I'm not sure it does, it feels rude#2021-09-1312:49Alex Miller (Clojure team)I assumed you were spec'ing this to include with tools.analyzer.jvm, where it would be fine#2021-09-1312:50Ben SlessThat was my intention, yes#2021-09-1312:50Alex Miller (Clojure team)seems like it would be the greatest utility to the most people as either part of that library or as a separate contrib lib#2021-09-1312:51Alex Miller (Clojure team)currently tools.analyzer still supports pretty old clojure versions, older than spec#2021-09-1312:51Ben SlessSo it would make sense as a contrib lib#2021-09-1312:52Alex Miller (Clojure team)I think so#2021-09-1312:52Alex Miller (Clojure team)I'd have to ask Rich but I don't think he would object to that#2021-09-1312:52Alex Miller (Clojure team)we've taken the approach of appending ".specs" for stuff like this, so it would presumably be tools.analyzer.jvm.specs#2021-09-1312:53Ben Sless:thumbsup:#2021-09-1312:54Ben SlessIf I'm doing it with the intent of it becoming a contrib library I should probably start with a tools.analyzer.specs then merge the jvm stuff on top of it#2021-09-1313:03Alex Miller (Clojure team)I mean, there isn't really anything but .jvm, so I think it would be fine to include it all in one#2021-09-1313:04Alex Miller (Clojure team)I can get a repo set up this week for you#2021-09-1322:36Franco GasperinoWhat is the recommend route for evaluating the result of a clojure.spec.gen.alpha/check result summary set with the test framework (runners, assertions, and family) of clojure.test?
The test.check library appears to have defspec, but it's unclear if it's a good fit to glue to spec/fdef generative, as examples seem to favor it's own clojure.test.check.generators in the guide examples.#2021-09-1323:49Alex Miller (Clojure team)I would recommend not doing that and separating your generative tests from your unit tests#2021-09-1400:05Michael Gardnercould you elaborate on this?#2021-09-1400:53Alex Miller (Clojure team)unit tests are generally fast and you run them a lot. generative tests are usually slow and you run them less often, they also often require most setup (for things like instrumentation, mocking etc).#2021-09-2120:02johanatani have something like the following:
(deftest specd-functions-pass-check
(stest/instrument)
(macros/filtered-check `a-namespace/a-function (num-tests 1))
(macros/filtered-check `b-namespace/a-function (num-tests 3))
#2021-09-2120:02johanatanwhere macros/filtered-check is the following:#2021-09-2120:03johanatan(defmacro filtered-check [sym opts]
`(let [fltr# (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
check# (fn [res#]
(cljs.test/is (= true (-> res# :clojure.spec.test.check/ret :result))
(goog.string/format "spec check failure:\r\n%s"
(with-out-str (cljs.pprint/pprint res#)))))]
(when (or (= fltr# "all")
(clojure.string/includes? (clojure.string/join "/" [(namespace ~sym) (name ~sym)]) fltr#))
(let [check-res# (clojure.spec.test.alpha/check ~sym ~opts)]
(clojure.spec.test.alpha/summarize-results check-res#)
(if (nil? check-res#)
(cljs.test/is false "stest/check result was nil. did you pass it any valid symbols?")
(doall (map check# check-res#)))))))
#2021-09-2120:03johanatan^^ clojurescript obviously#2021-09-2120:03johanatan(designed for figwheel-main autotesting)#2021-09-2120:04johanatanalso i don't write "unit tests" as they are subsumed by property-based tests#2021-09-2120:04johanatanthe "filtering" feature takes care of the "slow and don't run often" aspect#2021-09-1323:50Alex Miller (Clojure team)some people have written code to wrap spec check tests into clojure.test, but I don't have that at hand#2021-09-1402:51Franco Gasperinook thank you#2021-09-1402:52Franco Gasperinofollowing up on that - do you organize a generative fdef in the same namespace as the function it's associated with, or with a peer testing namespace similar to a unit test?#2021-09-2120:05johanatanpersonally i write the fdef right beside the defn#2021-09-2120:05johanatanbut that is probably a matter of personal opinion#2021-09-2120:06johanatani do have a few "integration tests" that span multiple functions / namespaces defined in the test/ area itself#2021-09-2120:06johanatane.g., "roundtrip" style tests#2021-09-2119:55johanatanis there ever a situation where the :ret value passed to a (failing) fdef :fn checker would diverge from the value you get when you run the function in question with the associated :args manually ?#2021-09-2119:55johanatan(I'm seeing this currently and it's a bit baffling)#2021-09-2119:56johanatanthese are the reported values:
:cljs.spec.alpha/value
{:args {:maps ({:A/A ""} {:A/A ""})}, :ret {:A/A " "}},
#2021-09-2204:29colinkahnI think unless itās being conformed into that different value it will be the same. You could try doing:
(s/conform (:ret (s/get-spec `your-function)) {:A/A ""})
as a sanity check to make sure your spec isnāt changing it during conforming.#2021-09-2218:14johanatanthe :ret spec here is simply map?
#2021-09-2218:15johanatanbut for completeness:#2021-09-2218:15johanatanutils.utils=> (s/conform (:ret (s/get-spec `-merge-attribs)) {:A/A ""})
{:A/A ""}
#2021-09-2218:15johanatancould this be a bug in clojure spec ? it is in alpha after all. @U064X3EF3 ?#2021-09-2218:17Alex Miller (Clojure team)sorry, can you re-explain?#2021-09-2218:17johanatanis there ever a situation where the :ret value passed to a (failing) fdef :fn checker would diverge from the value you get when you run the function in question with the associated :args manually ?#2021-09-2218:18johanatanthese are the reported values:
:cljs.spec.alpha/value
{:args {:maps ({:A/A ""} {:A/A ""})}, :ret {:A/A " "}},
#2021-09-2218:18johanatanthis is the manual repl invocation of the function in question:
utils.utils=> (-merge-attribs '({:A/A ""} {:A/A ""}))
{:A/A ""}
the manual repl invocation is the "correct" one#2021-09-2218:19johanatanNotice that the one reported by spec includes a \space and the one returned from the manual invocation is merely the empty string.#2021-09-2218:20johanatan(the :ret that is)#2021-09-2218:25Alex Miller (Clojure team)it seems like you're asking me a question about your function, not spec#2021-09-2218:26Alex Miller (Clojure team)maybe there is more here than you can "see" - that "empty" string could have a Unicode non-breaking space or something in it#2021-09-2218:26Alex Miller (Clojure team)or zero-width space, whatever that weird unicode thing is#2021-09-2218:28Alex Miller (Clojure team)in general, my answer to your original question is no (unless you're using spec :stub instrumentation or something)#2021-09-2218:45johanatani think you misunderstood.#2021-09-2218:46johanatanspec says that for some input a specific return value is returned. when i run that function manually in the repl with the exact same input a different value is returned.#2021-09-2218:46johanatani.e., "" != " "#2021-09-2218:46johanatanregardless what that character is#2021-09-2218:46johanatanempty string is empty string#2021-09-2218:46johanatan" " is not empty string#2021-09-2218:47Alex Miller (Clojure team)a 1-character string with a Unicode zero-width space character looks like an empty string#2021-09-2218:47johanatanah gotcha#2021-09-2218:47johanatanis there any way to get the raw characters to be printed ?#2021-09-2218:47johanatanfrom what i've seen spec seem to report escape sequences for those#2021-09-2218:47johanatanlike \b, \f etc etc#2021-09-2218:47Alex Miller (Clojure team)not those#2021-09-2218:48Alex Miller (Clojure team)U+200B#2021-09-2218:48johanatanah, ok. thanks. any other characters like that i should know about ?#2021-09-2218:48Alex Miller (Clojure team)well, that's one I've had this problem with. there are probably others#2021-09-2218:49johanatanhehe š too bad https://clojuredocs.org/clojure.core/char-escape-string doesn't exist in cljs (although seems that wouldn't cover all of these anyway)#2021-09-2218:49Alex Miller (Clojure team)you add a debug in your :fn function to print the count of the input strings or map char etc#2021-09-2218:49johanatanš thx#2021-09-2218:50Alex Miller (Clojure team)I'm just guessing here - this is a possible answer that matches what I see, but you should validate#2021-09-2218:50johanatanyes of course#2021-09-2218:51Alex Miller (Clojure team)these kinds of things happen with generative tests :)#2021-09-2218:55johanatanunfortunately i think it might be something else:#2021-09-2218:55johanatancomparing v: '1 1', and: '1 1', count v: 5, count joined: 5 false
#2021-09-2218:56johanatanooh, there's two non-printable chars in there#2021-09-2218:56johanatanand they're probably in opposing positions#2021-09-2218:58johanatanfound it. silly mistake.#2021-09-2218:58johanatani added a spurious reverse
on the array of values i was comparing against#2021-09-2218:59johanatanmight be worth thinking about how to better report string values (adding perhaps unicode escapes for these as well as the platform default escapes)#2021-09-2219:00johanatanthanks for your help!#2021-09-2219:07johanatanfyi... i had used simple-type-equatable
as my inner-type
for a recursive map gen. if i had used instead simple-type-printable-equatable
, this probably wouldn't have happened#2021-09-2119:56johanatanthis is the manual repl invocation of the function in question:#2021-09-2119:56johanatanutils.utils=> (-merge-attribs '({:A/A ""} {:A/A ""}))
{:A/A ""}
#2021-09-2119:56johanatanthe manual repl invocation is the "correct" one#2021-09-2119:58johanatan(i have deleted the target/ directory and restarted the repl so it should not be a stale code issue)#2021-09-2203:23johanatananyone?#2021-09-2506:09apbleonardWhen conforming a s/multispec (with clojure.spec.alpha) is it possible to be returned which multimethod dispatch value was used, similar to the "tags" returned when conforming an s/or?#2021-09-2507:50dgb23Not sure if I understand the question:
user=> (require '[clojure.spec.alpha :as spec])
nil
user=> (defmulti tag ::tag)
#'user/tag
user=> (defmethod tag :foo [_] (spec/keys :req [::tag]))
#object[clojure.lang.MultiFn 0x522b2631 "
#2021-09-2507:55dgb23Do you mean that after conforming the above, you somehow forget that :user/tag
was used to dispatch and you want to explicitly encode that in the conformed value?#2021-09-2511:45apbleonardWell not quite. In your example I'd like the dispatch value :foo
to be explicitly encoded in the conformed result.#2021-09-2512:21apbleonard...as per https://www.cognitect.com/cognicast/103: "In spec, there is valid question mark, just a validator that's Boolean. But, the more interesting function is conform, which takes the data and a spec, and gives you a de-structured, labeled version, if you will, of the data where every branching point that was detected, the result of it was labeled. If you had some arbitrary complex predicate that you needed to satisfy and it was one of threeāX or Y or Zāand it matched Y, there'll be a key there that says Y is what they supplied. Your code won't need to figure that out again because the spec did that, de-structured it, and labeled it. There's a path system in spec. It's part of the design. Everywhere there's a branch, there's a label. "#2021-09-2514:45apbleonardI guess I need an s/or
. I liked the clear dispatching and the hierarchical keys offeredby multispecs though.#2021-09-2709:36arohnerAre there any libraries for turning XML XSD definitions into specs?#2021-09-2820:52johanatanis there a way to get at the unconformed value in an fdef :fn :ret value ?#2021-09-2820:52johanatani.e., the non-marked up, pristine original return value#2021-09-2820:54johanatanah, looks like: https://clojuredocs.org/clojure.spec.alpha/unform may do it#2021-10-0118:05bortexzIn spec 2, one can select from inside a collection of schemas instead of a single schema. Is there a way to make that work with collection-like maps (e.g. a sorted-map timeseries indexed on timestamp, specced with map-of
) to select from the values of such map?#2021-10-0317:19Carlois clojure.spec.gen.alpha/let
not available in clojurescript?#2021-10-0317:28CarloThis page https://clojurians-log.clojureverse.org/test-check/2019-02-26
and this https://github.com/clojure/test.check/blob/88eebf105fbb01db17ce12a3bff207cc395270b9/src/test/clojure/clojure/test/check/test.cljc#L18
seem to imply that I should be able to use it, but I don't see how#2021-10-0317:30CarloIf I try require
[clojure.spec.gen.alpha :as gen :include-macros true :refer-macros [let]]
it will say that there is no macro named let
#2021-10-0318:39Carlo@U04V15CAJ just because of the previous link#2021-10-0318:42colinkahnIt doesn't because let
is a macro and can't be lazy loaded (I believe that's the reason anyways). You can use the non-macro version, bind
which is part of clojure.spec.gen.alpha
#2021-10-0318:45colinkahnJust to clarify, let
is not available in the Clojure version of clojure.spec.gen.alpha
either.#2021-10-0319:09Carlo@U0CLLU3QT if the problem is the lazy loading, can I somehow preload it? Yes, I was trying to use let
because it seems cleaner wrt bind
and fmap
which is what I'm using now. So, are the tutorials outdated? Was let
removed from clojure.spec.gen.alpha
at some point? I thought I used it in the past š®#2021-10-0319:10Carlomoreover, where is this code https://github.com/clojure/test.check/blob/master/src/test/clojure/clojure/test/check/test.cljc#L1054 getting the gen/let
?#2021-10-0321:03Carlook, I'm so dumb, I should just have imported clojure.test.check.generators
where the let macro I'm interested in is actually defined :man-facepalming:. Thanks for the help!#2021-10-0321:04colinkahnThose examples look like just test.check, which specs gen namespace lazy load behind statically defined functions. It's a design decision in spec, that all the functions in the gen namespace can be used without loading clojure.test.check.generators until needed (for example people could define specs for use in production code but only incur loading the test.check.generators namespace in their test suite where they'll use that functionality). In ClojureScript, it's the same but there is no true lazy loading, but you only need to manually require the test.check namespaces when you need them.#2021-10-0321:04colinkahn^ but yeah if you don't care about this stuff then just import test.check generators like you just posted š#2021-10-0321:12Carloawesome! Now I understand why the re-export was needed and what's the problem with re-exporting the let
macro. Thanks again @U0CLLU3QT#2021-10-0317:43mafcocincoI briefly used spec.alpha
but didnāt end up doing much with it. Iām interested in using spec2
for message validation in a pub/sub system. Can someone point me to the current āstate of the artā/example use cases/best practices/etc. for using spec2
?#2021-10-0317:58bortexzCurrently spec 2 is a work in progres and not ready for production use, so there is not much to be found, but if youāre still interested to give it a try, thereās some info on the githubās repo wiki (https://github.com/clojure/spec-alpha2/wiki), and the talk where Rich talks about it https://www.youtube.com/watch?v=YR5WdGrpoug&ab_channel=ClojureTV#2021-10-0322:28seancorfieldSpec 2 still has a lot of bugs and has gone through a lot of changes -- with a number of additional changes still planned. We have been very heavy users of Spec (1) since it first dropped -- in production code -- and for about three months we ran a branch of our code tracking Spec 2 but it just became intractable to keep tracking it and working around the bugs...#2021-10-0403:49mafcocincogot it. I will use spec 1 then. thanks.#2021-10-0501:55mafcocincoafter a bit of research, I decided to use metosin/malli
. Seems to have all the features I need and I like the data-driven orientation.#2021-10-2822:19peterdeeHi! At one time there was a function spec* in spec-alpha2 that allowed programmatic definition of specs. I donāt see that in the current SHA version. Is it still possible to define specs programmatically?#2021-10-2822:22seancorfieldAre you looking for register
@peterd? https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/alpha/spec.clj#L429#2021-10-2822:36peterdeeI am looking at some code I wrote about two years ago. (Wish I could recall!) I was using both s/spec* and s/register. I was using backquoted forms or variable values with spec*. Hmmmā¦ It looks like the spec* calls are typically wrapped in a s/register, but s/register takes a spec, not a body (form) so my guess is that register isnāt going to allow programmatic operation???#2021-10-2822:40Alex Miller (Clojure team)You probably want resolve-spec or maybe create-spec depending on what you're doing#2021-10-2822:41Alex Miller (Clojure team)resolve-spec takes a form and returns a spec object#2021-10-2822:42peterdeeUgh! How did I miss that? Yes, I think resolve-spec.#2021-10-2822:42peterdeeThanks!#2021-10-2822:44Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/commit/bab9b91d2b8490e9c7bdbf3b1500636b28bb4e75#2021-10-2822:50peterdeeThanks. My program is happy again.#2021-10-2822:52peterdeeGuess I should have looked at the commit log!#2021-10-2823:06Alex Miller (Clojure team)I think https://clojure.github.io/spec-alpha2/ is relatively synced to the code too#2021-10-2823:10peterdeeSimple programmatic access is a nice feature. Iām wrote a parser for MiniZinc, a popular open-source combinatorial solver language; it has its own language for defining data. (Will open source my parser soon). The specs allow you to catch users errors that the parser misses.#2021-11-0215:57Drew VerleeIs it possible to have spec generators for two different contexts?
E.g in one i want my user's uuids to be a random uuid, and in another for them always to be the same value.#2021-11-0215:58Alex Miller (Clojure team)there are several functions that accept an override generator map#2021-11-0215:59Alex Miller (Clojure team)so you could swap out at point of use (during exercise, instrumentation, etc)#2021-11-0215:59Drew VerleeThanks alex, I'm reading the docs now.#2021-11-0315:19timoHi, anyone has an idea how to generate spec from json-schema?#2021-11-0315:38vlaaadOh, I wanted to make a library that does that#2021-11-0315:38vlaaadLast time I checked no solutions existed#2021-11-0320:14qqqI posted the following in #clojure , and @hiredman pointed out clojure-spec satisfies the properties
Consider the problem of String -> AbstractSyntaxTree. Solving this problem is called Parsing, and we can solve it via Regex + LL / LALR / Peg / Parsec / ... . In particular, we can view Regex / Peg as "DSLs" for String -> AbstractSyntaxTree
Now, suppose we are trying to define a DSL -- which can be viewed as a subset of s-exprs. For example, Scheme is a "s-expr DSL"; Lisp is a "s-expr DSL", edn is a "s-expr DSL", clojure is a "s-expr DSL" -- in all 4 cases, all valid data values look like s-exprs.
Question: What is the Regex / Peg equiv for doing "s-expr -> DSLs" ? I.e. what is a "pattern description language" for specifying what subset of s-exprs are valid exprs in the DSL ?
#2021-11-0320:15qqqA followup to this then is: Regex is backed by DFA / NFA theory. Other parsing techniques is backed by (limited subset of) Context Free Grammars. Is there a "theory" behind clojure-spec, or is it driven largely by battled tested useful tactics ?#2021-11-0320:17hiredmanthe theory is basically the same as parsing where parsing is sort of logically defined over any sequence of tokens of any type, but we usually think of parsing as over a sequence of character or string tokens#2021-11-0320:18hiredmanit is parsing but over lazy seqs of lazy seqs (which introduces another novel kind of recursion)#2021-11-0320:18hiredmanparsing over a complex token type#2021-11-0320:19hiredmanfrom a parsing viewpoint spec is usually just used as a recognizer "does this match the grammar or not" but it can be used as a full parser#2021-11-0402:54qqqThere are three types of input data:
[a] sequence of chars (standard parsing)
[b] sequence of tokens (basically parsing, but after the lexing stage)
[c] TREE of tokens (what we have)
I can see how [a] == [b]; but I don't see how one define a generalization that covers both [b] and [c]. I'm not saying one does not exist, just that I'm not seeing it. Can you clarify on this ?#2021-11-0402:54qqq@hiredman: ^#2021-11-0402:55hiredmana tree is a recursion of sequences, so just take the parsing over a sequence and apply it recursively#2021-11-0402:57qqqCFG is a language that defines patterns over sequences of tokens. What defines patterns over trees ?#2021-11-0402:58hiredmanif you make tokens terminals or another cfg#2021-11-0402:58hiredmanbecause the tokens are complex and also have a structure to be parsed#2021-11-0403:00qqqAs far as I can see Cfg[Token] and Cfg[Cfg[Token]] both define patterns over sequences; can you point me at a research paper / book ? I need a more detailed exposition.#2021-11-0403:01hiredmanno,I've never seen this written up#2021-11-0403:01hiredmanbut like, if you write out a grammar, say in bnf, everything is a terminal or a rule, and the terminals are the tokens#2021-11-0403:03hiredmanif instead of putting in, like, literal 'if' in the grammar as the token, you say, the terminal is actually a member of this set of sequences and describe the set as a cfg#2021-11-0403:11qqqI think the core issue here is Vec<Vec<T>> does not represent the full space of Tree<T>; so this idea of Cfg[Cfg[Token]] seems to not recognize Trees.#2021-11-0416:20respatializedCan you give an example value that satisfies Tree<T>
but not Vec<Vec<T>>
?#2021-11-0404:12hiredmana tree Tree<T> = Leaf<T> | Node<List<T>>#2021-11-0404:13hiredman(to make up some terrible type notation)#2021-11-0404:13hiredmanerrr#2021-11-0404:14hiredmanTree<T> = Leaf<T> | Node<List<Tree<T>>>#2021-11-0404:14hiredmanSo you walk the list in the node parsing it, where each token type is itself tree to be matched against some other parser#2021-11-0416:20respatializedCan you give an example value that satisfies Tree<T>
but not Vec<Vec<T>>
?#2021-11-0423:48qqq@afoltzm: All leaves in a Vec<T> have depth 1. All leaves in a Vec<Vec<T>> have depth 2.#2021-11-0423:49qqq@hiredman: Here I think is the problem
Cfg[T] pattern language over Vec<T>
Cfg[Cfg[T]] is a pattern language over Vec<Vec<T>>#2021-11-0423:50hiredmanthat depends on exactly what the internals of CFG are#2021-11-0423:51hiredmanVec<Vec<T>> is not a tree, so it is not what we have#2021-11-0423:51hiredmanedn/clojure data is a tree#2021-11-0423:51hiredmanso you have a recursive data type like Tree<T> = Leaf<T> | Node<List<Tree<T>>>#2021-11-0423:52hiredmanand one way to "parse" such a thing is to recursively parse everywhere the datatype does#2021-11-0423:54hiredmanso you can imagine a parsing that walks through List<String> building up a parse tree, and a fundamental operation of such a parser is doing an equality comparison of the strings in the list with the strings that are the terminals of the grammar#2021-11-0423:55hiredmannow say, what if you grammar is defined in some dsl/library that has some built in "functions" like "is-if-string?" that runs true when given the string "if"#2021-11-0423:56hiredmanso you go to your grammar and replace the if terminal with a "call" to is-if-string?, so instead of the parser doing an equality check it calls the predicate on the token#2021-11-0423:57hiredmannow change from string tokens to arbitrary data structures, and use a predicate like three-element-vector-all-0?#2021-11-0423:59hiredmannow imagine you have a higher order function 'member' that takes a set, and returns a predicate that returns true if its argument is in the set#2021-11-0423:59hiredmanwhat is a cfg? it is a description of a (possibly infinite) set that contains the strings of your language#2021-11-0500:01hiredmanso you have a cfg where the terminals are predicates, and the predicates can be defined as themselves being a cfg#2021-11-0502:46qqq@hiredman: I agree with you that a pattern language for Tree<T> exists. I also agree that this pattern language has derivation rules, where some rules are choices of the form choiceA | choiceB | choiceC | ... ; and other nodes are of some form that builds up trees. I am not sure that calling this CFG-like is correct.#2021-11-0505:38hiredmanIt is a CFG#2021-11-0505:42hiredmanRegexs describe sets, CFGs commonly do not exhaustively list all terminals instead defining sets of terminals via regexes. CFGs describe sets. Given a cfg, swap the sets described by regexes for sets described by cfgs. You've swapped sets for sets, just a different set builder notation.#2021-11-0505:45hiredman(I am well aware that CFGs are more powerful than regexes, that just means that CFGs can describe more sets than regexes can, but they are still sets)#2021-11-0505:58qqqPoint me to one text book / peer reviewed paper that shows how to use CFG to pattern math over trees.#2021-11-0506:01seancorfield@qqq I think you're being very belligerent here. If you want to find peer reviewed papers etc, you should use Google/Bing/whatever and find them yourself instead of demanding that others here do your research for you.#2021-11-0506:01seancorfieldDoes https://www.sciencedirect.com/science/article/pii/S0196885816300598 come close to what you're looking for?#2021-11-0506:02seancorfield(that was just the first vaguely relevant seemingly result from a very, very brief search that you could easily do yourself)#2021-11-0506:05qqq@seancorfield: The link you posted has nothing to do with parsing.#2021-11-0506:06qqqMy point here is: CFG, as traditionally used in CS, https://en.wikipedia.org/wiki/Context-free_grammar , is for List -> Tree. Besides @hiredman, I have never heard anyone claim that CFG can be used to parse Trees -- and that such a textbook / peer reviewed paper does not exist.#2021-11-0506:08hiredmanThe CFGs you are looking at are already being used to parse simple trees#2021-11-0506:09hiredmanIn that the terminals are sets of lists of character, not just sets of characters#2021-11-0506:10qqq@seancorfield: Would it be less "very belligerent" if I instead wrote it as:
"Point me to one text book / peer reviewed paper that shows how to use CFG to pattern match over trees, and I'll be convinced your definition makes sense." ?
Here, @hiredman is making a claim that standard usage of CFG does not claim. I am asking him to back up him claim with peer reviewed article, rather than his own claims.#2021-11-0506:11hiredmanA cfg is not a function with the type list -> tree, you should maybe read that Wikipedia article a little closer#2021-11-0506:12qqqGiven, an alphabet Sigma, CFGs define a subset of Sigma^* which can be mapped do an AST specified by the derivation rules.#2021-11-0506:13qqqNote that Sigma^* is over Vec<Sigma>, not a Tree.#2021-11-0506:14qqqIf you can point me to one peer reviewed theoretical CS resource that claims that CFGs recognize subsets of Tree<Sigma>, I'll concede the argument.#2021-11-0506:15qqqOtherwise, I am maintaining my claim that CFGs are only used to recognize Sigma^*, which is Vec<Sigma> in our discussion so far.#2021-11-0506:15hiredmanThey don't need to recognize Tree<Sigma>, if you accept that tokens in you "language" can be any type, not just strings, then you make you token type Tree#2021-11-0506:16hiredmanSo sigma is a set whose elements are trees#2021-11-0506:17qqqSigma, in typical CS literature is either {0, 1}, or alphanumeric. Is there anyone else in the CS Theory community that uses the definition you use ?#2021-11-0506:18hiredmanSure, because parsers typically parse text, but just because it's typical that doesn't mean that is all there is#2021-11-0506:19qqq@seancorfield in DMs asked me to drop this conversation, so I'm leaving it here as is.#2021-11-0506:19hiredmanA cfg is a logical/mathematic object, where "string" is an abstract notion that can be anything#2021-11-0506:27qqqI'm making one more statement because I think I can resolve our disagreement:
Traditional CFGs have type signature: Vec<Sigma> -> bool
I am looking for a pattern language to specify functions of type signature: Tree<Sigma> -> bool
@hiredman if you "add trees to Sigma" as you suggest, CFGs still have type signature Vec<Sigma> -> bool
the only difference is that Sigma is now defined as
Sigma = Letter | Tree ...
Now, in this case, I will concede that the CFG is "recognizing lists of trees", but this CFG is not recognizing individual trees. I.e., the type signature is Vec<Tree> -> bool; whereas I am after Tree -> bool.#2021-11-0512:44Alex Miller (Clojure team)Does this have anything to do with spec? Can we move further conversation to #off-topic or something?#2021-11-0515:37jerger_at_ddaHi, is there a way to generate a good documentation from speced functions?
I know the discussion about spec meta from last year but did'nt find any update ... anything new ?#2021-11-0515:42Alex Miller (Clojure team)the doc generator we use for core and contrib (autodoc) does make docs for specs#2021-11-0515:44Alex Miller (Clojure team)but autodoc is frustratingly hard to use in practice#2021-11-0515:44jerger_at_ddado you know what authodoc version clojure uses?#2021-11-0515:44jerger_at_ddaI can find a tag 1.1.2 but that's 5 years old ...#2021-11-0515:44Alex Miller (Clojure team)yeah :)#2021-11-0515:46Alex Miller (Clojure team)you can see an example with the spec doc at https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/defn#2021-11-0515:46jerger_at_ddaš#2021-11-0515:46jerger_at_ddasaw this already š#2021-11-0515:47jerger_at_ddaIf clojure doc is generated with 1.1.2, I will give the autodoc-lein an update to use the 1.1.2 version also ...#2021-11-0515:47Alex Miller (Clojure team)autodoc is conceptually well structured (split into separate programs to "collect" data and export docs, but super difficult to use well in practice#2021-11-0515:48jerger_at_ddayea .. a read about#2021-11-0515:49jerger_at_ddado have a link for the setup clojure uses itself ?#2021-11-0515:49Alex Miller (Clojure team)https://github.com/clojure/clojure-api-doc#2021-11-0515:50Alex Miller (Clojure team)but Clojure's use of autodoc is particularly weird#2021-11-0515:50Alex Miller (Clojure team)it's using 1.1.2 though#2021-11-0515:55jerger_at_ddacool - thanx. If we find our way through I will give feedback ... maybe we did some steps in right direction valuable for others :-)#2021-11-0515:56Alex Miller (Clojure team)I would love to take the good stuff in autodoc and turn it into something actually usable (Clojure's specific usage of it for core and contrib is additional layer of bespoke weirdness over the top)#2021-11-0515:56Alex Miller (Clojure team)but it's hard for that to ever get enough priority#2021-11-0515:57jerger_at_ddaWe will spend one day on this ... let's see wether there is going sth on š#2021-11-0515:42Eugenwhat spec (or related) libraries are there for testing database services are there ?
I heard about such a library on a podcast but don't remember which one.
Context:
I would like to test a Java ERP solution and services will depend on data.
I am curious if generative testing can help with this.#2021-11-0518:04Colin P. HillWhat do you want the library to do?#2021-11-0707:12Eugenhi, I would like to do db integration tests and insert a bunch of related records based on the relationship betwe them.
I want to test user roles - and that requires user-role record but also depends on user records and maybe account or permission records.
These should be generated as well .#2021-11-0707:12Eugenkind of along those lines#2021-11-0707:12EugenI remember a library designed for this purpose - but don't remember which#2021-11-0813:15Colin P. HillIt sounds like you might actually be describing a library made at my company: https://github.com/reifyhealth/specmonstah
Obviously a bit of conflict of interest in me highlighting it, but if it helps, I haven't personally worked on it and haven't yet even learned how to use it š
#2021-11-0822:24Eugenyep, that is the one#2021-11-0822:24Eugenthanks#2021-11-0517:04qqq@alexmiller: Valid point, future (if any) debates on whether CFGs can recognize trees will be moved elsewhere. If you don't mind, I do have a question for you. 'spec' is basically a pattern language which, among other things, can be viewed as a shorthand for describing functions with type signature 'Tree -> bool' . (Given a spec, some s-exp trees are accepted; others are rejected.) Question: in the design of spec, is there some formal theoretical foundation spec is built upon (say, the tree equiv of regex / cfg / ll parsing / lr parsing / ...), or has it largely been a collection of useful techniques (i.e. we should be able to use it to validate pattern A, pattern B, pattern C, ... ).#2021-11-0517:53Alex Miller (Clojure team)Spec is built on two logical foundations - sets (maps as sets of attributes) and regular expressions (not regex as string matching but the more general idea). The latter impl is based on Matt Mightās work with regular expression derivatives, but that's really an implementation detail. This is not about trees or parsing, it's about whether a value is in the predicative set described by the spec. As it is inherently predicative, and predicates can be anything, this is I think much broader than pattern matching.#2021-11-0517:54Alex Miller (Clojure team)From day 1, the ideas of being predicate based, and automatically generative were considered to be required.#2021-11-0518:02Alex Miller (Clojure team)specs work on values, there is no s-exp tree#2021-11-0518:02Alex Miller (Clojure team)there's no explicit notion of trees at all#2021-11-0518:04Alex Miller (Clojure team)specs are descriptors/validators of sets of accepted values. some specs are composite and describe composite values. conceptually, I guess you could say those are trees, but that is more implicit than explicit.#2021-11-0518:05Alex Miller (Clojure team)if specs are "is this value in the set?", generators are "give me example values in that set", so these are intrinsically related#2021-11-0519:36Colin P. HillGiven a map and a spec that describes it, is there a way to yield a map which only contains those keys which were specified? I don't want to write my spec so that it rejects unspecified keys, but I also don't want to write what is effectively another description of the map shape in the select-keys
that I'll need before handing the map down to the database.#2021-11-0519:42Alex Miller (Clojure team)no, I know people have made things around spec#2021-11-0519:42Alex Miller (Clojure team)in spec 2 we have added a validation flag to validate in a "closed" mode#2021-11-0519:43Alex Miller (Clojure team)we very much believe this should not be a property of the spec (but it may be a useful property of an act of checking)#2021-11-0519:45Colin P. HillBig agree on that. I'm working with a custom extension to spec which adds closed map validations. It's been useful for making sure we don't hand things off in places where extra fields might be damaging, but there are other places where we want to describe the same shape while allowing it to be extensible. Currently trying to rewrite a use of it to follow some other strategy.#2021-11-0519:40qqqAre there any guarantees on the running time of spec? I.e. given a s-exp with at most N nodes and at most D depth, if there some big-Oh expression on the time it takes the spec validator to decide whether the s-exp belongs in the spec ?#2021-11-0519:41Colin P. HillA spec can be any arbitrary predicate, so not in the general case#2021-11-0519:41Alex Miller (Clojure team)given that predicates are arbitrary functions, no#2021-11-0601:41qqqSince spec allows arbitrary predicates, which can use arbitrary time / memory, how do most reason about the time/space usage of spec functions / algorithms ? Do they (a) not worry about it, since in most cases, it's fine or (b) have sufficient knowledge of spec internals to know at what time what predicate is called on what, etc ...#2021-11-0602:08seancorfield@qqq The same way you reason about any other functions in Clojure.#2021-11-0603:49qqq@seancorfield: Are you saying this because spec internals are so simple one should be able to reason about it's runtime like one reasons about map / filter / vec-ops ... ; or are you trying to end the conversation before the discussion on spec internals can start ?#2021-11-0603:58seancorfieldYou said "arbitrary predicates, which can use arbitrary time / memory" -- can you reason about those predicates?#2021-11-0604:00seancorfieldBecause if you start from the premise that the complexity of the predicates can be arbitrary, then you have to see that the complexity of Specs that are built on predicates can also be arbitrary, right?#2021-11-0604:03seancorfieldHow would you reason about any library that can take in arbitrary functions (predicates) and calls them on a data structure you pass in?#2021-11-0604:17seancorfieldTwo people have already said there are no guarantees on the run time of Spec -- and I agree -- and there are also no guarantees on the space usage (for the same reason: it's built on arbitrary predicates). But Spec isn't "magic", it's open source so anyone can inspect the source code and reason about it just as much as they can with any other open source library.#2021-11-0604:18seancorfieldI think, in general, people don't worry much about the internals of any library like that -- unless they see unpredictable or unacceptable time/space usage and then they profile their code and identify the root cause.#2021-11-0604:19seancorfieldBut with any of these complex libraries (rules engines and databases/query-based systems also come to mind), you just have to be careful about the complexity of the "queries" you run -- and there are nearly always pathological cases that catch us out. That's just kind of the way things are in software.#2021-11-0604:21seancorfieldAt work, we've used Spec very heavily ever since it was released. We use it for a wide variety of things (I've blogged about our usage of Spec at http://corfield.org if you're interested) -- and we do use it in production code for data validation. Most of the data structures we validate are pretty flat and, in general, relatively small -- so the Specs around them are relatively straightforward. And, in that context, we have seen no performance problems.#2021-11-0604:23seancorfieldBut there are (obviously) going to be data structures that are sufficiently complex that a full Spec is also going to be very complex and checking might be "slow". There might be values that have sufficiently complex semantics that a fully-fleshed out Spec might be "slow" -- but that's true whether you use Spec or whether you write the validation logic by hand (with the caveat that a generic system like Spec is always likely to have more overhead than hand-crafted code).#2021-11-0604:24seancorfieldDoes that help answer your questions @qqq?#2021-11-0604:51qqq@seancorfield That is a reasonable interpretation of my question, but not the way I was intending. Consider for a moment, a strict, non-lazy map. Let's call it smap. Let f be some arbitrary function. What is the running time of (smap f lst)
Now, you could make the argument that because f can take arbitrary memory/time, we can't say anything about (smap f lst). On the other hand, we can say, well it's running time is (reduce + (map (fn [x] (running_time 'f x)) lst)). One can argue about whether this is useful, but knowing the internals of how smap works definitely allows us to create some expression (in terms of f, lst) that expresses the running time of (smap f lst).
Now, if we look at https://clojure.org/guides/spec -- unless I am reading it incorrectly, it teaches the reader a mental model of "what value spec will return." What it does not seem to do however, is cover the internals of how spec computes the answer. I.e. it talks about "what", not "how".#2021-11-0604:52qqqSo now, what does this have to do with my earlier question of whether predicates can take arbitrary memory / time. Because spec allows this flexibility, it seems that the answer to "what is the running time of this spec query" depends on the predicates we pass it -- and this likely means that we need to figure out (1) how often will our predicate be called, (2) what data will it be called on -- and this requires a quite detailed knowledge of spec internals.#2021-11-0605:36seancorfieldBut this is true of any library that you pass both functions and data structures into and have it call those functions on those data structures. There's nothing novel about Spec in that regard.#2021-11-0604:55qqqLet's consider this example of Regex vs Spec. In normal regex, we can't specify arbitrary predicates. So it goes something like this: regex -> nfa -> dfa -> run DFA on input string. So we can say: well, the running time for a regex of size S on input of length N will be O(N & 2^S). [The DFA can be exponential in size of the NFA]. This is easy to assert without knowing too much about the internals of how the regex is matched.
On the other hand, in the case of spec, because we can pass arbitrary predicates; it seems to reason about spec's runtime / space, we have to figure out how often / when / what input the user specified predicates are called upon -- this is something which AFAIK can only be understood via understanding spec internals, i.e. HOW spec recognizes, rather than WHAT spec recognizes.
Does this sound reasonable? @seancorfield#2021-11-0605:36seancorfieldI already answered this in the thread that you either ignored or chose not to respond to.#2021-11-0605:00qqqOne practical question here is: do you ever do "untrusted user input => parse as edn => run spec on it"; do you know whether there exists some malicious input of size N which can cause spec to take 2^N (or even N^6) time ?#2021-11-0605:39seancorfieldJust parsing "untrusted user input" as EDN (or JSON) can be subject to malicious input attacks. That's why there are often recommendations to not do this -- but, practically speaking, people do this all the time. Adding Spec to the mix adds more processing and increases that risk.#2021-11-0605:04EddieSome of this is touched on in a few of the docstrings. https://clojuredocs.org/clojure.spec.alpha/every and https://clojuredocs.org/clojure.spec.alpha/coll-of come to mind. Both are intended to declare a collection where all the elements match a spec/predicate, but s/every
will sample the collection and s/coll-of
will exhaustively check each element.#2021-11-0605:11Alex Miller (Clojure team)I think most specs are pretty closely tied to the size of collections (s/coll-of will check all N elements of the coll, s/map-of will check all N entries of a map etc). if you're using regex specs, I would look to the regex derivatives papers for complexity bounds (https://matt.might.net/papers/might2011derivatives.pdf and related there but it's exponential in N). From a practical pov though, regexes are mostly used to describe syntax for things like macros, and in general most macro calls have small N (you don't invoke a macro with 100s of args or whatever).#2021-11-0605:12Alex Miller (Clojure team)"do you know whether there exists some malicious input of size N which can cause spec to take 2^N (or even N^6) time" - no, you don't know this, and it is possible.#2021-11-0605:36seancorfieldI already answered this in the thread that you either ignored or chose not to respond to.#2021-11-0605:39seancorfieldJust parsing "untrusted user input" as EDN (or JSON) can be subject to malicious input attacks. That's why there are often recommendations to not do this -- but, practically speaking, people do this all the time. Adding Spec to the mix adds more processing and increases that risk.#2021-11-0606:29qqqDoes spec do memoization / caching / dynamic programming? If not, can we generate 2^N time on O(N) size input as follows:
(s/def :foo (s/or :a something_that_uses_foo :c something_that_uses_foo))
then running it on s-expr of (1 (1 (1 (1 (1 (1 ... ))))))
This is similar to the cfg rule A => B | C , where B / C both use A. If we use plain backtracking, this can easily go exponential. In the case of strings, I believe packrat / peg gets around this issue by memoization -- using O(N * S) memory where N = length of input, S = # of states in grammar.
How does Spec handle this issue? Clearly it has to backtrack at s/or . The question is whether it memoizes results or if it just re-evaluates everything.#2021-11-0606:30qqq@seancorfield: I have a custom "untrusted string" -> s-exp parser that takes O(N) space, O(N) time and O(1) stack space. So the "untrusted user input => edn" step is not a problem for me. However, the "edn => spec" step currently scares me. (Among other things, the example just raised above).#2021-11-0607:11ikitommi@qqq here's an old post about using spec with untrusted inputs: https://quanttype.net/posts/2021-03-06-clojure-spec-and-untrusted-input.html#2021-11-0607:46qqq@ikitommi: Is this how the attack works? You pull you some dependency Foo that has defined some slow spec ::foo-slow-spec. You are unaware of these slow specs (and don't care because you're not explicitly using them). An attacker constructs a HTTP request that you run spec on. They add a field "::foo-slow-spec", which triggers the spec from Foo dependency. And the defense to this attack is to ensure that the incoming data only has keys explicitly mentioned in the spec?#2021-11-0608:07ikitommiyes. you could also have a slow spec in your codebase, as normal input, that is slow.#2021-11-1121:01dominicmIs there a version of flowing s/and for spec1 or a workaround that produces the same effect? (a version that doesn't pass conformed values to subsequent predicates, as described in https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new).#2021-11-1122:29colinkahn(s/and (s/nonconforming <your-spec>) ā¦)
#2021-11-1122:30colinkahnYouāll need to wrap each spec that you want to avoid conforming for in s/nonconforming
, more specifically#2021-11-1122:52dominicm@U0CLLU3QT you are wonderful, thank you kindly. You've got me out of a very tricky hole!#2021-11-2204:12Andrew ByalaHi, everyone. I have a beginner question about Spec's namespacing as it pertains to maps. I'd like to create two definitions within the namespace, such that the following passes:
(s/conform ::account {:account-id "abc123", :amount 50})
(s/conform ::transaction {:from-account "abc123", :to-account "def456", :amount 25})
Note that both definitions reference the same non-namespaced key of amount
, but I'd like to define the amount
of an account
to be an int between -100 and 100, and the amount
of a transaction
to be any positive int. Furthermore, I'd like to use the same spec for the account-id
on an account
, as well as both the from-account
and to-account
keys in a transaction
.
I can't see any way within a map to say that the value of a particular key corresponds to a specification by name. I'm trying to find a way to do something like this, but can't see how:
; This is not working code... I'm trying to find a way to get here.
(s/def ::transaction-amount pos-int?)
(s/def ::balance-amount #(<= -100 % 100))
(s/def ::account-id string?)
; Both :from-account and :to-account should be mapped to the spec for ::account-id
(s/def ::transaction (s/keys :req-un [[:from-account :as ::account-id]
[:to-account :as ::account-id]
[:amount :as ::transaction-amount]]))
; The :amount here is a ::balance-amount, but the :amount for a transaction is a ::transaction-amount
(s/def ::account (s/keys :req-un [::account-id
[:amount :as ::balance-amount]]))
#2021-11-2204:32seancorfield(s/def :account/amount ...)
(s/def :transaction/amount ...)
Then you can use those qualified names in the :req-un
part and it will match :account
in both cases, with different semantics.#2021-11-2204:36seancorfieldAlthough a lot of examples use ::
that's just shorthand for a qualified name that "happens to match" the current namespace -- but qualified keywords should really really reflect their domain semantics -- they don't need to match code namespaces.#2021-11-2204:36seancorfieldDoes that help @abyala?#2021-11-2204:39seancorfieldWith the account IDs, you can define a common ::account-id
and then just define the other names as aliases:
(s/def ::account-id ..)
(s/def ::from-account ::account-id)
(s/def ::to-account ::account-id)
And, again, as above you can use qualified names that match the domain semantics rather than ::
for the code namespace.#2021-11-2204:42Andrew ByalaNifty, @seancorfield. This seems to work for everything I'm looking for:
; This is reused for both transactions and accounts.
(s/def ::account-id string?)
(s/def :transaction/from-account ::account-id)
(s/def :transaction/to-account ::account-id)
(s/def :transaction/amount pos-int?)
(s/def ::transaction (s/keys :req-un [:transaction/from-account :transaction/to-account :transaction/amount]))
(s/def :account/account-id ::account-id)
(s/def :account/amount #(<= -100 % 100))
(s/def ::account (s/keys :req-un [:account/account-id :account/amount]))
#2021-11-2204:42Andrew ByalaIs that what you were suggesting?#2021-11-2204:43seancorfieldThat looks great, yes!#2021-11-2204:43seancorfieldAnd now you can see how the qualified keys provide separate semantic "namespaces" rather than code namespaces.#2021-11-2204:44Andrew ByalaYep, I see that. The piece I was missing was how :req-un
would strip that semantic namespace back off again, letting the actual key be whatever I wanted, Super helpful!#2021-11-2204:51Andrew ByalaSeparate question -- do you find you tend to create a separate namespace just to hold specs for other namespaces, especially if they're shared? Or do you tend to put specs within the namespaces containing other logic?#2021-11-2204:52seancorfieldSpecs for data structures tend to live in their own namespace, no code. Specs for functions live in the same namespace as the function. Roughly.#2021-11-2204:53seancorfieldI wrote about our use of Spec here https://corfield.org/blog/2019/09/13/using-spec/ based on what we do at World Singles Networks.#2021-11-2209:42vemvHas it been hammocked (officially or not) to give the notion of coverage to specs?
e.g. for an (s/or)
, verify at runtime, integrated with a given test suite, that all branches of the or
have been exercised.
(I know that generative testing is a thing! I want it the other way around here: start by code and reach the spec, not vice versa)#2021-11-2209:43vemvOne big use case, is that for any defn that returns s/or :result ,,, :error ,,,
I should rest assured that the test suite has coverage for the green and red paths of the spec alike#2021-11-2213:29Alex Miller (Clojure team)Haven't talked about it#2021-11-2213:52Colin P. HillIs there a writeup anywhere about the reason that instrument
doesn't validate :ret
and :fn
? I was wondering if this was considered and decided against, or there were some non-obvious difficulties. There's a library out there that does this, but I've got this nagging suspicion that it was left out of the first-party spec library for a reason.#2021-11-2214:00Alex Miller (Clojure team)https://clojure.org/guides/faq#instrument_ret#2021-11-2214:01Colin P. HillAh my bad, should have looked around for a FAQ, of course it's on there. Thank you!#2021-11-2422:11danieroux(s/def ::percentage (s/double-in :min 0 :max 100 :NaN? false :infinite? false))
(s/explain ::percentage (float 50))
#2021-11-2422:12danierouxGot tripped over by this just now. The data that came in was java.lang.Float. There is no s/float-in
as far as I can see?#2021-11-2503:46Drew Verleedouble-in says it specs a 64-bit floating point number
https://clojuredocs.org/clojure.spec.alpha/double-in
what did s/expain return?#2021-11-2503:47Alex Miller (Clojure team)Clojure doesn't really support floats in general (other than for interop). you can of course make your own spec for floats, or cast into a double etc.#2021-11-2508:21danierouxThank you, I will avoid using floats in Clojure then.#2021-11-2503:49Drew Verleemy brain just tripped over the fact a 64 floating point number isn't a float. err, or i guess it is, but not the 64 bit variety.
as an aside, are the other numbers just not buoyant?#2021-11-2920:00Oliver GeorgeHas anyone backported schema/select ideas from spec 2? I'd quite like them please.#2021-11-2920:10Alex Miller (Clojure team)I'm not aware of anyone that's done this, but as a warning the impl in spec 2 is pretty buggy and still a wip#2021-11-2920:25kenny(ish) ran out of time. Also buggy. https://github.com/ComputeSoftware/spec1-select#2021-11-2923:08Oliver GeorgeThank you both.#2021-11-3007:38Jakub HolĆ½ (HolyJak)Hello! How is it possible that a nillable spec (v1) fails with NPE? This works just fine and as expected:
366 (s/def ::org-ds (s/nilable #(satisfies? % jdbc.p/Connectable)))
367 (s/conform ::org-ds nil) ;=> nil
but when I check specs using s/instrument s s/fdef
then it fails in a test with NPE:
Caused by: java.lang.NullPointerException: null
at clojure.core$instance_QMARK___5404.invokeStatic (core.clj:144)
clojure.core$find_protocol_impl.invokeStatic (core_deftype.clj:536)
clojure.core$satisfies_QMARK_.invokeStatic (core_deftype.clj:569)
clojure.core$satisfies_QMARK_.invoke (core_deftype.clj:569)
jakub.specs$fn__224908.invokeStatic (specs.clj:367)
jakub.specs/fn (specs.clj:366)
clojure.spec.alpha$spec_impl$reify__2059.conform_STAR_ (alpha.clj:923)
clojure.spec.alpha$nilable_impl$reify__2556.conform_STAR_ (alpha.clj:1839)
clojure.spec.alpha$conform.invokeStatic (alpha.clj:164)
...
clojure.spec.alpha$deriv$fn__2425.invoke (alpha.clj:1537)
...
clojure.spec.alpha$regex_spec_impl$reify__2509.conform_STAR_ (alpha.clj:1703)
...
clojure.spec.test.alpha$spec_checking_fn$conform_BANG___3024.invoke (alpha.clj:121)
Why??? š#2021-11-3007:42flowthingHmm, doesn't satisfies?
take the protocol as the first argument?#2021-11-3010:47Jakub HolĆ½ (HolyJak)Thank you! You were right, incorrect arg order t osatisfies? seems to have been the problem#2021-11-3022:07johanatanin general, how flexible should specs be on the inputs that they accept / won't crash on. for example, in clojure.spec.alpha the kvs->map spec will crash if you pass it any non-collection:
=> (s/explain-data ::s/kvs->map [0])
nil
=> (s/explain-data ::s/kvs->map 0)
#object[Error Error: 0 is not ISeqable]
#2021-11-3022:07johanatanusing something such as: (s/and coll? ...)
would avoid this problem#2021-11-3022:07johanatan(for the definition of kvs->map)#2021-11-3023:25Alex Miller (Clojure team)Well that function is essentially an internal spec to support s/keys* so it's only used in a coll context#2021-12-0100:43johanatanyea, i had a few of those too but since these are registered "globally" they're fair game to my (spec-based) fuzzer. so, i'm making the ones in my project "defensive" against this (although it should never happen from production code)#2021-12-0100:52johanatani.e., this is where the inputs are coming from (with the one exceptional case i've found [which crashes] excluded):
(s/cat :spec (s/with-gen qualified-keyword? #(gen/elements
(clojure.set/difference
(set (keys (s/registry)))
#{::s/kvs->map})))
:val any?)
#2021-12-0108:06wontheone1Hello!
I made a function to return specs depending on arguments.
(defn- search-schema-spec-for-of-and-defaultValue [of defaultValue]
(s/keys :req-un [:build-api.search-schema/key
of
:build-api.search-schema/valueType
:build-api.search-schema/cardinality]
:opt-un [:build-api.search-schema/of
defaultValue
:build-api.search-schema/doc]))
Itās called like the following,
(search-schema-spec-for-of-and-defaultValue
:build-api.search-schema.listing/scope :build-api.search-schema.enum.one/defaultValue)
But I get the following error
Unexpected error (AssertionError) macroexpanding s/keys at (src/sharetribe/build_api/routes.clj:136:3).
Assert failed: all keys must be namespace-qualified keywords
(every? (fn* [p1__1917#] (c/and (keyword? p1__1917#) (namespace p1__1917#))) (concat req-keys req-un-specs opt opt-un))
I just wanna factor out common parts and make different specs based on of
and defaultValue
as they are the differentiator, any advice on how to achieve?#2021-12-0109:23vlaaaduse macros or eval instead of defn ā s/keys is a macro#2021-12-0110:35wontheone1Thanks! @U47G49KHQ
I defined common spec separately and combined with s/and
and this seems to work good enough for me!
(def build-api-search-schema-common-spec
(s/keys :req-un [:build-api.search-schema/key
:build-api.search-schema/valueType
:build-api.search-schema/cardinality]
:opt-un [:build-api.search-schema/of
:build-api.search-schema/doc]))
(s/and build-api-search-schema-common-spec (s/keys :req-un [:build-api.search-schema.listing/scope]
:opt-un [:build-api.search-schema.enum.one/defaultValue]))
...etc...
#2021-12-0116:55colinkahnIf youāre combining map specs you might want to look at s/merge
as well#2021-12-0117:19wontheone1Thanks#2021-12-0800:50Drew Verlee> For this we have exercise
, which returns pairs of generated and conformed values for a spec
What if we just want the generated values?#2021-12-0800:51Drew Verleei can fish them out, but i'm guessing there is a convention rather then threading with calls to first.#2021-12-0802:00Alex Miller (Clojure team)you can call gen/sample if you just want generated values https://clojure.github.io/spec.alpha/clojure.spec.gen.alpha-api.html#clojure.spec.gen.alpha/sample#2021-12-0802:01Alex Miller (Clojure team)sorry, https://clojure.github.io/test.check/clojure.test.check.generators.html#var-sample is the docstring for that#2021-12-0803:49Drew Verleethanks alex. ill read that shortly!#2021-12-0804:38Drew Verleethat likely does what i need. it would be nice if i had away to easily jump from one doc to another.
that fns args are are just & args and the doc string is that its a lazy version of another fn. not much to go off! i guess i should just jump to the code?#2021-12-0804:41Drew Verleewell, im guessing that would be very educational
(lazy-combinators hash-map list map not-empty set vector vector-distinct fmap elements
bind choose one-of such-that tuple sample return
large-integer* double* frequency shuffle)
#2021-12-0804:41Drew Verleebut not very direct š#2021-12-0816:53borkdudeI'm working on better clojure.spec support for babashka. I'm testing expound with instrumentation. How do people generally get to see expound's output when running with instrumented functions? What do you have to configure, besides the explain printer? It seems you have to catch the ExceptionInfo and push the ex-data through s/explain-out yourself?#2021-12-0816:54borkdudeBesides that question: what are some libraries that leverage clojure.spec that I should test.#2021-12-0817:11delaguardoreitit with spec based coercions#2021-12-0817:12borkdudereitit doesn't work because it isn't pure Clojure, it has custom Java classes#2021-12-1222:58vemvhttps://github.com/nedap/speced.def perhaps, we've wanted to use it for bb scripting sometimes (@UHJH8MG6S)#2021-12-2221:53lucian303spec tools: https://cljdoc.org/d/metosin/spec-tools/0.10.5/doc/readme#2021-12-0823:03abrooksWe have a library that provides and conforms against specs for incoming complex values. Within the values can be maps and s/keys
will, as the doc string says:
> In addition, the values of all namespace-qualified keys will be validated (and possibly destructured) by any registered specs.
If the caller puts their own namespaced keys in the map (which is fine) and they have defined specs for these, those specs will be validated (and conformed) when we validate or conform the larger structure.
This creates problems in providing good error messages since, when we conform, we don't know if the caller has violated our spec or their own spec. There's no way to conform only our parts of the spec first to validate the shape of their input without also conforming and failing on their values. This leads to a very confusing error message that we can't help to resolve.#2021-12-0823:04abrooksIs there a sane work around for this? I'd really like an s/keys
option or similar that doesn't validate keys beyond those expressed.#2021-12-0823:26Alex Miller (Clojure team)a) you could narrow the data before conforming
b) hopefully you can differentiate by key namespace either for narrowing or for error gen?#2021-12-1714:30abrooks(a) doesn't work since we actually want the data preserved in the conformed result.
I'll look at (b) but would love to mention that having local specs (similar to local Clojure hierarchies) is really what I think I want as a user. You can have the global spec (just like global-hierarchy
) but also provide a local subset or curated spec for a particular context.#2021-12-1714:35Alex Miller (Clojure team)yeah, that's not in line with the spec rationale https://clojure.org/about/spec#_global_namespaced_names_are_more_important#2021-12-1718:38abrooksOh, I love namespaced names and the general global usage is fine but, as Stu and Rich have noted, a la carte features are valuable for application domain use cases not just programming domain use cases. I'd love to have spec for application domain use cases which is exactly what hierarchies allows for. Global for programming domain, a la carte for application domain. Often the specs overlap so you want to be able to re-use them between the two.#2021-12-1718:38abrooksmetosin/malli supports this, FWIW.#2021-12-1020:07Zach Mitchell, PhDI'm trying to represent a piece of data of the form [:some-keyword int int]
where the two int
s should not be equal. Is there a way to encode this relationship between the two int
s in a spec? Should the spec just encode the shape of the data and leave validating this relationship to something else?#2021-12-1020:22Alex Miller (Clojure team)specs can be any predicate so the answer to all such questions is generally yes :)#2021-12-1020:23Alex Miller (Clojure team)one option is to combine a first structural spec with a second value checking one with s/and#2021-12-1020:23Alex Miller (Clojure team)s/and flows conformed values so that can provide structure for the second one#2021-12-1020:24Alex Miller (Clojure team)(s/and (s/cat :k keyword? :a int? :b int?) #(not= (:a %) (:b %)))
something like that#2021-12-1020:40Zach Mitchell, PhDExcellent! Thank you#2021-12-1411:04Ben SlessWith multi specs, unlike with or
I can't get back the matching case when calling conform. What should I do in case I want to take actions based on the spec the input matches?#2021-12-1412:06jkxyzYou can use whatever dispatch function you use for the multi-spec to also branch in other parts of the code. The obvious case is a :type
key or something similar#2021-12-1415:33Ben SlessI was hoping I wouldn't need to retread this path because I considered it an implementation detail#2021-12-1416:39jkxyzYou could use conformer
to tag each branch#2021-12-1417:01jkxyzIf it's a closed set of branches you can use or
with a predicate that matches on the "type" -- which I suppose is probably the reason that multi-specs don't conform with a tag. If you're trying to parse a structure against an open set of branches you could end up with any value as the tag, so you'd need another multimethod to dispatch to code which handles the conformed values. If you don't expect consumers of the namespace to extend the spec I'd say just use or
#2021-12-1417:05jkxyzIf it really is just an implementation detail then I'd say use conformer
to conform to some useful value#2021-12-1518:08seancorfieldI haven't tried to repro this (and therefore haven't commented on it) but wondered if someone here could take a look? https://clojureverse.org/t/does-clojure-spec-supports-records/8428#2021-12-1518:46Alex Miller (Clojure team)Records will generally only work with :req-un / :opt-un - this is covered in the spec guide https://clojure.org/guides/spec#2021-12-1518:51Alex Miller (Clojure team)oh, this is https://clojure.atlassian.net/browse/CLJ-1975#2021-12-1613:04danieroux(require '[clojure.spec.alpha :as s])
(s/def ::not-in-map-spec pos-int?)
(s/def ::in-map-spec string?)
(s/def ::the-map
(s/keys :req [::in-map-spec]))
(def m
{::in-map-spec "yes"
::not-in-map-spec -13})
(s/valid? ::the-map m) ; => false
(s/explain ::the-map m) ; => -13 - failed: pos-int? in: [:user/not-in-map-spec]
#2021-12-1613:05danierouxThis surprises me, that a key thatās not in my spec, makes the spec invalid?#2021-12-1613:08Alex Miller (Clojure team)This is covered in the docstring for s/keys - all keys are checked#2021-12-1613:48danierouxHmm. Thatās a conundrum in my usage, which I assume questions my usage:
I have a big-map. I want to grab a part of it and save it away. Then use the rest to flow along. I canāt grab a part of it, because the rest is invalid.
Do I explicitly select-keys to save it away?#2021-12-1810:58ikitommiYou could also use add-on libraries like spec-tools: https://cljdoc.org/d/metosin/spec-tools/0.10.5/doc/spec-driven-transformations#transforming-map-specs#2021-12-1614:05Alex Miller (Clojure team)Yes, you can do that#2021-12-1619:55pavlosmelissinos(s/def :demo/m1 (s/keys :req-un [::a ::b ::c ::d ::e]))
(s/def :demo/m2 (s/keys :req-un [::a ::b ::c]))
Does spec provide a way to go from m1 to m2 programmatically?
(I suppose I'm looking for the inverse of s/merge
)#2021-12-1620:10Alex Miller (Clojure team)spec is not a transformation engine, it's a validation/verification library#2021-12-1620:10Alex Miller (Clojure team)or are you asking about spec transformation#2021-12-1620:12Alex Miller (Clojure team)assuming so, not really, it really encourages building up by composition, not by taking away#2021-12-1620:18pavlosmelissinosI meant spec transformation, yes.
I'm writing generative tests based on some s/keys
specs and I'll have to dissoc some of the keys after the generation anyway, so I wondered if I could skip that step. Definitely not a deal-breaker and the justification behind the design decision is sound, as always. Thanks. š#2021-12-2020:31wilkerluciohello, is there a place that explains how to properly implement the s/Spec
protocol? I'm not sure about some of the methods, also, is there a place that explains the :path
, :via
and :in
parts of the spec problems? (I see in the spec docs it talks about but doesn't mention exact those keys)#2021-12-2020:43Alex Miller (Clojure team)No, we don't really consider that public api#2021-12-2020:44Alex Miller (Clojure team)But if you have a question I'll try to answer it :)#2021-12-2021:52wilkerluciomostly about the :path
, :via
and :in
, what is expect on each and what makes it different from one another?#2021-12-2021:58Alex Miller (Clojure team)there is some info at https://clojure.org/guides/spec#_explain#2021-12-2022:03Alex Miller (Clojure team)although those are words in the error message that map to the keys:
ā¢ :path
- vector of keyword tag "path" segments (usually from :or, :cat, :alt, etc)
ā¢ :via
- vector of spec names from the root
ā¢ :in
- vector of data values from root to failing value (will be omitted if it's the root value)#2021-12-2022:08Alex Miller (Clojure team)there's an implicit tree being recursively walked here. the nodes are specs (name = :via element). Each edge has a name (:path element) and the value to validate (:in element)#2021-12-2022:08Alex Miller (Clojure team)in some cases, some of those are synthetic, created by the spec#2021-12-2022:38wilkerluciothe examples I'm seeing are mostly related to s/keys
, where I see things like this:
(s/def ::a string?)
(s/explain-data (s/keys :req [::a]) {::a 1})
{:clojure.spec.alpha/problems ({:path [:com.piposaude.model.bulk-insert-test/a],
:pred clojure.core/string?,
:val 1,
:via [:com.piposaude.model.bulk-insert-test/a],
:in [:com.piposaude.model.bulk-insert-test/a]}),
:clojure.spec.alpha/spec #object[clojure.spec.alpha$map_spec_impl$reify__1998
0xabd19ae
"
#2021-12-2022:38wilkerlucioin this case all :path
, :via
and :in
are the same value#2021-12-2022:38wilkerluciobut from what I got to you, is like:
:in
- path from root
:path
- local path
:via
- specs in the process
is that a fair simplification of them?#2021-12-2022:39wilkerlucioand do you have examples that can clear up the distinction?#2021-12-2117:37Alex Miller (Clojure team)they are all "paths", but :in
is data, :path
is tags, and :via
is specs#2021-12-2111:46timoHow do I activate specs in the repl? Running my tests with kaocha fail with spec-validation but in the repl these do not show up. :thinking_face:#2021-12-2111:54pavlosmelissinostake a look at instrumentation: https://clojure.org/guides/spec#_instrumentation_and_testing#2021-12-2112:05timothanks :thumbsup: didn't connect instrument
to what I need#2021-12-2312:57thumbnailHi! We noticed clojure.spec sometimes generating #inst "NaN-NaN-NaNTNaN:NaN:NaN.NaN-00:00"
(which seems invalid).
Upon investigation we noticed the [impl dumps a large-integer into js/Date.
](https://github.com/clojure/clojurescript/blob/a4673b880756531ac5690f7b4721ad76c0810327/src/main/cljs/cljs/spec/gen/alpha.cljs#L113-L114) which can yield the above result.
Is this to be expected?#2021-12-2313:53Alex Miller (Clojure team)I'd say that's a bug, can you file a question at https://ask.clojure.org ?#2021-12-2313:53Alex Miller (Clojure team)It should really be filed under the test.check library, which is where the code is#2022-12-2613:09misha@alexmiller greetings!
https://github.com/clojure/core.specs.alpha/commit/938c0a9725a284095baa2387dff1a29c3f1e26ac
I wonder, why not just do?
(s/or ::local-name ::local-name
::seq-binding-form ::seq-binding-form
::map-binding-form ::map-binding-form
Had to port some code leveraging core.specs from 1.10 to 1.9 due to these.
I get the "alpha" status and all.
But the question above is a general one.
I'd assume dispatch keywords are as part of a spec as the spec name itself, since it is not wild to assume there would be a bunch of code dependent on conformed values, eventually. Why not use the same qualified keys where available (sans e.g. inline specs)? Is there any specific reason or unobvious consequence?
Thanks!#2022-12-2613:30Alex Miller (Clojure team)We wanted to make path names better as they show up in error messages#2022-12-2613:32mishagood point!#2022-12-2613:32Alex Miller (Clojure team)Generally we don't use qualified names for paths as they are contextual in the parent spec #2022-12-2613:37mishaI can see how it often applies to regex
specs.
For most or
specs I found ::
way more suitable. Sans the ugly unreadable path, ofc.#2022-12-2613:45Alex Miller (Clojure team)The path is the whole point #2022-12-2622:46bortexzI would like to spec a coll-of maps that is sorted on a specific keyword, is there a simple way to do this?#2022-12-2701:27colinkahnSomething like (s/and (s/coll-of map?) #(apply <= (map :some-kw %)))
perhaps (tested using CLJS so not sure if Java can compare using <= on strings).#2022-12-2709:14bortexzI was trying to avoid the custom predicate, but I guess thereās no other way, thanks!#2022-01-0111:44Ho0manHi everyone, an extremely weird thing is happening using spec.alpha
... Please check this simple snippet. Why does the spec/explain-data
at the end try to validate the sample
record against the dfntn-spec
when I'm not even including it in the spec/keys
passed to it ??! Am I doing something wrong or is this a bug ? :
ā¢ Clojure Spec Version : [org.clojure/spec.alpha "0.2.194"]
(ns hermes.lib.system.alaki
(:require [clojure.spec.alpha :as spec]
[clojure.string :as clj-str]
[com.stuartsierra.component :as component]
[hermes.lib.system.utils :as utils]
[taoensso.timbre :as timbre]))
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
(defn multimethod-dispatch-fn
[x]
(let [system? (-> x :component/system? true?)]
(case system?
true (-> :systemized-component)
false (-> x :component/type))))
(defmulti dfntn-spec multimethod-dispatch-fn)
(defmulti create multimethod-dispatch-fn)
(spec/def :component/name simple-keyword?)
(spec/def :component/config
(spec/multi-spec dfntn-spec
multimethod-dispatch-fn))
(spec/def :component/deps
(spec/map-of simple-keyword? simple-keyword?))
(comment
(def sample {:component/type [:Schema :v-0-0-1]
:component/system? false
:component/name :schema-1
:component/config {:serde :nippy
:topics [{:topic-name "~~"
:partitions 1}]}
:component/deps {}})
(spec/explain-data
(spec/keys :req [:component/name
:component/deps
])
sample)
)
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
#2022-01-0112:36Ho0manHi everyone, here is another instance that I just don't get what's going on :
(ns hermes.lib.system.alaki
(:require [clojure.spec.alpha :as spec]
[clojure.string :as clj-str]
[com.stuartsierra.component :as component]
[hermes.lib.system.utils :as utils]
[taoensso.timbre :as timbre]))
;;------------------------------------------------------------------;;
(defn multimethod-dispatch-fn
[x]
(let [system? (-> x :component/system? true?)]
(case system?
true (-> :systemized-component)
false (-> x :component/type))))
(defmulti config-spec multimethod-dispatch-fn)
(defmulti create multimethod-dispatch-fn)
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
(spec/def ::component-config
(spec/multi-spec config-spec
multimethod-dispatch-fn))
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
(def sample
{:component/type [:Schema :v-0-0-1]
:serde :nippy
:topics [{:topic-name "~~"
:partitions 1}]})
(defmethod config-spec [:Schema :v-0-0-1] [& _] any?)
(spec/valid? (config-spec sample) sample)
(spec/explain-data (config-spec sample) sample)
(spec/valid? ::component-config sample)
(spec/explain-data ::component-config sample)
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
While the last spec/valid?
returns true the subsequent spec/explain-data
returns :
#:clojure.spec.alpha{:problems [{:path [[:Schema :v-0-0-1]], :pred hermes.lib.system.alaki/config-spec, :val {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}, :via [:hermes.lib.system.alaki/component-config], :in []}], :spec :hermes.lib.system.alaki/component-config, :value {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}}
ā¢ Spec Version : [org.clojure/spec.alpha "0.3.214"]
#2022-01-0113:52lassemaattaI think that the behavior you noticed in your first example is normal for s/keys
, see the docstring:
> In addition, the values of all namespace-qualified keys will be validated (and possibly destructured) by any registered specs.#2022-01-0310:54Ho0manYear you're right. Thanks a lot @U0178V2SLAY.#2022-01-0112:36Ho0manHi everyone, here is another instance that I just don't get what's going on :
(ns hermes.lib.system.alaki
(:require [clojure.spec.alpha :as spec]
[clojure.string :as clj-str]
[com.stuartsierra.component :as component]
[hermes.lib.system.utils :as utils]
[taoensso.timbre :as timbre]))
;;------------------------------------------------------------------;;
(defn multimethod-dispatch-fn
[x]
(let [system? (-> x :component/system? true?)]
(case system?
true (-> :systemized-component)
false (-> x :component/type))))
(defmulti config-spec multimethod-dispatch-fn)
(defmulti create multimethod-dispatch-fn)
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
(spec/def ::component-config
(spec/multi-spec config-spec
multimethod-dispatch-fn))
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
(def sample
{:component/type [:Schema :v-0-0-1]
:serde :nippy
:topics [{:topic-name "~~"
:partitions 1}]})
(defmethod config-spec [:Schema :v-0-0-1] [& _] any?)
(spec/valid? (config-spec sample) sample)
(spec/explain-data (config-spec sample) sample)
(spec/valid? ::component-config sample)
(spec/explain-data ::component-config sample)
;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
While the last spec/valid?
returns true the subsequent spec/explain-data
returns :
#:clojure.spec.alpha{:problems [{:path [[:Schema :v-0-0-1]], :pred hermes.lib.system.alaki/config-spec, :val {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}, :via [:hermes.lib.system.alaki/component-config], :in []}], :spec :hermes.lib.system.alaki/component-config, :value {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}}
ā¢ Spec Version : [org.clojure/spec.alpha "0.3.214"]
#2022-01-1020:15slipsetWe were discussing at work today, and this cropped up:
ardoq.core> (spec/def :foo/foo string?)
;; => :foo/foo
ardoq.core> (spec/def :foo/bar int?)
;; => :foo/bar
ardoq.core> (spec/def ::test (spec/keys :req [:foo/foo]))
;; => :ardoq.core/test
ardoq.core> (spec/valid? ::test {:foo/foo "foo" :foo/bar 1})
;; => true
ardoq.core> (spec/valid? ::test {:foo/foo "foo" :foo/bar "1"})
;; => false
In the last example, :foo/bar
is not mentioned in ::test
but seems to still be checked. What am I not seeing?#2022-01-1020:19mpenets/keys will cause all namespace qualified keys to be checked#2022-01-1020:23slipsetRight, itās there in the docstring:
> In addition, the values of all namespace-qualified keys will be validated
> (and possibly destructured) by any registered specs.
TIL.#2022-01-1020:26slipsetWhich still is a bit strange since then:
ardoq.core> (s/def :foo/foo string?)
;; => :foo/foo
ardoq.core> (s/def :foo/bar int?)
;; => :foo/bar
ardoq.core> (s/def ::test (s/keys :req []))
;; => :ardoq.core/test
ardoq.core> (s/valid? ::test {:foo/foo "foo" :foo/bar "1"})
;; => false
ardoq.core>
Which makes you wonder why even bother with :req
?#2022-01-1020:28mpenetYou still need to be able to express what key set is required#2022-01-1020:28mpenetI guess for opt it's debatable. It's mostly for gen there#2022-01-1020:29slipsetTrue#2022-01-1020:30mpenetAlso if you have a ns qualified key that points to a value that does not conform to the spec with the same key, no matter where, it's likely a smell. Integrant pushes toward this for instance, I am not a fan of that part of the lib.#2022-01-1020:32Alex Miller (Clojure team)opt is used for gen#2022-01-1114:08mpenetthat might be of interest here#2022-01-1310:52Jakub HolĆ½ (HolyJak)Is there some place I can check for the status of spec2 and whether there is any tentative timeline? š#2022-01-1312:57Alex Miller (Clojure team)No tentative timeline#2022-01-1314:28vlaaadhow about status?#2022-01-1314:44Alex Miller (Clojure team)not sure what you're looking for#2022-01-1314:44Alex Miller (Clojure team)it exists in development state#2022-01-1315:05vlaaadMaybe Not was given in November 2018. Itās been 3 years since the announcement of spec 2. In no way Iām trying to rush Clojure development ā I see a lot of stuff done during this time, particularly around tools deps, but Iām nonetheless curious if spec alpha 2 is delayed due to other areas having higher priority during these 3 years, or was there maybe some unexpected turn of events that resulted in spec design going into unforeseen direction? I remember reading something about function specsā¦ Curious how itās going there.#2022-01-1315:15Alex Miller (Clojure team)still thinking about how to express the relationships we want to express at the function spec level. the provide/consumes stuff in Maybe Not is part of it, and mostly realized in schema/select, but there are other aspects too like (for example) wanting to be able to (easily) say that the elements in the output of filter
are a subset of the elements in the input. and how to integrate that with the params in function definition in a clean way. I think we'll be spending more time on spec in '22.#2022-01-1314:10Ronny LiHi everyone, in https://m.youtube.com/watch?v=YR5WdGrpoug from 2018, Rich talks about defining a function that only requires a subset of a previously defined schema using s/select
. Is this available now? I'm asking because I looked through the spec docs but didn't see anything. I might've just missed it though.#2022-01-1314:38mpenetNope. There is a lib that attempted to bring that to spec1 but it's an experiment I think.#2022-01-1314:46Alex Miller (Clojure team)spec2 is a wip and has schema/select https://github.com/clojure/spec-alpha2 (docs: https://github.com/clojure/spec-alpha2/wiki/Schema-and-select - differs a bit from the talk)#2022-01-1817:53MatElGranHi all, cross-posting here from #beginners hoping to have some feedback to my (super clojure beginner) problem, hope itās ok. https://clojurians.slack.com/archives/C053AK3F9/p1642508701489300#2022-01-1910:27jerger_at_ddaWhile sitting in a activity-pub workshop ... does anybody know a clojure based activity-pub implementation?#2022-01-1915:30Ben SlessMaybe this would be better as a general #clojure or #find-my-lib q?#2022-01-1916:37jerger_at_ddathe reason to ask here was that I would expect spec to be the foundation. But I will try your suggestions š#2022-01-2903:08Drew Verlee@U3LRXRAT0 Are we talking about this? https://github.com/BasixKOR/awesome-activitypub ?
I can't see how spec will help with implementing a Protocal, spec helps validate clojure data....
Skimming this https://www.w3.org/TR/activitypub/ makes me think that all this protocal is just like a recommendation on how label your data so other actors can try to read it? Which other actors? Why would i want to send them messages?#2022-01-2909:38jerger_at_ddaYes, you found the intended activity pub :-)
Activity pub's data-structures are described by linked data = LD (a global way to describe data structures // labels but also composition & validity). Spec is deep inspired by LD, so their concepts maps perfectly together. That's the reason why I expect spec will play a relevant part in a clojure adaption of activity pub.
The value add of LD is to describe data in case of integration between systems. You can describe data in many other ways ... but I know no other way integrating more or better - so LDs value is that your not forced to invent an other (imperfect) way to express your integrated data structures.
ActivityPub adds a way to consume and publish data in a asynchronous and only temporarily connected world. So your systems avail. is not bound strictly to the availability of the systems integrated by yours. That's not trivial to implement ...#2022-01-3123:51hiredmanI looked at implementing the ld spec described algorithms in clojure at one point. The description of the algorithms were extremely imperative when I looked at them, constantly mutating maps, and pretty much assumed you were writing javascript#2022-01-3123:52hiredmanhttps://json-ld.org/spec/ED/json-ld-api/20121225/#context-processing kind of thing#2022-02-0110:01pavlosmelissinosOk so here's a scenario I'm not sure how to approach:
I've changed some specs and introduced a bug. Now my generative tests don't work because "Couldn't satisfy such-that predicate after X tries." I'm pretty sure I've identified the spec that contains the bug but I can't tell exactly why it's wrong.
My first idea was to generate a config (even if it doesn't satisfy all the predicates) and then use explain-data on it and navigate :clojure.spec/problems
. However, :clojure.spec/gen
and :clojure.test.check.generators/sample
refuse to even show me the generated data if it's invalid. Is there an alternative that would work in this case?#2022-02-0110:05lassemaattahave you tried generating values directly from that "possibly broken" spec instead of the whole config?#2022-02-0110:08lassemaattaI usually debug such problems by starting from the simplest case which generates successfully and then gradually add more restrictions until I find the spec/predicate, which fails to pass the data#2022-02-0110:26pavlosmelissinosSure but I was wondering if spec could assist with that.
The culprit was a custom predicate that's part of the property tests. It's used to ensure that the generated data satisfies certain properties (e.g. make sure that a key only appears once in a vector of maps).
However, in the process it introduced a regression (it used concat on a vector, which turned it into a list).
After some digging I figured it out and fixed it; I was just wondering if there was a less manual way to do it, with spec, to save me some time in the future.
As in:
1. generate the data
2. run explain-data
3. get back a map with s/problems
4. see something like ":v should be a vector, not a list"#2022-02-0114:21Alex Miller (Clojure team)There has been some work in test.check to provide better reporting and eventually I would like to help surface some of that through spec for cases like this#2022-02-0114:45pavlosmelissinosThat's good to know, thanks š
To be honest I'm a bit surprised there's no way to get the generated data if it doesn't satisfy all the predicates but I'm sure there's a reason it's like that. I mean I understand it's an all or nothing situation but on the other hand having the incomplete state would be useful for debugging.#2022-02-0114:52Alex Miller (Clojure team)there's good reason to verify generated values match the spec, but there is a general problem with not getting example failures if no matching generated value can be found#2022-02-0110:10karol.adamiecwith spec, is it good idiomatic usage/taste to say that for example a reduce over collection should satisfy a predicate?
in real terms, lets imagine we have a spec of a Security. And we build a holding. So each security, that is included in holding has a weight. And naturally all weights must sum up to a 100.
Is this too convoluted and going too far with spec? Goal is to use spec not only to validate inputs, but also generate artificial holdings, that make sense.#2022-02-0110:19vlaaadYou can have arbitrary predicates composed with s/and
, itās very idiomatic IMO#2022-02-0110:20vlaaade.g. your holding spec can be something like (s/and (s/coll-of ::security) holdings-add-up-to-100?)
where holdings-add-up-to-100?
is your predicate#2022-02-0110:20karol.adamiecnice. good to know... now off to write a generator š#2022-02-0110:20vlaaadyeah, that š#2022-02-0110:20lassemaattathat's the fun part š#2022-02-0110:21vlaaadbut I would say itās all idiomatic and good taste#2022-02-0110:21vlaaadespecially the generator#2022-02-0110:22karol.adamiecyeah, sounded like a good idea, but it is easy to get overexcited and go in directions one was not meant to travel in...#2022-02-0110:25vlaaadthat depends on where your specs are going to be used. if itās for testing the code that relies on āholdings add up to 100ā invariant, then it makes sense to invest in generator. If that code is not of super importance, you can have your generator to be a set of examples as a starting pointā¦#2022-02-0110:26karol.adamiecyeah, ideally we would use this extensively in unit tests. First, to catch edge cases that real data does not expose easily. and second to get rid of huge and obnoxius fixtures. š#2022-02-0110:27karol.adamiecbut a good point, generator writing is optional.#2022-02-0712:17Benjamin(defn foo [] 'fo)
(s/fdef foo :ret int?)
(stest/instrument `foo)
(foo)
I don't get how :ret
in fdef works. Why doesn't it throw when I do this?#2022-02-0712:19Lennart Buitret specs are not instrumented by stest/instrument
#2022-02-0712:19Benjaminah#2022-02-0712:22Lennart Buityou can use stest/check
to check the entire function spec, as per https://clojure.org/guides/spec#_testing#2022-02-0914:42BenjaminThere is no way in keys
to say "use this spec" like:
(s/def ::foo (s/keys :req [(::bar string?)]) )
something like this right? What is the most concise alternative?#2022-02-0914:52Alex Miller (Clojure team)you mean to specify an ad hoc spec for a key? no#2022-02-0914:53Alex Miller (Clojure team)(s/def ::bar string?)
(s/def ::foo (s/keys :req [::bar]))
#2022-02-0914:53Alex Miller (Clojure team)spec 2 has some facility for specifying ad hoc specs for unqualified keys in a schema#2022-02-0914:55Benjamin(s/def ::foo
(fn [m]
(and
(s/valid? ::foo-default-keys (dissoc m ::special))
(s/valid? ::foo-special-key (select-keys m [::special])))))
if I do this my design is wrong I feel like#2022-02-0915:00Alex Miller (Clojure team)why is ::special so special?#2022-02-0915:01Alex Miller (Clojure team)I find things that are hard to spec often point to problems in the code/data design#2022-02-0917:29AronWouldn't this statement be true in general? What I mean is, when something is problematic it's not enough to know it's problem in "code/ data design" because it's not clear how to unproblem it š#2022-02-1022:07Braden ShepherdsonI'm running into this with some fdef
specs in my tests. apparently there's some other problem that spec is endeavouring to explain?
https://clojure.atlassian.net/browse/CLJ-2481#icft=CLJ-2481#2022-02-1022:12Alex Miller (Clojure team)you can see in the comments this was closed as a dupe of https://clojure.atlassian.net/browse/CLJ-1975 which is still open#2022-02-1022:13Alex Miller (Clojure team)You can work around it by adding a :kind predicate to the s/coll-of#2022-02-1120:59QuestHow are you shimming Clojure.spec instrumentation in dev & test, particularly with deps.edn? I was using lein :injections
for this but find no equivalent mechanism in the deps.edn world.
A test fixture solves test
part, but I cannot find any version of this solution that doesn't require a (use-fixture ...)
at the top of every test namespace. Does someone have an alternative solution? (Recommend an improved test runner?)
Shimming in :dev
seems to have two unclean approaches.
1) Make the editor inject on cider-connect [which varies for each editor]
2) Make the app itself support spec instrumentation as part of its official config-driven initialization -- & wait until point of invocation to (require)
spec test.deps (as these aren't available in prod build)#2022-02-1121:02QuestClosest solution for dev
I've seen is https://underwhelm.net/blog/2019/12/19/tools-deps-injections/ , which works but is fragile#2022-02-1121:35QuestFrom reading around, seems the solution of #2 is the "best approach," but it's a heavier pitch to my new team than I'd like https://github.com/jeaye/orchestra/pull/52#2022-02-1215:07vemvas much as I'm a Lein fan / advanced user I really dislike :injections
- they don't map 1:1 to an actual Clojure future so often people end up devising non-clean solutions that other folks are certainly solving in an idiomatic way.#2022-02-1215:10vemvuse-fixtures
is fine and has the advantage that it will work with basically every test runner. You can always roll a tiny linter ensuring that each ns has this declaration.
Otherwise using a bespoke test runner seems acceptable, there are a few e.g. eftest
, circleci.test
, kaocha
, Polylith's.#2022-02-1417:36QuestHmm, I raised the use-fixtures PR but it changes 70 files. & you can't mix :use
& :each
fixtures in test. Making the linter check for the fixture is a good idea.
If the team balks I'll mention some of the other test runners; it might be a hard sell, but would be cleaner in the end.
Thanks for the thoughts & advice!#2022-02-1615:40Colin P. HillIs there any way to spec a map that can mostly be described with map-of, but has a few special keys with their own types? E.g.
{::special-1 {:I-should-be :a-map}
::special-2 :I-should-be-a-keyword
"standard-key-1" "standard-val-1"
"standard-key-2" "standard-val-2"
; ...
"standard-key-n" "standard-val-n"}
#2022-02-1615:42Colin P. Hill(This is a structure already in use that I'm speccing for documentation purposes. I wouldn't generally design something that functions both as a record and as a map of arbitrary well-typed values.)#2022-02-1615:43kennyDoes s/merge a s/map-of and s/keys help?#2022-02-1615:45Colin P. HillThat was my first thought, but I'm not seeing a way to write the map-of part. If I did (map-of string? string?)
, the special keys and values would fail validation.#2022-02-1615:46Alex Miller (Clojure team)yes, you can do this by spec'ing it as a collection of kv pairs and then describe the pairs. I have a blog post about the destructuring spec at https://cognitect.com/blog/2017/1/3/spec-destructuring that has this characteristic#2022-02-1615:48Alex Miller (Clojure team)the key is you want (s/every (s/or ...kv-tuple-shapes...) :into {})
#2022-02-1615:56Colin P. HillHmm, I've been defaulting to using coll-of rather than every, but I'm seeing that it blows up when I try to do that with :into {}
#2022-02-1616:07Alex Miller (Clojure team)blows up how?#2022-02-1616:34Colin P. Hill(s/conform (s/coll-of (s/cat :k string? :v string?) :into {}) {"foo" "bar" "baz" "bork"})
Execution error (Error) at (<cljs repl>:1).
nth not supported on this type cljs.core/PersistentArrayMap
=> :repl/exception!
Could be a quirk of the cljs implementation#2022-02-1616:39Alex Miller (Clojure team)no, that's why the example above uses s/every
instead#2022-02-1616:39Colin P. HillOkay, figured that might be the case, it's just surprising to me#2022-02-1616:39Alex Miller (Clojure team)coll-of assumes the coll is nth-able (which maps aren't). we may actually have a ticket to consider that, don't remember#2022-02-1716:29SimonHello
Very Senior Clojure Spec question here:
Iām new to Clojure-spec, but have been using Clojure for about a year now. During my thesis I implemented the compiler for Secure Guarded Commands by Flemming Nielson (my supervisor). https://link.springer.com/chapter/10.1007/978-3-030-41103-9_7
The language syntax allows the compiler to analyse the explicit, implicit, bypassing, correlation, sanitised, and indirect information flows. Such that to make sure at compile time that no information is leaked.
Would it be valuable to add similar information flow analysis to Clojure Spec?#2022-02-1716:32Alex Miller (Clojure team)what problem would this solve?#2022-02-1716:35Alex Miller (Clojure team)I guess I'm wondering if you're looking to extend spec or use the information in specs to do this analysis independently?#2022-02-1717:14SimonAt a minimum something like this:#2022-02-1717:22SimonI think i have some people doing something similar. https://dl.acm.org/doi/pdf/10.1145/3468264.3473127#2022-02-1717:28Alex Miller (Clojure team)that image makes me think you are wanting something even more fundamental, like a change to the language#2022-02-1717:29Simonnot neccessarily, I think we could also use a syntax that is closer to that of spec.#2022-02-1717:30Simonitās sort of an extension of a type checker.#2022-02-1717:31Simonand if clojure.spec checks types, then I think it would be possible to do?#2022-02-1717:32Alex Miller (Clojure team)well, spec is emphatically not a type checker :)#2022-02-1717:32Alex Miller (Clojure team)it's a predicate value system applied at runtime#2022-02-1717:34Alex Miller (Clojure team)some people have explored using it for static checks (https://github.com/arohner/spectrum and some work with https://github.com/clojure/core.typed)#2022-02-1717:35Alex Miller (Clojure team)but that's certainly not the conception or use of it in core#2022-02-1805:09Drew VerleeAlso, philosophically speaking, correctness isn't a property of types, but of ... <waves hands>...#2022-02-1813:50Colin P. HillWhat's the difference between a type system and a predicate value system? If a type is a set of values, and a set can be defined by a membership predicate, then it seems like these are the same thing implemented in different ways#2022-02-1918:05Ben SlessThe core predicates are, but can you represent in a type system a sorted collection? Represent constraints such as "the index keys set must be equal to the set of values associated with this key for all records"?
Predicates are way looser, you'll have to work hard to formalize an entire type system around them#2022-02-1918:07Ben SlessIf Phil Wadler and his friends are right, type systems are logic systems. Find a logic system that is predicative and you've found the appropriate type system#2022-02-2114:05Colin P. HillI can represent in a type system a sorted collection ā if the type system is implemented with predicates! That is to say, the distinction you're drawing seems to beg the question: if predicative systems count as type systems, then the answer to your question is trivially "yes".#2022-02-2114:06Colin P. HillStatic type checkers are always developing new capabilities. Dependent and refinement types are an area of active research, and Rust's linear types were theoretical until Rust implemented them. Before these implementations, it still would have been correct to describe those ideas abstractly and call them type systems.#2022-02-2114:07Colin P. HillSimilarly, coherent models which lack a total formalization, like runtime predicate checks, might also reasonably be called type systems, just as a statically typed language built without a theoretical model in mind already has a type system waiting to be described.#2022-02-2114:28Ben SlessI'm asking a different question - what is the logic system which corresponds to predicative types. There is some calculus associated with it.
The realization type systems correspond to a logical calculus system (such as lambda calculus) takes it far from type checkers. There is a formal theory there waiting to be discovered#2022-02-2114:47Colin P. HillI don't really know what logic system would correspond to it, but I'm not sure why we would need an answer to that question. We've had plenty of type systems whose formal theories were only developed after the implementations, haven't we?#2022-02-2114:51Ben SlessI don't think so. These developments have sometimes been independent and later converged under the same name, see HM type system, one logician one computer scientists. The field of logic, being less constrained, usually develops these first, but it is not a strict requirement#2022-02-2114:52Ben SlessMaybe predicate types will be the first#2022-02-2114:56Ben SlessMaybe you can reduce them to another system#2022-02-2115:22Colin P. HillThe existence of this paper would suggest to me that Java's type system was first built, and only later analyzed formally: https://www.researchgate.net/publication/220299005_A_Formal_Type_System_for_Java#2022-02-2115:23Colin P. HillA skim suggests to me that, as of that paper's publication, the formal analysis work wasn't even yet complete (but I might be wrong about that)#2022-02-2115:31Ben SlessI'm drawing a distinction between a specific type system implementation, a theoretical type system, and a logic system.
Could be that formal analysis was applied to Java's type system after it was written, but it must have corresponded to an existing logical system (or a yet undiscovered one)
https://www.youtube.com/watch?v=IOiZatlZtGU#2022-02-2119:38bronsa> Rust's linear types were theoretical until Rust implemented them
FYI there's an impl of linear types in this 1991 paper https://www.cs.utexas.edu/users/hunt/research/hash-cons/hash-cons-papers/BakerLinearLisp.pdf#2022-02-2206:03Ben SlessI think linear types relate to substructural logic
https://en.m.wikipedia.org/wiki/Substructural_logic#2022-02-2217:00Colin P. Hill> FYI there's an impl of linear types in this 1991 paper
I stand corrected! But I think my point is still valid, and I hope it's still clear#2022-02-2217:02Colin P. Hill> Could be that formal analysis was applied to Java's type system after it was written, but itĀ must haveĀ corresponded to an existing logical system (or a yet undiscovered one)
That last parenthetical note makes all the difference, imo. It means that one does not need to point to a known equivalent logical system to be able to say that something is a type system.#2022-02-2217:26Ben SlessThe idea underlying this is that there isn't a sound type system that doesn't have a corresponding logical system. You either find the one which already maps on to it, or discover a new one š#2022-02-2219:49Colin P. HillWell hold on, adding sound as a qualifier changes everything š Plenty of real-life type systems have been unsound, and have in some cases even been useful despite that.#2022-02-2219:51Colin P. HillWhich is why I don't think the question of logic systems bears upon my original question. Something can fail to correspond to any known logical system, or even be found to be unsound when you try to formalize it, while still being a type system.#2022-02-1813:50Colin P. HillHeck, some of the core predicates are type checks, just at runtime#2022-02-1820:28respatializedThis conversation seems like it's going to open Pandora's box:
http://lambda-the-ultimate.org/node/5604#2022-02-1820:29respatializedhttp://lambda-the-ultimate.org/node/412#2022-02-2318:43jjttjjLet's say you have to integrate with several providers by providing them an endpoint the receives json from them and must return json. The json structures each provider gives you and expect back are pretty different even though the essence of the operation is always sort of the same, though each platform has its quirks. I attempt to smooth over some of the differences, and build out my functionality on a common internally use data structure.
In these situations, do you bother spec'ing the structures given/expected by each provider? Or the internal common structure only? Or both?
Originally I thought I could get away with focusing on only my internal structure, but I'm finding the platforms are different enough in complicated ways that I really want specs for each one.
My usage of spec here is to aid in writing tests (not necessarily generative ones)#2022-02-2320:13Colin P. HillSince they don't turn domain changes into compilation errors like static types sometimes do, afaict the worst case scenario is that you decide later that keeping them up to date isn't worth the effort and just delete them. No harm in just writing them now if they're useful now, imo.#2022-02-2320:16Colin P. HillPersonally I'm a big fan of specs as a way of documenting data structures, even if I never use them for anything else. The fact that I can instrument them for tests and REPL sessions is just gravy.#2022-02-2403:38jjttjjMakes sense, thanks!#2022-02-2407:28Ben SlessBoth. This lets you have a contract with them on one hand,a and verify your transformation to the common model on the other#2022-02-2819:24zendevil.ethSo here we have :req and :opt to be used in s/keys. That doesnāt stop other keys tht are not listed in opt or req from being accepted right? How do you limit the keys that can be accepted to only the req and opt keys?#2022-02-2819:24Braden Shepherdsonthat's deliberately not supported by spec. use select-keys
or similar.#2022-02-2819:30zendevil.ethwhy this design decision?#2022-02-2819:39Braden Shepherdsonthe focus was on enforcing minimum needs.#2022-02-2819:40Braden Shepherdsonspec 2 does allow for this, because there are legitimate use cases for it. and the implicit checking of bonus keys with their own specs can be a DoS attack vector, if those other keys have slow specs.#2022-02-2819:40Braden Shepherdsonworse, since it's extra data getting validated, you can send perfectly valid requests that take tens of seconds or more to validate, but don't stand out in monitoring or logs as errors or unusually large...#2022-02-2819:41zendevil.ethwhatās the solution to the dos attack when comprehensively validating the input#2022-02-2819:42Braden Shepherdsonhttps://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#closed-spec-checking here's the spec 2 docs.#2022-02-2819:42zendevil.ethso we should use spec2 right?#2022-02-2819:43Braden Shepherdsonwell, perhaps not. it's still alpha. just mentioning it.#2022-02-2819:43zendevil.ethisnāt spec alpha too?#2022-02-2819:43Braden ShepherdsonI'd say the most straightforward thing is to just select-keys
first, then validate with spec 1.#2022-02-2819:43Braden Shepherdsonsure, but it's been battle tested in a way the spec 2 hasn't.#2022-02-2819:46Alex Miller (Clojure team)spec 2 has not been released, and you shouldn't use it for real stuff#2022-02-2819:55jjttjjStating the obvious, but you can always use a custom predicate to limit the spec in addition to s/keys
#2022-02-2822:09zendevil.ethLetās say Iām trying to conform a map to a spec. I want each key of the map to have values of a certain type. How can I write a spec for that?#2022-02-2822:10zendevil.ethI have this:
(s/def ::create-sessions-params (s/and (s/keys :opt-un [::students ::start_date ::start_time ::timezone ::instructor])))
#2022-02-2822:10zendevil.ethbut then for each key, I want to impose a type to the keyās value too#2022-02-2822:10zendevil.ethhow will I do that?#2022-02-2822:12ghadi(s/def ::instructor string?)
etc.#2022-02-2822:12ghadi(the s/and in your example doesn't do anything - remove it)#2022-02-2822:13ghadiI can read your question in several ways, might want to elaborate#2022-02-2822:21zendevil.eth@ghadi what would be the best way to restrict the keys to just the keys in the opt?#2022-02-2822:21ghadi?#2022-02-2822:21ghadirestrict what?#2022-02-2822:22zendevil.ethlike it shouldnāt conform if there are keys other than the ones in the :opt-un list#2022-02-2822:22zendevil.ethopt-un or req-un or opt or req#2022-02-2822:23ghadisame answer as in the thread above - transform lightly, then check#2022-02-2822:24ghadiselect-keys or similar#2022-02-2822:31zendevil.ethcan I use one of these libraries to achieve the effect rather than select keys:
https://github.com/metosin/spec-tools
https://github.com/wilkerlucio/spec-coerce#2022-02-2823:21zendevil.ethSo would this be good?:
(defn keys-conform? [m k]
(every? (set k) (keys m)))
(def create-sessions-keys [::students ::start_date ::start_time ::timezone ::instructor])
(s/def ::create-sessions-params (s/and (s/keys :opt-un create-sessions-keys) #(keys-conform? % create-sessions-keys)))
#2022-02-2823:21zendevil.ethkeys-conform makes sure that the keys in the map are only the ones that are in the keys vector#2022-02-2823:28seancorfields/keys
is a macro and cannot be passed a var like that, as I recall.#2022-02-2823:28Alex Miller (Clojure team)unfortunately, that won't work in s/keys#2022-02-2823:44zendevil.eththat should definitely be a feature in clojure spec#2022-02-2823:45zendevil.ethhow come we canāt use variables in it#2022-02-2823:47Alex Miller (Clojure team)spec forms are macros and at compile time, we wouldn't know your runtime key set. other choices have been made for spec 2. It is possible to retrieve the keys from the spec using specs on the spec forms themselves but it's pretty ugly to do so - most people end up just walking the form as data to do this#2022-02-2823:28Alex Miller (Clojure team)so you'd have to repeat the vector of keys there#2022-02-2823:29Alex Miller (Clojure team)btw, there is a faq entry about this here: https://clojure.org/guides/faq#exclusive_keys#2022-02-2823:30seancorfieldWhat we do is get the spec form (via the Spec API) and lift the keys out of it to use in other contexts -- we treat the Spec itself as the "system of record".#2022-02-2823:34zendevil.eth@seancorfield can you please elaborate?#2022-02-2823:51hiredmanuser=> (s/def ::f (s/keys :req [:a/x]))
:user/f
user=> (s/form ::f)
(clojure.spec.alpha/keys :req [:a/x])
user=>
#2022-02-2823:51hiredmanyou use s/form to inspect specs#2022-03-0111:35zendevil.ethcan someone explain to me why Iām seeing this error?
"res is " {:response {:status 400, :body "{\"spec\":\"(spec-tools.core/spec {:spec (clojure.spec.alpha/keys :req-un [:spec$34944/timezone :spec$34944/name]), :type :map, :leaf? false})\",\"problems\":[{\"path\":[],\"pred\":\"clojure.core/map?\",\"val\":null,\"via\":[],\"in\":[]}],\"type\":\"reitit.coercion/request-coercion\",\"coercion\":\"spec\",\"value\":null,\"in\":[\"request\",\"body-params\"]}", :headers {"Content-Type" "application/json; charset=utf-8", "X-XSS-Protection" "1; mode=block", "X-Frame-Options" "SAMEORIGIN", "X-Content-Type-Options" "nosniff"}}, :request {:protocol "HTTP/1.1", :remote-addr "127.0.0.1", :headers {"host" "localhost", "content-type" "application/x-www-form-urlencoded", "content-length" "39"}, :server-port 80, :content-length 39, :content-type "application/x-www-form-urlencoded", :uri "/api/schools", :server-name "localhost", :body #object[java.io.BufferedInputStream 0x79432e4b "java.io.BufferedInputStr
Iām making the request like so:
(require '[peridot.core :refer [content-type request session])
(-> session
(request "/api/schools"
:params {:name name
:timezone (or timezone "America/New_York")}
:request-method :post))
this is my reitit handler:
["/schools" {:swagger {:tags ["schools"]}}
["" {:post {:description "create a school"
:parameters {:body {:timezone string?
:name string?}}
:handler create-school-handler}}]
...]
FYI I have these three middleware ([reitit.ring.coercion :as rrc]):
rrc/coerce-exceptions-middleware
rrc/coerce-request-middleware
rrc/coerce-response-middleware
#2022-03-0111:37zendevil.ethI donāt see the error when I remove the middleware#2022-03-0119:46seancorfield@ps Based on \"pred\":\"clojure.core/map?\",\"val\":null
-- it's saying that it got null
via JSON but expected a (non-nil) hash map, if I'm reading it correctly. Maybe ask in #reitit?#2022-03-0120:12zendevil.eth@seancorfield why would it get null via json? When I remove the coerce middleware, the params are right there in the handler#2022-03-0120:17seancorfieldNo idea. That's why I suggested asking in #reitit (which I do not use).#2022-03-0120:18seancorfield(since it seems to be related to the reitit middleware, more than Spec)#2022-03-0917:40colinkahnAre there any libraries to use with spec for preparing data to be run against a spec? For example, we want to evolve a spec with a required field to keep contracts strong, but knowing this wonāt be compatible with old pre-existing stored data in the system. Weāre considering declaring (via some library) a default for when that new key is missing, preparing the data w/ those defaults, so it will validate against the spec. I know this is possible via something like spec-tools coercers, but since coupling coercing w/ specs seems like itās a known anti-pattern I assume using it for defaults in this way would be as well. This is kind of a round-about question about having strong specs for some places (think consumers vs producers), which I feel like is more of a solved thing in spec2 (use the select api to make certain things required in specific places), but without writing custom predicates I see no clear way to this (deeply) with the current version of spec.#2022-03-0917:58Colin P. HillI think the "right" solution per spec's design philosophy (at least at the time the first version was written ā maybe the thinking has moved since then) is that, if the new version of your spec will reject values that were valid under the old version of your spec, then you should simply create a new spec and leave the old one behind for compatibility, possibly with the new key added as an optional value.#2022-03-0920:40colinkahn@U029J729MUP thanks, having a compatibility spec makes sense as we update the stricter versions.#2022-03-1108:32ikitommiare there or could there be plans to add support for inline function specs into clojure.core? Microsoft has suggested an https://devblogs.microsoft.com/typescript/a-proposal-for-type-syntax-in-javascript/, based on TS. I would really like to see something similar for clojure, could be spec or something pluggable that could be implemented with malli too. now there is the Plumatic syntax (schema+malli), ghostwheel (spec), guardrails (spec), aave (malli) and many others.#2022-03-1108:34ikitommijust for docs, I think the community would be eager to built actual tooling on top of that (ides, linters, instrumentation etc)#2022-03-1113:12Alex Miller (Clojure team)Possibly, although those specs may be a different form than what we have now#2022-03-1609:19ikitommiI was hoping for something more concrete, but thanks anyway š#2022-03-1521:30Mark WardleHello all. Using clojure.spec a lot now, and also using for generative testing. Working really well, but in one domain I need to generate 'unique' [obv only in that test run] integer identifiers. Unfortunately, they also need to match a specific pattern and have a Verhoeff check digit. My spec and a custom generator is:
(defn gen-identifier
"A generator of identifiers of the specified type.
Parameters:
- t : one of :info.snomed/Concept :info.snomed.Description or
:info.snomed/Relationship."
[t]
(gen/fmap #(let [partition (rand-nth (seq (partitions-for-type t)))]
(Long/parseLong (verhoeff/append (str % partition))))
(s/gen (s/int-in 100000 Long/MAX_VALUE))))
https://github.com/wardle/hermes/blob/46cfee5b8005ffe8e26d86dcbbb239ba7b8ef01a/src/com/eldrix/hermes/rf2spec.clj#L18
I can do this for some by simply filtering out duplicates if I'm generating a single batch, but there are other times when that is not possible. I guess I could fallback to an incrementing counter and forego using a generator, or map through generated entities and override with an autoincrementing integer generator from an atom, but is there something in spec I'm missing? I've seen the list-distinct-by but that appears to offer guarantees only within the list obviously. I was thinking about something that starts with something based on time but then increments to satisfy the other generative predicates. Then it starts to feel a bit tricky! Anyone else faced this issue, or have any suggestions? I am missing something obvious? All suggestions welcomed. Thank you#2022-03-1522:20colinkahnWhen using test.check Iāve generated these using an atom as a counter in the past. Then itās a matter of using gen/fmap
where itās generator is all the different things that should be related, and the transform function assigns the ids to be consistent. The generator looks like this:
(def counter (atom 0))
(def gen-unique
(gen/no-shrink (gen/fmap (fn [_] (str (swap! counter inc))) (gen/return nil))))
Interested though if other people have different ways to do this. I donāt know if itās a downside, but itās extra coordination at the gen/fmap
to āreconcileā the parts (but Iāve found this necessary at some level for almost all generators iāve written).#2022-03-1522:46Mark WardleOh that looks really good thank you. I shall give that a try. Hadn't used gen/no-shrink either before! Thanks again. #2022-03-1715:46jeffparkerBoth the sound and video quality of Rich's Spec talk at LispNYC in Nov '16 are pretty challenging for the viewer. Here's a https://vimeo.com/689129571 version to reduce the friction. š#2022-03-1715:57Alex Miller (Clojure team)Any chance you'd be willing to offer that up to the clojuretv youtube channel?#2022-03-1716:12jeffparkerSure, we'll be adding it to the https://www.youtube.com/channel/UCv33UlfX5S4PKxozGwUY_pA/videos youtube channel as well. What's the best way to get it to clojuretv?#2022-03-1717:18richhickeyVery cool - thanks!#2022-03-1717:18richhickeyVery cool - thanks!#2022-03-2804:31Oliver GeorgeHello is there a way I can get better errors for something like this (below). Essentially, I'm looking to get-in
a deeply nested data structure to run a predicate test on something. Wondered if someone has written some kind of spec-in
which I can use.
(s/explain
(s/and (s/cat :s map?)
(s/and (s/conformer #(get-in % [:s :db :route :route-params :uuid])) string?))
[{:db {:route {:route-params {:uuid 1}}}}])
1 - failed: string?
Ideally, the explain-data should include a path which would indicate where in the nested data I should look.#2022-03-2806:07colinkahnI think to control exactly the message you want you'd need to define a deftype implementing the spec protocols (Specize and Spec). There isn't any documentation really for how to do that since those protocols are not meant for public use (afaik).
If you're ok with the message being more verbose, predicate specs are kind of self describing. For example:
(s/def ::s (s/and (s/cat :s map?)
#(string? (get-in % [:s :db :route :route-params :uuid]))))
(s/explain ::s [{:db {:route {:route-params {:uuid 1}}}}])
Will give you:
{:s {:db {:route {:route-params {:uuid 1}}}}} - failed: (string? (get-in % [:s :db :route :route-params :uuid])) spec: :cljs.user/s
#2022-03-2804:34Oliver George(this sort of thing gives me an incremental strategy to moving away from s/assert by mapping them over to s/fdef tests)#2022-03-2810:21onetomIs there a way to automatically rename the unqualified keys to their fully qualified equivalent, eg. during conformance?
For example:
(s/<something-like-conform> (s/keys :req-un [:ns1/a :ns2/b]) {:a 1 :b 2})
;; => {:ns1/a 1, :ns2/b 2}
The idea would be to maximize the amount of code, which uses fully-qualified keys.#2022-04-1908:28onetomI wonder if my question was unclear, or why hasn't it received any interaction?#2022-04-2704:47AronI would guess that the reason is simply people not having experience with this problem. Guessing again, I imagine that because the mapping from unqualified to qualified is always dependent on the context (could be that more than one place has unqualified that map to the same qualified, or could be that there are lots of qualified keywords but only a few places where there are unqualified versions of the, these two scenarios are indistinguishable just when looking at the keyword), and so such mappings are probably always written by hand, no one has tried to automate it. I would be very interested in the reality though, after all this guessing š
#2022-04-2708:14Linus EricssonI did experience the problem. However, in my case, I found that I probably tried to overuse the conforms functionality in spec.
However, there has been utility functions for update-keys
https://github.com/clojure/clojure/blob/master/changes.md#33-update-keys-and-update-vals
which could be used for qualifying keys. Also clojure.set/rename-keys could work`.`#2022-04-2704:47AronI would guess that the reason is simply people not having experience with this problem. Guessing again, I imagine that because the mapping from unqualified to qualified is always dependent on the context (could be that more than one place has unqualified that map to the same qualified, or could be that there are lots of qualified keywords but only a few places where there are unqualified versions of the, these two scenarios are indistinguishable just when looking at the keyword), and so such mappings are probably always written by hand, no one has tried to automate it. I would be very interested in the reality though, after all this guessing š
#2022-03-3018:26bortexzSometimes I find the best place to put fn specs as metadata on de fn, like tests
(defn a
{:spec (s/fspec ...)}
[]
...)
is this something being considered for spec2? technically you can already do this, is there tooling around using specs on metadata?#2022-03-3018:33Alex Miller (Clojure team)there are a lot of tradeoffs (var meta is evaluated in particular) and yes there are some things being considered here#2022-03-3022:34Oliver George@alexmiller have you peeked at the survey responses? Does it help justify prioritisation/investment in v2 spec?#2022-03-3022:46Alex Miller (Clojure team)I have, although I didn't really need the survey to tell me people are interested in spec :)#2022-03-3113:04Colin P. HillI imagine checking the survey to confirm that people want spec2 is rather like checking a thermometer to confirm that lava is, indeed, hot#2022-04-0703:27didibusI find I often have this scenario:
{:type :foo
:bar 1
:baz 2}
Where I want my spec to say that the keys are :type
, :bar
and :baz
, but also that the value of :type
must be :foo
. What's the most straightforward way to write that spec?
Bonus point where the generator for it properly constructs a map where :type
is always :foo
.#2022-04-0704:08Alex Miller (Clojure team)have you looked at multi-spec?#2022-04-0704:10Alex Miller (Clojure team)seems like it would be a great match here#2022-04-0705:31didibusMulti-spec is actually where I have this problem, because the generator doesn't generate the correct :type
#2022-04-0705:34didibusLike if we take the spec guide example: https://clojure.org/guides/spec#_multi_spec
Assume that instead of:
(s/def :event/type keyword?)
we have
(s/def :event/type #{:event/search :event/error})
If you try to generate an :event/event
, they won't be valid to any of the type of events most of the time, because the generator doesn't know which of the :event/type to pick for the different multi-specs#2022-04-0707:45flowthingI'd like to write a spec for a clojure.data.xml data structure and use said spec for generation as well. For example, say I have something like this:
{:tag :foo
:content [{:tag :bar
:content ["..."]}]}
I'm not sure what the best way to do that would be, though. That is, how do I write (spec/def ::foo ,,,)
that specifies that :tag
needs to be :foo
and :content
must conform to the ::bar
spec, etc.
Is some combination of spec/keys
and spec/and
the way to go?#2022-04-0708:47lassemaattaIt's been a while since I've used spec so this might be horribly wrong, but it might also solve both your problems š š§µ#2022-04-0708:47lassemaatta#2022-04-0708:48lassemaatta#2022-04-0708:51flowthingThank you! I'll give that a go. I tried flailing about with multi-spec
a bit, but I didn't think of the merge into a base element spec, that's clever. :thumbsup:#2022-04-0708:51lassemaattaI tested this for at least two seconds, so caveat emptor š#2022-04-0708:51flowthingSure thing, no worries. š#2022-04-0709:19flowthingActually, it's probably easier to write a spec for the Hiccup syntax using tuples and/or regexp ops.#2022-04-0709:21flowthing(spec/def ::messageId
(spec/tuple #{:messageId} string?))
(spec/def ::from
(spec/tuple #{:from} string?))
(spec/def ::soap/Header
(spec/tuple #{::soap/Header}
::messageId
::from
;; etc
))
(spec/def ::soap/Envelope
(spec/tuple #{::soap/Envelope}
::soap/Header))
(->
(spec/gen ::soap/Envelope)
(gen/generate)
(xml/sexp-as-element)
(xml/emit-str)
(pretty-print-xml-string)
(println))
;;=>
<a:Envelope xmlns:a="">
<a:Header>
<messageId>I67NC7</messageId>
<from>733R0T1MGsY9cnx943</from>
</a:Header>
</a:Envelope>
#2022-04-0715:00lassemaatta@U0K064KQV the example I gave above might offer one way to use generators with multi-specs. but as I said, I'm no expert in this stuff so there may be better ways to do it#2022-04-1211:52Mark WardleRealise this will be mostly a matter of opinion, but where do you place your function specifications? I tried them in a dedicated specs namespace, but I'm preferring them just above the function definition. Are there any disadvantages to keeping the function spec and the function together in the same namespace?#2022-04-1212:12Colin P. HillI strongly prefer colocating them with the functions. Tends to make the namespacing more meaningful and communicative, and (as a fringe benefit) means you're jumping around less when you change things.#2022-04-1212:31Mark WardleThanks! That's helpful. #2022-04-1213:41Colin P. HillThe only disadvantage that comes to mind is that it might lead to one namespace owning a spec which is later revealed to most properly belong elsewhere, perhaps in a more general-purpose namespace ā but this is just a specific case of the general unsolved problem of changing knowledge and needs. It can also be worked around by moving the spec to the "right" place, leaving behind an s/def
in the old place that just aliases the spec under the old name, and documenting the latter as deprecated.#2022-04-1213:42Mark WardleThanks Colin. Makes sense!#2022-04-1215:01seancorfieldAnother "vote" for having function specs directly above the function they describe. We use separate .specs
namespaces for data specs. I talk a bit about that in https://corfield.org/blog/2019/09/13/using-spec/#2022-04-1215:03Mark WardleThanks Sean. That's a helpful blog post thank you. All makes sense!#2022-04-1212:13Colin P. HillIf there are some specs used by a number of functions, I'll sometimes just put those at the top of the file rather than immediately before some particular function.#2022-04-1908:25onetomat https://youtu.be/dtGzfYvBn3w?t=6346 there is a slide, saying:
> ā¢ DO NOT use check
, instrument
or (turn on) assert
in production
> ā¢ they are for making sure your program works - use in advance
> ā¢ but spec can play a runtime role
> ā¢ validating inputs from external systems or users
The args conformance and exception throwing feature of instrument
seems like a very useful guard to have even for production code, especially, when it's checking relationships between the input arguments. It would help us pinpoint code-paths in our code, which is not covered by tests or specs.
Q1: what does "validating inputs from external systems or users" refer to?
using function :pre [...]
conditions?
what spec functions should i use, if not s/assert
?
it seems like that the unconditional s/assert* was made to be used in
:pre` conditions...
Q2: is there a way to do something like instrument
, but only for checking :args
, so i wouldn't need to repeat such logic as function :pre
conditions?#2022-04-1909:04Linus EricssonQ1: I would suggest using a construct like
(when-not (s/valid ::spec data)
(throw (ex-info "error in..." {:error-data error-data}))
because asserts (and pre-posts i think) can be disabled by various options to the JVM. Be explicit with your throws after validation.
Q2: I would not recommend using instrument
or pre
when validating input from users or other systems. Rather make this validation a separate step in the processing of data. The way to handle incorrect data can differ between various systems and sources of data.#2022-04-1911:54onetomi didn't know u can turn off :pre
& :post
conditions with *assert*
, thanks!#2022-04-2005:42Arondo I have to throw if something is not valid?#2022-04-2006:12SantiagoYou can also use something like Failjure instead of throwing, but I think the general way is to make use of JVM exceptions#2022-04-2014:30Linus EricssonRe throw or not - it depends on how your application handles errors in inputdata. Usually this is very application specific. Usually it's not good to use exceptions for flow control.#2022-04-2101:59AronThere is no reason for validation or for tests to throw, not in cases that I expect.#2022-04-2003:20onetomi wanted to generate some datomic lookup-refs with spec, but s/cat
always returns a seq
, not a vector
.
is there some built-in / idiomatic / recommended way to spec vectors?
i saw there are :kind
and :into
options for s/coll-of
, but i couldn't combine it with s/cat
; i always got Couldn't satisfy such-that predicate after 100 tries.
error.
i found a recommendation here:
https://gist.github.com/mhuebert/8fdeedae57bf797778054dcf8f33ab8b
but my gut feeling tells me, that there might be a simpler way.#2022-04-2003:26Alex Miller (Clojure team)unfortunately, there is not a simpler way - it is very challenging (in spec 1) to spec vectors and get all of conform, unform, gen etc to work. we've fixed this in spec 2 but I don't have a good answer in spec 1.#2022-04-2013:27KelvinWould s/tuple
work?#2022-04-2013:46Alex Miller (Clojure team)for some limited cases, yes (it's fixed size and can only be positionally described, not with the regex ops)#2022-04-2020:50onetomI just started looking into spec2.
Found this page, which specifically mentions how to solve my issue with non-flowing s/and-
:
https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha
(s/def ::vcat (s/and- (s/cat :i int?) vector?))
Thanks @U064X3EF3 for the great docs!#2022-04-2021:03Alex Miller (Clojure team)not mentioned there, but that's actually reified for you in s/catv (vector-only) and s/cats (sequence-only) in spec 2#2022-04-2021:04Alex Miller (Clojure team)I don't think that ever made it back to those docs, but they're there#2022-04-2021:05Alex Miller (Clojure team)and before you ask, no I would not recommend using spec 2 at this time, it has a number of bugs :)#2022-04-2022:51Cam SaulIs there a way to pass along/access/use the overrides from a top-level call to s/gen
inside a custom generator defined with s/with-gen
or (s/spec ... :gen ...)
?
Example.
Supposed I want to generate some sort of query with the keys :database
and :table
. I'd like to have a custom generator for :table
so its values are based on the value of :database
... however if I use an overriden generator for :database
That generator is invisible from inside my custom generator.
(s/def ::database string?)
(s/def ::table
(s/with-gen
string?
(fn []
(gen/bind
(s/gen ::database)
(fn [db]
(gen/fmap (fn [table]
(str db \. table))
gen/string))))))
(s/def ::query
(s/keys :req-un [::database ::table]))
(gen/sample (s/gen ::query {::database (fn [] (gen/return "my_database"))}))
;; =>
[{:database "my_database", :table "."}
{:database "my_database", :table "7.$"}
{:database "my_database", :table "T2."}
{:database "my_database", :table "xv.HĀØ"}
{:database "my_database", :table "aB.Ćn"}
{:database "my_database", :table "Q3MCh.ĆĆ”"}
{:database "my_database", :table "F68.Ā·%:"}
{:database "my_database", :table ".>Ƭ"}
{:database "my_database", :table "jBk.ĆP ĀµrfL"}
{:database "my_database", :table "d.Ā„Ā¶Ā«Ć¢\b"}]
Is there some way to do this? I know I could do the whole thing with test.check.generators/let
or bind
but I'd be missing out on all the niceness from spec maps since AFAIK there's nothing like s/keys
in test.check
#2022-04-2023:20Cam SaulI guess I could get it working by redefining the entire spec with s/def
instead of trying to pass an override for the generator, or maybe just with-redefs
-ing clojure.spec.alpha/registry-ref
itself... both feel icky but since this just for generative testing it's probably not the end of the world.#2022-04-2023:22Cam SaulAnother option might be to replace all my specs with functions that return specs based on previous state, but for map specs I guess I'd have to use Spec Alpha 2 so I could programmatically create them with schema
or the like. Maybe that's a bit better. But creating a bunch of ad-hoc specs programatically actually seems like more work than if I forgot about spec altogether and just used test.check.generators
directly#2022-04-2115:03colinkahnI don't think there's any way provided in spec to make your custom generator see the override (you could get creative and bind a dynamic var, but that's outside of specs functionality and probably a bit unexpected). test.check does have hash-map, which will generate a map of keys to values like (gen/hash-map :database db-gen :table table-gen)
. I would probably use fmap though here and do something like:
(gen/fmap (fn [[db-str table-str]] {:table (str db-str \. table-str) : database db-str})
(gen/tuple (gen/return "my_database") gen/string))
Most things can be solved using fmap and then reconciling the generated parts in the fmap fn somehow. This gives better shrinking as well than using bind/let.#2022-04-2121:20Cam SaulI tried binding dynamic vars at every possible place but couldn't figure out how to get it to work consistently.
Thanks, I think I'll just stick to using test.check
stuff without spec for now. I managed to put together a generator that works sort of like s/keys
with :opt
or :opt-un
, not sure how well it's going to handle the shrinking. I'll have to wrap my head around that a bit more#2022-04-2614:55Dave RussellHello! Is there a good way to find all currently instrumented vars? It looks like (st/unstrument)
will return anything that has been unstrumented, so that's a way to tell (sort-of), but for our use-case we'd like to:
ā¢ Preserve current instrumentation state
ā¢ Instrument all vars
ā¢ Do something
ā¢ Restore previous instrumentation state
Which allows us to enable instrumentation when, e.g., running tests but keeping any instrumentation state in the REPL#2022-04-2615:08Colin P. HillLooks like no, unless you want to access a private var and risk breakage later. The instrumented vars are kept in a global atom.
https://github.com/clojure/spec.alpha/blob/f23ea614b3cb658cff0044a027cacdd76831edcf/src/main/clojure/clojure/spec/test/alpha.clj#L150#2022-04-2615:09Alex Miller (Clojure team)isn't there a function for this in stest?#2022-04-2615:09Colin P. HillDoesn't appear to be. There's one that does the opposite though, turning off instrumentation within a dynamic scope.#2022-04-2615:10Alex Miller (Clojure team)There's instrumentable-syms and checkable-syms but I guess not instrumented-syms#2022-04-2615:11Dave RussellYep š instrumented-syms
would have been perfect#2022-04-2615:11Dave RussellRight now we basically have:
(let [currently-instrumented-vars (st/unstrument)]
(set/difference (set (st/instrument)) (set currently-instrumented-vars)))
As a way to figure out what syms we should restore#2022-04-2615:12Colin P. HillA with-instrument-enabled
macro parallel to the with-instrument-disabled
would have been pretty slick, to let this be done in a call scope rather than with global state changes.#2022-04-2615:30Alex Miller (Clojure team)feel free to file requests at https://ask.clojure.org ...#2022-04-2615:39Dave RussellWill do -- thanks!#2022-04-2615:48Dave Russellhttps://ask.clojure.org/index.php/11816/support-checking-instrumented-syms-clojure-spec-test-alpha#2022-05-0512:30pavlosmelissinosI have some kind of user configuration in the form of edn files. They're quite complex and rigorously spec'd, so I want to help the users identify potential problems (e.g. typos, settings that are no longer used etc), to help them manage this complexity.
Assuming that it's not feasible at the moment to simplify the maps and given that spec doesn't support closed maps, what's the best way to do the above? There's a bunch of multi-specs in the specs that seem to break libraries like spell-spec and spec-tools.
Example:
given (s/def ::m (s/keys :req [:foo] :opt [::bar]))
I want to get some warning for (def m {:foo 1 :baar 1})
Reasoning:
In my use case it's more likely that baar is a typo, than a correct key that is used somewhere else.
I have no way of knowing without asking the user, so how can I do that?
I understand that closed maps are an antipattern in Clojure but the use case is real unfortunately. Happy to consider alternatives in any case! Thanks!#2022-05-0512:34Alex Miller (Clojure team)https://clojure.org/guides/faq#exclusive_keys#2022-05-0512:38pavlosmelissinosOooh, didn't think of that, thanks! š#2022-05-0604:13West(s/def ::params (s/or :string string?
:map map?))
(defn is-null? [val]
(or (empty? val)
(= "null" val)
(nil? val)))
(s/def ::non-null-params
(s/and ::params
#(not (is-null? %))))
(gen/sample (s/gen ::non-null-params))
Why does ::non-null-params
still give me empty maps and empty strings?#2022-05-0604:23seancorfieldBecause s/or
produces a pair. Either [:string s]
or [:map m]
#2022-05-0604:23seancorfieldis-null?
could have [[_ val]]
as its arglist and it will work.#2022-05-0604:24seancorfieldOr you could test (is-null? (second %))
#2022-05-0604:25WestAh ok. Then clearly I didn't want to use s/or
I wanted to choose one or the other.#2022-05-0604:25seancorfield(technically it's a MapEntry
so (val %)
is "more correct" than (second %)
#2022-05-0604:25seancorfields/or
is how you do it, but it returns a tagged pair to tell you which branch matched.#2022-05-0604:30West(s/def ::non-null-params
(s/and ::params
#(not (is-null? (val %)))))
Thanks Sean.
It's working just as expected!#2022-05-0604:31seancorfieldI think Alex has said that Spec 2 will have a non-conforming or
that doesn't do that, or at least some easy way to avoid it...#2022-05-0604:33West"Will have", as in "when it's released", or "is being talked about". Can I test it now?#2022-05-0604:35seancorfieldIt's only available via git deps and it's kind of buggy.#2022-05-0604:36WestAlright, maybe an adventure for another day.#2022-05-0604:36seancorfieldWhat you can do with Spec 1 is used the undocumented nonconforming
function, which is currently documented for Spec 2 (but might go away -- it will still be in Spec 1 tho'):
(s/def ::params (s/nonconforming (s/or :string string? :map map?)))
#2022-05-0604:37seancorfieldSpec 1 isn't going away. Spec 2 will most likely morph into the actual "core" Spec at some point and folks will either migrate from Spec 1 to "Spec" or use both side-by-side.#2022-05-0604:39seancorfieldFor a while at work, we maintained a branch of our code base migrated to use Spec 2, and help Alex test stuff, but it was such a moving target and several places were buggy and/or incomplete so after several months we abandoned trying to keep our branch updated and decided to wait for Spec 2 to mature.#2022-05-0604:40seancorfieldSpec 2 has a lot of improvements over Spec 1 but it is also quite different in several places.#2022-05-0604:41WestEither way, I'm excited to see how it turns out.#2022-05-0817:39Benjaminhttps://developers.google.com/discovery/v1/reference/apis wouldn't it be cool to generate spec from this?#2022-05-0817:55kennyYou definitely could. Cognitectās aws-api does just that from the AWS api structured docs (smithy files). I'm not aware of anyone doing it for GCP but no reason you couldn't. Since the AWS api -> spec part of aws-api isn't open source and I needed part of that code, I wrote a version available here https://github.com/kennyjwilli/smithy-spec. Could be a starting point for GCP. #2022-05-0817:56BenjaminI'll think about it#2022-05-1610:11pithyless@U02CV2P4J6S I remember seeming something like this on Slack - perhaps it was this? https://github.com/ComputeSoftware/gcp-api#2022-05-1610:12pithylessOh, I just realized this is by @U083D6HK9 facepalm#2022-05-1610:14pithyless<meme of Homer Simpson slowly backing away>#2022-05-0923:18nodenameWhen I pull the problems from an exception that I receive from spec when running a test, sometimes I see :pred nil?
and sometimes I see
:pred #object[clojure.core$nil_QMARK_
0x5c45bb7f
"
This is making it hard to write the test properly. Any thoughts?
Or is there a better way to check that a certain test results in a certain set of problems?#2022-05-0923:46Alex Miller (Clojure team)can you share any more info?#2022-05-0923:47Alex Miller (Clojure team)the intention is to return nil?
there#2022-05-1000:29nodenameOh I think I see my problem, itās not in the return value but in my is
predicate. The returned info did say nil?
. It works if I say
(is (= problems
...
:pred 'nil?
...))
with a quote before the nil?
in my own data structure.#2022-05-1107:00Mark WardleDear all, is the spec registry cleared between tests? I was just bitten by a surprising 'spec not found' exception during instrumentation in a project that uses one of my other projects as a library.
My tests run successfully from the command-line, but when run in a brand-new, just loaded REPL, they fail because a specification was not found. It's an easy fix, as I haven't included a require, but surprised my tests didn't catch it. Finding it quite easy to accidentally omit the necessary requires as one flits back and forth between namespaces so the specs get loaded and then everything works. It's only in the context of trying to use as a library that I got an exception.#2022-05-1107:08seancorfieldThis seems no different to working with multimethods or protocols where you need to require the namespaces for the various implementations ahead of use?#2022-05-1107:55Mark WardleI guess so. But in that situation, my tests would fail. Here they worked, perhaps because the specs were loaded from one namespace, persist into a test on another namespace. At least, I presume that is what is happening.#2022-05-1108:30seancorfieldIf you're running multiple tests in the same process, state will persist, just as it would with multimethods or protocols.#2022-05-1109:45Mark WardleThanks. I suppose trying to reset the global registry prior to running a test namespace might work, but sounds more complicated. I guess it an issue because I have specifications not linked to a clojure namespace reflecting my domain - so one doesn't notice easily if you don't have a require in a specific namespace.#2022-05-1109:47Mark WardleIs there an argument in favour of a test runner resetting global state before running a namespace?That would, I think, fix the issue.#2022-05-1109:53Mark WardleI'm finding it difficult that running all tests fails to throw an exception, but specifying a single namespace does throw an exception: I understand why this happens, but it is confusing, at least to me!#2022-05-1116:05seancorfieldI don't think that a test runner could realistically "reset (all) global state)" -- partly because the test runner itself maintains global state about what tests it ran and what failures and successes (and some of that state is baked into clojure.test
directly) unless you arrange to run the tests for each namespace in a separate process, which will add a huge amount of overhead to running your test suite.#2022-05-1116:07seancorfieldAt work, we run tests for each artifact's project in separate processes (via the Polylith test runner). Well, isolated classloader instances. We also have our build.clj
perform each test run in a separate process (a separate JVM process) so we can test modules in isolation in our monorepo.#2022-05-1116:09seancorfieldThe Spec registry contains all the clojure.core.specs
to, BTW.#2022-05-1118:10Mark WardleThanks Sean. I didn't know that. So trying to reset the internal atom might not be such a good idea. Just shows the issues with global state :)#2022-05-1911:30Ho0manHi everyone,
I have a fully specd and complex data structure x-pack
where I want to override the ::iid
generator for all its sub-entities so they can all share the same value (they all use ::i-commons/iid
), However as the supersession rule in spec/gen
explains I am not able to override ::i-commons
. Any suggestions for a workaround? Or am I getting something very wrong?
(spec/def ::___x-pack___
(spec/keys
:req-un [::i-commons/iid
,,,]))
(spec/def ::x-pack
(spec/spec ::___x-pack___
:gen (fn []
(->> (spec/gen ::___x-pack___)
(gen/fmap map->X-Pack)))))
,,,,
(gen/generate (spec/gen ::x-pack {::i-commons/iid #(gen/return "ID1")})
Thanks#2022-05-2809:31BenjaminHi, was there not a way to add a spec to the docstring of a function?>#2022-05-2811:42jumarI'm not sure about that but some tools like Cider show it when you try to show a docstring for the function.#2022-05-3010:02teodorluYou could change the docs yourself by changing the var metadata:
https://clojuredocs.org/clojure.core/alter-meta%21#2022-05-3112:50folconI've been reminding myself how to use spec and one thing I can't remember is how to get better errors from this sort of thing?
(defn load-order [order-book {:keys [side id] :as order}]
{:pre [(s/valid? :order/order order)]}
(assoc-in order-book [side id] order))
Which does give me this:
Execution error (AssertionError) at fruit-economy.sim.market/load-order (market.clj:23).
Assert failed: (is (s/valid? :order/order order))
But considering it's a spec, I was expecting a little more?
I've hacked it a bit by wrapping it in an is
, which gives me:
FAIL in () (market.clj:2)
expected: (s/valid? :order/order order)
actual: (not
(s/valid?
:order/order
{:price 1,
:size 425.0,
:side :sell,
:id 10229,
:good-kw :inventory}))
But I'm sure there's a better way?
EDIT:
@jcf suggested using fdef
to specify args and instrument, so I've swapped to doing that and calling instrument on debug as so in my core
ns:
(when (debug?)
(let [instrument (requiring-resolve 'clojure.spec.test.alpha/instrument)]
(instrument 'fruit-economy.sim.market/load-order)))
To be honest this is pretty great, but perhaps there's something I'm missing? So I thought I'd pop the question here!
REF: https://github.com/Folcon/fruit-economy/commit/a92b4b5329eabd7347240804f04224b6440ebdb2#2022-05-3117:28jcfYou rang? š#2022-05-3117:28folconOh not ringing š, attributing š#2022-05-3117:29jcfThanks for the shout out, @U0JUM502E! š#2022-06-0507:59Bart KleijngeldI have a map of which I'd like to specify two optional keys, but they must not exist both. I'm trying to express this in spec but I'm having a hard time. You can't use and
and or
in the :opt
part of the s/keys
spec either, so that makes it particularly hard.
Can someone help me out?#2022-06-0512:41Bart KleijngeldI've come up with a solution, but would love to hear from more experienced users.
(defn only-one-key-of [kws]
(fn [m]
(not
(set/subset? kws
(into #{} (keys m))))))
(s/def ::some-map
(s/and
(s/keys :req [:a]
:opt [:b :c :d])
(only-one-key-of [:b :c])))
(comment
(s/valid? ::some-map {:a 1 :b 2 :c 3}) ;; correctly gives `false`
)
#2022-06-0513:14Alex Miller (Clojure team)I think that's the right approach#2022-06-0513:17Bart KleijngeldNice. Thanks for your feedback#2022-06-0813:26roltsmall warning: only-one-key-of
might be a bit misleading, it won't give the expected result if there are more than 2 keywords#2022-06-0813:27Bart KleijngeldYes, it was not very well-defined, it also didn't for just one parameter. I've redone the implementation to be more robust now. Thanks for pointing it out though š#2022-06-1418:10Drew VerleeI don't understand what spec is trying to tell me here. Isn't {:keys ...
valid clojure?
-- Spec failed --------------------
(... ... [{:keys [http-client max-retries backoff retry?], :as client} ...] ...)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
should satisfy
vector?
-- Relevant specs -------
:clojure.core.specs.alpha/arg-list:
.... (more spec info...)
#2022-06-1418:14Victor Maia AldecƓaI don't understand either, but you can try to use this lib to get better spec messages:
https://github.com/bhb/expound#2022-06-1418:15Drew Verleethat is expound formatted š#2022-06-1418:20Alex Miller (Clojure team)Do you have the message without expound?#2022-06-1418:32Christian JohansenThrowing in a guess: maybe you forgot to put optional arguments in a vector? (defn blabla [a b & {:keys [c d]}] ,,,)
#2022-06-1420:36Drew Verleei could go back and re-create the errors it if it helps. The minimal change that made the errors go away was to
(defn foo [{:keys [x] :or {::x "x"}] x)
to
(defn foo [{:keys [x] :or {x "x"}] x)
note the :or {::x
to to just x. that is, the value of the hashmap of the :or arg has to be a symbol not a key.
This same error seemed to generate several different expound messages (at least thats how it seems to me).#2022-06-1420:51Alex Miller (Clojure team)the keys of :or
are always the symbols being bound, which must be unqualified, so neither :x
or ::x
are valid in that spot#2022-06-2416:18joost-diepenmaatHope Iām not spamming people, but I would love to get some feedback on a tool we made to help with creating spec generators: https://git.sr.ht/~jomco/proof-specs proof-specs
can be run as a leiningen alias or from the repl and will exercise all data specs (keyword specs) in a set of namespaces and report on any errors. Weāre using it ourselves as part of our test suite to ensure we always write generating specs and to make it easier to find out which part of complex specs are broken.
Thanks for any feedback :-)#2022-06-2416:24joost-diepenmaatMain reason we wrote the thing is that sometimes spec generators just error out with very little context making it hard to figure out which specs even work. proof-specs
will at very least give you a list of every spec that wonāt generate which helps a lot in figuring out whatās going on.#2022-06-2422:09raymcdermottThis might be a silly question but is there a s-exp spec that can be applied recursively?#2022-06-2422:20Alex Miller (Clojure team)No, and I think you can quickly come up with many reasons why#2022-06-2423:18raymcdermottSure. Like I said, was a silly question. I just wanted to make sure I hadn't missed anything before moving on other options.#2022-06-2709:36reefersleepBut I guess youāre free to make a function that does recursive stuff and use it in a spec? š#2022-06-2709:39raymcdermottyes, also specs can be applied recursively even if it's silly šŗ#2022-06-2711:38borkdude@U04V5V0V4 What are you looking for? Something like tree-seq + spec? This is what grasp basically does:
https://github.com/borkdude/grasp#2022-06-2712:10raymcdermottok - thanks, I remember talking about it with you but I've never looked at the code š#2022-06-2712:13raymcdermottto be more straightforward: I'm looking to conform all of Clojure. That's what I'm really looking to do.#2022-06-2718:59raymcdermott@U04V15CAJ one small thing I noticed in your reify
spec is that you use (s/cat :reify #{'reify} ...
why do you use a set rather than just the symbol 'reify
? Cos that's what I do and it works but maybe set gets you something else that I'm missing?#2022-06-2719:01borkdudeBecause of how spec works?
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/valid? 'foo 'foo)
Execution error at user/eval154 (REPL:1).
Unable to resolve spec: foo
user=> (s/valid? (s/cat :foo 'foo) ['foo])
Execution error at user/eval156 (REPL:1).
Unable to resolve spec: foo
user=> (s/valid? #{'foo} 'foo)
true
user=> (s/valid? (s/cat :foo #{'foo}) ['foo])
true
#2022-06-2719:03borkdudeI.e. I want to match the literal symbol 'reify
#2022-06-2719:20raymcdermottyes ... actually it's neater than #(= 'reify %)
which is the ugly thing I have š:skin-tone-3:#2022-06-2719:51reefersleepI use sets for comparison all the time. it's great!#2022-06-2719:52raymcdermottyeah, seems a ton more elegant and more transparent than a function#2022-06-2720:14Alex Miller (Clojure team)oh but it is a function ;)#2022-06-2720:25raymcdermottho ho#2022-06-2807:38reefersleepSometimes it can make sense to make it a var as well. Like
(def valid-foo? #{:x :y :z})
(valid-foo? :x) ;=> :x
#2022-06-2422:11raymcdermottEg (s/cat symbol? s/+ any?) but recurring #2022-06-2717:01souenzzoYou can give it a name, then recur to this name:
(s/def ::foo
(s/cat :k keyword?
:v map?
:others (s/? ::foo)))
=> :user/foo
(s/valid? ::foo
[:a {} :b {} :c {}])
=> true
(s/valid? ::foo
[:a {} :b {} :c])
=> false
(s/valid? ::foo
[:a {} :b {} :c []])
=> false
#2022-06-2718:55raymcdermottha, yes - actually that's what we ended up doing! š#2022-06-2718:56raymcdermottthank you @U2J4FRT2T#2022-06-2422:12raymcdermott[on the phone so syntax ain't great]#2022-06-2821:26Braden Shepherdson(I'm not sure if there's a better place for specmonstah questions; please direct me elsewhere if there is.)
I'm using the (superb) specmonstah library to generate a big chunk of test data. One can use :constraints {:some_field #{:uniq}}
to ensure a particular field is unique.
however, sometimes the field is only unique within some more limited namespace. is there any way to write a custom constraint? I want a field that's unique among all those that share the same parent (another field).#2022-07-0407:21Martynas MHey.
Is there a good way to spec an atom
that has a list of functions? š Is there at least a way to spec an atom?#2022-07-0407:23vlaaadAny predicate can be used as a spec#2022-07-0407:26Martynas MAh, I see. Thanks.#2022-07-0407:28vlaaade.g.
(s/def ::fns (s/coll-of ifn?))
(s/def ::my-atom
(s/and (s/conformer
#(if (instance? clojure.lang.Atom %)
@%
::s/invalid))
::fns))
(s/valid? ::my-atom (atom [dec identity inc])) ; => true
#2022-07-0407:29vlaaadmaybe this use of conformer
is not idiomatic. tbh Iām not sure whatās an idiomatic use of conformer
#2022-07-0407:31vlaaadI think this is not idiomatic, because spec is a ālibrary [that] specifies the structure of dataā (as per https://clojure.org/guides/spec), and atoms are state, not data#2022-07-0407:32vlaaadso maybe you could consider not speccing atoms? š#2022-07-0407:33Martynas MI think I'll not do it.#2022-07-0513:26Colin P. HillAtoms are stateful containers of data. It often makes a lot of sense to spec atoms, especially if they're a highly structured central data store for your application ā similar to having a formal schema for a database. The atom
function even has an option for including a validator, which could be built on spec (though be mindful of performance and consider making it no-op in prod).
https://clojuredocs.org/clojure.core/atom#example-567add80e4b01f598e267e8f#2022-07-0513:28Colin P. HillA naive (in the sense that it doesn't check function signatures) spec for a list of functions might be (s/* ifn?)
#2022-07-0414:18Eric DvorsakIn the core.specs, isn't ::map-special-bindings
always going to be a subset of ::map-bindings
? how would ::map-special-bindings
spec ever be used given that it's only part of :map-binding-form
spec which is defined as (s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))
https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L29-44#2022-07-0414:22kwladykahttps://clojure.wladyka.eu/posts/form-validation/#trick-about-mirroring-s-def-and-via
here is use case showing the difference (s/def ::created ::instant)
vs (s/def ::created (s/and ::instant))
#2022-07-0414:23kwladykaI donāt know the exact code which you are asking, but I know the difference between this too#2022-07-0414:26Eric DvorsakI'm not sure I understand your answer, do you mean the intention might just be to produce better error messages?#2022-07-0414:27kwladyka:via [:foo/created],
vs :via [:foo/created :foo/instant],
#2022-07-0414:28kwladykaI mean without s/and
you miss the information about the full path, because it cut corners to make path simpler#2022-07-0414:28kwladykas/and
keep full path and this is ONLY ONE WAY to get this information#2022-07-0414:33Eric Dvorsakok, although there's no and
involved in the code I linked to (clojure.core.spec.alpha), it's a merge of a spec map-bindings
with another map-special-binding
after double checking it seems to me like the only point of map-special-binding
is to keep the map-binding-form
spec open to new keywords#2022-07-0414:36kwladykaI donāt know this part of the code, so I canāt really give you deeper answers#2022-07-0514:58LycheeseI have a map which needs to have at least one of three possible keys. Sometimes one of those keys is present but with an empty map. I want to not count that as fulfilling the requirement in the or
in :test/map
but also not invalidate the entire map if one of the other two keys is present and valid. Any way of doing that without first dissoc'ing those keys with empty values?
(s/def :test/b ;; c and d look similar
(s/and not-empty
(s/map-of number? (s/multi-spec mm :test/type))))
(s/def :test/map
(s/keys :req [:test/a
(or :test/b
:test/c
:test/d)]
:opt [:test/e
:test/f]))
#2022-07-0515:01Alex Miller (Clojure team)with spec, your only option would be a conformer that manipulates the map ahead of the spec. or you can s/and an arbitrary predicate that does anything you want.#2022-07-0808:57LycheeseHow would I go about making a certain key necessary when that key can exist at different depths in a map (and it only needs to exist at least once)? Do I need to normalize the map and put it at one level or is there another way?#2022-07-0810:36reefersleepUse clojure.walk
to see if exists at least once?#2022-07-0811:48LycheeseHow would I go about incorporating that into a spec validity check?#2022-07-0811:56reefersleep(s/def ::guarantee-key-in-map (write-your-fn-here))
(s/def ::my-map
(s/and ::guarantee-key-in-map
(s/keys :req [:other-key
... and so on
#2022-07-0812:10reefersleepsomething like that?#2022-07-0812:11reefersleep(s/valid? ::my-map some-map)
#2022-07-0812:11reefersleepI donāt write that many specs, so donāt copy verbatim š it probably doesnāt compile#2022-07-0812:13reefersleepOr simpler, without the map thing:
(s/def ::guarantee-key-in-map (write-your-fn-here))
(s/valid? ::guarantee-key-in-map some-map)
#2022-07-0812:18reefersleepThe key takeaway is that your spec is something separate from e.g. s/keys
. I donāt think you can use s/keys
to say arbitrary things about certain values in your map. But you can do so with a separate spec that you can s/and
#2022-07-0812:19reefersleepAnybody, feel free to take the wheel and show @U02FM0NNZAB how Iām wrong š#2022-07-0812:59LycheeseI think I only now got what Alex Miller was alluding to in the post above. Feel kinda dumb now. Thank you^^
I think I got my test case working the way I want it to. Now I just need to remember where I wanted to use this š
#2022-07-1719:59apbleonardIf I had a map with :billing-address
and :shipping-address
keys whose associated values were expected to be valid against the same ::address
spec, could I express that in spec/keys
directly - i.e. without creating intermediary ::billing-address
and ::shipping-address
specs? Also not sure if (spec/def ::billing-address ::address)
or something like it, is an option?#2022-07-1720:42Alex Miller (Clojure team)you would need to do the latter, which is an option#2022-07-1720:43Alex Miller (Clojure team)it's often convenient to have both structural specs (like address) and named attributes that refer to those#2022-07-1722:20apbleonardOk fab - thanks @alexmiller Probably seen that in code many times but not really thought about it much. Trying to build a API data dictionary at work that promotes specifying attributes over HTTP request/responses bodies (to avoid the repetition that brings.) Looks like I will need to list these named specs in the dictionary as well as the attributes themselves.#2022-07-2808:13Ben SlessThis is a bit more relevant to generators, but how can I use the generated result from one generator as the baseline for another?
I want to be able yo generate sequences representing events in time or request and response, and for all of them to share an ID, for example#2022-07-2808:26lassemaattaperhaps https://clojure.github.io/test.check/clojure.test.check.generators.html#var-bind? or the let
variant, as hinted in the docstring#2022-07-2808:28lassemaattaor perhaps fmap#2022-07-2808:35lassemaattahttps://www.youtube.com/watch?v=F4VZPxLZUdA presentation by Gary Fredericks gives nice examples on how to compose generators#2022-07-2808:38Ben SlessJust guessing because I haven't looked yet, but bind sounds exactly right, thanks!
My intuition is from what it means for implementing logic engines, where bind is akin to conjunction#2022-07-2808:42Ben SlessSo let's just run this through, I have a generator for maps, and I want some values from g1 to continue through g2, most brutal way would be merging these values on top of what comes out of g2, but there surely is a better way#2022-07-2808:52lassemaattaI'm no expert on the matter, but I remember once doing something like (->> (gen/tuple random-id-gen seq-of-requests-gen) (fmap (fn [[id reqs]] (map #(assoc % :id id) reqs))))
when I just wanted to "give me some random requests with the same random id" etc.#2022-07-2814:57pppaultuple into fmap is also a solution i used for a map generator#2022-07-2814:59pppauli made a recursive map generator based on example data so my gen looks like (fmap (tuple (map (elements elements))))
and basically you can replace any elements with another fmap (self similar structure)#2022-07-2814:59pppauli saw bind when looking for my solution, but i didn't see how it was going to be better than fmap#2022-07-2815:04pppaulmerging values from previous generators is ok, those other gens are able to shrink, fmap is a bit inefficient, but maybe you could make a new generator that merging for your problem. so long as your fmap isn't doing a lot of work, i don't think it'll be a problem#2022-07-2909:53Ben SlessIs there a way to recover the keys from a map generator before it runs?#2022-07-2914:38pppauli don't know what that means. recover something before it runs doesn't even sound like it makes sense logically#2022-07-2914:38Ben SlessIf I have a map generator, can I know what keys it will generate before running it?#2022-07-2914:38pppaulyes#2022-07-2914:39Ben SlessCool, how?#2022-07-2914:39pppaul(gen/map (gen/elements ...list-of-keys) (gen/elements ...list-of-vals))#2022-07-2914:40Ben SlessI mean going the other way, I already have the map generator, I want to take it apart#2022-07-2914:40pppaulthere is also (gen/hash-map)#2022-07-2914:41pppauli don't get it. are you saying that you are using someone else's gen, that has fixed keys, and you can't see what keys those are from their code?#2022-07-2914:43lassemaattaI was browsing through the generator code today and I got the impression that generator instances are opaque so you can't for example access the source spec. or at least that's my understanding of it#2022-07-2914:45pppaulyou could make a data structure that creates gens, then put that on as meta-data. but you would be creating a wrapper around the whole gen library at that point#2022-07-2915:43Ben SlessIt was a matter of efficiency. If I want to use bind to inject values from one generator to the next, there's no reason to generate those values in the second generator as well, right?#2022-07-2917:21lassemaattathis is just a guess (and perhaps not a good idea), but if you really want to do that kind of optimization (e.g. the value is really expensive to generate and you know the qualified kw you want to replace) then perhaps you could leverage the overrides
arg of clojure.spec.alpha/gen
:thinking_face: for example something not completely unlike this: (bind random-id-gen (fn [id] (c.s.a/gen ::coll-of-maps {::id (fn [] (return id))})))
, where you don't want to execute the default generator for ::id for each map#2022-07-2918:34Ben SlessIt's not critical, maybe I'm overly concerned about stepping into pathological situations#2022-08-0108:16Mark WardleI simply have functions that return generators that wrap another generator and replace the values in the generated map (using fmap) with the explicitly defined ones. Eg see https://github.com/wardle/hermes/blob/main/src/com/eldrix/hermes/rf2.clj I have lots of cases where I need to be careful to use the same identifier in many generated structures. #2022-08-0420:19Stel AbregoHowdy y'all, I'm wondering if it's possible to "bubble up" a spec error from s/every
. Here's what I mean:
(s/def :nuzzle/title string?)
(s/def :nuzzle/rss? boolean?)
(s/def :nuzzle/page-entry (s/keys :req [:nuzzle/title] :opt [:nuzzle/rss?]))
(s/def :nuzzle/user-config
(s/and
(s/keys :req [:nuzzle/base-url])
(s/every (fn [[key _]] (or (keyword? key) (vector? key))))
(s/every (fn [[key val]] (or (not (vector? key))
(s/valid? :nuzzle/page-entry val))))))
(s/explain
:nuzzle/user-config
{:nuzzle/base-url ""
[:blog-posts :test-post] {:nuzzle/title 234 :nuzzle/rss? true}})
For every key-value pair in the config map where the key is a vector, I want to validate the associated value against a spec (in this case :nuzzle/page-entry
). I'm using s/every
because vector keys can be arbitrary but their values must conform to :nuzzle/page-entry
.
The problem is that when a schema error occurs for a :nuzzle/page-entry
(for example :nuzzle/title
is not a string), I get an s/explain
output like this:
[[:blog-posts :test-post] #:nuzzle{:title 234, :rss? true}] - failed: (fn [[key val]] (or (not (vector? key)) (valid? :nuzzle/page-entry val))) in: [1] spec: :nuzzle/user-config
Is there any way to refactor this to get a better error message about the specific problematic key-value pair (`:nuzzle/title 234`)?#2022-08-0420:29Alex Miller (Clojure team)I don't know if this will help but you may find the techniques used in https://cognitect.com/blog/2017/1/3/spec-destructuring to be helpful here#2022-08-0420:30Alex Miller (Clojure team)that is, treating the map as a coll of an s/or of k/v tuple types (rather than as a map)#2022-08-0420:32Stel Abrego@U064X3EF3 I will read up on this, thank you!#2022-08-0517:53Dan BoitnottIs there a function to test whether a value is valid and conformed to a spec? For example, given (s/valid? ::myspec val) => true
, is there an equivalent such that (s/<something>? ::myspec (s/conform ::myspec val)) => true
, or alternatively a way to generate a spec for the conformed value such that (s/valid? (s/<something> ::myspec) (s/conform ::myspec val)) => true
?
The problem I'm trying to solve is that I have a family of functions which expect to receive conformed values but, because user inputs are involved, need to confirm the inputs are conformed.#2022-08-0517:54Alex Miller (Clojure team)if you have conformed values, then s/unform should give you the unconformed value#2022-08-0517:54Alex Miller (Clojure team)so (s/valid? ::myspec (s/unform ::myspec conformed-value))
?#2022-08-0517:56Dan BoitnottCool! Lemme give that a shot. Thanks!#2022-08-0518:00Dan BoitnottWorks, thanks! So, is this an idiomatic use of spec? Or am I running into this because I'm misusing conform?#2022-08-0518:06Alex Miller (Clojure team)this is an api provided for un-conforming values, so seems idiomatic to me#2022-08-0522:38Dan BoitnottIn case it's at all interesting, here's what I came up with:
(defn conformed-or-throw
"If x is a valid and conformed returns x. If x is valid and un-conformed,
returns (spec/conform spec x). Otherwise throws an ex-info with msg."
[spec msg x]
(if (s/valid? spec x) (s/conform spec x)
(let [unformed (try (s/unform spec x)
(catch Exception _ (throw-with-spec spec (str msg " (unable to unform)") x)))]
(if (s/valid? spec unformed) x
(throw-with-spec spec (str msg " (unformed)") unformed)))))
#2022-08-0522:39Dan BoitnottWhere throw-with-spec
is:
(defn throw-with-spec [spec msg x]
(throw (ex-info (str msg "\n" (exp/expound-str spec x)
"-------------------------\n"
(with-out-str (pprint x))
"-------------------------")
{:spec spec :x x})))
#2022-08-1717:36Stel AbregoIs dynamically redefining specs good practice? I want to check if certain values in a config map match values from another location in the config map so I'm using this function:
(s/def :nuzzle/author keyword?)
;; Dynamically redefine :nuzzle/author based on config
(defn redefine-author-spec [config]
(s/def :nuzzle/author (s/and keyword? (-> config :nuzzle/author-registry keys set))))
It totally works for my use case but I'm wondering if this is something that is typically not recommended.#2022-08-1717:52phronmophobicI couldn't find anything directly related, but it does seem to go against the spirit of an open system with global semantics:
https://clojure.org/about/spec#_unify_specification_in_its_various_contexts
> Specs for data structures, attribute values and functions should all be the same and live in a globally-namespaced directory.
https://clojure.org/about/spec#_informational_vs_implementational
> Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment. The best and most useful specs (and interfaces) are related to purely information aspects. Only information specs work over wires and across systems. We will always prioritize, and where there is a conflict, prefer, the information approach.#2022-08-1718:13Alex Miller (Clojure team)I would not recommend it due to the redefinition, but dynamically creating a spec is not inherently bad and can be useful. I think my question then is whether this is really a spec or something weaker.#2022-08-1718:13Alex Miller (Clojure team)you also don't necessarily have to put the spec in the registry#2022-08-1718:14Alex Miller (Clojure team)that is, you could just use s/valid?
on a spec you create and pass around#2022-08-1718:14Alex Miller (Clojure team)(won't work when validating s/keys with namespaced keys)#2022-08-1722:24Stel Abrego@U064X3EF3 That makes sense. I really just want the amazing bhb/expound
error messages so I'm trying to shove stuff into specs that maybe shouldn't be there. I think I could use a local spec instead. Thank you!#2022-08-1912:32Yehonathan SharvitIāve heard Rich saying a couple of times that map specs should be open. Does a similar advice apply for APIs of REST endpoints?
Should we allow extra keys (not mentioned in the payload schema) in request payloads?
If yes, should we drop extra keys or should we pass them down the stream?
If no, why?#2022-08-1913:03ikitommiIMO definitely don't pass anything extra by default from untrusted sources, it's a security risk (e.g. allowing a :credit-checked true
to be set to fresh orders). In Java, as it's mostly closed/classes and the JSON-libs have either "strip extras" or "fail on extras". With clojure & open maps, you need an extra tool to do that automatically, given a spec/schema. I recall reitit-ring defaults to "strip extras" with all of spec, schema & malli (supports also "failing on extras" & "explicitly open" for all three)#2022-08-1913:08ikitommiI would use "explicitly open" only from internal (trusted) sources, and when the use case benefits from it.#2022-08-1914:55Yehonathan SharvitWhat do you think about rejecting request with extra keys?#2022-08-1915:23jumarThat's a bad idea I believe- with distributed systems you need flexibility and openness in what you accept.
That gives you more freedom in how you do deployments, etc#2022-08-2005:13didibusI think Rich has slightly changed his opinion on that matter, since Spec 2 allows closed map specs.
I've had conversations about it with the core team, especially around security use cases, and to validate maps from the producer side. And they seem to recognize those use cases.
But there's a time and place for both.#2022-08-2005:16didibusPersonally, on an API, I would drop them. If someone sent a well formed payload that just has more stuff on it, why reject the request?#2022-08-2020:58Yehonathan SharvitIāve heard people say that if you tolerate an extra field, you might have a forward compatibility issue in case you make this extra field part of the schema.#2022-08-2023:17didibusAt the end of the day, you got to decide what scenario you'd rather make annoying to your clients.
Do you annoy them by erroring on a request that has everything it needs, but also has more stuff?
Do you annoy them in the extremely rare scenario that you didn't error when they were giving you extra stuff, for years, letting them make calls without issues, and one day you might decide to make a field of the same exact name and break their superfluous data calls?#2022-08-2023:20didibusI prefer the former personally, because in practice I find there are more scenarios where that comes in useful.
Say there's a field you no longer need? Or say you're doing a deployment, and the client and server are upgrading to support a new field at the same time, if the client deploys first and calls with additional fields on server that still don't support that behavior, etc.
But honestly, I find this whole situation kind of unimportant, all these are pretty rare. Most clients tend to call with exactly the fields that the API asked for most of the time.#2022-08-2023:21didibusWell, actually I take that back. The upgrade situation is pretty common. Let me talk about it a bit more.#2022-08-2023:23didibusYou want to add a mandatory new field to the API...
How would you do that? The new code requires the new field, it's not optional, you want to make it mandatory.
But the current API doesn't take that field, and clients are calling that API without the field.
How do you time the deployment of new client code which now always include this new mandatory field, and the server code that mandates it?#2022-08-2023:25didibusIf you deploy the server code first, your validation will error at all current clients which have not upgraded to pass the new field. And even if you didn't validate, well the API wouldn't work, cause your new logic needs that field, it's mandatory.#2022-08-2023:26didibusBut if you have all clients upgrade first, they'll see errors, because the current server code doesn't allow additional fields.#2022-08-2023:26didibusIt's a catch 22#2022-08-2023:27didibusIf your current code though simply ignored additional fields, you're out of the pickle.
Tell all clients that a new server code will be deployed on some date, and require a mandatory additional field. Clients can start passing the data even if the current code ignores it. And when they all do, you can deploy the new server code.#2022-08-2105:30Yehonathan SharvitVery insightful scenario @U0K064KQV!#2022-08-2105:49Yehonathan SharvitIāve summarized your insights in a https://twitter.com/viebel/status/1561228623331397632?s=20&t=xJGKWSvqC6QDn42N8rgNvA @U0K064KQV.#2022-08-2111:44pithylessThere's one more common case: silently stripping extra params and returning success may hide incorrect usage of an API (which may or may not be obvious and discovered till much later).
Consider if we have an http endpoint where I added an optional filter query-param: /search?status=pending
If I'm incorrectly using the filter param (e.g. calling it with status
not current_status
) it would strip the invalid param and still return results (just not the ones I expected). Now you're relying on tests, etc. which may or may not fail; but also may fail much later and it may take longer to understand what went wrong.
I find it useful to define public APIs with strip-extras behavior (for all the reasons @U0K064KQV mentioned above), but to define them as strict and fail-fast when in development mode (for better developer experience and faster identification of contract inconsistencies)#2022-08-2112:25Yehonathan SharvitYeah. There is also an intent concern#2022-08-2121:18didibusI wonder if APIs should return warnings. Succeed, but warn :thinking_face:
I also wonder if clients should be allowed to choose the behavior they prefer, can the api take "mode=strict" for example.#2022-08-2121:50Yehonathan SharvitInteresting.#2022-08-2909:27dergutemoritzSorry for the late reply but I just came across this. I would like to add one more perspective: The scenario @U0K064KQV gave is a breaking change (i.e. requiring more from the client). In this case, I would rather provide a new API which requires the new param (possibly just with a v2
suffix. Maybe remove the old version after a transitional period if it's desirable to not keep it around indefinitely. Basically do the same thing as Rich recommends for library APIs here: https://youtu.be/oyLBGkS5ICk?t=1944
In practice, you end up with a similar flow as in the solution suggested by @U0K064KQV but arguably it's a bit more transparent what's going on, e.g. it's very easy to determine how many clients are using the new API already. Also, you're still protected from breaking clients who accidentally were sending this newly required param before.#2022-08-2916:31didibusYes, it's a breaking change where you will no longer handle the prior case. Upgrading to V2, generally implies V1 is still supported. If I do that, I too prefer versioning over making the new field optional.
But sometimes, especially for microservices, you will not continue to support the old way, and need a path to migration that doesn't require downtime or cause issues.#2022-08-3109:00dergutemoritzBut you have a transitional period of supporting both in your case, too (i.e. when some clients already send the to-be-required new param and others don't yet), it's just less explicit. I guess if you're sitting in the same organization as all consumers, it's not much of an issue as you can just tightly coordinate the upcoming breaking change. But if you're offering a service that's consumed by third parties outside of your control, I'd definitely prefer the explicit approach.#2022-08-3109:01dergutemoritzP.S.: Just re-watched the rest of the spec-ulation talk and found that it even has a section about (web) services, too, discussing that exact situation: https://youtu.be/oyLBGkS5ICk?t=3958#2022-08-3110:12Yehonathan SharvitWhat does Rich say about that? I cannot listen to the talk right now#2022-08-3110:29dergutemoritzBasically the same thing: Don't break an existing operation foo
, make a foo2
instead.#2022-08-3110:29dergutemoritz(break by requiring more)#2022-08-3114:54Yehonathan SharvitIn case the new argument is mandatory, Rich advises us to give a new name to the API. But what about the case where the new argument is optional and of a different type that the one sent by the client?#2022-08-3115:56dergutemoritzAh right, he doesn't discuss the forward compatibility issue. I for one prefer to not allow extra keys on that level for that reason, especially when we're talking plain JSON, i.e. non-namespaced keys. Also, it helps guard against typos for optional params (i.e. clients intending to provide an optional param but because it's misspelt, it's silently ignored). Of course, the latter could also be achieved by other means e.g. have a "warning" backchannel as suggested by @U0K064KQV above.#2022-08-3115:56didibus> But you have a transitional period of supporting both in your case, too (i.e. when some clients already send the to-be-required new param and others don't yet), it's just less explicit. I guess if you're sitting in the same organization as all consumers, it's not much of an issue as you can just tightly coordinate the upcoming breaking change. But if you're offering a service that's consumed by third parties outside of your control, I'd definitely prefer the explicit approach.
>
Ya, we agree. I'm talking in micro-service architecture, multiple services are part of a coherent system, maybe owned by one or more internal teams. You don't need to support multiple versions of each microservices internally, that's just too much to commit. You can force the consumers to take an update project for the new behavior. Once they did it they can go to prod, there's no transitory period where we support both, clients now pass extra fields, but the current server ignores them. Then you do deployment of the server and it now starts using those additional fields.
With paying 3rd party customers to a service, you probably can't force them all to migrate early, and probably need to support both behaviors for a while.#2022-08-3115:57dergutemoritzIndeed - context matters š#2022-08-3115:58didibusI still don't think that erroring on additional fields has advantages in either of those cases though, at least can't think of any here.#2022-09-0103:08Yehonathan Sharvit@U0K064KQV @U06GVE6NR Did you know that the HTTP Prefer Header could be set to either āhandling=strictā or āhandling=lenientā?
https://twitter.com/viebel/status/1565174064821551105?s=20&t=ijcp8FtsCUVa34sCvoCOAA#2022-09-0103:28didibusInteresting that was considered. I can see this having value, client can choose. They can even in devo be strict to test they didn't make typos and all, and in prod be lenient if they want.#2022-08-2016:57walterlHi. š I'm spec'ing a Ring
What is the
As I understand it, I would have to check the values manually and separately (unlike with
Nm. I see one can't use string keys with s/keys
.#2022-08-2109:20Alejandroafter reading a book called Domain Modeling Made Functional I wonder what would be a good way to attach type information to pieces of data. I'm not talking about a type system, but about reasoning of how data flows through components. I'd like have specs that claim a function only accepts data that went through some kind of processing. E.g., an email is parsed and validated on the edge of the system, but not confirmed yet by the user, and I'd like to make sure it's never used inappropriately. How would you do this? Using keywords (with inheritance, probably) or with records?#2022-08-2111:37Alex Miller (Clojure team)Seems like a perfect use of metadata#2022-08-2120:34AlejandroHm. That's interesting, I'll try it, thanks#2022-08-2121:32didibus> I'd like have specs that claim a function only accepts data that went through some kind of processing. E.g., an email is parsed and validated on the edge of the system, but not confirmed yet by the user, and I'd like to make sure it's never used inappropriately
This tends to be pretty trivial in Clojure and with Spec. You can use metadata, but I'd say in general, you don't even have too, since you'll probably be working with maps only for your model anyways, you can just add more data to them.
For example, on your User entity (which should be a plain Clojure map), just add a :type
key where the values are the various state of a User through your system.
So maybe you have a {:type :verified-user}
and a {:type :registered-user}
.
Now in each function, where you expect a :verified-user
, well just have a spec for that which will also assert that :type
is :verified-user.
This will probably be pretty close to the book you read, where I'm assuming they use custom ADT types to model this.
Alternatively, you can enhance the user with more info, and validate the info, such as adding an :email-verified true
on the User only after its been verified, then have a spec on the function that asserts that :email-verified
exists and is true on the user.#2022-08-2121:35didibusYou could make this metadata, but it's harder to use spec to validate meta, not super hard, but you kind of need two specs, one for the data and one for the meta.
The advantage of meta is if you don't want them to participate in equality. Like if you still want two users to be equal even if one is verified and the other is not.#2022-08-2122:21didibus(require '[com.myapp :as-alias app])
(require '[com.myapp.user :as-alias user])
(s/def ::user/type #{:registered-user :verified-user})
(s/def ::user/username string?)
(s/def ::user/email string?)
(s/def ::app/user
(s/keys :req-un [::user/type ::user/username ::user/email]))
(s/def ::app/verified-user
(s/and ::app/user #(= :verified-user (:type %))))
(s/def :app/registered-user
(s/and ::app/user #(= :registered-user (:type %))))
So now on functions that expect a :verified-user you can do:
(s/fdef somefn
:args (s/cat :verified-user ::app/verified-user)
:ret ::app/verified-user)
And on the function that verifies a registered-user you can do:
(s/fdef verify-user
:args (s/cat :registered-user ::app/registered-user)
:ret ::app/verified-user)
#2022-08-2122:30didibusAnd if different types of users also had different set of keys, you can multi-spec the ::app/user spec.#2022-08-2114:11popeyeWhen to use clojure spec ?#2022-08-2114:26danierouxhttps://clojure.org/about/spec#2022-08-2218:02lilactownis there a reason that there's no concrete Spec
type/class? I have a desire to extend a library I'm writing to dispatch on whether a value is a reified spec value, and I can't easily do that without taking a dependency directly on clojure.spec.alpha
#2022-08-2218:03lilactowni.e. the only way I can tell whether I've been passed a spec is by using s/spec?
#2022-08-2218:04Alex Miller (Clojure team)It's not a type#2022-08-2218:13Alex Miller (Clojure team)You could dynamically load and resolve that function #2022-08-2218:22lilactownthat doesn't work in CLJS#2022-08-2218:23lilactownin various places in clojure.spec.alpha an object is created via (reify Spec ,,,)
and returned, which is what I want to extend the behavior of#2022-08-2218:27lilactownI don't care about the name, just wishing there was a concrete type I could extend#2022-08-2218:49Alex Miller (Clojure team)sets are also specs, and functions. there is not "one type" to check for#2022-08-2218:53Alex Miller (Clojure team)the behavior is defined via the protocol. things like with-gen are a good example of extending by wrapping. but maybe there is some better way to do what you need, not sure#2022-08-2320:29vlaaadDoes something like that make sense?
(defmulti dispatch identity)
(defmethod dispatch :default [v]
(:spec v))
(s/valid?
(s/multi-spec dispatch :spec)
{:spec (s/keys :req-un [::a ::b]) :a 1 :b 2})
=> true
The real code in :default
branch would be more complex, but the idea is that I have a complex rule to infer spec from the value, and I don't want to encode it in a single predicate that executes the rule and calls s/valid?
inside because it will lose the error reporting/explanation. I.e. I don't want to do this:
(s/valid?
(s/spec #(s/valid? (:spec %) %))
{:spec (s/keys :req-un [::a ::b]) :a 1})
=> false
because (s/explain-data ...)
will just say that (s/valid? (:spec %) %)
is false and not that the :b
is missing#2022-08-2320:31vlaaadExplanation comparison:
(s/explain
(s/spec #(s/valid? (:spec %) %))
{:spec (s/keys :req-un [::a ::b]) :a 1})
=> {:spec ... :a 1} - failed: (valid? (:spec %) %)
(s/explain
(s/multi-spec dispatch :spec)
{:spec (s/keys :req-un [::a ::b]) :a 1})
=> {:spec ... :a 1} - failed: (contains? % :b)
#2022-08-2418:52vlaaadcheck this out:
(defmacro def-dynamic-spec [name & fn-tail]
(let [multi-name (symbol (str name "-multi-fn"))]
`(do
(defmulti ~multi-name (constantly nil))
(defmethod ~multi-name nil
a macro that creates a spec that dynamically infers a value spec using a function of the value, while preserving error reporting/explanation data#2022-08-2508:37vlaaadAny opinions?#2022-08-2508:37vlaaadAny opinions?#2022-08-2409:47Ishaan Guptahow far is the idea that spec2 schemas are just sets of keys gonna be taken? is there going to be a full set of set operations for schemas like there's a full set of regex operations for a sequence?#2022-08-2411:59Alex Miller (Clojure team)It may be take even less far than it is :) not sure yet#2022-08-2409:48Ishaan Guptais there any utility in a schema intersection?#2022-09-0702:58walterlFrom the https://clojure.org/guides/spec:
> Note that the :ret
and :fn
specs are not checked with instrumentation as validating the implementation should occur at testing time.
How can one enable :fn
and :ret
validation during tests, without using stest/check
?#2022-09-0705:08seancorfieldPerhaps https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite ?#2022-09-0713:20KelvinI use the Orchestra library to instrument :ret
(alongside the default :args
): https://github.com/jeaye/orchestra#2022-09-0920:30Cam SaulWhat's the easiest way to do something like a "soft cut" e.g. core logic conda
to lock a regex spec into validating an optional argument with a specific spec if it meets some condition, and failing otherwise?
Suppose I have a spec like this:
(s/def ::my-map
(s/keys :opt-un [...]))
(s/def ::my-spec
(s/cat :my-map (s/? ::my-map) :options (s/* (s/cat :k keyword? :v any?))))
If I pass in a map that fails the ::my-map
spec I get an error like
identity - failed: Extra input in: [0] spec: ::my-spec
because it then tries to use that argument as an option. How can I force it to validate maps with the ::my-map
spec?
I've tried stuff like (s/& map? ::my-map)
but that doesn't work either#2022-09-0921:28Alex Miller (Clojure team)just checking - you have an optional followed by kv pairs, and with what input do you get this error? something that's neither a map or keyword as first arg?#2022-09-0921:30Alex Miller (Clojure team)I'm not sure if there's an easy alternative for you - seems like you could possibly treat this as two specs based on even/odd arg count. it's an unusual usage but you could probably build an s/multi that switched based on that and used different specs#2022-09-0921:33Cam SaulI ended up doing something like
(s/&
(s/cat :my-map (s/? map?) :options (s/* (s/cat :k keyword? :v any?)))
(s/keys :opt-un [:my-map])
which works okay I guess but is a little icky I think#2022-09-0921:36Cam SaulHere's a more detailed example including something with neither a map nor keyword first arg. I just get an extra input error
(s/def ::k
integer?)
(s/def ::my-map
(s/keys :req-un [::k]))
(s/def ::my-spec
(s/cat :my-map (s/? ::my-map) :options (s/* (s/cat :k keyword? :v any?))))
(s/explain-str ::my-spec [{:k 100}])
(s/explain-str ::my-spec [{:k 100} :x :y])
;; => "Success!\n"
(s/explain-str ::my-spec [100])
;; => "(100) - failed: Extra input in: [0] spec: ::my-spec\n"
(s/explain-str ::my-spec [{}])
;; => "({}) - failed: Extra input in: [0] spec: ::my-spec\n"
#2022-09-0921:38Cam Sauldefmulti
is actually one real-world example of where you have a usage like this. The metadata attribute map is optional and after that it supports dispatch-fn
as a positional argument and then a few optional key-value arguments like :default
and :hierarchy
if I recall correctly.#2022-09-0921:42Cam SaulThis came up for me because I was working on a def
macro that did something similar (supports and optional attribute map and keyword options) and I tried to tweak the spec to validate one of the keys in the attribute map specifically and it made things fail with confusing errors. In the example above I want the last failure to tell me it's failing because :k
is not an integer?
#2022-09-0921:46Cam SaulThis is the working version I came up with:
(s/def ::my-spec
(s/& (s/cat :my-map (s/? map?) :options (s/* (s/cat :k keyword? :v any?)))
(s/keys :opt-un [::my-map])))
(s/explain-str ::my-spec [{}])
=> "{} - failed: (contains? % :k) in: [:my-map] at: [:my-map] spec: ::my-map\n"
I was wondering if maybe there was some easier way to do this, e.g. some way to define my so-called "soft cut" inline rather than having to wrap the whole thing in s/&
#2022-09-0921:30thiruIf I were to start using Spec today is it recommended to use Spec 1 or Spec 2? Also, my codebase is Clojure 1.8 and can't upgrade. The only issue I see with this is that ident?
is missing but that's easy enough to add. Is there any strong reasons to avoid using Spec with Clojure versions before 1.9?#2022-09-0921:31Alex Miller (Clojure team)spec 2 has not been released and is not under active dev at this moment#2022-09-0921:31Alex Miller (Clojure team)I think someone did have a backport of spec at one point (we are not maintaining anything official though)#2022-09-0921:32Alex Miller (Clojure team)https://github.com/tonsky/clojure-future-spec#2022-09-0921:33thiruExcellent, thank you for the super fast response Alex! š#2022-09-1214:18Kelvinhttps://clojurians.slack.com/archives/C1B1BB2Q3/p1662759068905259?thread_ts=1662759019.232099&cid=C1B1BB2Q3
Interesting - I wasnāt aware of this, does that mean spec2 is delayed indefinitely? (Posting outside of thread since I donāt want to derail)#2022-09-1214:34Alex Miller (Clojure team)It just means that at this moment we are not actively working on it#2022-09-1814:04maleghastHello All, was just wondering if there is already a tool out there for deriving spec from a data structure..? Basically I want to consume JSON and render it as EDN / Clojure and then reverse engineer a spec from that data structure.#2022-09-1814:24lispycloudsi knew about https://github.com/stathissideris/spec-provider
havent tried it in recent times#2022-09-1814:24maleghastThanks @U7ERLH6JX I will go take a look...#2022-09-2001:59Drew VerleeYeah, malli can do that too.
but in general, i would think three times before going that route.#2022-09-2018:30phronmophobicI'm trying to use spec to build a generator for valid destructure bindings. I'm starting with the specs in clojure.core.specs.alpha
.
(s/def ::seq-binding-form
(s/and vector?
(s/cat :forms (s/* ::binding-form)
:rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
:as-form (s/? (s/cat :as #{:as} :as-sym ::local-name)))))
Generating ::seq-binding-form
values struggles because it's just generating random vectors that hopefully satisfy the s/cat
. This seems to get fixed with (gen/fmap vec ...)
:
(s/def ::seq-binding-form
(s/with-gen
(s/and vector?
(s/cat :forms (s/* ::binding-form)
:rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
:as-form (s/? (s/cat :as #{:as} :as-sym ::local-name))))
#(gen/fmap
vec
(s/gen
(s/cat :forms (s/* ::binding-form)
:rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
:as-form (s/? (s/cat :as #{:as} :as-sym ::local-name)))))))
Unfortunately, the version using gen/fmap
now usually throws a stackoverflow exception. However, I noticed that if I ignore the vector requirement for ::seq-binding-form
, then the stack overflow exception goes away.
(s/def ::seq-binding-form
(s/cat :forms (s/* ::binding-form)
:rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
:as-form (s/? (s/cat :as #{:as} :as-sym ::local-name))))
Is there a way to prevent and stackoverflow exceptions and have it generate a vector?#2022-09-2018:37phronmophobicIt looks like spec2 might have better support for this. I don't have any real reason to stick with spec1, so I'll see if I can get this working with spec2.#2022-09-2018:37Ben SlessWhat if instead of fmap you use and conforming?#2022-09-2018:38phronmophobicI don't know about conforming, but I'll take a look#2022-09-2018:38Ben SlessMaybe it's called conform, lemme check#2022-09-2018:39phronmophobicI was checking out spec2 based off this previous thread, https://clojurians.slack.com/archives/C1B1BB2Q3/p1650425212914829?thread_ts=1650424802.839689&cid=C1B1BB2Q3#2022-09-2018:39Ben Slesshttps://clojuredocs.org/clojure.spec.alpha/conformer#2022-09-2018:40phronmophobicgiving it a try!#2022-09-2018:51phronmophobicI don't think that solves this use case. It still doesn't generate ::seq-binding-form
as a vector. It does convert it if I conform the value, but I think it's worth trying spec2#2022-09-2018:53Ben SlessGen doesn't emit conformed values?#2022-09-2018:59phronmophobicIt's totally possible I'm doing it wrong. My attempt was:
(s/def ::seq-binding-form
(s/and
(s/cat :forms (s/* ::binding-form)
:rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
:as-form (s/? (s/cat :as #{:as} :as-sym ::local-name)))
(s/conformer vec)))
#2022-09-2019:01Alex Miller (Clojure team)the specs need to be updated to have this work with spec 2#2022-09-2019:03Alex Miller (Clojure team)to use catv
instead of cat
etc#2022-09-2019:38phronmophobicIt works!#2022-09-2117:04phronmophobicJust to follow up, test.check
is magic. For the right problems, it finds tricky use cases that I would never think of.
> (let [[ & [ & [& [& [& [& hi]]]]]] [42]]
hi)
(42)
> (let [[& {:as m}] [:a 42]]
m)
{:a 42}
#2022-09-2117:09Alex Miller (Clojure team)generation isn't magic, but I'm pretty sure shrinking is#2022-09-2619:15kennyCurious - why does the Spec form for s/double-in
return a s/and
with a series of predicates attached for each option passed (e.g., :min, :max, etc), and something like s/every
does not include the https://github.com/clojure/spec.alpha/blob/13bf36628eb02904155d0bf0d140f591783c51af/src/main/clojure/clojure/spec/alpha.clj#L558-L567?
(s/form (s/double-in :min 2))
=> (clojure.spec.alpha/and clojure.core/double? (clojure.core/fn [%] (clojure.core/<= 2 %)))
(s/form (s/every string? :min-count 2))
=> (clojure.spec.alpha/every clojure.core/string? :min-count 2)
#2022-09-2619:18Alex Miller (Clojure team)This is a limitation of how double-in is written as a composite but without custom form support. This has actually been addressed in spec 2 (not sure if this particular one has been fixed there but it could be)#2022-09-2619:21kennyOk. So the desired output for s/double-in's (and its ilk) form would be similar to that of s/every -- returning the kv options passed to the form instead of the composite?#2022-09-2619:33Alex Miller (Clojure team)yep, I think the s/form should reproduce the original not expose how it's implemented#2022-09-2619:34Alex Miller (Clojure team)spec-alpha2 % clj
Clojure 1.10.1
user=> (require '[clojure.alpha.spec :as s])
nil
user=> (s/form (s/double-in :min 2))
(clojure.alpha.spec/double-in :min 2)
#2022-09-2619:34kennyMakes sense. Thanks for the response. #2022-09-2810:49joost-diepenmaatGood morning. I might be mis-remembering something, but isnāt there some analogue to (s/keys :req-un .. :opt-un ā¦) that works for strings instead of keywords?#2022-09-2810:50joost-diepenmaatI thought that (s/strs ā¦) would exist but noā¦#2022-09-2811:55Alex Miller (Clojure team)no#2022-09-2816:25joost-diepenmaatFair enough. Thanks. #2022-10-0115:27NiclasDoes anybody know of a lib that allows me to inline fn specs in the fnās metadata?
Looking for something like:
(defn myfn
{:spec (...)}
...)
ā¦ which would be equivalent to:
(defn myfn ...)
(s/fdef myfn ...)
#2022-10-0115:55teodorluI don't know of a library that does this specifically, but I think it should be doable with a macro.
Perhaps something like this:
;; idea: create a macro defn+spec
(defn+spec myfn
{:spec (,,,)}
[]
,,,)
;; so that it expands to
(do
(defn myfn
{:spec (,,,)}
[]
,,,)
(s/fdef myfn (:spec (meta #'myfn))))
#2022-10-0201:30colinkahnThere's https://github.com/danielcompton/defn-spec which has links to some other libs too in the README#2022-10-0211:42FiVoIs there a standard way to reuse specs for conformed values? Do I need write a spec for the "normal" as well as conformed version of the data? As an example:
(s/def ::tuple (s/and vector?
(s/cat :first identity :second identity)))
(def tuple (s/conform ::tuple [1 2]))
(defn foo [t] t)
(s/fdef foo :args (s/cat :t ::tuple))
(foo tuple) ; => error
I would like to reuse ::tuple
for my funciton spec.#2022-10-0214:05Alex Miller (Clojure team)Whatās the error? (and why not use s/tuple in that spec?)#2022-10-0214:42FiVoThe ::tuple
spec was just an example and maybe it was not clear what I am asking. I am mainly wondering if there is a way to avoid the ::tuple-conformed
spec in the following
(s/def ::tuple (s/and vector?
(s/cat :first int? :second int?)))
(def conformed-tuple (s/conform ::tuple [1 2]))
conformed-tuple
;; => {:first 1, :second 2}
(s/valid? ::tuple conformed-tuple)
;; => false
(s/def ::first int?)
(s/def ::second int?)
(s/def ::tuple-conformed (s/keys :req-un [::first ::second]))
(s/valid? ::tuple-conformed conformed-tuple)
;; => true
#2022-10-0214:44FiVoYou actually kind of answered my question here https://stackoverflow.com/questions/41686718/specs-for-conformed-specs-asts#2022-10-0214:55FiVoSo I think the question becomes how do people usually generate both specs for the above case?#2022-10-0219:30Brian BeckmanSOLVED!
Hello, speccers. Iām really stuck and would be very grateful for advice! Iām writing specs to generate test cases for the back end of a compiler (LPython [sic]; http://lpython.org). The compiler back-end accepts a kind of S-expression syntax (nice!?). TL;DR: I want to generate S-exprs like
(const 42 (I 4)) ; :head const, :val 42, :type (I 4)
I can easily get
A. [const 42 (I 4)]
; error: need round brackets outside
B. (const 42 I 4)
; error: need round brackets inside
C. (const 42 ((I 4)))
; error: too many brackets inside
but not what I want without un-nestable post processing.
DETAILS:
(I 4)
could be other types like (F 16)
and (C 32)
in actual code, so the :type spec
(it seems), must be a s/cat
to get the internal round brackets:
(s/def ::ttype (s/cat :head #{'I}, :bytes #{4}))
(->> ::ttype s/gen gen/generate) ; ~~~> (I 4)
I run into trouble right away trying to spec ::iconst
to generate
(const 42 (I 4))
A. If I spec ::iconst
as a tuple, I get a well-formed expression, but with square brackets outside!:
(s/def ::iconst (s/tuple #{'const} integer? ::ttype))
(->> ::iconst s/gen gen/generate) ; ~~~> [const 42 (I 4)]
B. If I spec ::iconst
as a s/cat
, the ::ttype
field gets spliced instead of appended:
(s/def ::iconst (s/cat :head #{'const} :value integer?
:type ::ttype))
(->> ::iconst s/gen gen/generate) ; ~~~> (const 42 I 4)
B. If I try a nested s/cat
for the ::type
field, itās no help:
(s/def ::iconst (s/cat :head #{'const} :value integer?
:type (s/cat :singleton ::ttype)))
(->> ::iconst s/gen gen/generate) ; ~~~> (const 42 I 4)
C. If I try s/coll-of :into () :count 1
, I get TOO MANY brackets!
(s/def ::iconst (s/cat :head #{'const} :value integer?
:type (s/coll-of ::ttype :into () :count 1)))
(->> ::iconst s/gen gen/generate) ; ~~~> (const 42 ((I 4)))
The only solution Iāve been able to see is a post-processing step, but this is EXTERNAL to the spec and doesnāt scale to my realistic recursively nested examples:
(s/def ::iconst (s/tuple #{'const} integer? ::ttype))
(->> ::iconst
s/gen
gen/generate
(apply list)) ; ~~~> (const 42 (I 4))
Z. Even s/with-gen
takes me back to the future:
(s/def ::iconst-with-gen
(s/with-gen ::iconst
(fn [] (tgen/let [iconst ::iconst]
(apply list iconst)))))
(->> ::iconst-with-gen s/gen gen/generate) ; ~~~> fails after 100 tries
Iām really stuck and would be most grateful for ideas!#2022-10-0219:51phronmophobicI think wrapping the ::type
spec with s/spec
will do what you want:
(s/def ::ttype (s/spec (s/cat :head #{'I}, :bytes #{4})))
(s/def ::iconst (s/cat :head #{'const} :value integer?
:type ::ttype))
(->> ::iconst s/gen gen/generate) ;; (const -241544 (I 4))
#2022-10-0219:54Brian BeckmanSOLVED! thanks!#2022-10-0219:57Brian BeckmanIs there a general rule-of-thumb I can add to my āintuition-bankā about this? I would have thought the s/spec
wrapper to be redundant!#2022-10-0220:15Brian BeckmanMaybe I should just think of it as making a ānestableā from a āspliceableā spec in s/cat
#2022-10-0221:11phronmophobicyea, it's not totally obvious. it is mentioned in the docs, https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/spec
> Can also be passed the result of one of the regex ops -
> cat, alt, *, +, ?, in which case it will return a regex-conforming
> spec, useful when nesting an independent regex.
https://clojure.org/about/spec#_defining_specs
> spec
should only be needed when you want to override a generator or to specify that a nested regex starts anew, vs being included in the same pattern.#2022-10-0222:53Brian BeckmanYouāre very generous! The doc is not emphatic about it, but it is clear. I found this SO that helped me quite a bit, too https://stackoverflow.com/questions/37583708/clojure-spec-alt-vs-or-for-sequence-spec#2022-10-0510:32popeyeI have output response where , response fields are different in different icons, Now I wanted to apply the validation for each input fields, Since this is dynamic in nature, Are we able to create a dynamic spec for validation ?#2022-10-0515:55colinkahnHave you looked at multi-spec
?#2022-10-0610:31pavlosmelissinosI have a (s/def ::items (s/coll-of ::item :min-count 1))
.
How do I create a new spec from it that has some extra constraints, e.g. (s/def ::items-2 (s/coll-of ::item :min-count 1 :gen-max 6))
without repeating the common ones?#2022-10-0612:05Alex Miller (Clojure team)I donāt think you can easily do that#2022-10-0612:12pavlosmelissinosAlright, thanks š
It would have been nice to have but I can work around it, so it's not a big deal.#2022-10-0620:27Cora (she/her)š:skin-tone-2:#2022-10-0620:29Cora (she/her)is there a way to spec maps but without defining entities for the keys?#2022-10-0620:29Alex Miller (Clojure team)map?
is a valid spec :)#2022-10-0620:29Cora (she/her)I mean a spec for a combination of specific keys with specific values#2022-10-0620:30Alex Miller (Clojure team)what kind of keys / what kind of values ?#2022-10-0620:30Alex Miller (Clojure team)you can s/and
any custom predicate you might have#2022-10-0620:31Cora (she/her)say I want a spec for a map of {:key1 "a string" :key2 :a-keyword}
without s/def :key1
and s/def :key2
#2022-10-0620:32Cora (she/her)I can write arbitrary functions for these but it would be nice if there were a way to go just {:key1 string? :key2 keyword?}
#2022-10-0620:33Alex Miller (Clojure team)user=> (s/def ::m (s/keys :req-un [:foo/key1 :foo/key2]))
:user/m
user=> (s/valid? ::m {:key1 "a string" :key2 :a-keyword})
true
#2022-10-0620:33Cora (she/her)that doesn't validate the values, though?#2022-10-0620:34Alex Miller (Clojure team)but no, you can't get the value check without also def'ing the key specs#2022-10-0620:34Cora (she/her)gotcha#2022-10-0620:34Cora (she/her)ok#2022-10-0620:34Cora (she/her)for reference, with malli you can do:
[:map
[:id string?]
[:tags [:set keyword?]]
[:address
[:map
[:street string?]
[:city string?]
[:zip int?]
[:lonlat [:tuple double? double?]]]]]
#2022-10-2715:32souenzzobtw
malli is more structural
https://en.wikipedia.org/wiki/Structural_type_system
spec is more nominal
https://en.wikipedia.org/wiki/Nominal_type_system
spec will force you to give a unique name to each piece of data that you have.
it is nice to have both available in clojure
not a better vs worst. each one has it own advantages and penalties#2022-10-2716:17Cora (she/her)for sure, I just greatly prefer structural and was working on a project that only had spec#2022-10-2717:28souenzzoI just greatly prefer named and I'm working on a project that only had malli
let's switch chairs š#2022-10-0620:35Cora (she/her)I'm on a project using spec, though, and was hoping for something similar. sounds like I need to define the key specs#2022-10-0620:35Cora (she/her)thanks, Alex#2022-10-0710:33reefersleepI mean, you could write your own spec-outputting function that works like you describe š#2022-10-0710:50reefersleepThis is not what you want, the failure message would not be very helpful. But just a back-of-the-napkin sketch to inspire you š
(defn map-spec [validation-map]
(s/def (fn [input]
(and (clojure.set/subset? (set (keys input))
(set (keys validation-map)))
(->> input
(filter (fn [[k v]]
(contains? (set (keys validation-map))
k)))
(map (fn [[k v]]
(let [validation-fn (get validation-map k)]
(validation-fn v))))
(every? identity))))))
#2022-10-1008:57reefersleepActually, it might be a bit difficult to get the usually good feedback messages from the spec validating functions unless you venture into macro territory, so itās probably a bit harder than the initial impression I gave. Still, might be an interesting exercise. But probably, itās a good idea to follow the conventions laid out by spec, as @U064X3EF3 says. Wrapping spec generation in macros can also deter tool usage (such as āgo to spec definition of keywordā in Cursive - I specifically lived through that š )#2022-10-1012:53popeyeI have a input for password , how to do a spec validation for password ?
:password {:type :encrypted}
`#2022-10-1120:31Brian BeckmanDear friends: Seeking explanation.
I noticed that certain with-gen specs describe as unknown
. To wit:
(s/def ::foo (s/with-gen (fn [x] (integer? x))
(fn [] (gen/return 42))))
(s/valid? ::foo 1234)
;; => true
(gen/generate (s/gen ::foo))
;; => 42
(s/describe ::foo)
;; => :clojure.spec.alpha/unknown
But here is one that works:
(s/def ::bar (s/with-gen integer?
(fn [] (gen/return 43))))
(s/valid? ::bar 2345)
;; => true
(gen/generate (s/gen ::bar))
;; => 43
(s/describe ::bar)
;; => integer?
What gives, please & thanks?#2022-10-1120:56Alex Miller (Clojure team)(fn [x] (integer? x))
is an anonymous function, which is just an opaque object so there is no way in Clojure to work backward from that to a form#2022-10-1120:57Alex Miller (Clojure team)with things like integer?
we are working some magic to see that's a function tied to a var and demunging a class name to recover it#2022-10-1120:58Alex Miller (Clojure team)in spec 2, we are able to get rid of all of that and better capture forms by having well separated symbolic and object layers, so eventually this will be improved#2022-10-1121:09Brian BeckmanThat makes sense, Alex. ty. I did notice that sometimes (specifically when there is no with-gen
) describe
can report an anonymous function:
(s/def ::baz (fn [x] (integer? x)))
(s/describe ::baz)
;; => (fn [x] (integer? x))
#2022-10-1121:16Alex Miller (Clojure team)in that case, s/def is a macro and can capture the form#2022-10-1121:17Alex Miller (Clojure team)with-gen
is a function and the args are evaluated before invocation#2022-10-1121:27Alex Miller (Clojure team)there is another option too that is a macro and can specify a custom generator:
user=> (s/def ::foo (s/spec (fn [x] (integer? x)) :gen (fn [] (gen/return 42))))
:user/foo
user=> (s/describe ::foo)
(fn [x] (integer? x))
#2022-10-1121:28Alex Miller (Clojure team)(although this is a different known problem in that it omits the custom gen)#2022-10-1122:02Brian Beckmangreat stuff. tyvm#2022-10-1203:00Brian BeckmanFeature Request: codox-friendly docstrings for specs š Iām up to a few hundred specs in my project now and would love to catalogue them automatically in the same way as I do defs and defns.#2022-10-1204:15Alex Miller (Clojure team)This is the most requested feature in the issue tracker#2022-10-1204:15Alex Miller (Clojure team)In scope for spec 2, but havenāt worked on it yet#2022-10-1212:38Brian BeckmanIn practice, I put docstrings in functions that validate or generate from the specs, so the lack is not a blocker.#2022-10-1419:39Brian BeckmanOBSERVATION: I noticed a handy feature: specs created via s/with-gen
can automatically filter out nil
. Is it generally true that such specs never produce nil
when the spec-part of s/with-gen
forbids nil
? I guess, stated that way, itās obvious that it should be generally true.
This is a nice feature for me because my project generators create trees with level-crossing semantical constraints that can go wrong in combinatorial ways. Any time anything goes wrong (e.g., 0 to a negative power somewhere in the tree), I just barf out nil
as if in a maybe monad (itās about 2% of the time, so itās fine). Itās nice that I get automatic filtering of the generated values at the spec level. Actually, itās brilliant! It let me strip out the maybe monad from my code (simplifying it greatly) and just rely on nil
punning and some->
and some->>
operations.
Consider the following:
(s/def ::nil-producing-spec
(s/or :nil nil? :int integer?))
(def nil-producing-generator
(s/gen ::nil-producing-spec))
(gen/sample nil-producing-generator)
;; => (nil nil -1 -2 nil 6 3 nil nil nil)
(s/def ::nil-rejecting-spec
(s/with-gen
integer?
(fn [] nil-producing-generator)))
(gen/sample (s/gen ::nil-rejecting-spec))
;; => (0 -4 -2 -5 0 -64 0 -4 6 0)
#2022-10-1502:20lassemaattaI think s/gen
checks the generated values and verifies they pass the spec (https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L287). You could swap integer?
with something like pos-int?
and see that it now filters out more than just nil
s.#2022-10-1818:54ghadispec derived generators automatically filter spec derived values by the validity of the spec#2022-10-1818:55ghadithis means that a great technique is to attach a sloppy generator to a specific spec#2022-10-1818:56ghadie.g. give a generator that generates all dates to a spec/predicate for holidays#2022-10-1820:11kpavIs there a way to generate only required or optional keys?#2022-10-1820:19kpave,g, Given this spec:
(s/def ::required-key string?)
(s/def ::optional-key string?)
(s/def ::foo (s/keys :req [::required-key]
:opt [::optional-key]))
I'd like a way to force generation of:
{:required-key "foo"}
And also:
{:required-key "foo"
:optional-key "bar"}
#2022-10-1820:29Alex Miller (Clojure team)the latter should be what you get by default - required and sometimes optional#2022-10-1820:29Alex Miller (Clojure team)you could filter with fmap to get the former#2022-10-1820:30kpavThis is assuming I am given an unknown spec. Basically, I'd like to know which keys are required, and which keys are optional#2022-10-1820:31Alex Miller (Clojure team)there are some 3rd party libraries that have functions to parse that from the spec form#2022-10-1820:32kpavah ok thanks! I'll take a look into those#2022-10-3003:20jmmkis it a bad idea to import/rename a spec. Or is there a better way to accomplish this
I have a few different something/client.clj
namespaces and I need to pass instances of the clients to a handler constructor. since the unqualified name client
conflicts, I create new names for them
e.g.
(ns myapp.api.client
(:require [clojure.spec.alpha :as s]))
(s/def ::client (partial satisfies? ApiClient))
(ns myapp.other.client
(:require [clojure.spec.alpha :as s]))
(s/def ::client (partial satisfies? OtherClient))
(ns myapp.handler
(:require [clojure.spec.alpha :as s]
[myapp.api.client :as ac]
[myapp.other.client :as oc]))
(s/def ::api-client ::ac/client)
(s/def ::other-client ::oc/client)
;; handler constructor accepts config with :api-client and :other-client
(s/def ::config (s/keys :req-un [::api-client ::other-client]))
#2022-10-3003:34seancorfieldOn the face of it, this seems reasonable to me... although that last line is missing ::
yes?#2022-10-3003:35seancorfield(and now your ::config
structure has to have unqualified keys called :api-client
and :other-client
-- is that what you intended?)#2022-10-3004:06jmmk> although that last line is missing ::
yes?
Yes, those keys should have ::
> (and now your ::config
structure has to have unqualified keys called :api-client
and :other-client
-- is that what you intended?)
It is what I intended, but I am not sure if my reasoning is sound. For a bit more context, Iām using integrant, so these client args will be passed from a āsystem mapā
I guess it boils down to wanting to use shorter/easier keys (while still disambiguating the two separate clients) to refer to these, both within the system config and within the handler.#2022-10-3011:37Christian JohansenYou are effectively throwing away the namespace that originally disambiguated them and then giving them new names. Why not just use namespaced keys instead? Less indirection, easier to understand what things are.#2022-10-3013:12jmmkIt feels a bit weird to me that map keys are tied to their values/semantics, but maybe that's the "spec way" and I just need some more time getting used to it?
it also feels verbose in the config and verbose to destructure because they both share the name client
and at some point I have to change the name anyway
;;;; rename
;; config
:myapp/handler
{:other-client #ig/ref :myapp.api/client
:api-client #ig/ref :myapp.other/client}
;; construct
(s/def ::api-client :ac/client)
(s/def ::other-client :oc/client)
(defmethod ig/init-key :myapp/handler
[_ {:keys [api-client other-client]}]
(handler api-client other-client))
vs
;;;; use namespaced keywords throughout
;; config
:myapp/handler
{:myapp.api/client #ig/ref :myapp.api.client
:myapp.other/client #ig/ref :myapp.other/client}
;; construct
(defmethod ig/init-key :myapp/handler
[_ config]
(let [{api-client ::ac/client
other-client ::oc/client} config]
(handler api-client other-client)))
#2022-10-3016:55seancorfieldI wouldn't bother destructuring them for that call -- I'd use (handler (::ac/client config) (::oc/client config))
#2022-10-3017:48jmmkWould it be fair to say then that if I am using a spec and want to pass instances (or things that conform to that spec) in a map that it makes the most sense to use the namespaced keyword for that, regardless of the extra verbosity?#2022-10-3018:36jmmkAnd a followup to that:
If preferring the full keyword is generally a good idea, what then is an appropriate use case for req-un
? Is it for things you donāt want/need to specify as precisely?
beyond the above potential misuse of it, I am using it for things like port
, host
config#2022-10-3019:33seancorfield@U08FV2128 Namespaced keywords are idiomatic and I think in the case you're describing -- passing two different configurations into another function -- it makes sense for the original hash map to use namespaced keywords. If you have keys that "belong to" or are "associated with" some specific concept and/or subsystem within your app, then namespaced keywords mean you can talk about those things everywhere within your app without any conflicts, and also see where the data comes from or is going to.#2022-10-3019:39seancorfieldUnqualified keywords are fine when there is no possibility of a conflict and/or there is no inherent "domain" associated with the key. I'd say, for a web app that starts up a server on a given port, listening to a specific host, then it's fine to use :port
and :host
there.
If you mean port/host for specific services you access from within your app, it's probably better to qualify them unless there's a strong reason not to. For example, interacting with a database via next.jdbc
is going to require unqualified keywords for :port
, :host
, etc so it might make sense to have :db/config
as a key in your overall config whose value is a hash map with :port
, :host
, etc.
Keywords don't need to match namespaces. At work we use qualifiers like wsbilling
and wsmessaging
to identify data belonging to those subsystems as high-level keys (but we also have a mix of full namespace qualified keys and unqualified keys across the apps for various contexts).#2022-10-3019:44jmmk> Keywords don't need to match namespaces. At work we use qualifiers like wsbilling
and wsmessaging
to identify data belonging to those subsystems as high-level keys (but we also have a mix of full namespace qualified keys and unqualified keys across the apps for various contexts).
How do you use those non-namespaced keywords with specs?#2022-10-3019:54seancorfield(s/def :wsbilling/id int?)
and then (s/def :wsbilling/transaction (s/keys :req [:wsbilling/id ..]))
-- is that what you're asking?#2022-10-3020:37Christian JohansenTo use non-namespaced keys with spec you use :req-un
and :opt-un
#2022-10-3022:05jmmk> Keywords donāt need to match namespaces
It didnāt click until I saw your example that specs can use arbitrary namespaces
@U04V70XH6 @U9MKYDN4Q thanks for all the advice.
I am getting back into a clojure project I wrote many years ago as a pile of spaghetti and trying to impose some order on it and use spec instead of my home-grown data validation/coercion#2022-11-0421:13jmmkI have read various comments/advice on where spec fits into the workflows of producing/consuming data, but Iām still not exactly clear on when and where āparsingā / ācoercionā should happen vs āvalidationā / āconformingā, and specās ability to parse or ādestructureā things (like a binary format for example) seems to blur the lines a little bit.
Iām also confused where tools like https://github.com/exoscale/coax come in
Example:
I request/receive some data payload and convert it into an EDN structure more or less matching the shape of the raw payload (e.g. json deserialization)
I only care about a subset of the fields in the payload, and I need to:
ā¢ ensure the right fields are present
ā¢ pull them out of the source data, parse them, and put them into my applicationās representation
Which parts is spec appropriate for? Would it make sense to use multiple specs to check both the raw data and my application representation?
Specifically regarding parsing, should a spec pull apart
ā¢ string value containing delimited values like v1/v2/v3
ā¢ date string mm/dd/yyyy
or should those use normal string/date handling functions to parse them, and leave the spec as a simple regex to ensure the format looks correct (I suspect the answer is āit dependsā, but Iām not sure where to draw the line)#2022-11-0501:21jmmkI think I have misunderstood the sequence specs and thought I could apply them to strings.
However, the v1/v2/v3
really is a sequence but in string format, which again brings up the question of when the parsing should happen.#2022-11-0502:09jmmkIt feels like the spec validation is not very useful before parsing because parsing implicitly includes validation
Iām leaning towards:
ā¢ parse input data into internal structure, parsing/converting fields as needed
ā¢ write spec against the internal structure. validate transformed data before passing it among internal functions
I still donāt see the use case for the coercion libs#2022-11-0502:39jmmkAfter reading https://groups.google.com/g/clojure/c/Tdb3ksDeVnU/m/ryc28SteAgAJ and I consider how the fully qualified keywords work in spec, itās a bit clearer, but I still donāt have a fully-formed idea of how to move between two data types with the help of spec/coax#2022-11-0515:50mpenetCoax is mostly used to convert from json to edn for us, either from http requests or inside db fields. In our case we cannot serialize as edn (or transit/fressian&co) because other languages have to be able to use this data as well. It does no validation tho, it will coerce if it can/has-to otherwise it leaves values as is.#2022-11-0515:50mpenetIt never calls valid? or conform (there is a helper fn that combines both)#2022-11-0515:51mpenetIirc it uses almost nothing from spec internally, and thatās intentional. Itās strictly a transformation pass infered from a spec form/coax-registry#2022-11-0515:52mpenetUsually if you want/need validation that comes after#2022-11-0515:59mpenetA common (imho) mistake I often see in the wild is trying to tie your specs with coercion, using s/and and conformers. That turns things into franken-specs quickly, makes conforming all kind of broken when you need it and basically close the doors to have multiple coercion strategies for a spec depending on usage context#2022-11-0516:00mpenetThe way spec is designed, doing coercion has to be a separate pass imo#2022-11-0516:02mpenetSome other validation engines/libs have different strategies. But that has to be thought beforehand, allowing to compose operations when values are āvisitedā.#2022-11-0516:07mpenetPersonally I quite like the spec+coax combo, I might be biased (I wrote coax), but we use it heavily at work and itās been quite frictionless so far#2022-11-0516:50jmmk@U050SC7SV so a coax pass will run and transform some fields to match the desired spec before the data is ever used?#2022-11-0516:52mpenetYes#2022-11-0516:53mpenetItās a necessary evil in our case#2022-11-0516:54mpenetIf you can use transit/edn etc itās better#2022-11-0517:13jmmkIām dealing with multiple third-party APIs which makes this more interesting because the data format and structure is not easily usable without large transformations
I currently do mostly āmanualā parsing, but Im trying to simplify it and add spec into the equation where appropriate#2022-11-0519:54jmmk@U050SC7SV do you use āreverseā coercions to go the opposite direction, i.e. get data back into in the format necessary to interact with the other system
I guess itās really just another spec with a different qualifier e.g. :application/field
might go from string to int and :other-system/field
might go from int to string#2022-11-0519:55mpenetYes in one particular case. Itās one of the good things with coax. Coercion rules are first class#2022-11-0519:56mpenetYou can have as many as you want per spec. Itās an argument to coerce calls basically#2022-11-0519:57mpenetSo same spec. Different coercers#2022-11-0612:09walterl@U08FV2128 I have the same question, so rather than providing an outright answer, I'll report on what I'm currently trying. I validate user input from the HTTP API with spec A, coerce it to my app-internal representation manually (will have a look at coax), and validate the app-internal data in the rest of my app with spec B.
So, e.g., JSON input field :quantity
is validated with (s/def :order.input/quantity (s/and string? #(re-matches #"^\d+$" %)))
, and coerced into field :order/quantity
, validated with (s/def :order/quantity nat-int?)
.
This models the two "sides" of the HTTP API interface with separate specs:
User ā[spec A]ā API ā[spec B]ā Rest of the system
I have no idea if this is The Right Way, but it works well, while being fairly verbose, even a little tedious.#2022-11-0612:30jmmk@UJY23QLS1 that sounds similar to what I have come up with so far.#2022-11-0611:35eighttrigramsHey,
I've got a data structure (which I cannot change) like
[{:identifier "abc" :other-key-1 "abc"}
{:identifier "SELF" :content {}}
{:identifier "def" :other-key-2 []}]
I would like to do the following:
1. Determine if there is exactly one item in the vector of maps which has the :identifier "SELF"
.
2. For that item, determine if there exists the key :content
3. Make sure its value is a map (whose properties I also would like to spec)
Can anyone think of a good way of doing this in spec or give me pointers of how to think about this?#2022-11-0612:34walterl(require '[clojure.spec.alpha :as s])
(def d [{:identifier "abc" :other-key-1 "abc"}
{:identifier "SELF" :content {}}
{:identifier "def" :other-key-2 []}])
(defn- self-id? [{:keys [identifier]}] (= "SELF" identifier))
(s/def :self/identifier #{"SELF"})
(s/def :self/content (s/keys)) ; TBD
(s/def ::self (s/keys :req-un [:self/identifier :self/content]))
(s/def ::not-self (s/or :!m (complement map?) :!self (complement self-id?)))
(s/explain-data ::self (first (filter self-id? d)))
(s/def ::d (s/and (s/coll-of map?)
(s/cat :before (s/* ::not-self)
:self ::self
:after (s/* ::not-self))))
(s/explain-data ::d d)
(s/conform ::d d)
; {:before [[:!self {:identifier "abc", :other-key-1 "abc"}]],
; :self {:identifier "SELF", :content {}},
; :after [[:!self {:identifier "def", :other-key-2 []}]]}
#2022-11-0614:42eighttrigramsThis is fantastic! Thank you so much, @UJY23QLS1#2022-11-0614:44walterl> "...pointers of how to think about this"
Thing of it like regex: you want to match 0 or more non-self maps, followed by a self map, followed by 0 or more non-self maps.
(An s/not
spec would've been handy here.)#2022-11-0614:47eighttrigramsYeah, like a regex. It totally makes sense. The docs even stated this, but just didn't come up with this in the moment.#2022-11-0614:47eighttrigrams> An s/not
spec would've been handy here.
:thinking_face: Interesting. I see, for the :not-self
...#2022-11-0614:51walterlYeah, I would've liked to do (s/not ::self)
rather than having to define a negation of ::self
manually, because now they could drift out of sync.#2022-11-0614:54eighttrigramsYes. I have an intuition that it should generally be possible to make such a thing. One would need to account for the different, uhm, don't know the name, the different things like s/keys
, s/cat
and so on. The negation of the s/keys
was an s/or
with complement
. Not in the mood to solve that puzzle right now, but certainly an interesting challenge š#2022-11-0912:34prncWould anyone be able to share a way / āworkaroundā for of adding docs to specs?
Has anyone gone beyond a map keyword -> docstring? ā which is what Iām about to do, so asking here for some lessons learnt :)
(Iām assuming this must have been done / attempted a number of times (since it is the most requested feature for spec2 IIUC but I have not run into any specific examples, caveats or advice))#2022-11-0912:48mpenetessentially: bake your own metadata registry#2022-11-0912:48mpenetthere's nothing built-in#2022-11-0914:25Alex Miller (Clojure team)this is one of the most highly voted requests we have and it's still on our list for spec 2#2022-11-1019:11Ishaan Guptawhat's up with this spec-alpha2 behaviour?#2022-11-1019:27Alex Miller (Clojure team)seems like it's whether it evals to a var or not so probably a bug#2022-11-1019:27Alex Miller (Clojure team)if you can drop it on http://ask.clojure.org I'll make a ticket and take a look at some point#2022-11-1113:43jmmkIs there a better way to accomplish this? I find in each place I am using conform
, I want it to fail (throw) if it is invalid
(defn conform! [spec v]
(let [conformed-value (s/conform spec v)]
(if (s/invalid? conformed-value)
(throw
(ex-info
"value did not conform to spec"
(s/explain-data spec v)))
conformed-value)))
#2022-11-1114:00Alex Miller (Clojure team)Looks good to me#2022-11-2408:59ChrisIs anyone aware of any projects that convert JSON-schema to specs? An internet search only turns up projects based on Java libs, but it would be preferable to have a spec created. (Most preferable would be to create the schema from a spec, but probably a much harder problem given the relative expressiveness!)#2022-11-2409:35tatutI have some old code (not up to current json schema) https://github.com/tatut/json-schema you can use that as a predicate#2022-11-2409:35tatutbut it doesnāt create clojure specs EDIT: nevermind, you wanted schema from specs#2022-11-2410:19ChrisI wanted either š
This is neat, thanks for sharing. Might be a good base for a ->spec version if I canāt find an existing implementation. #2022-11-2417:03jmayaalvthere is also spec-tools
https://github.com/metosin/spec-tools/blob/master/docs/04_json_schema.md#2022-11-2417:28Chris Exactly what I wanted, thank you!#2022-11-2916:51Ben LiebermanWhat is the most idiomatic way to spec "if this key exists in a map, that key should not exist?" I'm finding this a bit more confusing than I initially expected. A short example:
;; map with either short or long name but not both
(def a-map {:short-name "foo"}) ; should be valid
(def a-bad-map {:short-name "foo" :long-name "foobar"}) ; should be invalid
I've tried various combinations of s/and
, not
and other core fn's without luck.#2022-11-2916:59Ben Liebermanmy latest try, for example, blows up the stack:
(s/def ::short-route-name (s/and string? #(s/invalid? (s/conform ::long-route-name %))))
(s/def ::long-route-name (s/and string? #(s/invalid? (s/conform ::short-route-name %))))
I also tried using or
but that fails too.#2022-11-2917:05pavlosmelissinosspec favors open maps by design; it doesn't really support statements like "this key shouldn't exist"#2022-11-2917:06jkxyzYou can use a predicate like #(not (and (:foo %) (:bar %)))
#2022-11-2917:07Ben LiebermanI saw that Malli has both an enum spec and an option to close maps but I want to learn spec before I give Malli a try, even though both those features would be useful to me here#2022-11-2917:11pavlosmelissinosThere are also third party libraries, like https://github.com/bhauman/spell-spec that "close" specs if you really want that functionality.#2022-11-2917:12pavlosmelissinosI wouldn't try to model "negative" specs from within spec, it's not designed for that.#2022-11-2917:13Ben LiebermanWell I guess I feel slightly less silly then for finding this pretty non-obvious#2022-11-2917:14pavlosmelissinosAn interesting approach is that followed by cognitect's aws-api; instead of saying "this is what a valid response doesn't look like", it specs anomalies: https://github.com/cognitect-labs/anomalies#2022-11-2917:18Ben LiebermanI'll have to check that out, thanks @UEQPKG7HQ#2022-12-2113:53vemvhttps://clojure.org/guides/spec#_entity_maps says:
> Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys.
I'm curious, does this behavior also exist in spec2?
(I never invested time in spec2 at all as it's still wip)#2022-12-2113:55vemvThe broader question that prompted me to ask this is: is such behavior a universally good idea?
I have no opinion from my side. I don't recall it ever bothering me. Perhaps there's the performance/security consideration of checking unforeseen keys?
I wouldn't mind about that either given that I can remove extraneous keys at the edges (with some or other approach - there are a few)#2022-12-2113:58Alex Miller (Clojure team)yes, it exists, although s/keys is really deprecated in favor of schema/select in spec 2#2022-12-2113:59Alex Miller (Clojure team)but there is also the notion of closed spec checking which limits things somewhat (and plumbing to add other such options during checking)#2022-12-2114:02vemv> yes, it exists, although s/keys is really deprecated in favor of schema/select in spec 2
So it also exists when using select
in a vanilla fashion, right?#2022-12-2115:20Alex Miller (Clojure team)I think so? can't remember#2022-12-2316:07valerauko(s/coll-of ::thing
:kind vector?
:min-count 1
:distinct true)
why does this cause a boxed math warning? (unchecked inc)#2022-12-2413:05Ben SlessListening to Alex's answers on spec2 on the Clojure Stream on modeling relationships between functions' inputs and outputs, have you considered using a relational language? E.g. datalog?#2023-01-1015:45Ben LiebermanIn the (presumably unlikely) case where test/check
does not find any failures in a function I've spec'ed, what is the expected output? e.g., I have a function foo
and I check it like so
(->> (test/check 'foo) test/summarize-results)
In my case I'm just getting back a map of {:total 0}
which seems suspect to me given the docs for these functions.#2023-01-1015:48Alex Miller (Clojure team)maybe `foo instead of 'foo ?#2023-01-1015:49Alex Miller (Clojure team)(that is, a resolved symbol name)#2023-01-1015:50Alex Miller (Clojure team)I suspect it's not finding foo and thus not testing anything, see the guide too https://clojure.org/guides/spec#_testing#2023-01-1015:50Ben Liebermanoh, right. I forgot about that syntax and thought the ` was just an italicized '#2023-01-1015:51Ben LiebermanThanks!#2023-01-2019:18Colin P. HillDoes anyone have any usage examples where s/and
's chained conforming behavior was useful? I find I almost always have to work around it with a lambda or s/nonconforming
because the functions and specs I'm composing the s/and
from don't expect conformed input, and that makes me wonder if I'm missing something about the intended usage patterns.#2023-01-2019:34colinkahnConceptually the same, clojure.spec.alpha itself uses that for keys*
using &
- https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1810#2023-01-2020:00Alex Miller (Clojure team)I find it is often useful when chaining predicates. the main thing that makes it not useful most often are when you put something in it that gives you a conformed value that differs from the original (particularly s/or). I'd be curious what the most common cases are for you when it's an issue. in spec 2, there is a non-flowing s/and-
(name subject to change)#2023-01-2021:03Colin P. HillI'll dig around in my code in a minute to see if I can find a succinct and reasonable example ā my current project isn't so much a useful product as it is a playground to get a feel for what can be done with specs beyond simple validation, so most of the stuff in my immediate reach is kinda ramshackle and/or at the edges of what is intended. But yeah, s/or
is the main culprit. Non-transforming conformations aren't an issue ā if every spec in the chain simply returned its argument, s/and
would be indistinguishable from its non-flowing variant.#2023-01-2021:07Colin P. HillBut also, non-transforming conformations are less interesting. Conformation is one of the main things I'm trying to get a feel for right now. I found the Jira tickets where you discouraged one of the spec-tools maintainers from viewing specs as a coercion engine, so atm I'm trying to explore what kinds of transformations are supported, and how they can be used.#2023-01-2021:12Alex Miller (Clojure team)I think itās likely that we will add a nonconforming or option - thatās the most common need. Mostly wondering if thereās anything else#2023-01-2021:16colinkahn> Conformation is one of the main things I'm trying to get a feel for right now.
From various talks I've interpreted conformation being suggested in two places:
ā¢ To make use of existing specs (like the keys*
example conforms the tuples into a map for the existing keys
spec).
ā¢ In macros, where you're parsing at compile time for syntax reasons.#2023-01-2021:17colinkahn(I think those were probably talks by @U064X3EF3 so I'm sure he could say if those are accurate or not š)#2023-01-2021:20Colin P. Hill> Mostly wondering if thereās anything else
Probably not. Maybe alt
, being the regex equivalent of the same. The other transformers don't really make things more deeply nested than they were before, so they're not usually an inconvenience. The case that prompted the question is a little unusual ā I have a multi-spec in an and
, so while there's a contract on what that multi-spec is supposed to represent, I can't necessarily be sure how it will conform (and since conformation output can be non-obvious I don't necessarily want to include that in the contract and count on it being obeyed). Easy solution in nonconforming
, but it got me thinking that I may not rightly understand why and
is designed the way it is and might not be using the tool in the best way.#2023-01-2021:26Alex Miller (Clojure team)the purpose of conforming is to tell you why and how a value matched a spec#2023-01-2021:26Colin P. Hill> In macros, where you're parsing at compile time for syntax reasons.
Parsing is kinda what I've been fixing on for my current understanding (not just macros, could also be for data-oriented APIs). It seemed like standard conformations all aimed to describe the implicit structure of their input while generally leaving the scalars at the leaves of the tree alone ā exactly what a parser does with a stream of tokens, just here with potentially more initial structure than just a stream.#2023-01-2021:27Alex Miller (Clojure team)the "why" part is why s/or and s/alt tell you which path was taken, why s/cat separates it's concatenated parts, etc#2023-01-2021:28Alex Miller (Clojure team)s/and's flowing is extremely useful when chaining predicates (where typically the value is not not modified, just passed through) ala (s/and int? pos? even?)
#2023-01-2021:31Colin P. HillHow does that differ from how the non-flowing variant will behave? I thought conforming with a predicate function was essentially an identity operation, so here even?
receives the same value that pos?
did#2023-01-2021:32Alex Miller (Clojure team)https://insideclojure.org/2019/12/06/journal/#2023-01-2021:33Alex Miller (Clojure team)the original value (not the conformed value) is passed through s/and-#2023-01-2021:35Alex Miller (Clojure team)for specs that are pred fns, doesn't affect conform (but does affect gen)#2023-01-2021:35Colin P. HillOh interesting. I'm not immediately seeing how it affects gen, could you say more on that?#2023-01-2021:36Alex Miller (Clojure team)it's explained in that blog#2023-01-2021:36Colin P. HillAh sorry, skimmed poorly#2023-01-2021:36Alex Miller (Clojure team)well s/and- is - it only gens from the first predicate. s/and gens, then filters#2023-01-2021:37Alex Miller (Clojure team)I expect you will often need a custom generator with s/and-#2023-01-2021:41Colin P. HillDoes this mean that and-
will be an exception to the general rule that a spec's generator retries generation until it gets something that conforms to the (whole) spec? Or does and
do some more proactive filtering ahead of that step with its 2nd through last terms?#2023-01-2021:51Alex Miller (Clojure team)I don't think it's an exception - they will all try 100 times and all spec generators check s/valid? on the generated values#2023-01-2021:51Alex Miller (Clojure team)s/and-, by not using anything but the original pred, is less likely to generate valid values though#2023-01-2021:53Colin P. HillGot it, that makes sense. I wasn't aware that the original flavor built its underlying pre-verification generator out of more than one of its terms#2023-01-2021:54Colin P. HillI'll have to read the implementation at some point, because it's not immediately obvious to me how that could even be done :thinking_face:#2023-01-2021:54Alex Miller (Clojure team)generate values from the first pred. only keep ones that satisfy 2nd pred. only keep those that satisfy 3rd pred, etc.#2023-01-2021:55Colin P. HillJust a chain of such-thats so the generation tooling handles filtering before the spec does its final check?#2023-01-2021:55Alex Miller (Clojure team)yes, so it helps to think about the ordering of preds in s/and#2023-01-2021:56Alex Miller (Clojure team)this is covered in https://clojure.org/guides/spec#_using_sand_generators#2023-01-2021:56Colin P. Hillack, my poor skimming embarrasses me again#2023-01-2021:56Alex Miller (Clojure team)I forgive you, there is a lot to read :)#2023-01-2511:06BenjaminHi, I would like to extract a spec from a function spec (`s/fdef`). I realized since the spec is speccing an arglist this might be a bit challenging. Right now I wanted to extract the 2nd arg spec always.
Context: https://github.com/WorksHub/leona/issues/4#2023-01-2511:10Benjamin(defn droid-resolver [ctx query value])
(s/def ::int int?)
(s/def ::string string?)
(s/def ::any any?)
(s/fdef droid-resolver
:args (s/cat :a ::int :b ::string :c ::any)
:ret ::int)
(let [[_ _ _ _ b-spec]
(s/form (:args (s/get-spec #'droid-resolver)))]
b-spec)
:leona.core-test/string
#2023-01-2511:11Benjaminthis would not fly would it? It would fail if someone defines a spec differently than cat
?#2023-01-2513:13Alex Miller (Clojure team)Yes, it would fail if that spec changed#2023-01-2513:14Alex Miller (Clojure team)Thereās not really a way to get the spec of a specific arg like that#2023-01-2513:19BenjaminOk thanks!#2023-02-0117:59Ben Liebermanquestion about resolving specs: is there a way to require a spec out of the global registry as if it were a regular var in a ns? I have discovered that I can't do something like (require [foospec.bar :refer [baz]])
presumably because baz
is not interned in the foospec.bar
ns but is instead in the registry? Is that correct?#2023-02-0118:13Alex Miller (Clojure team)correct, specs are not vars#2023-02-0118:14Alex Miller (Clojure team)you could put a spec in a var with (def my-spec (s/get-spec ::foo))
if you wanted to for some reason#2023-02-0118:15Ben Liebermanno, not necessary, I'm fine with it being in the registry. So if require the whole foospec.bar
ns say as fspec
and then resolve ::fspec/baz
its just looking in the registry for that fully qualified name?#2023-02-0118:16Ben Liebermanthat's how I have been doing it at any rate#2023-02-0118:16Alex Miller (Clojure team)not sure I understand "require the whole foospec.bar
ns say as fspec
" but fspecs are also just specs in the registry (but their keys are fully-qualified symbols, not keywords)#2023-02-0118:18Alex Miller (Clojure team)s/fdef
just creates a spec with s/fspec
, then adds it to the registry (which is just a map) with the fq symbol as the key#2023-02-0118:18Ben Liebermansorry, that was a poor choice of alias on my part#2023-02-0118:18Ben LiebermanI am not talking about function specs, just any spec in general#2023-02-0118:19Alex Miller (Clojure team)then yes, that is how it works#2023-02-0120:16seancorfield@U03QBKTVA0N Don't forget there's :as-alias
now in a require so you can create an alias for resolving Spec names without having to actually load a real namespace.#2023-02-0120:18seancorfield(ns using.specs
(:require [i.do-not.exist :as-alias idne]))
::idne/bar ;=> i.do-not.exist/bar
and no i/do_not/exist.clj
file is needed for this.#2023-02-0123:35Ben Liebermanoh thanks @U04V70XH6 I did not know about as-alias
. That's helpful.#2023-02-0709:13djtangohello I seem to remember that there was a spec for Clojure (ie someone wrote spec to describe the clojure syntax). Did I imagine this? I'm struggling to google for it#2023-02-0709:15dgb23you might be thinking of this:
https://github.com/clojure/core.specs.alpha#2023-02-0709:16djtangothank you - this is the one š#2023-02-1518:58stopaHey team, noob question. Consider the following object:
(s/def ::type #{:foo :bar})
(s/def ::obj (s/keys :req [::type ::a ::b] :opt [::c]))
How would I express something like: If ::type
is :foo, ::c
is required, but otherwise it's not?#2023-02-1519:00Alex Miller (Clojure team)you can s/and
a function that tests any property you like#2023-02-1519:00Alex Miller (Clojure team)but you might also want to look at s/multi-spec
https://clojure.org/guides/spec#_multi_spec#2023-02-1519:01stopaNice! Thanks @U064X3EF3!#2023-02-1603:18stopaHey @U064X3EF3 one more question:
(s/def ::type #{:biz :baz})
(s/def ::foo string?)
(s/def ::bar string?)
(s/def ::kux string?)
(s/def ::common (s/keys :req [::type ::foo ::bar]))
(s/def ::biz-obj ::common)
(s/def ::baz-obj
(s/merge ::common
(s/keys :req [::kux])))
(defmulti type-mm ::type)
(defmethod type-mm :biz [_] ::biz-obj)
(defmethod type-mm :baz [_] ::baz-obj)
(s/def ::obj (s/multi-spec type-mm ::type))
(gen/generate (s/gen ::obj)))
Here I make it so if the type is :baz, then ::kux is required.
But one thing I did want to do:
Sometimes I want to generate the different instances of the objects -- i.e ::biz-obj and ::baz-obj
But the way I wrote it, I would get an invariant. If I wrote:
(gen/generate (s/gen ::biz-obj)))
I would sometimes get values with ::type :baz
This is because I don't constraint the type in this layer.
I guess I have two questions: Is this the right approach? And if so, how would I go about properly constructing ::biz-obj and ::baz-obj?#2023-02-1617:29Alex Miller (Clojure team)well you could constrain the type in those layers#2023-02-1617:29Alex Miller (Clojure team)or make a generator that inserted the expected type#2023-02-1617:29Alex Miller (Clojure team)using gen/fmap#2023-02-1815:11stopaOne question for my learning: How would I constrain the type in those layers?
(s/def ::biz-obj (s/keys :req [::type ::foo ::bar]))
(s/def ::baz-obj
(s/merge ::biz-obj
(s/keys :req [::kux])))
Since these are both s/keys
, and they both must take ::type
as one of the validations, how would I say "for this map, ::type
is just this part"?
Or did you have something else in mind?#2023-02-2117:16stopaA noob naming question, that I guess doesn't really have an answer, but I thought I'd try anyways.
Consider:
(s/def ::triple (s/cat :e string? :a string? :v string?))
Now I may call this vec a triple:
["e", "a", "v"]
But what should I call the (s/conform ::triple)
version of it?
{:e "e", :a "a", :v "v"}
I am thinking parsed-triple
or conformed-triple
, but I wonder if there's a current idiom. If I have some functions that work on one type vs another, I worry about wonky names.#2023-02-2117:33delaguardonamed-triple#2023-02-2117:33delaguardoor just record#2023-02-2117:33nbardiukhttps://clojure.org/guides/spec#_collections mentions Conforms to map with named keys based on the cat tags
. I would add more naming options like tagged-triple
or named-triple
. Python has named tuple
which looks similar#2023-02-2117:34stopaOo nice idea! Thanks team#2023-02-2117:36Ben SlessNamed tuple#2023-02-2208:42serioganamed tuple
may be confusing as far as element order is not garanteed#2023-02-2209:15delaguardoWhy should order matter after conforming with the spec?#2023-02-2209:19dgb23If you're already using tripple as a name, then name it tagged or named tripple as well instead of tuple IMO#2023-02-2209:19dgb23tripple is more precise than tuple#2023-02-2209:20seriogaThe word tuple
means āfinite ordered listā. https://en.wikipedia.org/wiki/Tuple
It can be vague that named
predicate drops the āorderedā characteristic. :-)#2023-02-2209:21dgb23From the same article:
> Many programming languages offer an alternative to tuples, known as https://en.wikipedia.org/wiki/Record_(computer_science), featuring unordered elements accessed by label.#2023-02-2209:23seriogabtw python's named tuples are ordered and support access by index.#2023-02-2209:23dgb23record tripple or just record seem to be good names š#2023-02-2209:25delaguardoIt is even matching with clojure.core/record š#2023-02-2414:30pavlosmelissinosGiven an incomplete map and its spec, can I get spec to give me back the list of missing keys (required and optional)? With explain-data I can get the required keys but what about optional ones?
Context: I'm playing with the idea of (dynamic) spec-driven config presets, i.e. there's a process that creates named incomplete maps and then another process/user selects a preset and fills-in the gaps.#2023-02-2818:12stopaHey team! I am making a graphql-like query language, that looks something like this:
{"users" {"$" {"where" {"bookshelves.books.title" "The Count of Monte Cristo"}}
"bookshelves" {}}
"books" {}}
Here, I have forms
, which have an option-map
, and further children.
What I would want here is a combination of keys
(to say what $
should be), and map-of
, to describe that it can have children.
I wasn't sure how to do this, so the best approach I could think of, was to massage the data into a kind of vector, like:
[["users", {"where" {"bookshelves.books.title" "The Count of Monte Cristo"}}, ["bookshelves", {}, []]],
["books", {}, []]]
So then I could type this vec, like so:
(s/def ::where (s/map-of string? ::triple-model/value))
(s/def ::option-map (s/keys :opt-un [::where]))
(s/def ::form (s/cat
:k string?
:option-map ::option-map
:children (s/coll-of ::inner-form)))
(s/def ::forms (s/coll-of ::form))
Is this the idiomatic way to do things with spec, or would you do it differently?#2023-03-0819:31phronmophobic> I wasn't sure how to do this, so the best approach I could think of, was to massage the data into a kind of vector, like:
I'm not totally sure I understand what you're trying to do, but massaging data before validation sounds like an anti-pattern.#2023-03-0819:49phronmophobicmaybe something like:
(s/def ::options (s/keys))
(s/def ::form
(s/coll-of
(s/or
:options (s/tuple #{"$"}
::options)
:sub-form (s/tuple string? ::form))
:into {}))
#2023-04-0715:01stopa(realized I never responded -- I think this could work! Thank you @U7RJTCH6J)#2023-03-1718:22Joerg SchmueckerHi, Is there a project that shows best practices for spec? Not a sample but some lib that is actually used. Thanks#2023-03-1719:11seancorfieldSpec can be used in a lot of different ways so I wouldn't expect any single project to be a good exemplar of it... Is there some specific aspect of Spec that you're seeking guidance on?#2023-03-1720:38Joerg SchmueckerJust basics, how do I spec parameters on a function best. What are typical project setups, i.e. are the specs in the code or the tests. Do you instrument specs in prodcution or not (or when)? Just all the things that make the idea workable.#2023-03-1720:39Joerg SchmueckerI think any well āspec-dā code base woudl be a good starting point. I just donāt know how to google for one.#2023-03-1720:54phronmophobicThose are all good questions. https://grep.app is pretty good for finding code in the wild, https://grep.app/search?q=s/def.#2023-03-1720:55Joerg SchmueckerThank you,. I will give that a try.#2023-03-1720:55seancorfieldhttps://corfield.org/blog/2019/09/13/using-spec/ talks about the various ways we use Spec at work. Happy to answer any follow-up Qs.#2023-03-1720:57Joerg SchmueckerSean, I did read your blog. Great content. For me it isnāt obvious how to get from the concept to the practical implementation.#2023-03-1720:58seancorfieldI suspect there's a lot more use of Spec in large application codebases and those are nearly all closed source.#2023-03-1720:59Joerg SchmueckerI would love to see the Nubank code base š#2023-03-1721:00Joerg SchmueckerI work with a lot of FSI customer and I am trying to figure out if we can simplify some of those problems.#2023-03-1722:49dgb23> Do you instrument specs in prodcution or not (or when)?
I personally wouldn't use instrumentation outside of development.
If you use spec for API/request validation, or generally stuff that comes from the outside of your program, then you want to be somewhat rigorous with it and nail it down to sufficient detail.
Otherwise I would say just use it when it helps, incrementally. For example it can be useful as a thinking tool, to design the shape of your data. Or you can use conform to parse things into a richer structure.
But it's really just that, a tool.
I played around with it a lot and looked at how people use it in OSS projects. It's very nice and composable, and you can do a lot with it. I think playing around with it is a good way to get familiar with what you can do.
Then forget about it, just write code and it will introduce itself naturally.#2023-03-1722:55dgb23Here's are more involved specs (tools.deps):
https://github.com/clojure/tools.deps/blob/master/src/main/clojure/clojure/tools/deps/specs.clj
Specs can also be small and informal, basically just a set of names
https://github.com/cognitect-labs/anomalies/blob/master/src/cognitect/anomalies.cljc#2023-03-1722:58seancorfieldAnother example from a library: https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/specs.clj -- all about instrumenting library functions during development#2023-03-1723:16Joerg SchmueckerFantastic just what I needed. Thank you so much.#2023-03-1723:26dgb23> Generative scenario testing
This is a very interesting section. I've been thinking about something similar in the back of my head for a while. I named them workflows, but I like "scenario" very much as a term here!
> Deriving code from specs
I've been struggling with something similar for quite a while and was never quite happy with my results. Have you been using form
and conform
to parse the specs themselves?#2023-03-1723:30seancorfieldI'm not in front of my work codebase -- I'm taking every Friday off this month -- but I'm pretty sure we use form
and walk the result to find the s/keys
definition and "parse" that to derive everything we need. If you remind me on Monday, I can provide more detail.#2023-03-1723:38dgb23Ah right, its a specific subset of possible specs that make sense in that context#2023-03-3014:03practicalli-johnhttps://practical.li/clojure/clojure-spec/ covers how I use clojure.spec along with practical examples and coding videos
I've also recently been using spec with Reitit to validate API requests and responses (which I will be writing up this month)#2023-03-3105:46Hugh PowellHi folks, has anyone managed to generate Swagger 2 output for specs that places definitions in the :definition
top level key, rather than in-lining them in the :parameters
and :response
keys inside :paths
. It looks like https://github.com/metosin/ring-swagger#more-complete-example, but https://github.com/metosin/spec-tools/blob/master/docs/05_swagger.md#full-example https://github.com/metosin/spec-tools/blob/master/src/spec_tools/swagger/spec.cljc#L81.#2023-04-1216:17ikitommiInterested in PR to spec-tools to add support?#2023-04-1309:02Hugh PowellHere's my https://github.com/metosin/spec-tools/pull/274. Let me know what you think.#2023-04-2320:50karol.adamiecHi,
I get the mysterious "Additional data must be non-nil."
Below is a sample. What do i do wrong?
(defn mean
[numbers]
(when-let [num-count (seq numbers)]
(/ (reduce + 0 num-count) (count num-count))))
(defn within-range?
[avg coll]
(if (empty? coll)
true
(let [min-val (apply min coll)
max-val (apply max coll)]
(<= min-val avg max-val))))
(defn avg-unchanged-by-order?
[avg coll]
(let [shuffled-coll (shuffle coll)
shuffled-avg (mean shuffled-coll)]
(= avg shuffled-avg)))
(s/fdef mean
:args (s/cat :numbers (s/nilable (s/coll-of (s/double-in :NaN? false :infinite? false))))
:ret (s/nilable number?)
:fn (fn [{:keys [args ret]}]
(if-let [coll (:numbers args)]
(and (within-range? ret coll)
(avg-unchanged-by-order? ret coll))
true)))
(stest/check `mean)
#2023-04-2321:04phronmophobicThe docs suggest using stest/abbrev-result
to get more info about what happened when it failed, https://clojure.org/guides/spec#_testing. Have you tried looking at that info?#2023-04-2321:13karol.adamiecif i do as per docs (stest/abbrev-result (first (stest/check
mean`) i do not see any more relevant information. There is the sample case that "failed", but it is correct input and if i give it to mean
or its validators/helpers everything seems fine :/#2023-04-2321:14phronmophobicWhat is the failing sample?#2023-04-2321:23karol.adamiecit is a bit random. sometimes it says spec failed and collection is [ 1.0000004768371582 1.000000238418579 2.147483648E9 ]
and calculated mean is 7.158278833333336E8
other times there is nothing inside, other than the message in exception "additional data must not be null".#2023-04-2321:24karol.adamiecstacks point at explainInfo
--- Contents:
0. clojure.spec.test.alpha$explain_check
1. invokeStatic
2. "alpha.clj"
3. 390#2023-04-2321:26karol.adamiechttps://clojurians-log.clojureverse.org/clojure-spec/2019-08-02#2023-04-2321:27karol.adamiecfound some old @U064X3EF3 explanations about similiar thing... but unsure if that still applies...#2023-04-2321:30phronmophobicone thing that is suspicious is that mean
is called in the validation function. I would try to narrow down where the problem might be. Examples:
ā¢ run stest/check
with just the :args
specification
ā¢ run with just the :ret
ā¢ run with just :fn
ā¢ run with just :fn
and only check within-range?
ā¢ etc #2023-04-2321:39karol.adamiecyeah, that mean call maybe sth... i changed the avg-unchanged-by-order?
to be a dummy that always returns true and now 1000 tests pass...#2023-04-2321:40karol.adamiecthanks š#2023-04-2321:41phronmophobicI think thereās a way to write those sorts of checks, but Iām not sure how to do it off the top of my head.#2023-04-2321:43phronmophobicTheyāre definitely supported by test.check #2023-04-2321:43karol.adamieci would think calling function under test is ok... it is not like recursion...#2023-04-2321:46karol.adamiecyeah, all pass if i return true, but still do all the calcs and logic. so just calling mean
seems to not be the issue#2023-04-2323:01phronmophobicAnother thing that just popped into my head is that your test expects the order of the numbers not to make a difference in the calculated mean, but that's not true for all floating point numbers!#2023-04-2323:02phronmophobichttps://stackoverflow.com/questions/21373865/different-results-when-adding-same-doubles-in-different-order#2023-04-2323:02phronmophobicnot sure why that didn't occur to me earlier :doh:#2023-04-2405:14karol.adamiecYou saved my bacon sir!
Swapped the spec to be ints, not doubles, and behold it is rock solid now.
That mysterious error pops up when the spec is non-deterministic, which Alex has said years ago more less.
So a better error message would be nice š#2023-04-2406:03karol.adamiecSo a new question arises, what now?
What would be best way to move forward?
1. drop the invariant (yikes...)
2. replace the generator in spec to gen ints, even though we say doubles...
3. use a tolerance delta in the invariant <= my favourite i think... but adds so much noise ;/#2023-04-2422:53phronmophobicIt depends on your goals. If you just want a function that calculates the mean of a sequence of numbers correctly, then I would use a library. However, there might be other goals (like learning spec) where you might do something different.#2023-04-2422:58phronmophobicThere are lots of fiddly trade offs when working with floats. Most libs that work with floats use a number of tricks like comparing within some tolerance for tests. Another trick is to pre sort the numbers before summing to improve accuracy and consistency. Iām not an expert here which is why I would just use a library.#2023-04-2506:21karol.adamiecyeah, the goal is to use spec to learn. i am not implementing new statistical package... yet :D#2023-04-2506:24karol.adamieci went with tolerance delta.
The biggest takeaway is: if your spec is non-deterministic, you will get the mysterious exception mentioned above.
Easy trap to fall into and burn, but once burned unlikely to happen again. And seeing that error immediately points in right direction. Typical newbie problem...#2023-04-2507:45phronmophobicI actually haven't used s/fdef
. For these types of things, I usually go straight to https://clojure.org/guides/test_check_beginner. The nice thing is that you can create separate tests for each property (eg. within-range, avg-unchanged-by-order?) and run them independently for however long it takes to feel comfortable that your implementation works. For avg-unchanged-order?
, doing one shuffle only tests a small part of the test space. I would probably write it to use https://github.com/clojure/math.combinatorics to test a broader part of the input space.#2023-04-2507:50phronmophobicFor test.check
, there's a way to get the RNG seed to make tests consistent and reproducible. There's probably also a way to do that with s/fdef
so that avg-unchanged-order?
is consistent/reproducible and avoids the problem you were running into.#2023-04-2507:56karol.adamiecThanks for ideas. Will think on that.
One point intrigued me especially.
You say:
_I actually haven't used `s/fdef` . For these types of things, I usually go straight to <https://clojure.org/guides/test_check_beginner|test.check>. The nice thing is that you can create separate tests for each property (eg. within-range, avg-unchanged-by-order?) and run them independently for however long it takes to feel comfortable that your implementation works._
One could add `s/fdef` to helpers as well too.
It seems to mee then, that desired test "pyramid" would be:
1. s/fdef for most important and public facing parts
2. test.check for these as well,
3. exaple based classic unit tests, to present few cases to readers. It gets hard to get a glance of "shapes" involved without examples.#2023-04-2507:59karol.adamiecwill try to roll with this and see how my perceptions change š#2023-04-2508:00karol.adamieci do feel there is this a bit awkward dichotomy with spec/gen and test.check/gen i.e. some things are wrapped, but sooner or later (rather sooner) one HAS TO land in test.check land and learn it good.#2023-04-2508:01karol.adamiecOTOH, using test.check for double generator, where there is a nice s/double-in in spec feels wrong... so i end up doing things twice... and have extra pass => can i do that using spec namespace only...#2023-04-2508:03phronmophobicI still use stuff like s/def
and s/double-in
. You can get the generator to use with test.check
via s/gen
.#2023-04-2508:06karol.adamiecin official docs they do mention that one should use test.check as last order of biz. as that will force that dependency on prod build.
So it seems like the spirit of what i should do is try to avoid test.check in my prod namespaces, and reach for that only in testing namespaces.
Although i have to say, i do not really follow the argument there, as spec depends on it anyway... so who cares whether i include that explicit or it is sucked in in deps of deps...#2023-04-2508:08phronmophobicspec dynamically loads it so you can load spec without test.check#2023-04-2508:11karol.adamiecyes it does. so i do not fully understand this:
There are three ways to build up custom generators - in decreasing order of preference:
1. Let spec create a generator based on a predicate/spec
2. Create your own generator from the tools in clojure.spec.gen.alpha
3. Use test.check or other test.check compatible libraries (like https://github.com/gfredericks/test.chuck)
The last option requires a runtime dependency on test.check so the first two options are strongly preferred over using test.check directly.#2023-04-2508:12karol.adamiectest.check bytecode ends up in my prod slim and shiny build anyway...
What is the perspective here that i miss? Maybe some more intricate interactions when designing libs based on libs...#2023-04-2508:13karol.adamiecOr just a rule of thumb to make code as future proof as possible as this will be the direction?#2023-04-2508:13phronmophobicI don't think test.check
should show up unless you explicitly include it.#2023-04-2508:14phronmophobicmaybe some other dependency is pulling it in?#2023-04-2508:17karol.adamiechmm... i might have misunderstood the wording then. i assumed that spec is using test.check namespace internally#2023-04-2508:18karol.adamieci have not inspected the build image btw, just to be clear#2023-04-2508:19karol.adamiecbut what you say makes more sense for sure š#2023-04-2508:19karol.adamiecand clears my confusion, so this is 99% correct i think#2023-04-2508:19phronmophobicIt does use test.check
, but only for functionality starting at generators and later, https://clojure.org/guides/spec#_generators#2023-04-2508:20phronmophobicwhich you won't usually need at runtime#2023-04-2508:20karol.adamiecspec generators rely on the Clojure property testing library https://github.com/clojure/test.check. However, this dependency is dynamically loaded#2023-04-2508:20karol.adamiecyes, to that was the part that set my mind on it#2023-04-2508:21karol.adamiecso, i hack away happily using spec namespace, but ... it will load in test.check at some point even in prod. even without me saying it has to#2023-04-2508:21karol.adamiecso we go back to the 1.2.3. above. why would that matter?#2023-04-2508:22karol.adamiecanyway. i can live with not knowing for now.
Thank You for help š#2023-04-2508:23phronmophobicIf your prod code doesn't use generators or things that need them, then you don't need test.check
as a prod dependency and it won't be included#2023-04-2508:24phronmophobichttps://github.com/clojure/spec.alpha/blob/ad06cdc7407c11990c7e93206133fb14eb62cacf/src/main/clojure/clojure/spec/gen/alpha.clj#L55#2023-04-2508:25phronmophobicThat's what all the delay and dynaload stuff is doing.#2023-04-2508:27phronmophobicand if test.check
isn't included and you try to use generators, you'll get an exception, https://github.com/clojure/spec.alpha/blob/ad06cdc7407c11990c7e93206133fb14eb62cacf/src/main/clojure/clojure/spec/gen/alpha.clj#L40#2023-04-2508:37karol.adamiecnice.
š#2023-05-2619:28djtangoI'm using metosin data specs and have a datastructure like this:
[{:type "foo" :message "foo-msg"}]
In my code I have various maps of type->msg
(def errors-a
{:foo "foo-msg-a"
:bar "bar-msg-a"})
(def errors-b
{:foo "foo-msg-b"
:bar "bar-msg-b"})
(defn keyset [m]
(->> m keys (map name) (into #{})))
(defn ->error-spec [errors]
[{:type (s/spec (keyset errors))
:message string?}])
s/spec expects a macro and so s/explain-data
outputs the symbol (e.g. keyset) instead of the set inlined. I've tried writing a macro to inline the set produced but have been getting errors to do with the mixing of macros and trying to evaluate values at compile time. Anyone able to help me write a macro so (s/spec (keyset errors))
inlines to (s/spec #{"foo" "bar"})
as this makes the explain-data most understandable#2023-04-2516:08karol.adamiecI have spent some time searching for information about clojure.spec.alpha.test/check
and how to wrap it up so it is used in a test run.
So how can I wrap around an above (stest/check
mysymbol)` in a (deftest)
?#2023-04-2516:24lassemaatta(disclaimer: it's been a while since I've used spec) isn't the return value of stest/check
just data? Can't you just verify it?#2023-04-2516:25karol.adamiecyes i can. i can wrap it up and dress in an it
body, and request the return data result key to be a :success. or whatever. It just seems like making a bit of a piruette...#2023-04-2516:26karol.adamiecso i lost my confidence and start questioning my reasons...#2023-04-2516:29lassemaattaThere's https://clojure.org/guides/test_check_beginner#_defspec in test.check
for when you have generators providing the values and the actual function under test. but I don't think that's applicable to fspec
's (or if there's some similar defsomething macro) for them.#2023-04-2516:30karol.adamiecyes. defspec seems like "half way there"#2023-04-2516:31karol.adamiecIt very well mignt be that fspec is not really a 'testing' area but more for instrumentation and repl.#2023-04-2516:37lassemaattaI'm not sure I'd agree. My understanding is that instrumentation is there to verify that other code invokes your function correctly (ie. it checks the :args
part of fspec), but verifying that your function actually does what it's fspec promises (the :ret
and :fn
parts ) is the responsibility of s/check
(= testing)#2023-04-2516:54karol.adamiec(t/deftest mean-invariant-properties
(t/testing "Should pass fspec exercise"
(t/is (nil? (:check-failed (stest/summarize-results (stest/check `mean)))))))
#2023-04-2516:54karol.adamiecso this works. but it does not look 'good'#2023-04-2516:55karol.adamiecusually stuff that looks wrong, or is too much work ends up being me trying to force round pegs in square holes.
hence me asking š#2023-04-2517:06lassemaattaLooks ok to me :man-shrugging: I guess you could try minimizing the code if you really want to; remove the t/testing
part and perhaps create some helper fn, which does the e.g. the (nil? (:check-failed (stest/summarize-results
part to make it shorter.#2023-04-2517:08karol.adamiecyeah, but that is not even "correct". it also might throw and there is another keyword for checking that...
It is not a big deal wrapping it in small macro and attaching it by invoke and pass of a namespace...
I am just surprised and worried that is not there.#2023-04-2517:10karol.adamiecmore worried. š
but thanks for sharing your views on that. š#2023-04-2517:10karol.adamiecflying solo often has me questioning my understanding, especially for new areas š#2023-04-2517:21lassemaattaa very similar discussion from a few years ago: https://groups.google.com/g/clojure/c/0UjSFT926vg š#2023-04-2517:24lassemaattabut yeah, it seems there's no out-of-the-box integration with fdef
and clojure.test
. Though it seems some test libraries (kaocha) bridge the gap and try to make it easier. But in any case I wouldn't feel very worried about it. My guess is that stest/check
was designed to have a strict scope and return the data as data and it's up to you to decide what to do with that data.#2023-04-2516:19karol.adamiecor the reason i can not find that is that is not supposed to be done?#2023-04-2516:19karol.adamiecOne can argue that property tests as invaraibly a bit random are not a great fit for pipelines...?#2023-04-2520:30Ben Liebermanare pre/post maps in a function my best bet for spec'ing input/output of parsing datetimes from strings? for instance I have this relatively simple conversion (-> "2023-10-27T09:00:00.000Z" Instant/parse (OffsetDateTime/ofInstant (ZoneId/systemDefault)))
but unfortunately the API I'm consuming does not include the Z
at the end. Is it better to have two specs, one a regex pattern for input and one a call to #(instance? OffsetDateTime %)
as output?#2023-05-1118:56Mark WardleHi. Could you use an s/or
spec and conform
, and then do what is needed based on the result of that conform? You could provide different predicates using different regexps (or even use instance? so your fn can take in lots of different things) and then conform, and then use the result to operate on the result. You'd parse one way for one type, or another for another type. Or am I misunderstanding your question? It sounds as if this isn't necessarily a spec question but a polymorphism question in which you want to act on a variety of different possible inputs?#2023-05-1119:12Ben LiebermanI think the input will always be uniform unless the controller of this API is changing it, which is out of my hands. I think using conform
could be the right path forward, but I'd have to go back and look at this code. Haven't had a chance in a while. Thanks for the suggestion.#2023-05-1119:16Mark WardleI didnāt spot that your question was from April sorry! #2023-05-1119:16Ben LiebermanNo problem! I appreciate the reply š:skin-tone-3:#2023-05-0513:52wcohenare there any best practices or existing examples for trying to deploy cljs.spec.test.alpha/check with a jsdom environment available (without jumping to shadow yet)#2023-05-1015:02Henrique PrangeI have created specifications for a file entity, with separate definitions for documents and images. To differentiate between the two, I used s/or
to define the :file/content-type
according to the media type. However, Iād like to ensure that the :file/content-type
matches the media type of the file being checked, depending on whether it is an image or a document. Specifically, I would like to ensure that image files are of content-type "image/jpeg"
or "image/png"
only, and that document files are of content-type "application/pdf"
or "image/jpeg"
only. Below is the specification I have created:
(s/def :file/content-type (s/or :doc #{"application/pdf" "image/jpeg"}
:img #{"image/jpeg" "image/png"}))
(s/def :file/height int?
(s/def :file/width int?
(s/def :file/doc (s/keys :req [:file/content-type]))
(s/def :file/img (s/keys :req [:file/content-type :file/height :file/width]))
Can someone please advise on how I can ensure that image files and document files have the appropriate content type?#2023-05-1118:17Mark WardleHi Henrique... (Are you using Clojure with WO now?)
You can use s/and for this kind of stuff with any predicate(s) you want. If you want, instead of using two different specs, you can combine them into a single spec and then have a multi-spec on the type:
(s/def :file/content-type #{"application/pdf" "image/jpeg" "image/png"})
(s/def :file/height int?)
(s/def :file/width int?)
(s/def :file/type #{:img :doc})
(defmulti file-spec :file/type)
(defmethod file-spec :img [_] (s/and
(s/keys :req [:file/type :file/content-type :file/height :file/width])
#(#{"image/jpeg" "image/png"} (:file/content-type %))))
(defmethod file-spec :doc [_] (s/and
(s/keys :req [:file/type :file/content-type])
#(#{"application/pdf" "image/jpeg"} (:file/content-type %))))
(s/def ::file (s/multi-spec file-spec :file/type))
Here I'm assuming the file data comes with :file/type and file:content-type. Alternatively, you can define two specs as you did and use whichever one you want.
When I started, I thought I'd specify everything with clojure spec, but there is a long tail with diminishing returns, so I'll often not specify some things, knowing I can supplement with runtime checks and use spec a la carte.#2023-05-1118:22Mark WardleThis is what it would look like with two separate specs meaning that your code has to make the choice as to which type to check against:
(s/def :file/content-type #{"application/pdf" "image/jpeg" "image/png"})
(s/def :file/height int?)
(s/def :file/width int?)
(s/def :file/doc (s/and (s/keys :req [:file/content-type])
#(#{"application/pdf" "image/jpeg"} (:file/content-type %))))
(s/def :file/img (s/and (s/keys :req [:file/content-type :file/height :file/width])
#(#{"image/jpeg" "image/png"} (:file/content-type %))))
(s/conform :file/doc {:file/content-type "application/pdf"})
(s/conform :file/img {:file/content-type "application/pdf"})
(s/conform :file/doc {:file/content-type "image/jpeg"})
(s/conform :file/img {:file/content-type "image/jpeg" :file/height 100 :file/width 100})
#2023-05-1213:16Henrique PrangeHey Mark! Great to hear from you! Iām no longer working with WebObjects, and Iām now using Clojure and Datomic on my current project.
Thanks for your help with the spec question. Iām using spec mostly to validate that my Datomic entities are complete and consistent, and itās been working really well so far. I didnāt know that I could use defmulti
with spec, which is really helpful! Ultimately, I ended up following your second suggestion and making separate specifications for images and documents using s/and
.
Thanks again for your advice!#2023-05-1214:04Mark WardleNo problem at all. I still have a legacy WO application in production that I'm replacing bit by bit over time with new Clojure based components. If I squint, I see the older dynamism of the objective C WO and the key value coding / key paths form WO in Clojure and its general approach, although I sometimes miss the batteries-included approach on occasions, I definitely feel I'm not fighting the framework as much here.#2023-05-1702:11john2xI'm trying to understand if this is an okay usage for spec, and if it's possible.
I have user spec like so:
(s/def ::email string?)
(s/def ::id string?)
(s/def ::user (s/keys :req [::email ::id]))
I'm going to store this user in DynamoDB, but to do that I need to transform from DynamoDB's shape to my ::user
spec.
So far I have record->user
and user->record
helper functions to transform from one to the other. It works, but I need to manually type in the mapping if I add new attributes.
Can I do this with spec? e.g. create a spec for the DynamoDB record based on ::user
, and have the transformation functions auto-created as well?
Or is there another approach I should look into?#2023-05-1716:39phronmophobic> have the transformation functions auto-created as well
Are the transformation doing something more than removing the namespace of the keyword?#2023-05-1716:45phronmophobic> Can I do this with spec? e.g. create a spec for the DynamoDB record based on ::user
, and have the transformation functions auto-created as well?
Spec is agnostic to where the data comes from or is going to. It's possible to write this sort of functionality in clojure, but I might worry about artificially creating an https://en.wikipedia.org/wiki/Object%E2%80%93relational_impedance_mismatch.
It's hard to recommend anything specific without knowing more about the use case.
ā¢ How do you plan to use Dynamo?
ā¢ Are interacting with existing data or do you have more control over the format used to store data?
ā¢ Are you connecting to Dynamo via https://github.com/Taoensso/faraday or some other option?#2023-05-1723:45john2x> Are the transformation doing something more than removing the namespace of the keyword?
Basically this
(defn user->record
"Transform from ::models.user/user to DynamoDB record data that can be written"
[user]
{:id {:S (::models.user/id user)}
:email {:S (::models.user/email user)}})
(defn record->user
"Transform raw DynamoDB record data to ::models.user/user spec"
[data]
{::models.user/email (get-in data [:email :S])
::models.user/id (get-in data [:id :S])})
I guess this is my OOP background leaking. But I would prefer if I'm using and passing around ::user
maps everywhere, and only use the DynamoDB {:id {:S "foo"}}
shape on the edges.#2023-05-1723:47john2x> ā¢ How do you plan to use Dynamo?
> ā¢ Are interacting with existing data or do you have more control over the format used to store data?
> ā¢ Are you connecting to Dynamo via https://github.com/Taoensso/faraday or some other option?
Nothing fancy. Just intending to use it as a document store. I do have full control over the data. And I'm using com.cognitect.aws/api
#2023-05-1723:53phronmophobicWhat is the :S
key?#2023-05-1723:54john2xIt's just dynamodb's way of saying that the value is a string#2023-05-1723:54phronmophobicIt seems like the translation to/from could be automatic rather than manual. Is there a reason not to just iterate through all the keys in the provided map and strip/append the keyword namespace info?#2023-05-1723:59john2xyeah I think that could work. It might get tricky if I use more complex specs/shapes (e.g. nested). Fortunately I don't see this work doing anything too fancy with the schema for now#2023-05-1800:01phronmophobicFor inspiration, you can check out how https://github.com/seancorfield/next-jdbc interfaces with databases. Another higher level API that might provide some ideas is datalevin, which also provides a clojure interface over a key/value store, https://github.com/juji-io/datalevin#use-as-a-key-value-store.#2023-05-1914:13EdI've definitely written something that converts back and forth between DDB's representation and clojure's literal data. A quick google found this that you could use for inspiration: https://github.com/doo/clj-dynamodb/blob/master/src/clj_dynamodb/convert/to_dynamodb.clj (nothing to do with me - all my impl's have been closed source)#2023-05-1914:19EdBut I think it's generally a better idea to explicitly convert your data structures from DDB land to Clojure data and back with function calls, rather than rely on something like spec to coerce the data from one structure into another. I think that reduces the utility of the spec.#2023-05-1922:36john2xYeah, I'm starting with just manually writing the conversion for now. Thanks for all the suggestions#2023-05-2412:32OknoLombardahow is this possible? valid?
returns false, meaning v doesn't conform to spec, but explain
prints "Success!" as though it does#2023-05-2412:51Alex Miller (Clojure team)in general, cases of this are bugs#2023-05-2412:51Alex Miller (Clojure team)if you have one, please put it on https://ask.clojure.org#2023-05-2414:43OknoLombardaposted minimal example there
https://ask.clojure.org/index.php/12957/inconsistent-results-from-valid-explain-when-trying-values#2023-05-2414:50Alex Miller (Clojure team)Can you add the spec? (preferably not tied to metosin stuff). Maybe you can (s/form ::sp)
to see it?#2023-05-2415:04OknoLombardawill this do?
case> (s/form (ds/spec ::sp cards))
(spec-tools.core/spec
{:spec (clojure.spec.alpha/keys :req-un [:case$sp/cards]),
:type :map,
:leaf? false,
:name :case/sp})
case> (s/form :case$sp/cards)
(spec-tools.core/spec
{:spec
(clojure.spec.alpha/coll-of
(spec-tools.core/spec
{:spec clojure.core/int?,
:type :long,
:leaf? true,
:name :case$sp/cards})
:into
[]),
:type :vector,
:leaf? false,
:spec-tools.core/synthetic? true})
#2023-05-2415:24OknoLombardaI was told it might be issue of spec-tools itself, so I posted it on their github repo as well
https://github.com/metosin/spec-tools/issues/275#2023-06-0112:17gregHi, I noticed this behaviour that even if the keyword is not explicitly included in s/key, it is implicitly included:
(s/def :person/name string?)
(s/def :person/surname string?)
(s/def :person/email string?)
(s/def ::profile
(s/keys :req [:person/name
:person/surname]))
(comment
(s/explain ::profile {:person/name "Tom"
:person/surname "Wilkins"
:person/age 29})
; => Success!
(s/explain ::profile {:person/name "Tom"
:person/surname "Wilkins"
:person/email nil})
; => nil - failed: string? in: [:person/email] at: [:person/email] spec: :person/email
In the above example :person/email is not included in ::profile but the spec is complaining about it.
Can it be disabled?#2023-06-0113:09Alex Miller (Clojure team)This is by design (see the docstring of s/keys)#2023-06-0113:20gregThank you @U064X3EF3, I read it.
Can I compose a spec to validate a map in some other way so these namespaced spaces are not implicitly included?#2023-06-0113:22gregThe problem I'm facing is that I can't control the data I receive, and in the codebase I work with, there is lots of specs namespaced by category, here and there, often duplicated (another issue with overriding entries in registory). I'm trying to tackle these issues one by one, instead of making big changes in one go.
One of these namespace specs conflicts with keys used in maps I try to validate.#2023-06-0113:56Alex Miller (Clojure team)In short, no, with s/keys. You can instead select-keys the data to just the keys you want to check before validating#2023-06-0114:22gregYes, unfortunately this strategy would not work with instrumenting functions š#2023-06-0301:22gregOk, I managed to find a workaround to this problem. What I did was writing a wrapper to wrap Malli schema under Spec protocol. That map validation is performed by Malli despite the instrumentation is executed via Spec api.
This way I managed to avoid s/keys behaviour.
The other issue I had in the same repo was that these subspecs (`:person/name`, :person/surname
, etc) were defined multiple times in different parts of the project resulting in overriding global registry. Thanks to that wrapper I also managed to avoid that problem.#2023-06-0518:34joshchoIs there a way to browse specs nicely in Emacs? Maybe a minibuffer that shows all the specs defined, or a way to quickly jump to specs.#2023-06-0607:58lassemaattahttps://docs.cider.mx/cider/usage/misc_features.html#browsing-the-clojure-spec-registry#2023-06-0717:48KelvinIs there a good way to dynamically create new s/or
specs at runtime? As s/or
is a macro, it feels almost impossible to do that, hence Iād have to go down to or-spec-impl
which is very much discouraged:
(defn json-schema-or-spec
"Create a `s/or` spec from a coll-valued JSON Schema `type`"
[schema-types]
(let [pairs (map (fn [schema-type]
(case schema-type
"null" [:null ::null]
"boolean" [:boolean ::boolean]
"integer" [:integer ::integer]
"number" [:number (type->spec "number")]
"string" [:string (type->spec "string")]
"array" [:array (type->spec "array")]
"object" [:object (type->spec "object")]))
schema-types)
keys (mapv first pairs)
preds (mapv second pairs)]
(s/or-spec-impl keys preds preds nil))
(or alternatively end up creating a case
statement with 128 different cases)#2023-06-0717:54Alex Miller (Clojure team)you can write that case with a macro around s/or :)#2023-06-0717:55Alex Miller (Clojure team)but maybe a different kind of spec is better, like s/multi-spec?#2023-06-0717:56KelvinWriting a macro around s/or
was my first attempt:
(defn- schema-type->pair
[type->spec schema-type]
(case schema-type
"null" `[:null ::null]
"boolean" `[:boolean ::boolean]
"integer" `[:integer ::integer]
"number" `[:number (~type->spec "number")]
"string" `[:string (~type->spec "string")]
"array" `[:array (~type->spec "array")]
"object" `[:object (~type->spec "object")]))
(defmacro coll-schema-spec
[type->spec schema-types]
(let [schema-type->pair (partial schema-type->pair type->spec)]
`(s/or ~@(mapcat schema-type->pair schema-types))))
#2023-06-0717:57KelvinUnfortunately it did not go very well: I ended up encountering a lot of
Don't know how to create ISeq from: clojure.lang.Symbol
errors because I was passing in schema-types
as a variable#2023-06-0718:00Alex Miller (Clojure team)well I think you're on the right track there, but you don't want ~@ - this is one of those cases that might be easier as literal construction with list
cons
etc#2023-06-0718:01Alex Miller (Clojure team)~ turns off quoting and turns eval back on, but you don't want to eval#2023-06-0718:02Alex Miller (Clojure team)also, schema-type->pair
is just a map#2023-06-0718:13KelvinStill getting the same error; I think the fundamental issue is that schema-types
is a coll whose values are only known as runtime, so trying to fit it inside any compile-time macro like s/or
will only result in failure.#2023-06-0718:13Kelvin(And itās these times when I wished we used malli in our projects)#2023-06-0718:19Alex Miller (Clojure team)can you just register these schemas at runtime then?#2023-06-0718:19Alex Miller (Clojure team)think about it as dynamically making static specs, not statically making dynamic specs#2023-06-0718:20KelvinSo you actually get that already with the ::null
, ::boolean
, and ::integer
specs#2023-06-0718:21KelvinThe issue comes with those other specs whose implementations depend on the JSON Schema user input (the type->spec
function elides a lot of complexity)#2023-06-0718:22KelvinFor example with array
and object
youāre basically creating s/coll-of
and s/keys
specs from user input - not really conducive for registering static specs#2023-06-0718:29Alex Miller (Clojure team)not conducive at compile time, but totally fine at runtime#2023-06-0718:37Ben SlessYou can dispatch on the count of schema-types to unrolled calls to s/or#2023-06-0718:38KelvinI tried doing that, but that didnāt work either#2023-06-0718:38KelvinSince s/or
requires keyword literals as keys, not variables representing those keywords#2023-06-0718:39Ben SlessOh right#2023-06-0718:39Ben SlessWell you can use eval#2023-06-0718:40KelvinI used eval to pass in my schema-types
variable to the macro, but that gave a ācanāt eval localsā error#2023-06-0718:45KelvinAnyways, going back to @U064X3EF3ās point, Iām not sure what he meant by āregister these schemas at runtimeā since when I think of registering specs, I automatically think of s/def
#2023-06-0718:45KelvinI guess you can also generate the spec keywords at runtime, but given the potentially infinite schema values that does not seem to be a good idea#2023-06-0718:50Alex Miller (Clojure team)ultimately, the spec registry is a map in your Clojure runtime. you can put things in it anytime, not just when you load a namespace that calls s/def#2023-06-0718:50Alex Miller (Clojure team)(all of this is more readily available in the spec 2 api, both making specs and registering them without s/def)#2023-06-0718:51KelvinAnd spec 2 is supposed to have more support for dynamic, data-driven specs like the varardic s/or
Iām trying to make, right?#2023-06-0719:02KelvinAnyways I think Iām going to go with or-spec-impl
as my near-term solution#2023-06-0719:03KelvinIt works like a charm just like any other spec, and thereās not really any downside other than a strongly worded docstring#2023-06-0720:44Alex Miller (Clojure team)spec 2 has a data form you can use to make a spec, and a non-macro way to register a spec#2023-06-1113:14Sturmis it possible to use spec to validate maps where keys are strings? eg. to check {"name" "Joe" "age" 66}
to make sure "name"
and "age"
are present and that they match string?
and int?
respectively. I might be missing something obvious, but I can only see examples with keyword-based maps.#2023-06-1113:18Alex Miller (Clojure team)You can use map-of but not s/keys#2023-06-1113:20Sturmthanks @U064X3EF3, I'll have a play with map-of#2023-06-2221:01Chris LesterI have a strange error (because I can't find any references to this) on having a spec from another namespace that is only required and used in a (comment ...) block causing a compile failure. Is that a special case similar to unknown reader tags in the comment block that will cause it to fail since it attempts to parse that anyway?#2023-06-2221:06seancorfieldYou mean this reader error:
> clj
Clojure 1.11.1
user=> (comment
(require '[foo.bar :as quux])
::quux/wibble
Syntax error reading source at (REPL:4:0).
Invalid token: ::quux/wibble
user=>
#2023-06-2221:07seancorfieldThat's because the contents of the comment
must be syntactically valid Clojure tokens, and because the require
is runtime and not evaluated, the auto-resolving keyword prefix ::quux
cannot be expanded.#2023-06-2221:10seancorfieldYou could either add the require to your ns
(even tho' it wouldn't be used elsewhere -- you could use :as-alias
instead of :as
if you don't want it to be actually loaded assuming you're on a recent enough version of Clojure) or you could use the fully-qualified Spec name instead of using the alias.#2023-06-2221:25Chris LesterThanks Sean, makes sense, I added a partner dev who cleaned up that namespace since it wasnāt used (coming from a typed language and the non use of that ns bugged him).#2023-07-0401:46Joerg SchmueckerIs it possible that spec and test.check do have some incompatibilities? I am getting cannot cast Generator to fn exceptions when using specs in the tests.#2023-07-0401:54hiredmanSpec generators are test.check generators wrapped in a 0 art function#2023-07-0401:54hiredmanArg#2023-07-0408:12Joerg SchmueckerSo how do I use a spec generator in test.check? Unwrap them or use the āsurfaceā provided by āspec.alphaā.#2023-07-0408:13Joerg SchmueckerMore importantly, how would i find out about htat? Without having to bother a human.#2023-07-0520:14MegaMattWondering why this code produces the prn statements that it does.
(let [shape {:path1 true :path2 {:nested true}}
path-exists? (fn [p o]
(let [p (map keyword (str/split p #"\."))
missing? (= :missing (get-in o p :missing))]
(prn o)
#_(not missing?)
true))]
(s/def ::something (s/and (s/or :path1-exists (partial path-exists? "path1"))
(s/or :path2-exists (partial path-exists? "path2.nested"))
(s/or :path3-exists (partial path-exists? "path2.nested2"))))
(s/valid? ::something {:path1 :here :path2 {:nested :here :nested2 :blah}}))
Res:
{:path1 :here, :path2 {:nested :here, :nested2 :blah}}
[:path1-exists {:path1 :here, :path2 {:nested :here, :nested2 :blah}}]
[:path2-exists [:path1-exists {:path1 :here, :path2 {:nested :here, :nested2 :blah}}]]
Iāll post the why iām doing this in a thread in case others can suggest a better approach.#2023-07-0520:16MegaMattIām making a macro which takes a list of dot delimited strings and makes a spec which ensures the shape of the data
eg. (defshape ["a.b.c", "a.d"])
would ensure you have something like
{:a {:b {:c "test"}} :d true}
#2023-07-0520:17MegaMattand sure, just s/and would work but the errors are not helpful to the developer thus i wanted s/or so that i can have a keyword that indicates the problem.#2023-07-0520:25seancorfields/and
is conforming, passing the conformed value into the second and subsequent specs.#2023-07-0520:25seancorfieldI think if you wrap each s/or
in s/nonconforming
that will produce what you want.#2023-07-0520:26MegaMattthank you.#2023-07-0520:46MegaMattUpdate:point_up:: s/nonconforming solved my issue.#2023-07-1207:39Joerg SchmueckerHow would I spec an higher order function such as map
. The function parameter passed in should validate against
(s/def ::f (s/fspec :args (s/cat any?) :ret any?))
but if I try to validate (valid? ::f (fn [i] (* i 2)))
then this fails because it will generate non numerical values according to the above spec.#2023-07-1213:01colinkahnYou can replace your fspec with fn?
or ifn?
#2023-07-1213:48Joerg Schmueckeryes, I can but then I donāt check that the signature/shape matches. The real signature is something like this:
(s/fspec :args (s/cat :state any?) :res (s/cat :state any :result any?))
I always want to get a vector with 2 or more elements back.#2023-07-1222:49Joerg SchmueckerI know, itās much more than a type system! This is the first thing I did find that would be easier with a static type system. Thatās why I mentioned it.#2023-07-2823:11johanatanI would put a runtime guard in the body of the function. #2023-07-2823:11johanatanReturn nil when wrong input #2023-07-2823:12johanatanPretty sure this is how I handled this in the past #2023-07-2605:18Joerg SchmueckerSorry, another question that I couldnāt find a good answer for via G. How do I spec a nested structure?
Here is the code snippet that I tried.
(s/def ::my-spec (s/cat :a-b (s/cat :s string? :i int?) :s2 string?))
(s/explain-data ::my-spec [["test" 1] "world"])
;; => #:clojure.spec.alpha{:problems [{:path [:a-b :s], :pred clojure.core/string?, :val ["test" 1], :via [:ct-test/my-spec :ct-test/my-spec], :in [0]}], :spec :ct-test/my-spec, :value [["test" 1] "world"]}
I guess I could use a tuple in the spec and that would resolve it but thatās more limiting that s/cat
.
Different question? Is there any good way to put these answers into Google?#2023-07-2605:55phronmophobicI think you can get the nesting you want by wrapping the inner s/cat
with s/spec
.
https://clojurians.slack.com/archives/C1B1BB2Q3/p1664745079479859?thread_ts=1664739041.707149&cid=C1B1BB2Q3#2023-07-2605:57phronmophobic(require '[clojure.spec.alpha :as s])
(s/def ::my-spec (s/cat :a-b (s/spec (s/cat :s string? :i int?)) :s2 string?))
(s/valid? ::my-spec [["test" 1] "world"]) ;; true
#2023-07-2707:22Joerg Schmueckergotcha thanks!#2023-08-1506:10Lidor CohenHello everyone š
I made this function to generate maps with a given depth and width:
(s/def ::any (s/or :s string? :i int?))
(defn map-gen [x y]
(->> (repeatedly x #(-> keyword?
s/gen
g/generate))
(map (fn [k] [k (if (pos? y)
(map-gen x (dec y))
(-> ::any
s/gen
g/generate))]))
(into {})))
but for some reason it is very slow, what am I misusing here?#2023-08-1608:14pithylessYour recursion is generating an exponentially increasing large map, so the time is also going to go up. For example:
(map-gen 10 y)
where y is 1 to 5 goes up in time:
y=1 "Elapsed time: 14.155375 msecs"
y=2 "Elapsed time: 50.327042 msecs"
y=3 "Elapsed time: 748.101916 msecs"
y=4 "Elapsed time: 2843.250333 msecs"
y=5 "Elapsed time: 28140.05775 msecs"
#2023-08-1608:15pithylessSo, it would be helpful if you specify how big are the maps you're generating, what kind of time you're seeing, and what kind of time you'd expect (since "slow" and "fast" are relative).#2023-09-0214:20kokonutHi, I just realized that spec's instrumentation doesn't really check :ret
and :fn
field. I also read some discussions that say we should be able to use test/check
. But is there any way to enable those fields on repl in development phase (not unit testing)?#2023-09-0214:53nikolavojicichttps://github.com/jeaye/orchestra#2023-09-0215:42gnlAlso https://github.com/fulcrologic/guardrails#2023-09-0219:41kokonutThanks.#2023-09-0619:06mgSo it seems like maybe spec2 is dead in the water? Haven't seen any announcements related to this in years and the repo hasn't had a major change since 2021...#2023-09-0619:12seancorfieldNot dead. Stalled. Last I heard, Rich was still thinking about how to integrate specs into function definitions in a way he liked. Alex has said a couple of times that spec2 will likely become core spec (non-alpha) at some point.#2023-09-0620:55gnlGiven his recent announcement and departure from Nubank, might there be reason to cautiously hope...? š#2023-09-0620:58gnlP.S. Perhaps you could nudge him to take a look at the Guardrails/Ghostwheel syntax as a particularly succinct alternative ā having that in core would be pretty fantastic. Not that I consider this development very likely, but you don't ask, you don't get. š#2023-09-0619:15mgAh, thanks for the update. Maybe we should chip in for a nicer hammock#2023-09-0710:41ikitommiRe: function definitions. I wish Clojure would get a spec how to define inline types in the core + optional tooling for enforcement (spec, schema, malli). In js-land: https://tc39.es/proposal-type-annotations/#2023-09-0912:30Jacksson Enrique Mosquera Rivas#2023-09-0916:18seancorfieldIt looked like you got answers to this question in #C053AK3F9 - are you still stuck?#2023-09-2109:03vemv(One of those "just curious" questions š)
Is there an intrinsic reason why spec (1) is slower than Malli? It's relatively common to hear that it's not performant enough to be used in something like a web handler. However it doesn't sound too crazy to implement some sort of drop-in replacement for a few key functions like conform
.
One would still use the regular spec/def
for definitions - the custom checker could simply read the original registry.#2023-09-2110:43nikolavojicicNever had a significantly slow spec such that it matters for web app... For data generation yes, it can be slow and can be fixed by tweaking generators. Is there an example?#2023-09-2111:38vemv'That matters' is the tricky part... for many webapps a 1ms delay doesn't appear to matter, but if you are trying to serve thousands of requests per second, it becomes the bottleneck#2023-09-2115:12seancorfieldWe use conform
heavily in our main REST API at work and it's "fast enough". I guess if you have an app where Spec's performance matters and Malli satisfies while Spec does not, then use Malli?
We've never seen Spec as a bottleneck. JDBC stuff, yes. Elastic Search stuff, yes. Spec, never.#2023-09-2115:21vemvI once participated in a project where we used, roughly speaking, spec for code (fdefs and the like), and malli for data.
Wasn't bad, I might even enjoy exercising all those skills on a good day, but it doesn't seem ideal.
I'm a spec1 fan and would rather use it everywhere... but its performance is theoretically inferior. Again, you can say the same about e.g. Rails or Python web servers. In reality CPU is rarely the bottleneck, as you hint... but some of us enjoy squeezing the last drop of performance.
Knowing it can do fast coercion, validation, etc can help in the occasional team-wide decision making processes.#2023-09-2115:24vemvReflecting a bit, a later project used Malli in a more advanced way. I had the impression that you could go as far as building "your own spec1" but in Malli terms.
Would seem a clearer path than hacking on spec1, as it is a stalled project.#2023-09-2322:33jasonjcknw.r.t conform
vs decode/encode
in malli a lot more optimization work is done at 'compile time' (in the sense of constructing a chain of lambda functions to arrive at a a parser)
e.g. in malli , it's possible your decoder/conform compiles to an identity function#2023-09-2322:34jasonjckn#2023-09-2322:36jasonjcknThis is partly due to different designs, because the construction of the decoder/parser takes into account the structure of the input, if you know you're only trying to create morphisms between JSON <-> EDN , you can optimize more#2023-09-2322:38jasonjcknPersonally, i'm a huge fan of malli, i think its closer to rich hickey's definition of simplicity than spec version 1, since specs/schema are just data, ... this could change with spec version 2, but i'm a convert for now.#2023-10-1716:44Ho0manHi everyone,
I wanna spec sth like this :
[[:__ :TXC :BTC :ALT :PAXG ]
[:TT1 0M 0M 0M 0M ]
[:TT2 0M 0M 0M 0M ]
[:TT3 0M 0M 0M 0M ]
[:TT4 0M 0M 0M 0M ]
[:Others 0M 0M 0M 0M ]
]
but the following spec won't work :
(spec/def ::X-Header-Row (spec/cat :dont-care #{:__} :exposure-id-s (spec/+ ::Exposure-ID)))
(spec/def ::X-Row (spec/cat :g-id ::Group-ID :weights (spec/+ decimal?)))
(spec/def ::X (spec/cat :header ::X-Header-Row :rows (spec/+ ::X-Row)))
What is it that I'm not getting about how spec/cat
works and how can I spec that table ?
Thanks a lot in advance#2023-10-1716:55Alex Miller (Clojure team)won't work .... because?#2023-10-1717:39Alex Miller (Clojure team)regex ops combine to spec one collection, so you need to do something to separate the outer collection and the inner collection "levels"#2023-10-1717:40Alex Miller (Clojure team)probably the shortest path to that is to wrap s/spec around the ::X-Row cat#2023-10-1811:51Ho0manThanks a lot, Alex#2023-11-0803:16cflemingI have a case where conform
returns :clojure.spec.alpha/invalid
, but explain
returns āSuccess!ā. Am I missing something?
(def test-data (rest '(defn F "s||C_)C^?" {} ([[& {:keys [], :strs [a2p*1.+lNz3+9.-fc? !2_!T2*8_.aw7.H14*7 B_-T*E5+3lKy-gE Ca++-F*_G58!7+? VM.3-E**S*R+09eW f-F?J-D_?r*xdjb.D]} :as kQ.zi] & yeok5+8+3_O_.mXJZ] "MyN-Rmu[Y" nil :Wd--IU9M*! ":*ULe,h" :p-:?6+__S- J91*) "EcO")))
=> #'cursive.extensions.specs/test-data
(s/conform :clojure.core.specs.alpha/defn-args test-data)
=> :clojure.spec.alpha/invalid
(s/explain :clojure.core.specs.alpha/defn-args test-data)
Success!
#2023-11-0803:22hiredmanhttps://ask.clojure.org/index.php/13390/surprising-behaviour similar recent ask#2023-11-0803:23cflemingYeah that looks similar.#2023-11-0803:26cflemingIāve also found that generating code samples using the core specs doesnāt work well - it does work sometimes but not others, and Iām not sure why:
(gen/sample (s/gen ::core/bindings))
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
#2023-11-0803:49cflemingIāve commented on the ask with some cases. Looks to me like the cases should be valid, so perhaps the bug is in conform.#2023-11-0803:58hiredmanThe way s/and composes for generators is unlikely to get you a working generator for more complex specs, basically it takes the generator first spec, and then uses the rest of the specs in the and as a predicate to filter out generated values that don't match the entire s/and#2023-11-0803:59hiredmanIt is limited to 100 tries#2023-11-0803:59cflemingHmm, I see. Thanks for the explanation.#2023-11-0804:00hiredmanI think there might be something you can bind to increase such-that's limit, but in general if you hit it you usually need to write a custom generator instead of relying on s/and's composed one#2023-11-0804:01cflemingIs there a good way to figure out which spec is problematic? And can I override generators for the built-in specs?#2023-11-0804:01cflemingThe stacktrace didnāt seem especially useful.#2023-11-0804:03hiredmanspec has a way to define specs with custom generators, which is helpful for your own specs, for existing specs I sort of recall some mechanism for overriding but I am not sure#2023-11-0804:03hiredmanas far as figuring out which spec, I don't know#2023-11-0804:03cflemingOk, thanks. Iām currently generating these forms with my own code, and then just checking the result against the spec.#2023-11-1600:03cflemingIām trying to spec a map which I thought was homogeneous (i.e. key type matching a spec -> value type matching a spec), so I used map-of. Now it turns out that these maps can occasionally have other entries where the key is a constant keyword and then the value is something different to the other value type. I canāt figure out the best way to spec this - any advice?#2023-11-1601:03Alex Miller (Clojure team)it's a bit cumbersome but you can spec the map as a collection of tuple (entry) types#2023-11-1601:04Alex Miller (Clojure team)https://www.cognitect.com/blog/2017/1/3/spec-destructuring is a complicated example of this#2023-11-1601:06cflemingI see, so Iād define two s/tuple types, and then a coll-of :kind map? using s/or over the two types?#2023-11-1601:06Alex Miller (Clojure team)(s/def ::map-bindings
(s/every (s/or :mb ::map-binding
:nsk ::ns-keys
:msb (s/tuple #{:as :or :keys :syms :strs} any?)) :into {}))
(s/def ::map-special-binding
(s/keys :opt-un [::as ::or ::keys ::syms ::strs]))
(s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))
is kind of the crux of that - an s/merge of special keys via s/every with an s/keys#2023-11-1601:12cflemingOk, thanks for the pointer, I think Iāll need some time to digest that but I can figure it out from there.#2023-11-1601:14Alex Miller (Clojure team)that example is probably more complicated than what you need but that's the main idea#2023-11-1601:15cflemingCan I s/merge an s/map-of with an s/keys?#2023-11-1601:15Alex Miller (Clojure team)not successfully unless the map-of uses keyword keys#2023-11-1601:15cflemingIt doesnāt, unfortunately.#2023-11-1601:16Alex Miller (Clojure team)it's merging the requirements of the two specs so any data has to satisfy both#2023-11-1601:17Alex Miller (Clojure team)note that the tuple side (s/every of s/tuple) has an :msb branch that is effectively matching all of the known kws with any?#2023-11-1601:17Alex Miller (Clojure team)and then the s/keys side is s/opt so it's just ignoring anything else#2023-11-1601:23cflemingSo when I do an s/merge, every entry has to match both sides of the merge? Iām not sure I understood that.#2023-11-1601:30Alex Miller (Clojure team)yes#2023-11-1601:30Alex Miller (Clojure team)you are merging the requirements of the spec#2023-11-1601:32Alex Miller (Clojure team)so, it's kind of like s/and, but sub specs are effectively checked in parallel whereas s/and flows data through and checks sub specs serially#2023-11-1601:33Alex Miller (Clojure team)this also has implications for s/conform but you may not care about that#2023-11-1601:33cflemingHereās my little test case:
(s/def ::normal-key string?)
(s/def ::normal-val string?)
(s/def ::constant (s/map-of ::normal-key ::normal-val))
(s/def ::foo (s/coll-of string? :kind vector?))
(s/def ::special-case (s/keys :req-un [::foo]))
(s/conform ::constant {"foo" "bar"})
(s/conform ::special-case {:foo ["test"]})
(s/def ::mixed (s/every (s/or :constant (s/tuple ::normal-key ::normal-val)
:outlier (s/tuple #{:foo} ::foo))))
(s/def ::end-result (s/merge ::mixed ::special-case))
(s/conform ::end-result {"foo" "bar"})
(s/conform ::end-result {:foo ["test"]}))
#2023-11-1601:36cflemingSo I started with a map of string to string using ::constant
. Then I realised I need ::special-case
too. However my ::end-result
canāt conform {"foo" "bar"}
#2023-11-1601:37cfleming(s/explain ::end-result {āfooā ābarā})
{āfooā ābarā} - failed: (contains? % :foo) spec: :cursive.extensions.specs/special-case#2023-11-1601:40Alex Miller (Clojure team)req-un has to be opt-un there#2023-11-1601:40Alex Miller (Clojure team)as string/string entries won't match that#2023-11-1601:41cflemingAh, of course - that works, thanks!