-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Labels
easyThis is expected to be easy.This is expected to be easy.
Description
It would be nice to be able to pass measure_names::Vector{Strin} or something along these lines to evaluate and benchmark. This would be particularly useful for passing on anonymous functions to measure. Currently, only meta_data` can help, but it's not ideal:
julia> benchmark([ce]; measure=[((x;kwrgs...) -> feature_sensitivity(x, 1))], meta_data=[Dict(:measure_name=>"sens_1")])
Benchmark(1×7 DataFrame
Row │ sample num_counterfactual variable value measure_name target factual
│ Base.UUID Int64 String Float32 String Int64 Int64
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ 50193892-009d-11f0-2009-f1b3b93e7d48 1 #35 1.34479 sens_1 2 1, nothing)For non-anonymous functions, the function name is used as expected, but it might still be nice to be able to specify something more readable:
julia> sens1(x;kwrgs...) = feature_sensitivity(x, 1)
sens1 (generic function with 2 methods)
julia> benchmark([ce]; measure=[sens1])
Benchmark(1×8 DataFrame
Row │ sample num_counterfactual variable value generator model t ⋯
│ Base.UUID Int64 String Float32 Symbol Symbol I ⋯
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ f23365a6-009f-11f0-1c9a-1bc824dccac1 1 sens1 1.34479 GradientBasedGenerator(nothing, … CounterfactualExplanations.Model… ⋯
2 columns omitted, nothing)Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
easyThis is expected to be easy.This is expected to be easy.