Skip to content

Option to specify measure names in evaluate and benchmark #514

@pat-alt

Description

@pat-alt

It would be nice to be able to pass measure_names::Vector{Strin} or something along these lines to evaluate and benchmark. This would be particularly useful for passing on anonymous functions to measure. Currently, only meta_data` can help, but it's not ideal:

julia> benchmark([ce]; measure=[((x;kwrgs...) -> feature_sensitivity(x, 1))], meta_data=[Dict(:measure_name=>"sens_1")])
Benchmark(1×7 DataFrame
 Row │ sample                                num_counterfactual  variable  value    measure_name  target  factual 
     │ Base.UUID                             Int64               String    Float32  String        Int64   Int64   
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────
   150193892-009d-11f0-2009-f1b3b93e7d48                   1  #35       1.34479  sens_1             2        1, nothing)

For non-anonymous functions, the function name is used as expected, but it might still be nice to be able to specify something more readable:

julia> sens1(x;kwrgs...) = feature_sensitivity(x, 1)
sens1 (generic function with 2 methods)

julia> benchmark([ce]; measure=[sens1])
Benchmark(1×8 DataFrame
 Row │ sample                                num_counterfactual  variable  value    generator                          model                              t 
     │ Base.UUID                             Int64               String    Float32  Symbol                             Symbol                             I 
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ f23365a6-009f-11f0-1c9a-1bc824dccac1                   1  sens1     1.34479  GradientBasedGenerator(nothing,   CounterfactualExplanations.Model    
                                                                                                                                            2 columns omitted, nothing)

Metadata

Metadata

Assignees

No one assigned

    Labels

    easyThis is expected to be easy.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions