- using R version 4.3.0 (2023-04-21)
- using platform: x86_64-apple-darwin20 (64-bit)
- R was compiled by
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
GNU Fortran (GCC) 12.2.0
- running under: macOS Ventura 13.3.1
- using session charset: UTF-8
- checking for file ‘mlr3fairness/DESCRIPTION’ ... OK
- checking extension type ... Package
- this is package ‘mlr3fairness’ version ‘0.3.2’
- package encoding: UTF-8
- checking package namespace information ... OK
- checking package dependencies ... OK
- checking if this is a source package ... OK
- checking if there is a namespace ... OK
- checking for executable files ... OK
- checking for hidden files and directories ... OK
- checking for portable file names ... OK
- checking for sufficient/correct file permissions ... OK
- checking whether package ‘mlr3fairness’ can be installed ... [8s/14s] OK
See the install log for details.
- checking installed package size ... OK
- checking package directory ... OK
- checking ‘build’ directory ... OK
- checking DESCRIPTION meta-information ... OK
- checking top-level files ... OK
- checking for left-over files ... OK
- checking index information ... OK
- checking package subdirectories ... OK
- checking R files for non-ASCII characters ... OK
- checking R files for syntax errors ... OK
- checking whether the package can be loaded ... [1s/3s] OK
- checking whether the package can be loaded with stated dependencies ... [1s/2s] OK
- checking whether the package can be unloaded cleanly ... [1s/2s] OK
- checking whether the namespace can be loaded with stated dependencies ... [1s/2s] OK
- checking whether the namespace can be unloaded cleanly ... [1s/2s] OK
- checking loading without being on the library search path ... [1s/2s] OK
- checking startup messages can be suppressed ... [1s/3s] OK
- checking dependencies in R code ... OK
- checking S3 generic/method consistency ... OK
- checking replacement functions ... OK
- checking foreign function calls ... OK
- checking R code for possible problems ... [9s/16s] OK
- checking Rd files ... [1s/1s] OK
- checking Rd metadata ... OK
- checking Rd cross-references ... OK
- checking for missing documentation entries ... OK
- checking for code/documentation mismatches ... OK
- checking Rd \usage sections ... OK
- checking Rd contents ... OK
- checking for unstated dependencies in examples ... OK
- checking contents of ‘data’ directory ... OK
- checking data for non-ASCII characters ... [0s/1s] OK
- checking LazyData ... OK
- checking data for ASCII and uncompressed saves ... OK
- checking installed files from ‘inst/doc’ ... OK
- checking files in ‘vignettes’ ... OK
- checking examples ... [10s/19s] OK
- checking for unstated dependencies in ‘tests’ ... OK
- checking tests ... [11s/20s] OK
Running ‘testthat.R’ [10s/20s]
- checking for unstated dependencies in vignettes ... OK
- checking package vignettes in ‘inst/doc’ ... OK
- checking running R code from vignettes ... [1s/2s] NONE
‘debiasing-vignette.Rmd’ using ‘UTF-8’... [0s/0s] OK
‘measures-vignette.Rmd’ using ‘UTF-8’... [0s/1s] OK
‘reports-vignette.Rmd’ using ‘UTF-8’... [0s/0s] OK
‘visualization-vignette.Rmd’ using ‘UTF-8’... [0s/0s] OK
- checking re-building of vignette outputs ... [91s/181s] ERROR
Error(s) in re-building vignettes:
--- re-building ‘debiasing-vignette.Rmd’ using rmarkdown
Pandoc is required to build R Markdown vignettes but not available. Please make sure it is installed.
2023-07-10 23:46:26.428 R[80044:2290430804] XType: Using static font registry.
Warning in file.info(x, extra_cols = FALSE) :
expanded path length 4561 would be too long for
<h1 id="introduction-fairness-pipeline-operators">Introduction: Fairness Pipeline Operators</h1>
<p>Given we detected some form of bias during bias auditing, we are often interested in obtaining fair(er) models.
There are several ways to achieve this, such as collecting additional data or finding and fixing errors in the data.
Assuming there are no biases in the data and labels, one other option is to debias models using either <strong>preprocessing</strong>, <strong>postprocessing</strong> and <strong>inprocessing</strong> methods.
<code>mlr3fairness</code> provides some operators as <code>PipeOp</code>s for <code>mlr3pipelines</code>.
If you are not familiar with <code>mlr3pipelines</code>, the <a href="https://mlr3book.mlr-org.com/pipelines.html">mlr3 book</a> contains an introduction.</p>
<p>We again showcase debiasing using the <code>adult_train</code> task:</p>
<pre><code class="language-r">library(mlr3)
library(mlr3fairness)
librar [... truncated]
--- finished re-building ‘debiasing-vignette.Rmd’
--- re-building ‘measures-vignette.Rmd’ using rmarkdown
Pandoc is required to build R Markdown vignettes but not available. Please make sure it is installed.
Warning in file.info(x, extra_cols = FALSE) :
expanded path length 17990 would be too long for
<h1 id="fairness-measures">Fairness Measures</h1>
<p>Fairness measures (or metrics) allow us to assess and audit for possible biases in a trained model.
There are several types of metrics that are widely used in order to assess a model’s fairness.
They can be coarsely classified into three groups:</p>
<ul>
<li>
<p><strong>Statistical Group Fairness Metrics</strong>: Given a set of predictions from our model, we assess for differences in one or multiple metrics across groups given by a <em>protected attribute</em> [@fairmlbook; @hardt2016equality].</p>
</li>
<li>
<p><strong>Individual Fairness</strong>: Basically requires that similar people are treated similar independent of the protected attribute [@dwork2012].
We will briefly introduce individual fairness in a dedicated section below.</p>
</li>
<li>
<p><strong>Causal Fairness Notions</strong>: An important realization in the context of Fairness is, that whether a process is fair is o [... truncated]
--- finished re-building ‘measures-vignette.Rmd’
--- re-building ‘reports-vignette.Rmd’ using rmarkdown
Pandoc is required to build R Markdown vignettes but not available. Please make sure it is installed.
Quitting from lines 52-54 [build_modelcard_example_for_vignette] (reports-vignette.Rmd)
Error: processing vignette 'reports-vignette.Rmd' failed with diagnostics:
pandoc version 1.12.3 or higher is required and was not found (see the help page ?rmarkdown::pandoc_available).
--- failed re-building ‘reports-vignette.Rmd’
--- re-building ‘visualization-vignette.Rmd’ using rmarkdown
Pandoc is required to build R Markdown vignettes but not available. Please make sure it is installed.
2023-07-10 23:48:42.299 R[18209:2290505996] XType: Using static font registry.
Warning in file.info(x, extra_cols = FALSE) :
expanded path length 8617 would be too long for
<pre><code class="language-r">library(mlr3)
library(mlr3fairness)
library(mlr3learners)
</code></pre>
<h1 id="why-we-need-fairness-visualizations">Why we need fairness visualizations:</h1>
<p>Through fairness visualizations allow for first investigations into possible fairness problems in a dataset.
In this vignette we will showcase some of the pre-built fairness visualization functions.
All the methods showcased below can be used together with objects of type <code>BenchmarkResult</code>, <code>ResampleResult</code> and <code>Prediction</code>.</p>
<h1 id="the-scenario">The scenario</h1>
<p>For this example, we use the <code>adult_train</code> dataset.
Keep in mind all the datasets from <code>mlr3fairness</code> package already set protected attribute via the <code>col_role</code> “pta”, here the “sex” column.</p>
<pre><code class="language-r">t = tsk("adult_train")
t$col_roles$pta
#> [1] "sex"
</code></pre [... truncated]
--- finished re-building ‘visualization-vignette.Rmd’
SUMMARY: processing the following file failed:
‘reports-vignette.Rmd’
Error: Vignette re-building failed.
Execution halted
- checking PDF version of manual ... [7s/13s] OK
- DONE
Status: 1 ERROR
- using check arguments '--no-clean-on-error '