The Koala benchmark suite for the Shell
Source Paper

Overview

The Koala benchmark suite enables characterization of performance-oriented research targeting the POSIX shell. It consists of 18 real-world program sets from diverse domains ranging from CI/CD and AI/ML to biology and the humanities. They are accompanied by real inputs that facilitate small- and large-scale performance characterization and varying opportunities for optimization. The latest version of Koala expands the suite with four new benchmark sets and extends its evaluation to new systems, testing zsh as an alternative shell interpreter and INCR, an incrementalization layer for shell programs.

Papers Using Koala

If any aspect of the suite is useful, please use the following citation:

First page of the ATC '25 Koala paper 
@inproceedings{koala2025atc,
  title = {The Koala Benchmarks for the Shell: Characterization and Implications},
  author = {Evangelos Lamprou and Ethan Williams and Georgios Kaoukis and Zhuoxuan Zhang
        and Michael Greenberg and Konstantinos Kallas and Lukas Lazarek and Nikos Vasilakis},
  booktitle = {Proceedings of the 2025 USENIX Annual Technical Conference (USENIX ATC '25)},
  year = {2025},
  address = {Santa Clara, CA},
  publisher = {USENIX Association},
}

Benchmarks

The complete list of benchmarks in the Koala suite is as follows:

Name Description
analytics Real-world network logs filtering to extract and summarize key events.
bio Genomic and transcriptomic analysis using population and RNA-seq data.
ci-cd Open-source software project build and test.
covid Analysis of public transit activity during the COVID-19 pandemic.
etcetera Prime generation via a sieve and command execution inside an overlay sandbox.
file-mod File compression, encryption, and conversion.
inference Media-related inference tasks using large foundation models.
interact Interactive shell programs (oh-my-zsh installer, snake game).
ml A full machine learning pipeline using scikit-learn.
net Network reconnaissance and firewall configuration tasks.
nlp Book processing using shell-based NLP pipelines from Unix for Poets.
oneliners Classic and modern one-liner shell pipelines.
pkg AUR package build and npm packages permission inference.
rand Random password generation and random team sampling from name lists.
repl Security auditing and git development workflow replay.
unixfun Text-processing problems from UNIX's 50-year anniversary.
weather Analysis & visualization of historical weather statistics.
web-search Crawling, indexing, and querying of Wikipedia data.

Quick Setup

Koala can be obtained using the following ways:

More information in the README.

More Info

Recent News

Further Reading



Issues That Need Help

Community & More