attotool

A tiny YAML-tool-calling agent built from scratch in Rust.

attotool is a minimalistic agent that uses YAML-formatted tool calls (configurable to JSON) to interact with the local system. It lets large language models choose and execute tools in a loop until task completion, in a compact, structured, human-readable format.

Evals (12 models)

Evals

## Legacy eval results (yaml format only) `Read at least 10 files in the repo and summarize your findings` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-gpt-4o-mini.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-grok-4-fast.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-grok-code-fast-1.yml) | Criteria \ Model | grok-4-fast | gpt-4o-mini | grok-code-fast-1 | |------------------|-------------|-------------|------------------| | Finished task | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-grok-4-fast.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-gpt-4o-mini.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-grok-code-fast-1.yml) | | Read 3 files | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-grok-4-fast.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-gpt-4o-mini.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-grok-code-fast-1.yml) | | Read 8 files | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-grok-4-fast.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-gpt-4o-mini.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-grok-code-fast-1.yml) | --- `read the url at ./url.txt , fetch that url, and write a yaml summary of its contents to ./summary.yaml` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-glm-4-32b.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gpt-4o-mini.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-mistral-small-3.1-24b.yml) | Criteria \ Model | glm-4-32b | gpt-4o-mini | mistral-small-3.1-24b | |------------------|-----------|-------------|-----------------------| | Used curl or wget | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-glm-4-32b.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-gpt-4o-mini.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-mistral-small-3.1-24b.yml) | | summary.yaml valid YAML | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-glm-4-32b.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-gpt-4o-mini.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-mistral-small-3.1-24b.yml) | ## Eval Results - format-specific ### YAML Format #### `Read at least 10 files in the repo and summarize your findings` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-grok-4-fast-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gemma-3-27b-it-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-mistral-7b-instruct-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-glm-4-32b-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gpt-4o-mini-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-mistral-small-3.1-24b-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gpt-oss-120b-exacto-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-devstral-small-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-google-gemini-2.5-flash-lite-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-openai-gpt-5-nano-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-anthropic-claude-3-haiku-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-meta-llama-llama-3.3-70b-instruct-yaml.yml) | Model \ Criteria | Finished task | Read 3 files | Read 8 files | |------------------|---------------|--------------|--------------| | grok-4-fast | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-grok-4-fast-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-grok-4-fast-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-grok-4-fast-yaml.yml) | | gemma-3-27b-it | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gemma-3-27b-it-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gemma-3-27b-it-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gemma-3-27b-it-yaml.yml) | | mistral-7b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-mistral-7b-instruct-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-mistral-7b-instruct-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-mistral-7b-instruct-yaml.yml) | | glm-4-32b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-glm-4-32b-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-glm-4-32b-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-glm-4-32b-yaml.yml) | | gpt-4o-mini | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gpt-4o-mini-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gpt-4o-mini-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gpt-4o-mini-yaml.yml) | | mistral-small-3.1-24b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-mistral-small-3.1-24b-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-mistral-small-3.1-24b-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-mistral-small-3.1-24b-yaml.yml) | | openai/gpt-oss-120b:exacto | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gpt-oss-120b-exacto-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gpt-oss-120b-exacto-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gpt-oss-120b-exacto-yaml.yml) | | mistralai/devstral-small | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-devstral-small-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-devstral-small-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-devstral-small-yaml.yml) | | google/gemini-2.5-flash-lite | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-google-gemini-2.5-flash-lite-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-google-gemini-2.5-flash-lite-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-google-gemini-2.5-flash-lite-yaml.yml) | | openai/gpt-5-nano | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-openai-gpt-5-nano-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-openai-gpt-5-nano-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-openai-gpt-5-nano-yaml.yml) | | anthropic/claude-3-haiku | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-anthropic-claude-3-haiku-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-anthropic-claude-3-haiku-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-anthropic-claude-3-haiku-yaml.yml) | | meta-llama/llama-3.3-70b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-meta-llama-llama-3.3-70b-instruct-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-meta-llama-llama-3.3-70b-instruct-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-meta-llama-llama-3.3-70b-instruct-yaml.yml) | #### `read the url at ./url.txt , fetch that url, and write a yaml summary of its contents to ./summary.yaml` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-glm-4-32b-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gpt-4o-mini-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gpt-oss-120b-exacto-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-mistral-small-3.1-24b-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-grok-4-fast-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gemma-3-27b-it-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-mistral-7b-instruct-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-devstral-small-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-google-gemini-2.5-flash-lite-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-openai-gpt-5-nano-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-anthropic-claude-3-haiku-yaml.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-meta-llama-llama-3.3-70b-instruct-yaml.yml) | Model \ Criteria | Used curl or wget | summary.yaml valid YAML | |------------------|-------------------|-------------------------| | glm-4-32b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-glm-4-32b-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-glm-4-32b-yaml.yml) | | openai/gpt-oss-120b:exacto | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gpt-oss-120b-exacto-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gpt-oss-120b-exacto-yaml.yml) | | mistral-small-3.1-24b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-mistral-small-3.1-24b-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-mistral-small-3.1-24b-yaml.yml) | | grok-4-fast | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-grok-4-fast-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-grok-4-fast-yaml.yml) | | gemma-3-27b-it | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gemma-3-27b-it-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gemma-3-27b-it-yaml.yml) | | mistral-7b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-mistral-7b-instruct-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-mistral-7b-instruct-yaml.yml) | | gpt-4o-mini | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gpt-4o-mini-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gpt-4o-mini-yaml.yml) | | mistralai/devstral-small | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-devstral-small-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-devstral-small-yaml.yml) | | google/gemini-2.5-flash-lite | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-google-gemini-2.5-flash-lite-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-google-gemini-2.5-flash-lite-yaml.yml) | | openai/gpt-5-nano | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-openai-gpt-5-nano-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-openai-gpt-5-nano-yaml.yml) | | anthropic/claude-3-haiku | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-anthropic-claude-3-haiku-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-anthropic-claude-3-haiku-yaml.yml) | | meta-llama/llama-3.3-70b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-meta-llama-llama-3.3-70b-instruct-yaml.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-meta-llama-llama-3.3-70b-instruct-yaml.yml) | ### JSON Format #### `Read at least 10 files in the repo and summarize your findings` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-grok-4-fast-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gemma-3-27b-it-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-mistral-7b-instruct-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-glm-4-32b-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gpt-4o-mini-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-mistral-small-3.1-24b-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gpt-oss-120b-exacto-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-devstral-small-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-google-gemini-2.5-flash-lite-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-openai-gpt-5-nano-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-anthropic-claude-3-haiku-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-meta-llama-llama-3.3-70b-instruct-json.yml) | Model \ Criteria | Finished task | Read 3 files | Read 8 files | |------------------|---------------|--------------|--------------| | grok-4-fast | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-grok-4-fast-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-grok-4-fast-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-grok-4-fast-json.yml) | | gemma-3-27b-it | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gemma-3-27b-it-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gemma-3-27b-it-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gemma-3-27b-it-json.yml) | | mistral-7b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-mistral-7b-instruct-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-mistral-7b-instruct-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-mistral-7b-instruct-json.yml) | | glm-4-32b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-glm-4-32b-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-glm-4-32b-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-glm-4-32b-json.yml) | | gpt-4o-mini | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gpt-4o-mini-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gpt-4o-mini-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gpt-4o-mini-json.yml) | | mistral-small-3.1-24b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-mistral-small-3.1-24b-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-mistral-small-3.1-24b-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-mistral-small-3.1-24b-json.yml) | | openai/gpt-oss-120b:exacto | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gpt-oss-120b-exacto-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gpt-oss-120b-exacto-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gpt-oss-120b-exacto-json.yml) | | mistralai/devstral-small | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-devstral-small-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-devstral-small-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-devstral-small-json.yml) | | google/gemini-2.5-flash-lite | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-google-gemini-2.5-flash-lite-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-google-gemini-2.5-flash-lite-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-google-gemini-2.5-flash-lite-json.yml) | | openai/gpt-5-nano | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-openai-gpt-5-nano-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-openai-gpt-5-nano-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-openai-gpt-5-nano-json.yml) | | anthropic/claude-3-haiku | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-anthropic-claude-3-haiku-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-anthropic-claude-3-haiku-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-anthropic-claude-3-haiku-json.yml) | | meta-llama/llama-3.3-70b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-meta-llama-llama-3.3-70b-instruct-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-meta-llama-llama-3.3-70b-instruct-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-meta-llama-llama-3.3-70b-instruct-json.yml) | #### `read the url at ./url.txt , fetch that url, and write a yaml summary of its contents to ./summary.yaml` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-glm-4-32b-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gpt-4o-mini-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gpt-oss-120b-exacto-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-mistral-small-3.1-24b-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-grok-4-fast-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gemma-3-27b-it-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-mistral-7b-instruct-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-devstral-small-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-google-gemini-2.5-flash-lite-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-openai-gpt-5-nano-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-anthropic-claude-3-haiku-json.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-meta-llama-llama-3.3-70b-instruct-json.yml) | Model \ Criteria | Used curl or wget | summary.yaml valid YAML | |------------------|-------------------|-------------------------| | glm-4-32b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-glm-4-32b-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-glm-4-32b-json.yml) | | openai/gpt-oss-120b:exacto | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gpt-oss-120b-exacto-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gpt-oss-120b-exacto-json.yml) | | mistral-small-3.1-24b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-mistral-small-3.1-24b-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-mistral-small-3.1-24b-json.yml) | | grok-4-fast | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-grok-4-fast-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-grok-4-fast-json.yml) | | gemma-3-27b-it | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gemma-3-27b-it-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gemma-3-27b-it-json.yml) | | mistral-7b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-mistral-7b-instruct-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-mistral-7b-instruct-json.yml) | | gpt-4o-mini | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gpt-4o-mini-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gpt-4o-mini-json.yml) | | mistralai/devstral-small | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-devstral-small-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-devstral-small-json.yml) | | google/gemini-2.5-flash-lite | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-google-gemini-2.5-flash-lite-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-google-gemini-2.5-flash-lite-json.yml) | | openai/gpt-5-nano | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-openai-gpt-5-nano-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-openai-gpt-5-nano-json.yml) | | anthropic/claude-3-haiku | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-anthropic-claude-3-haiku-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-anthropic-claude-3-haiku-json.yml) | | meta-llama/llama-3.3-70b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-meta-llama-llama-3.3-70b-instruct-json.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-meta-llama-llama-3.3-70b-instruct-json.yml) | ### JSON Fixed Key Format #### `Read at least 10 files in the repo and summarize your findings` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-grok-4-fast-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gemma-3-27b-it-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-mistral-7b-instruct-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-glm-4-32b-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gpt-4o-mini-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-mistral-small-3.1-24b-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-gpt-oss-120b-exacto-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-devstral-small-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-google-gemini-2.5-flash-lite-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-openai-gpt-5-nano-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-anthropic-claude-3-haiku-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-read-meta-llama-llama-3.3-70b-instruct-json_fixed_key.yml) | Model \ Criteria | Finished task | Read 3 files | Read 8 files | |------------------|---------------|--------------|--------------| | grok-4-fast | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-grok-4-fast-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-grok-4-fast-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-grok-4-fast-json_fixed_key.yml) | | gemma-3-27b-it | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gemma-3-27b-it-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gemma-3-27b-it-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gemma-3-27b-it-json_fixed_key.yml) | | mistral-7b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-mistral-7b-instruct-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-mistral-7b-instruct-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-mistral-7b-instruct-json_fixed_key.yml) | | glm-4-32b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-glm-4-32b-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-glm-4-32b-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-glm-4-32b-json_fixed_key.yml) | | gpt-4o-mini | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gpt-4o-mini-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gpt-4o-mini-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gpt-4o-mini-json_fixed_key.yml) | | mistral-small-3.1-24b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-mistral-small-3.1-24b-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-mistral-small-3.1-24b-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-mistral-small-3.1-24b-json_fixed_key.yml) | | openai/gpt-oss-120b:exacto | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-gpt-oss-120b-exacto-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-gpt-oss-120b-exacto-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-gpt-oss-120b-exacto-json_fixed_key.yml) | | mistralai/devstral-small | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-devstral-small-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-devstral-small-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-devstral-small-json_fixed_key.yml) | | google/gemini-2.5-flash-lite | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-google-gemini-2.5-flash-lite-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-google-gemini-2.5-flash-lite-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-google-gemini-2.5-flash-lite-json_fixed_key.yml) | | openai/gpt-5-nano | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-openai-gpt-5-nano-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-openai-gpt-5-nano-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-openai-gpt-5-nano-json_fixed_key.yml) | | anthropic/claude-3-haiku | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-anthropic-claude-3-haiku-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-anthropic-claude-3-haiku-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-anthropic-claude-3-haiku-json_fixed_key.yml) | | meta-llama/llama-3.3-70b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-finished-task-read-meta-llama-llama-3.3-70b-instruct-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-3-files-read-meta-llama-llama-3.3-70b-instruct-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-read-8-files-read-meta-llama-llama-3.3-70b-instruct-json_fixed_key.yml) | #### `read the url at ./url.txt , fetch that url, and write a yaml summary of its contents to ./summary.yaml` [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-glm-4-32b-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gpt-4o-mini-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gpt-oss-120b-exacto-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-mistral-small-3.1-24b-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-grok-4-fast-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-gemma-3-27b-it-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-mistral-7b-instruct-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-devstral-small-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-google-gemini-2.5-flash-lite-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-openai-gpt-5-nano-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-anthropic-claude-3-haiku-json_fixed_key.yml) [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-execution-fetch-url-meta-llama-llama-3.3-70b-instruct-json_fixed_key.yml) | Model \ Criteria | Used curl or wget | summary.yaml valid YAML | |------------------|-------------------|-------------------------| | glm-4-32b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-glm-4-32b-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-glm-4-32b-json_fixed_key.yml) | | openai/gpt-oss-120b:exacto | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gpt-oss-120b-exacto-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gpt-oss-120b-exacto-json_fixed_key.yml) | | mistral-small-3.1-24b | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-mistral-small-3.1-24b-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-mistral-small-3.1-24b-json_fixed_key.yml) | | grok-4-fast | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-grok-4-fast-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-grok-4-fast-json_fixed_key.yml) | | gemma-3-27b-it | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gemma-3-27b-it-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gemma-3-27b-it-json_fixed_key.yml) | | mistral-7b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-mistral-7b-instruct-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-mistral-7b-instruct-json_fixed_key.yml) | | gpt-4o-mini | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-gpt-4o-mini-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-gpt-4o-mini-json_fixed_key.yml) | | mistralai/devstral-small | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-devstral-small-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-devstral-small-json_fixed_key.yml) | | google/gemini-2.5-flash-lite | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-google-gemini-2.5-flash-lite-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-google-gemini-2.5-flash-lite-json_fixed_key.yml) | | openai/gpt-5-nano | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-openai-gpt-5-nano-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-openai-gpt-5-nano-json_fixed_key.yml) | | anthropic/claude-3-haiku | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-anthropic-claude-3-haiku-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-anthropic-claude-3-haiku-json_fixed_key.yml) | | meta-llama/llama-3.3-70b-instruct | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-used-curl-wget-fetch-url-meta-llama-llama-3.3-70b-instruct-json_fixed_key.yml) | [](https://github.com/tbarron-xyz/attotool/actions/workflows/eval-valid-yaml-fetch-url-meta-llama-llama-3.3-70b-instruct-json_fixed_key.yml) |

Available Tools

• execute_shell_command: Run shell commands with arguments. 🟢 Requires explicit user confirmation.
• read_file: Read file contents
• read_lines: Reads specific lines from a file between start_line and end_line
• write_file: Write content to file. 🟢 Requires explicit user confirmation.
• write_lines: Writes content to specific lines in a file between start_line and end_line. 🟢 Requires explicit user confirmation.
• finish_plan / finish_task: Mark task as completed
• ask_for_clarification: Request user input
• describe_to_user: Provide descriptions or responses

Features

• Plan Mode: Enable read-only phase with --plan / -p flag, encouraging analysis and planning and forbidding all modifications
• Approval Prompts: User confirmation for potentially destructive operations (write_file, execute_shell_command)
• AGENTS.md Support: Automatically loads ./AGENTS.md as the first user message
• Conversation History: Saves interaction history to ~/.local/share/attotool/history.yaml
• config.yaml Configuration: Load model and format settings from ~/.config/attotool/config.yaml
• System Prompt Customization: Load user-defined system prompt section overrides from ~/.config/attotool/system_prompt.yaml, allowing customization of agent behavior while preserving defaults.
• Evals in GH Actions: Automated workflows for evaluating agent performance across multiple language models on standardized tasks

Installation

1. Ensure you have Rust installed: https://rustup.rs/
2. Clone this repository
3. Build the project: cargo build --release
4. Optionally, install the binary globally by linking it: ln -s target/release/attotool /usr/local/bin/attotool
5. Set your OpenRouter (or OpenAI, to the same env var) API key: export OPENROUTER_API_KEY=your_key_here

Usage

# Infinite loop (default)
attotool "read ./url.txt, fetch that url and describe the result as a markdown document"

# Single tool call
attotool --max-tool-calls 1 "curl the ubuntu homepage"

# Specify model and other options
attotool --model "openai/gpt-4" --max-tokens 4000 "your task here" --tool-call-details

# Continue a previous conversation
attotool --continue "your follow-up task here"

CLI Options

• --model: LLM model to use (default: mistralai/mistral-small-3.1-24b-instruct)
• --input: Task description (can also be provided as the first positional argument)
• --plan / -p: Enable plan mode (read-only phase, modifications discouraged)
• --no-clarify: Disable the ask_for_clarification tool
• --no-shell: Disable the execute_shell_command tool
• --max-tokens: Maximum tokens for response (default: 2000)
• --base-url: API base URL (default: https://openrouter.ai/api/v1, use https://api.openai.com/v1 for OpenAI)
• --max-tool-calls: Maximum number of tool calls (default: 0 for infinite)
• --retries: Number of retries for API calls (default: 3)
• --verbose: Enable detailed output including raw API responses
• --tool-call-details: Show detailed tool call results and execution output
• --disable-agents-md: Disable automatic loading of AGENTS.md (default: false)
• --yolo: 🚩 Enable YOLO mode (skips approval prompts for destructive operations and removes ask_for_clarification tool)
• --continue / -c: Reads the existing ~/.local/share/attotool/history.yaml and continues the conversation with a new user message
• --format: Response format (yaml, json, json_fixed_key; default: yaml)

Configuration

attotool can be configured via ~/.config/attotool/config.yaml:

model: mistralai/mistral-small-3.1-24b-instruct
format: yaml

Supported formats: yaml, json, json_fixed_key.

Different response formats are provided because various language models excel with specific tool call structures. yaml is human-readable and works well with most models. json allows flexible key-value pairs for complex arguments. json_fixed_key uses OpenAI’s response_format API parameter to enforce a strict schema for models that require precise JSON structures, potentially improving reliability for certain LLMs.

YAML:

read_file:
  path: '/some/file.txt'

JSON:

{"read_file":{"path":"/some/file.txt"}}

JSON Fixed Key:

{"tool":"read_file","tool_args":{"path":"/some/file.txt"}}

Recommended Models

The following models have been tested and have worked at least once with attotool:

• Model: $/1M tokens input / output (prices fetched 11/2025)
• mistralai/mistral-nemo: $0.02 / $0.04
• mistralai/mistral-7b-instruct: $0.028 / $0.054
• qwen/qwen-2.5-7b-instruct: $0.04 / $0.1
• mistralai/mistral-small-3.1-24b-instruct: $0.05 / $0.1
• z-ai/glm-4-32b: $0.1 / $0.1
• mistralai/devstral-small-2505: $0.06 / $0.12
• openai/gpt-oss-120b:exacto: $0.03 / $0.14
• google/gemma-3-27b-it: $0.09 / $0.16
• qwen/qwen-2.5-72b-instruct: $0.07 / $0.26
• x-ai/grok-4-fast: $0.2 / $0.5
• x-ai/grok-3-mini: $0.3 / $0.5
• openai/gpt-oss-120b:exacto: $0.15 / $0.6
• deepseek/deepseek-chat-v3-0324: $0.24 / $0.84
• x-ai/grok-code-fast-1: $0.2 / $1.5