Configuration Guide
Tracer config controls rollout jobs: which tasks to run, which agent scaffold
to use, how many jobs to launch, and whether to convert trajectories for SFT.
Most runs only need a few runtime_info.input fields. Leave meta_info,
repository pins, environment paths, and output declarations alone unless you are
changing the block contract itself.
Common fields
| Area | Edit when | Main fields |
|---|---|---|
| Model endpoint | You want Tracer to call a different model or gateway. | llm_api.api_key, api_base_url, model |
| Task source | You are receiving SWE/terminal tasks, an external local pool, or an HF snapshot. | task_source.provider, local_block, dataset_name, terminal_tasks_dir, split |
| Rollout scale | You are doing a smoke run, a batch run, or a full production run. | harbor_job.n_tasks, n_concurrent, max_retries, timeout_multiplier |
| Agent scaffold | You switch between Claude Code, OpenCode, or OpenHands SDK. | agent.name, version, runtime_image, runtime_host_path |
| Remote machine | You want Tracer to run rollouts on another Docker server. | meta_info.resources.ip, user, pwd, directory |
| Trainer handoff | You need training-ready data after rollout. | sft_conversion.enabled, scaffold, tokenizer_name, out_dir |
An example reference
Start with local Curator output, a small task cap, Claude Code, and SFT conversion enabled. This gives you a complete mini-run that can hand data to Trainer.
runtime_info: input: llm_api: api_key: <YOUR_API_KEY> api_base_url: https://your-openai-compatible-endpoint/v1 model: openai/Qwen3.6-35B-A3B task_source: provider: local dataset_name: ../curator/artifacts/merged_swe_tasks split: train harbor_job: jobs_dir: artifacts/jobs n_concurrent: 4 n_tasks: 10 max_retries: 2 timeout_multiplier: 5 agent: name: custom-claude-code version: 2.1.118 runtime_image: docker.io/jierun/c-cc-2.1.118:v0.1 runtime_host_path: artifacts/agent-runtime/claude-code max_turns: 80 temperature: 0.7 sft_conversion: enabled: true scaffold: auto tokenizer_name: Qwen/Qwen3.5-35B-A3B-Base out_dir: artifacts/sft_dataUse /tracer:check after editing. It catches missing task sources, bad model
endpoints, unavailable Docker/runtime images, and conversion config mistakes
before /tracer:run spends tokens.
Configuration breakdown
Remote execution
Tracer runs locally by default. If you want rollouts to happen on a remote
CPU/Docker node, fill meta_info.resources so the operating skill knows where
to log in and which block directory to use.
meta_info: resources: ip: 203.0.113.12 # local = current machine; otherwise SSH target user: ubuntu # SSH user for the remote server pwd: null # prefer SSH keys or your private secret channel directory: /home/ubuntu/LegoFlow/blocks/tracer public_storage: nullUse a remote node when local Docker capacity, network placement, or long-running rollouts make the current machine inconvenient. The remote directory should point at the Tracer block checkout, not just the repository root.
Task source
Tracer accepts Harbor task pools from the selected local producer, an external local directory, or a prebuilt Hugging Face snapshot. TRAJGEN_CONFIG selects a complete private profile; relative profile filenames resolve from this block. An omitted local_block keeps the SWE curator default; explicit invalid selectors are rejected.
- From Curator Tasks:
runtime_info: input: task_source: provider: local dataset_name: ../curator/artifacts/merged_swe_tasks split: train- From Terminal Curator:
runtime_info: input: task_source: provider: local local_block: terminal-curator terminal_tasks_dir: ../terminal-curator/artifacts/tasks harbor_job: n_tasks: 2 n_concurrent: 1 max_retries: 0Terminal input uses terminal_tasks_dir, leaving the SWE dataset_name intact.
Its independent dependency edge is active only for
{task_source.provider: local, task_source.local_block: terminal-curator} and
consumes terminal-curator.output.harbor_tasks_dir. A nonempty manifest is
required; missing or invalid listed tasks fail preparation. Before inference,
the producer skill must audit instruction/tests/reference-solution semantics,
record corrections, then repeat fresh Oracle/NOP validation. A runtime pass or
manifest alone does not prove that audit.
For an unrelated local pool, set local_block: external and dataset_name;
this selects neither producer. For local sources, a manifest filters IDs when
present; sources outside the terminal contract retain the valid-directory
fallback when absent.
- From Hugging Face Task Dataset:
runtime_info: input: task_source: provider: huggingface dataset_name: Lego-X/<dataset-name> split: trainHF preparation downloads a raw prebuilt-task snapshot. split participates in
profile/cache identity but does not filter a dataset-library split here.
Staging ownership and refresh
The public harbor_job.dataset_path defaults to artifacts/tasks/<dataset>
(terminal-curator for terminal input). harbor_job.dataset can override the
dataset label. Preparation atomically points the public path at an owned version
under artifacts/tasks/.legoflow-staging-v1/, keyed by resolved source, manifest,
settings, and file metadata. start.sh uses the fixed version returned by that
invocation, even if another preparation advances the public pointer.
Local task directories remain links. Terminal links resolve to immutable producer
batches; mutable external/SWE contents may still change in place. HF files are
materialized copies and a matching owned copy can be reused without download;
prepare_tasks.sh --overwrite requests a fresh download. An existing real
dataset directory or unrelated symlink is never replaced, including with that
flag. Choose a fresh dataset_path, or move old staging yourself after checking
active jobs. Retained versions are removed only by explicit cleanup, so preserve
versions referenced by active jobs.
Smoke run
Use a small n_tasks and low concurrency when checking a new endpoint,
runtime image, or task source. This is usually helpful for debugging or getting familiar with the whole pipeline.
runtime_info: input: harbor_job: n_concurrent: 1 n_tasks: 5 max_retries: 0 timeout_multiplier: 5Production rollout
For production, raise concurrency only after /tracer:check passes and a smoke
run has produced valid trajectories.
runtime_info: input: harbor_job: n_concurrent: 8 n_tasks: null # run every staged task max_retries: 2 timeout_multiplier: 5Agent version presets
The agent fields move as a set. Pick one row and keep agent.name,
agent.version, agent.runtime_image, agent.runtime_host_path, and
sft_conversion.scaffold aligned.
| Scaffold | agent.name | agent.version | agent.runtime_image | agent.runtime_host_path | sft_conversion.scaffold |
|---|---|---|---|---|---|
| Claude Code | custom-claude-code | 2.1.118 | docker.io/jierun/c-cc-2.1.118:v0.1 | artifacts/agent-runtime/claude-code | claude_code or auto |
| OpenCode | custom-opencode | 1.18.7 | docker.io/jierun/c-oc-1.18.7:v0.1 | artifacts/agent-runtime/opencode | open_code or auto |
| OpenHands SDK | custom-openhands-sdk | 1.33.0 | docker.io/jierun/c-oh-sdk-1.33.0:v0.1 | artifacts/agent-runtime/openhands-sdk | openhands_sdk or auto |
Use auto for a fresh run that uses the same agent.name in the current
config. Use an explicit scaffold key when reconverting an older job whose
scaffold may differ from the current config.
Handoff with neighboring blocks
Tracer normally receives tasks from Curator and sends converted LF data to Trainer.
Receive Curator tasks
task_source.dataset_name is the path Tracer actually reads. The dependency
entry documents that this value comes from Curator's merged_tasks_dir, so
root-level checks can validate the handoff.
meta_info: dependencies: from: task_source.dataset_name: from: curator.output.merged_tasks_dir when: {task_source.provider: local, task_source.local_block: curator}runtime_info: input: task_source: provider: local dataset_name: ../curator/artifacts/merged_swe_tasksProduce Trainer-ready data
Trainer usually reads artifacts/sft_data/<job>/lf.json. Keep conversion on
when Tracer should produce that file after rollout.
runtime_info: input: sft_conversion: enabled: true scaffold: auto tokenizer_name: Qwen/Qwen3.5-35B-A3B-Base out_dir: artifacts/sft_data output: sft_data_dir: path: artifacts/sft_data trajectory_format: artifacts/sft_data/<job>/lf.jsonField reference
| Field | Required? | Default | When to change |
|---|---|---|---|
llm_api.api_key | Yes | "" | Supply OPENAI_API_KEY or use a private config. Never commit it. |
llm_api.api_base_url | Yes | "" | Supply OPENAI_API_BASE_URL for your OpenAI-compatible endpoint. |
llm_api.model | Yes | openai/Qwen3.6-35B-A3B | Change when testing a different teacher model. |
litellm_proxy.port | Yes | 4003 | Change only if the local port is already occupied. |
task_source.provider | Yes | local | Use local for Curator output; use huggingface for a published dataset. |
task_source.local_block | For local | curator | Choose curator, terminal-curator, or external; omission alone defaults to curator. |
task_source.terminal_tasks_dir | For terminal | ../terminal-curator/artifacts/tasks | Path to the terminal producer publication and manifest. |
task_source.dataset_name | Except terminal | ../curator/artifacts/merged_swe_tasks | Change to another local task directory or dataset id. |
harbor_job.n_concurrent | Yes | 8 | Tune for endpoint capacity, Docker capacity, and API rate limits. |
harbor_job.n_tasks | No | 40 | Use 5 or 10 for smoke runs; use null to run all staged tasks. |
harbor_job.max_retries | Yes | 2 | Lower for fast checks; raise only for unstable infrastructure. |
harbor_job.timeout_multiplier | Yes | 5 | Increase when agents time out on valid tasks. This is a multiplier, not seconds. |
agent.name | Yes | custom-claude-code | Switch coding scaffold. Change the full agent row together. |
agent.version | Yes | 2.1.118 | Must match the selected scaffold and runtime image tag. |
agent.runtime_image | Yes | docker.io/jierun/c-cc-2.1.118:v0.1 | Must match the selected scaffold and version. |
agent.runtime_host_path | Recommended | artifacts/agent-runtime/claude-code | Use the pre-extracted runtime path when available. |
sft_conversion.enabled | Yes | true | Keep true when Trainer will consume Tracer output. |
sft_conversion.scaffold | Yes | auto | Keep auto for fresh runs; set explicit key when reconverting old jobs. |
sft_conversion.tokenizer_name | Yes | Qwen/Qwen3.5-35B-A3B-Base | Align with the Trainer base model. |
sft_conversion.max_instances | No | null | Cap conversion for quick checks; null converts all successful rollouts. |
env_extra.HARBOR_EXCLUDE_TASKS | No | excluded_tasks.txt artifacts/processed_tasks.yaml | Add task ids or files to skip known-bad or already processed tasks. |
meta_info.resources.ip | No | local | Set a host/IP when Tracer should run on a remote Docker node. |
meta_info.resources.user | For remote | null | SSH user for the remote node. |
meta_info.resources.directory | For remote | null | Tracer block working directory on the remote node. |
meta_info.resources.public_storage | No | null | Optional public artifact location for dashboards or sharing. |
Complete config.yaml
Use this as the complete reference after you understand the few fields above.
blocks/tracer/config.yaml
meta_info: name: tracer label: "Tracer" description: Generates raw agent trajectories with Harbor for downstream SFT data conversion. parent: legoflow blocks: {} dependencies: from: task_source.dataset_name: from: curator.output.merged_tasks_dir when: {task_source.provider: local, task_source.local_block: curator} task_source.terminal_tasks_dir: from: terminal-curator.output.harbor_tasks_dir when: {task_source.provider: local, task_source.local_block: terminal-curator} to: raw_trajectories_dir: to: trainer.input.source.job_dir when: {trainer.input.source.type: harbor_job} resources: ip: local # CPU-only; runs on any node with Docker available user: null pwd: null directory: null description: Local CPU/Docker node. Remote SSH fields are ignored when ip is local. public_storage: null repositories: harbor: url: https://github.com/LegoX/harbor.git branch: main commit: 73fd120e731217b38f31667932ea26a28cdd4fff path: repos/harbor readonly: true legoflow_trace_crafter: url: https://github.com/LegoX/LegoFlow-Trace-Crafter.git branch: main commit: 5e9575a63f59b704c98776399a9502443918e356 # origin/main HEAD (LegoFlow-Trace-Crafter rename) path: repos/LegoFlow-Trace-Crafter readonly: true environment: harbor_uv: artifacts/env/harbor-uv litellm_uv: artifacts/env/litellm-venv # separate venv; harbor env resets to litellm 1.83.0 legoflow_trace_crafter_uv: artifacts/env/legoflow-trace-crafter-uv legoflow_trace_crafter_extras: [llm] litellm: python_version: "3.13" litellm_version: "1.83.14"runtime_info: input: llm_api: api_key: "" # supplied by OPENAI_API_KEY; never commit a real key api_base_url: "" # supplied by OPENAI_API_BASE_URL model: "openai/Qwen3.6-35B-A3B" description: Raw upstream model API used to configure per-job LiteLLM proxy startup. protocols: [openai_compatible, anthropic_compatible] served_via: per_job_litellm_proxy input_cost_per_token: 0.0000021 output_cost_per_token: 0.0000084 litellm_proxy: config_template: scripts/serve_llm/litellm_config.example.yaml port: 4003 master_key: legoflow-local-proxy-key task_source: provider: local # local | huggingface local_block: curator # curator | terminal-curator | external (used only for local) dataset_name: ../curator/artifacts/merged_swe_tasks terminal_tasks_dir: ../terminal-curator/artifacts/tasks split: train harbor_job: jobs_dir: artifacts/jobs # exported by start.sh as TRAJGEN_HARBOR_JOBS_DIR n_concurrent: 8 n_tasks: 40 # null = all manifest tasks max_retries: 2 timeout_multiplier: 5 agent: name: custom-claude-code # Supported pinned versions: custom-claude-code=2.1.118, custom-opencode=1.18.7, custom-openhands-sdk=1.33.0. version: 2.1.118 runtime_image: docker.io/jierun/c-cc-2.1.118:v0.1 runtime_host_path: artifacts/agent-runtime/claude-code # pre-extracted; see /tracer:check reference max_turns: 80 temperature: 0.7 sft_conversion: enabled: true # true = start.sh runs convert step after the Harbor job scaffold: auto # auto derives from agent.name; explicit choices: claude_code | open_code | openhands_sdk | terminus2 tokenizer_name: Qwen/Qwen3.5-35B-A3B-Base # align conversion with trainer.model.model_name_or_path out_dir: artifacts/sft_data max_instances: null exclude_repos_file: "" # "" = repo default artifacts/excluded_repos.txt reasoning_check_mode: adaptive reasoning_content_ratio_threshold: 0.2 env_extra: HARBOR_EXCLUDE_TASKS: excluded_tasks.txt artifacts/processed_tasks.yaml # exclusion sources; literal ids also accepted output: raw_trajectories_dir: description: Harbor job directories with LiteLLM trajectory logs path: artifacts/jobs trajectory_format: "artifacts/jobs/<job>/<task>/agent/litellm-trajectory.jsonl" sft_data_dir: description: LF-format JSON arrays converted from Harbor trajectories, ready for LLaMA-Factory SFT. path: artifacts/sft_data format: llamafactory_lf_v0.1 trajectory_format: "artifacts/sft_data/<job>/lf.json"