Installation

Get started with Orchestrated by installing the SDK and CLI tool.

Installing the CLI

The orcha CLI tool is the recommended way to get started with Orchestrated. It lets you initialize projects, run evaluations, upload scorers, and manage your cloud configuration.

curl -fsSL https://console.getorchestrated.com/sdk/install.sh | bash

This installs the orcha command globally. Verify the installation:

orcha --version

Setting Up Your Project

Once the CLI is installed, you can set up your project in two ways:

Option 1: Using orcha init (Recommended)

Navigate to your project root and run:

orcha init

This will:

• Create an evals/ directory with example evaluation files
• Install the orchestrated SDK and required dependencies
• Generate a first.eval.ts file to get you started

Option 2: Manual SDK Installation

If you prefer to set up your project manually, install the SDK using your package manager:

pnpm add orchestrated

Verify Installation

If you used orcha init

Run the auto-generated example evaluation:

orcha eval evals/first.eval.ts

This will execute a simple Levenshtein distance evaluation to verify everything is working correctly.

Manual verification

Create a simple evaluation file to test your setup:

import { Levenshtein } from "autoevals";
import { Eval } from 'orchestrated'

await Eval("Hello World", {
  data: [
    { input: "What is 2+2?", output: "4", expected: "4" },
    { input: "What is 3+3?", output: "6", expected: "6" },
  ],
  scores: [Levenshtein],
})

Run it with:

orcha eval test.eval.ts

You should see a progress bar and evaluation results with scores.