示例

结合 CLI 各项功能的端到端工作流示例，包括代理驱动的用法

线索增强管道

从搜索中采集 LinkedIn 资料，用 LLM 分析进行增强，并将结果加载到 PostgreSQL 中。

Create the pipeline

```yaml dataset.yaml
name: lead-enrichment
description: Find and qualify engineering leads

sources:
  - id: search
    endpoint: /api/linkedin/search/users
    params:
      keywords: "Head of Engineering"
      count: 200
    parallel: 1

  - id: profiles
    endpoint: /api/linkedin/user
    dependency:
      from_source: search
      field: urn.value
    input_key: user
    parallel: 5
    rate_limit: "10/s"
    on_error: skip

  - id: qualified
    type: llm
    dependency:
      from_source: profiles
      field: name
    llm:
      - type: classify
        categories: "high_priority,medium,low"
        output_column: lead_score
      - type: enrich
        add:
          - "seniority:junior/mid/senior/lead/executive"
          - "is_technical:boolean"
          - "team_size:small/medium/large"
      - type: summarize
        max_length: 30
        output_column: quick_bio

storage:
  format: parquet
  path: ./data/
```

Run the collection

```bash
anysite dataset collect dataset.yaml
```

Load into PostgreSQL

```bash
anysite dataset load-db dataset.yaml -c pg
```

Query the results

```bash
anysite db query pg --sql "
  SELECT name, headline, lead_score, seniority, quick_bio
  FROM qualified
  WHERE lead_score = 'high_priority' AND is_technical = true
  ORDER BY name
" --format table
```

---

竞争对手监控

跟踪竞争对手公司及其员工和近期动态。

Create a competitors file

```text competitors.txt
anthropic
openai
google-deepmind
mistral-ai
```

Define the pipeline

```yaml competitor-monitor.yaml
name: competitor-monitor
description: Weekly competitor intelligence

sources:
  - id: companies
    endpoint: /api/linkedin/company
    from_file: competitors.txt
    input_key: company
    parallel: 2
    refresh: always

  - id: recent_posts
    endpoint: /api/linkedin/company/posts
    dependency:
      from_source: companies
      field: urn.value
    input_key: company
    parallel: 3
    rate_limit: "10/s"
    refresh: always

  - id: post_analysis
    type: llm
    dependency:
      from_source: recent_posts
      field: text
    llm:
      - type: classify
        categories: "product_launch,hiring,partnership,thought_leadership,other"
        output_column: post_type
      - type: summarize
        max_length: 30
        output_column: summary

storage:
  format: parquet
  path: ./data/

schedule:
  cron: "0 9 * * 1"

notifications:
  on_complete:
    - url: "https://hooks.slack.com/services/xxx"
```

Generate and install the weekly schedule

`dataset schedule` 会打印一行 crontab 内容 — 它本身不会运行任何东西：
```bash
anysite dataset schedule competitor-monitor.yaml --incremental --load-db pg
```
用 `crontab -e` 添加打印出的这一行（或改用 `--systemd` 生成 systemd 单元）。
见 [Scheduling](/docs/cli/datasets/scheduling?lang=zh)。

Analyze the data

```bash
anysite dataset query competitor-monitor.yaml --sql "
  SELECT c.name as company, pa.post_type, COUNT(*) as count
  FROM post_analysis pa
  JOIN companies c ON pa.company_id = c.urn_value
  GROUP BY c.name, pa.post_type
  ORDER BY c.name, count DESC
" --format table
```

---

多平台调研

针对一组人物采集 LinkedIn、Twitter 和 GitHub 上的数据，然后合并结果。

Define the pipeline

```yaml research.yaml
name: multi-platform-research
description: Cross-platform person research

sources:
  - id: linkedin_profiles
    endpoint: /api/linkedin/user
    from_file: people.txt
    input_key: user
    parallel: 3
    rate_limit: "10/s"
    on_error: skip

  - id: twitter_profiles
    endpoint: /api/twitter/user
    from_file: twitter_handles.txt
    input_key: user
    parallel: 3
    rate_limit: "10/s"
    on_error: skip

  - id: github_profiles
    endpoint: /api/github/user
    from_file: github_users.txt
    input_key: user
    parallel: 3
    on_error: skip

  - id: all_profiles
    type: union
    sources: [linkedin_profiles, twitter_profiles, github_profiles]

storage:
  format: parquet
  path: ./data/
```

Collect and query

```bash
anysite dataset collect research.yaml

anysite dataset query research.yaml --sql "
  SELECT * FROM all_profiles
" --format csv --output unified_research.csv

anysite dataset profile research.yaml
```

---

快速单行命令

# Enrich a single profile and save to database
anysite api /api/linkedin/user user=satyanadella -q --format jsonl | \
  anysite db insert mydb --table profiles --stdin --auto-create

# Batch process a CSV of companies
anysite api /api/linkedin/company --from-file companies.csv --input-key company \
  --parallel 5 --rate-limit "10/s" --on-error skip \
  --format csv --output company_profiles.csv

# Search and export in one command
anysite api /api/linkedin/search/users keywords="AI researcher" count=100 \
  --format csv --output ai_researchers.csv

# Quick database query
anysite db query pg --sql "SELECT name, headline FROM profiles WHERE headline LIKE '%CEO%'" \
  --format table

---

代理工作流模式

当 AI 代理（或脚本）需要决定如何驱动 CLI 时，这是一个有用的决策树：

One-off lookup of 1-5 items?
  → anysite api (ad-hoc call)

Batch from a known list?
  Small (< 20)  → anysite api --from-file
  Large (20+)   → dataset pipeline with a from_file source

Chaining multiple endpoints (search → detail → posts)?
  → dataset pipeline with dependent sources

Needs to run repeatedly (daily, weekly)?
  → dataset pipeline + schedule + incremental

One-time large collection?
  → dataset pipeline (progress tracking, error recovery, Parquet storage)

以上示例中都适用的实用默认原则：

• 编写调用或管道配置之前，始终先发现：用 anysite describe <endpoint> 获取准确的参数名和类型 — 永远不要凭猜测。
• 对批处理数据源来说，parallel: 3-5 和 on_error: skip 是合理的默认值。
• 在首次正式采集前，先对新管道进行 dry-run：anysite dataset collect dataset.yaml --dry-run。
• 重复运行时使用 --incremental，避免重新采集已有的输入值。
• anysite dataset guide --json 会以结构化数据的形式给代理提供完整的管道配置参考 — 见 Agent Protocol。

本 CLI 通过代理协议进行通信：结构化 JSON 输出、退出码，以及 两级发现载荷。当从管道或子进程中调用时，大多数子命令会自动切换为 JSON 输出。