dspy-skills

왜 쓰나요

핵심 기능

DSPy signatures / modules / predictors encoded as patterns
Teleprompter (BootstrapFewShot, MIPRO) usage guidance
Evaluation + metric design
RAG pipeline scaffolds with DSPy retrievers
Opinionated integration with common LLM providers

라이브 데모

실제 사용 모습

dspy-skill.replay ▶ 준비됨

0/0

설치

작성자 저장소 · Claude Skills 공식 문서

사용 사례

실전 활용법: dspy-skills

How to build your first DSPy program and optimize it

👤 ML engineers and applied researchers ⏱ ~90 min advanced

언제 쓸까: You have a task where prompt quality matters and want a systematic way to improve it.

흐름

Define the signature

Task: classify support tickets into {billing, technical, account}. Give me a DSPy signature and a simple Predict module.✓ 복사됨

→ Signature + module code
Write an eval

Add an evaluation set of 50 labeled examples and an accuracy metric.✓ 복사됨

→ Eval harness with metric callable
Optimize

Run BootstrapFewShot to compile the module against the eval set.✓ 복사됨

→ Compiled predictor + improved score

결과: A DSPy-optimized predictor that beats a hand-written prompt, with reproducible code.

함정

Eval too small — optimizer overfits — Minimum 100–200 examples; hold out a true test set
Metric doesn't capture what you care about — Spend on metric design before on model choice

함께 쓰기: filesystem

Build a RAG pipeline with DSPy

👤 Engineers building retrieval-augmented systems ⏱ ~120 min advanced

언제 쓸까: You want modular, optimizable RAG rather than a hand-wired chain.

흐름

Define the modules

Create a DSPy RAG pipeline: RetrieveThenRead with ColBERTv2 or a local retriever.✓ 복사됨

→ Modular pipeline with separate retrieval and generation
Optimize end-to-end

Write an eval on our QA set and run MIPRO to improve.✓ 복사됨

→ Compiled pipeline with score delta

결과: A RAG pipeline you can improve by changing the eval, not by rewriting prompts.

함정

Retriever quality caps end-to-end quality — Evaluate retrieval separately (recall@k) before optimizing generation

함께 쓰기: local-rag

조합

다른 MCP와 조합해 10배 효율

dspy-skill + local-rag

Plug a local retriever into DSPy's RAG modules

Swap the ColBERT retriever for a local-rag MCP as the retrieval source.✓ 복사됨

dspy-skill + filesystem

Organize DSPy programs, evals, and artifacts in a repo

Lay out a DSPy project with programs/, evals/, and artifacts/ directories.✓ 복사됨

도구

이 MCP가 노출하는 것

도구	입력	언제 호출	비용
signature-design	task spec	Start of any DSPy program	0
module-authoring	signatures + flow	After signatures	0
teleprompter-optimization	module + eval	After eval is ready	LLM tokens during optimization
evaluation-harness	task data	Before optimizing	0

비용 및 제한

운영 비용

API 쿼터: LLM tokens dominate during optimization runs
호출당 토큰: Can be high — optimizations may invoke the LLM hundreds of times
금액: depends on provider
팁: Use cheap models during teleprompter runs, upgrade only for final evaluation

보안

권한, 시크릿, 파급범위

자격 증명 저장: LLM provider keys in env vars

데이터 외부 송신: LLM provider endpoints

문제 해결

자주 발생하는 오류와 해결

Teleprompter seems to make things worse

Check metric correctness; use a held-out set; widen the example pool.

Optimization eats your budget

Cap max_bootstrapped_demos and use cheaper models during search.

대안

dspy-skills 다른 것과 비교

대안	언제 쓰나	단점/장점
prompt-architect-skill	You want prompt-level craft, not DSPy's programmatic approach	Hand-crafted vs optimized