rag-architect

[{"code":"env_var","match":"COHERE_API_KEY","path":"SKILL.md"},{"code":"api_key_mention","match":"api_key","path":"references/embedding-models.md"},{"code":"api_key_mention","match":"api-key","path":"references/embedding-models.md"},{"code":"oauth","match":"OAuth2","path":"references/rag-evaluation.md"},{"code":"oauth","match":"OAuth","path":"references/rag-evaluation.md"},{"code":"install_cmd","match":"pip install ragas","path":"references/rag-evaluation.md"},{"code":"install_cmd","match":"pip install trulens-eval","path":"references/rag-evaluation.md"},{"code":"api_key_mention","match":"api_key","path":"references/retrieval-optimization.md"},{"code":"api_key_mention","match":"api-key","path":"references/retrieval-optimization.md"},{"code":"oauth","match":"OAuth2","path":"references/retrieval-optimization.md"},{"code":"api_key_mention","match":"api_key","path":"references/vector-databases.md"},{"code":"api_key_mention","match":"api-key","path":"references/vector-databases.md"},{"code":"payment","match":"Pricing","path":"references/vector-databases.md"}]

# Chunking Strategies

---

## Strategy Comparison Matrix

| Strategy | Best For | Chunk Quality | Implementation Complexity |
|----------|----------|---------------|---------------------------|
| **Fixed-size** | Simple documents, logs | Low-Medium | Simple |
| **Recursive character** | General text, articles | Medium | Simple |
| **Sentence-based** | Conversational, Q&A | Medium-High | Medium |
| **Semantic** | Technical docs, manuals | High | Medium |
| **Document-aware** | Structured content (MD, HTML) | High | Medium |
| **Agentic/Contextual** | Complex documents | Very High | Complex |
| **Late chunking** | Long-context embeddings | High | Medium |

---

## When to Use Each Strategy

### Fixed-Size Chunking
```
Best For:
- Log files and structured data
- Quick prototyping
- When content has no natural structure
- Baseline comparison

When to Avoid:
- Technical documentation
- Content with semantic units (paragraphs, sections)
- When context preservation matters
```

### Recursive Character Splitting
```
Best For:
- General articles and blog posts
- Mixed content types
- Default starting point for most RAG
- LangChain/LlamaIndex default

When to Avoid:
- Highly structured docu

# Embedding Models

---

## Model Comparison Matrix

| Model | Dimensions | Max Tokens | Strengths | Provider |
|-------|------------|------------|-----------|----------|
| **text-embedding-3-large** | 3072 (or 256-3072) | 8191 | Best quality, flexible dims | OpenAI |
| **text-embedding-3-small** | 1536 (or 256-1536) | 8191 | Cost-effective, good quality | OpenAI |
| **embed-english-v3.0** | 1024 | 512 | Excellent compression, fast | Cohere |
| **embed-multilingual-v3.0** | 1024 | 512 | 100+ languages | Cohere |
| **voyage-large-2** | 1536 | 16000 | Long context, code-aware | Voyage AI |
| **voyage-code-2** | 1536 | 16000 | Code retrieval specialist | Voyage AI |
| **BGE-large-en-v1.5** | 1024 | 512 | Open source, high quality | BAAI |
| **BGE-M3** | 1024 | 8192 | Multi-lingual, multi-granularity | BAAI |
| **E5-large-v2** | 1024 | 512 | Strong benchmark performance | Microsoft |
| **GTE-large** | 1024 | 512 | Good general-purpose | Alibaba |
| **all-MiniLM-L6-v2** | 384 | 256 | Fast, lightweight | Sentence Transformers |
| **nomic-embed-text-v1.5** | 768 | 8192 | Long context, open weights | Nomic AI |

---

## When to Use Each Model

### OpenAI text-embedding-3-large
```
Best For

# RAG Evaluation

---

## Evaluation Framework Overview

| Framework | Focus | Strengths | Use Case |
|-----------|-------|-----------|----------|
| **RAGAS** | RAG-specific metrics | Faithfulness, relevance | Production RAG evaluation |
| **TruLens** | LLM app observability | Tracing, feedback functions | Debugging and monitoring |
| **LangSmith** | LangChain ecosystem | Traces, datasets, testing | LangChain projects |
| **Custom** | Specific requirements | Full control | Domain-specific needs |

---

## Core Metrics

### Retrieval Metrics

| Metric | Formula | What It Measures |
|--------|---------|------------------|
| **Precision@k** | Relevant in top-k / k | Are retrieved docs relevant? |
| **Recall@k** | Relevant in top-k / Total relevant | Did we get all relevant docs? |
| **MRR** | 1 / Rank of first relevant | How quickly do we find relevant? |
| **NDCG@k** | DCG@k / IDCG@k | Is ranking order correct? |
| **Hit Rate** | Queries with relevant in top-k / Total queries | Binary success rate |

### Generation Metrics

| Metric | What It Measures |
|--------|------------------|
| **Faithfulness** | Is answer grounded in retrieved context? |
| **Answer Relevance** | Does answer a

# Retrieval Optimization

---

## Optimization Techniques Overview

| Technique | Impact | Complexity | When to Use |
|-----------|--------|------------|-------------|
| **Hybrid Search** | High | Medium | Always for production |
| **Reranking** | High | Low | Top-k refinement |
| **Query Expansion** | Medium | Medium | Ambiguous queries |
| **HyDE** | Medium-High | Medium | Concept-heavy retrieval |
| **Metadata Filtering** | High | Low | Multi-tenant, categorical |
| **Query Decomposition** | Medium | High | Complex questions |
| **Contextual Compression** | Medium | Medium | Long retrieved chunks |

---

## Hybrid Search (Vector + Keyword)

### Reciprocal Rank Fusion (RRF)

```python
from dataclasses import dataclass
from typing import Callable

@dataclass
class SearchResult:
    id: str
    text: str
    score: float
    source: str  # "vector" or "keyword"

def reciprocal_rank_fusion(
    vector_results: list[SearchResult],
    keyword_results: list[SearchResult],
    k: int = 60,
    vector_weight: float = 0.5
) -> list[SearchResult]:
    """
    Combine vector and keyword results using RRF.
    k is a constant that reduces the impact of high rankings (typically 60).
    """

# Vector Databases

---

## Database Comparison Matrix

| Feature | Pinecone | Weaviate | Qdrant | Chroma | pgvector |
|---------|----------|----------|--------|--------|----------|
| **Hosting** | Managed only | Managed + Self-hosted | Managed + Self-hosted | Self-hosted (cloud beta) | Self-hosted |
| **Hybrid Search** | Yes (sparse-dense) | Yes (BM25 + vector) | Yes (sparse vectors) | Limited | Manual (+ pg_trgm) |
| **Filtering** | Excellent | Excellent | Excellent | Basic | SQL-native |
| **Max Dimensions** | 20,000 | Unlimited | 65,535 | Unlimited | 2,000 |
| **Pricing Model** | Per-vector/query | Per-node | Per-node | Free (OSS) | Free (extension) |
| **Multi-tenancy** | Namespaces | Multi-tenant class | Collections + payloads | Collections | Schema/RLS |
| **Best For** | Enterprise SaaS | Semantic apps | High-performance | Prototyping | Postgres shops |

## When to Use Each

### Pinecone
```
Best For:
- Enterprise RAG with strict SLAs
- Teams wanting zero infrastructure management
- Applications needing sparse-dense hybrid search
- High-volume production with predictable costs

When to Avoid:
- Cost-sensitive projects (expensive at scale)
- Need for self-hosting or data resi

---
name: rag-architect
description: Designs and implements production-grade RAG systems by chunking documents, generating embeddings, configuring vector stores, building hybrid search pipelines, applying reranking, and evaluating retrieval quality. Use when building RAG systems, vector databases, or knowledge-grounded AI applications requiring semantic search, document retrieval, context augmentation, similarity search, or embedding-based indexing.
license: MIT
metadata:
  author: https://github.com/Jeffallan
  version: "1.1.0"
  domain: data-ml
  triggers: RAG, retrieval-augmented generation, vector search, embeddings, semantic search, vector database, document retrieval, knowledge base, context retrieval, similarity search
  role: architect
  scope: system-design
  output-format: architecture
  related-skills: python-pro, database-optimizer, monitoring-expert, api-designer
---

# RAG Architect

## Core Workflow

1. **Requirements Analysis** — Identify retrieval needs, latency constraints, accuracy requirements, and scale
2. **Vector Store Design** — Select database, schema design, indexing strategy, sharding approach
3. **Chunking Strategy** — Document splitting, overlap, semant