Development Guide¶

This guide provides instructions for setting up your development environment, running tests, and contributing to the Tenets project.

1. Initial Setup¶

Prerequisites¶

• Python 3.9+
• Git
• An activated Python virtual environment (e.g., venv, conda).

Fork and Clone¶

1. Fork the repository on GitHub.
2. Clone your fork locally:
   Bash

   git clone https://github.com/jddunn/tenets.git
   cd tenets
   

Install Dependencies¶

Install the project in "editable" mode along with all development dependencies. This allows you to modify the source code and have the changes immediately reflected.

pip install -e ".[all,dev]"

Set up Pre-Commit Hooks¶

This project uses pre-commit to automatically run linters and formatters before each commit.

pre-commit install

Alternative Installs¶

If you only need core + dev tooling (faster):

pip install -e ".[dev]"

pip install -e .

Verifying the CLI¶

tenets --version
tenets --help | head

If the command is not found, ensure your virtualenv is activated and that the scripts (Windows) or bin (Unix) directory is on PATH.

1.1 Building Distribution Artifacts (Optional)¶

You typically do NOT need to build wheels / sdists for day‑to‑day development; the editable install auto-reflects code edits. Build only when testing packaging or release steps.

python -m build               # creates dist/*.whl and dist/*.tar.gz
pip install --force-reinstall dist/tenets-*.whl  # sanity check install

To inspect what went into the wheel:

unzip -l dist/tenets-*.whl | grep analysis/implementations | head

1.2 Clean Environment Tasks¶

pip cache purge        # optional: clear wheel cache
find . -name "__pycache__" -exec rm -rf {} +
rm -rf .pytest_cache .ruff_cache .mypy_cache build dist *.egg-info

1.3 Using Poetry Instead of pip (Optional)¶

Poetry can manage the virtual environment and extras if you prefer:

poetry install -E all -E dev   # full feature + dev toolchain
poetry run pytest              # run tests
poetry run tenets --help       # invoke CLI

poetry update

poetry add --optional rich

## 2. Running Tests

The test suite uses `pytest`. We have a comprehensive configuration in `pytest.ini` that handles most settings automatically.

### Running All Tests
To run the entire test suite:
```bash
pytest

Running Tests with Coverage¶

To generate a test coverage report:

pytest --cov

To view the interactive HTML report:

# On macOS
open htmlcov/index.html

# On Windows
start htmlcov/index.html

# On Linux
xdg-open htmlcov/index.html

3. Required / Optional Secrets¶

Configure these in GitHub: Settings → Secrets and variables → Actions.

Additional environment driven configs (rarely needed): | Variable | Effect | |----------|-------| | TENETS_CACHE_DIRECTORY | Override default cache directory | | TENETS_DEBUG | Enables verbose debug logging when true |

Security tips: - Grant least privilege (PyPI token scoped to project if possible) - Rotate any credentials annually or on role changes - Prefer Trusted Publishing over long‑lived API tokens once stable

4. Code Style and Linting¶

We use ruff for linting and formatting. The pre-commit hook runs it automatically, but you can also run it manually:

# Check for linting errors
ruff check .

# Automatically fix linting errors
ruff check . --fix

# Format the code
ruff format .

5. Building Documentation¶

The documentation is built using MkDocs with the Material theme.

Installing Documentation Dependencies¶

# Install MkDocs and theme
pip install mkdocs mkdocs-material

# Or if you installed with dev dependencies, it's already included:
pip install -e ".[dev]"

Serving Documentation Locally¶

FAST Development Mode (Recommended for editing docs)¶

# Use the lightweight dev config with dirty reload for FASTEST iteration
mkdocs serve -f mkdocs.dev.yml --dirtyreload

# Without dirty reload (still faster than full build)
mkdocs serve -f mkdocs.dev.yml

mkdocs.dev.yml differences: - Disables heavy plugins: No API generation, no mkdocstrings, no minification - Faster rebuilds: Skips expensive operations - Dirty reload: Only rebuilds changed pages (not entire site) - Perfect for: Writing/editing documentation content

Full Production Mode (for testing final output)¶

# Full build with all features including API docs generation
mkdocs serve

# Serve on a different port
mkdocs serve -a localhost:8080

# Serve with verbose output for debugging
mkdocs serve --verbose

# With clean rebuild
mkdocs serve --clean

The development server includes: - Live reload: Changes to docs files automatically refresh the browser - API docs generation: Auto-generates from Python docstrings - Full theme features: All navigation and search features enabled

Building Static Documentation¶

# Build the static site to site/ directory
mkdocs build

# Build with strict mode (fails on warnings)
mkdocs build --strict

# Build with verbose output
mkdocs build --verbose

# Clean build (removes old files first)
mkdocs build --clean

Documentation Structure¶

docs/
├── index.md           # Homepage
├── overrides/        # Custom HTML templates
│   └── home.html     # Custom homepage
├── styles/           # Custom CSS
│   ├── main.css
│   ├── search.css
│   └── ...
├── assets/           # Images and screenshots
│   └── images/
└── *.md             # Documentation pages

API Documentation Generation¶

The API documentation is auto-generated from Python docstrings using mkdocstrings and gen-files plugins.

How it works:¶

1. docs/gen_api.py script runs during build:
2. Scans all Python modules in tenets/
3. Generates markdown files with ::: mkdocstrings syntax

4. Creates navigation structure in api/ directory

5. mkdocstrings plugin processes the generated files:

6. Extracts docstrings from Python code
7. Renders them as formatted documentation
8. Includes type hints, parameters, returns, examples

Regenerating API docs:¶

# Full build with API generation (automatic)
mkdocs build

# Or serve with API generation
mkdocs serve  # Uses mkdocs.yml which has gen-files enabled

# Skip API generation for faster dev
mkdocs serve -f mkdocs.dev.yml --dirtyreload

Writing Good Docstrings for API docs:¶

def example_function(param1: str, param2: int = 0) -> bool:
    """Short summary of what this function does.

    Longer description with more details about the function's
    behavior, use cases, and any important notes.

    Args:
        param1: Description of first parameter
        param2: Description of second parameter (default: 0)

    Returns:
        Description of return value

    Raises:
        ValueError: When something goes wrong

    Example:
        >>> example_function("test", 42)
        True
    """

Making Documentation Changes¶

1. For content/markdown: Edit files in docs/ directory
2. For API docs: Update docstrings in Python source files
3. Preview changes:
4. Fast: mkdocs serve -f mkdocs.dev.yml --dirtyreload
5. Full: mkdocs serve
6. Test the build: mkdocs build --strict
7. Check for broken links in the browser console

Deploying Documentation¶

# Deploy to GitHub Pages (requires push permissions)
mkdocs gh-deploy

# Deploy with custom commit message
mkdocs gh-deploy -m "Update documentation"

# Deploy without pushing (dry run)
mkdocs gh-deploy --no-push

The site will be available at https://[username].github.io/tenets/.

2. Making Changes¶

Follow the coding standards: - Write clean, readable code - Add comprehensive docstrings (Google style) - Include type hints for all functions - Write tests for new functionality

3. Committing Changes¶

We use Conventional Commits:

# Interactive commit
make commit  # or: cz commit

# Manual commit (must follow format)
git commit -m "feat(analyzer): add support for Rust AST parsing"

Commit types: - feat: New feature - fix: Bug fix - docs: Documentation changes - style: Code style changes (formatting) - refactor: Code refactoring - perf: Performance improvements - test: Test additions or changes - chore: Maintenance tasks

4. Running Tests¶

# Run all tests
make test

# Run fast tests only
make test-fast

# Run specific test file
pytest tests/test_analyzer.py

# Run with coverage
pytest --cov=tenets --cov-report=html

5. Code Quality Checks¶

# Run all checks
make lint

# Auto-format code
make format

# Individual tools
black .
isort .
ruff check .
mypy tenets --strict
bandit -r tenets

6. Pushing Changes¶

# Pre-commit hooks will run automatically
git push origin feature/your-feature-name

7. Creating a Pull Request¶

1. Go to GitHub and create a PR
2. Fill out the PR template
3. Ensure all CI checks pass
4. Request review from maintainers

Testing¶

Test Structure¶

tests/
├── unit/              # Unit tests
│   ├── test_analyzer.py
│   ├── test_nlp.py
│   └── test_scanner.py
├── integration/       # Integration tests
│   ├── test_cli.py
│   └── test_workflow.py
├── fixtures/         # Test data
│   └── sample_repo/
└── conftest.py      # Pytest configuration

Writing Tests¶

"""Test module for analyzer functionality."""

import pytest
from tenets.core.analysis import CodeAnalyzer


class TestCodeAnalyzer:
    """Test suite for CodeAnalyzer."""

    @pytest.fixture
    def analyzer(self):
        """Create analyzer instance."""
        return CodeAnalyzer()

    def test_analyze_python_file(self, analyzer, tmp_path):
        """Test Python file analysis."""
        # Create test file
        test_file = tmp_path / "test.py"
        test_file.write_text("def hello():\n    return 'world'")

        # Analyze
        result = analyzer.analyze_file(test_file)

        # Assertions
        assert result.language == "python"
        assert len(result.functions) == 1
        assert result.functions[0]["name"] == "hello"

Test Markers¶

# Run only unit tests
pytest -m unit

# Skip slow tests
pytest -m "not slow"

# Run tests requiring git
pytest -m requires_git

Code Quality¶

Style Guide¶

We follow PEP 8 with these modifications: - Line length: 100 characters - Use Black for formatting - Use Google-style docstrings

Type Hints¶

All functions must have type hints:

from typing import List, Optional, Dict, Any


def analyze_files(
    paths: List[Path],
    deep: bool = False,
    max_workers: Optional[int] = None
) -> Dict[str, Any]:
    """
    Analyze multiple files in parallel.

    Args:
        paths: List of file paths to analyze
        deep: Whether to perform deep analysis
        max_workers: Maximum number of parallel workers

    Returns:
        Dictionary containing analysis results
    """
    ...

Docstrings¶

Use Google-style docstrings:

def calculate_relevance(
    file: FileAnalysis,
    prompt: PromptContext,
    algorithm: str = "balanced"
) -> float:
    """
    Calculate relevance score for a file.

    Uses multi-factor scoring to determine how relevant a file is
    to the given prompt context.

    Args:
        file: Analyzed file data
        prompt: Parsed prompt context
        algorithm: Ranking algorithm to use

    Returns:
        Relevance score between 0.0 and 1.0

    Raises:
        ValueError: If algorithm is not recognized

    Example:
        >>> relevance = calculate_relevance(file, prompt, "thorough")
        >>> print(f"Relevance: {relevance:.2f}")
        0.85
    """
    ...

Documentation¶

Building Documentation¶

# Build docs
make docs

# Serve locally
make serve-docs
# Visit http://localhost:8000

Writing Documentation¶

1. API Documentation: Auto-generated from docstrings
2. User Guides: Written in Markdown in docs/
3. Examples: Include code examples in docstrings

Documentation Standards¶

• Use clear, concise language
• Include code examples
• Add diagrams where helpful
• Keep it up-to-date with code changes

Debugging¶

Debug Mode¶

# Enable debug logging
export TENETS_DEBUG=true
tenets make-context "test" . --verbose

# Or in code
import logging
logging.basicConfig(level=logging.DEBUG)

Using VS Code¶

.vscode/launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug tenets CLI",
            "type": "python",
            "request": "launch",
            "module": "tenets.cli.main",
            "args": ["make-context", "test query", "."],
            "console": "integratedTerminal"
        }
    ]
}

Common Issues¶

1. Import errors: Ensure you've installed in development mode (pip install -e .)
2. Type errors: Run mypy to catch type issues
3. Test failures: Check if you need to install optional dependencies

Contributing Guidelines¶

Before You Start¶

1. Check existing issues and PRs
2. Open an issue to discuss large changes
3. Read the architecture documentation

Code Review Checklist¶

• Tests pass locally
• Code is formatted (black, isort)
• Type hints are present
• Docstrings are complete
• Tests cover new functionality
• Documentation is updated
• Commit messages follow convention
• No security issues (bandit)

Getting Help¶

• Open an issue for bugs
• Start a discussion for features
• Join our Discord (coming soon)
• Email: team@tenets.dev

Release & Versioning¶

Releases are automated. Merging conventional commits into main (from PRs) is all you normally do.

Branch Model¶

Workflows (high level)¶

1. PR merged into main.
2. version-bump.yml runs:
   • Collects commits since last tag
   • Determines next version:
   • Major: commit body contains BREAKING CHANGE: or type suffixed with !
   • Minor: at least one feat: or perf: commit (performance treated as minor to signal impact)
   • Patch: any fix, refactor, chore (unless a higher bump already chosen)
   • Skip: only docs / test / style commits (no release)
   • Updates pyproject.toml
   • Appends a section to CHANGELOG.md grouping commits (Features / Performance / Fixes / Refactoring / Chore)
   • Commits with message chore(release): vX.Y.Z and creates annotated tag vX.Y.Z
3. Tag push triggers release.yml:
   • Builds wheel + sdist
   • Publishes to PyPI (token or Trusted Publishing)
   • (Optional) Builds & publishes Docker image (future enablement)
   • Deploys docs (if configured) / updates site
4. release-drafter (config) ensures GitHub Release notes reflect categorized changes (either via draft or final publish depending on config state).

You do NOT run cz bump manually during normal flow; the workflow handles versioning.

Conventional Commit Expectations¶

Use clear scopes where possible:

feat(ranking): add semantic similarity signal
fix(cli): prevent crash on empty directory
perf(analyzer): cache parsed ASTs
refactor(config): simplify loading logic
docs: update quickstart for --copy flag

Edge cases: - Multiple commit types: highest precedence decides (major > minor > patch) - Mixed docs + fix: still releases (fix wins) - Only docs/test/style: skipped; no tag produced

First Release (Bootstrap)¶

If no existing tag: 1. Merge initial feature set into main 2. Push a commit with feat: initial release (or similar) 3. Workflow sets version to 0.1.0 (or bump logic starting point defined in workflow)

If you need a different starting version (e.g. 0.3.0): create an annotated tag manually once, then subsequent merges resume automation.

Manual / Emergency Release¶

Only when automation is blocked:

git checkout main && git pull
cz bump --increment PATCH  # or MINOR / MAJOR
git push && git push --tags

Verifying a Release¶

After automation completes:

pip install --no-cache-dir tenets==<new_version>
tenets --version

tenets distill "smoke" --max-tokens 2000 --mode fast --stats || true

Troubleshooting¶

Philosophy¶

Keep main always shippable. Small, frequent releases reduce risk and keep context fresh for users.

Advanced Topics¶

Adding a New Language Analyzer¶

1. Create analyzer in tenets/core/analysis/:

   Python

   class RustAnalyzer(LanguageAnalyzer):
       language_name = "rust"
   
       def extract_imports(self, content: str) -> List[Import]:
           # Implementation
           ...
   

2. Register in analysis/analyzer.py:

   Python

   analyzers['.rs'] = RustAnalyzer()
   

3. Add tests in tests/unit/test_rust_analyzer.py

Creating Custom Ranking Algorithms¶

1. Implement algorithm:

   Python

   class SecurityRanking:
       def score_file(self, file, prompt):
           # Custom scoring logic
           ...
   

2. Register algorithm:

   Python

   @register_algorithm("security")
   class SecurityRanking:
       ...
   

3. Document usage in docs/api.md

Happy coding! 🚀 Remember: context is everything.