`normalizer`¶

Full name: tenets.core.prompt.normalizer

normalizer¶

Entity and keyword normalization utilities.

Provides lightweight normalization (case-folding, punctuation removal, singularization, lemmatization when available) and tracks variant mappings for explainability.

Classes¶

EntityNormalizer¶

Normalize entities/keywords and record variant mappings.

Functions¶

normalize_list¶

Python

normalize_list(items: List[str]) -> Tuple[List[str], Dict[str, Dict[str, List[str]]]]

Normalize a list and return unique canonicals + per-item metadata.

RETURNS	DESCRIPTION
`Tuple[List[str], Dict[str, Dict[str, List[str]]]]`	(canonicals, meta_by_original) where meta contains steps and variants.

Source code in tenets/core/prompt/normalizer.py

Python

def normalize_list(items: List[str]) -> Tuple[List[str], Dict[str, Dict[str, List[str]]]]:
    """Normalize a list and return unique canonicals + per-item metadata.

    Returns:
        (canonicals, meta_by_original) where meta contains steps and variants.
    """
    norm = EntityNormalizer()
    canonicals: List[str] = []
    meta: Dict[str, Dict[str, List[str]]] = {}

    seen = set()
    for item in items:
        res = norm.normalize(item)
        if res.canonical not in seen:
            canonicals.append(res.canonical)
            seen.add(res.canonical)
        meta[item] = {"steps": res.steps, "variants": res.variants}

    return canonicals, meta