Prompt Learning

Prompt Learning is a Python SDK and CLI that improves an LLM prompt from natural-language feedback rather than numeric scores. It builds on meta prompting, where an LLM rewrites prompts, but replaces scalar signals such as pass or fail with textual feedback: annotations, rule reminders, and explanations of what went wrong. The prompt changes while the model weights stay fixed, so the approach also applies after deployment.

The loop uses three roles. An agent runs the task with the current prompt, an evaluator LLM finds failures and writes feedback, and a meta-prompt LLM revises the prompt from that feedback. You supply a dataset with input variables, the model's outputs, and one or more feedback columns, or you run your own evaluator functions to generate the feedback first. Optional annotations analyze why errors happened to guide the rewrite.

Besides whole prompts, it can optimize a ruleset, which suits coding agents whose behavior is steered by a list of rules. Prompts can be plain strings, message lists, or Phoenix prompt versions. Datasets are split into token-aware batches to fit the model's context window, and a budget in US dollars caps spending, with per-batch and running cost estimates in verbose mode.

A separate image command generates several images from a prompt with Google's image models and can collect human feedback, and an image evaluator uses Gemini vision to score prompt adherence, visual quality, artistic appeal, and consistency.