PROPANE: Prompt design as an inverse problem

Rimon Melamed1, Lucas H. McCabe1,3, Tanay Wakhare2, Yejin Kim1
H. Howie Huang1, Enric Boix-Adsera2
1George Washington University   2Massachusetts Institute of Technology   3LMI
Paper Code

Abstract

Carefully-designed prompts are key to inducing desired behavior in Large Language Models (LLMs). As a result, much human labor is dedicated to engineering prompts that tailor LLMs to specific use-cases, raising the question of whether this process can be automated. In this work, we propose an automatic prompt optimization framework, PROPANE, which requires no user intervention and solves the inverse problem of finding a prompt that induces semantically similar outputs to a fixed set of examples. We further demonstrate that PROPANE can be used to (a) improve existing prompts, and (b) discover semantically obfuscated prompts that transfer between models.

BibTeX

@misc{melamed2023propane,
        title={PROPANE: Prompt design as an inverse problem},
        author={Rimon Melamed and Lucas H. McCabe and Tanay Wakhare and Yejin Kim and H. Howie Huang and Enric Boix-Adsera},
        year={2023},
        eprint={2311.07064},
        archivePrefix={arXiv},
        primaryClass={cs.CL}
  }