Thanks for your comment, I took a look at your example, but i’d say that is addressing a different issue—constrained output tokens, not ingestion of input tokens. I also wanted to avoid scaffolding approaches since i’m zero shotting, I don’t want to use a chained series of prompts or chunking, I want to submit a single prompt.
I’m looking for any techniques similar to including an index of the prompt sections (like in a book with a list of the chapters) for the prompt and some character strings that differentiate the prompt’s sections. Here’s an example of the top of my prompt:
Time Now: 2025-05-09 21:46:07
=== System Context === Character Count: 5903
1. INTRODUCTION
[intro text]
2. SYSTEM STATE AND PROMPT STRUCTURE
When you run, the prompt sent to the LLM includes a detailed description of your current state and operational context. This ‘self’ is assembled from various dynamic and static sources. Below is a list of the key dynamic sections derived from your state files and other data sources, along with how they are processed for the prompt:
=== Your Goals ===
Source:
state_files/goals.json
Content: All current goals.
=== Previous Thought ===
Source:
state_files/previous_thought.txt
Content: The full ‘thought’ section from your previous run’s LLM output. This file is overwritten each run.
=== Previous Actions and Outcomes ===
Source:
state_files/previous_actions_outcomes.json
Content: The actions you decided to take in the previous run and the outcomes of executing them. This file is overwritten each run.
So the prompt includes the what sections are present throughout and what characters separate the sections: “=== prompt section title ===”.
This technique improves coherence over long context windows.
Here’s the Replit CEO Amjad Masad confirming what i’ve seen (timestamp: 36:45). “After 32k tokens, reasoning and a lot of benchmarks tank”