ollie_

• ollie_May 13, 2025, 4:41 PM

  1 point

  0

  on: How do I de­sign long prompts for think­ing zero shot sys­tems with dis­tinct equally dis­tributed prompt sec­tions (mis­sion, goals, mem­o­ries, how-to-re­spond,… etc) and how to main­tain llm co­her­ence?

  Here’s the Replit CEO Amjad Masad confirming what i’ve seen (timestamp: 36:45). “After 32k tokens, reasoning and a lot of benchmarks tank”

• ollie_May 12, 2025, 11:42 AM

  3 points

  0

  in reply to: mishka’s comment on: How do I de­sign long prompts for think­ing zero shot sys­tems with dis­tinct equally dis­tributed prompt sec­tions (mis­sion, goals, mem­o­ries, how-to-re­spond,… etc) and how to main­tain llm co­her­ence?

  Thanks for your comment, I took a look at your example, but i’d say that is addressing a different issue—constrained output tokens, not ingestion of input tokens. I also wanted to avoid scaffolding approaches since i’m zero shotting, I don’t want to use a chained series of prompts or chunking, I want to submit a single prompt.

  I’m looking for any techniques similar to including an index of the prompt sections (like in a book with a list of the chapters) for the prompt and some character strings that differentiate the prompt’s sections. Here’s an example of the top of my prompt:

  Time Now: 2025-05-09 21:46:07

  === System Context === Character Count: 5903

  1. INTRODUCTION

  [intro text]

  2. SYSTEM STATE AND PROMPT STRUCTURE

  When you run, the prompt sent to the LLM includes a detailed description of your current state and operational context. This ‘self’ is assembled from various dynamic and static sources. Below is a list of the key dynamic sections derived from your state files and other data sources, along with how they are processed for the prompt:

  === Your Goals ===

  Source: state_files/goals.json

  Content: All current goals.

  === Previous Thought ===

  Source: state_files/previous_thought.txt

  Content: The full ‘thought’ section from your previous run’s LLM output. This file is overwritten each run.

  === Previous Actions and Outcomes ===

  Source: state_files/previous_actions_outcomes.json

  Content: The actions you decided to take in the previous run and the outcomes of executing them. This file is overwritten each run.

  So the prompt includes the what sections are present throughout and what characters separate the sections: “=== prompt section title ===”.

  This technique improves coherence over long context windows.

[Question] How do I de­sign long prompts for think­ing zero shot sys­tems with dis­tinct equally dis­tributed prompt sec­tions (mis­sion, goals, mem­o­ries, how-to-re­spond,… etc) and how to main­tain llm co­her­ence?

• ollie_Jan 31, 2025, 3:52 PM

  3 points

  4

  on: Deep­Seek: Don’t Panic

  I believe that opensource advancements like R1 will drive wider adoption of ai systems.

  I think that the pricing models will change soon. Everyone talks about cost per million tokens to contact a hosted service, but I think it’ll switch to be cloud costs to provide infrastructure that can run models. Virtual machines running something like ollama.

  This solves another huge problem, privacy and how prompt data is handled. If you’re using an api to a hosted service you need to have a very good understanding of how your submitted prompt data is handled. This is key for organisations. I feel like the lack of understanding here is preventing widespread adoption, especially for communication tools that handle sensitive data.

  For example, you could run Deepseek R1 using ollama on an Azure virtual machine (nc series) that you pay per hour for, and then your cost isn’t based on usage of your ai. Right now it’s expensive to provision the infra to support decent models, but these costs fall continuously.

  I can imagine a world where organisations provision cloud infrastructure in their environments running open source models.

  https://​​learn.microsoft.com/​​en-us/​​azure/​​virtual-machines/​​sizes/​​gpu-accelerated/​​nc-series?tabs=sizebasic

  https://​​huggingface.co/​​deepseek-ai/​​DeepSeek-R1

• ollie_Jan 5, 2025, 4:27 PM

  1 point

  0

  on: How i’m build­ing my ai sys­tem, how it’s go­ing so far, and my thoughts on it

  If you’re downvoting, could you say why? I’m new to this site but there are very few posts on ai like my own. There’s a few on different ai tools, but what I have isn’t a tool, the output generated by it is for itself.

• ollie_Jan 5, 2025, 4:19 PM

  1 point

  0

  in reply to: CstineSublime’s comment on: How i’m build­ing my ai sys­tem, how it’s go­ing so far, and my thoughts on it

  It started out with the idea to make a system that can improve itself, without me having to prompt it to improve itself. The “question” is the prompt. And crafting the prompt is very difficult. So it’s an experiment in constructing a question that a system can use to improve itself without having to explicitly say “improve your codebase” or “give yourself more actions or freedoms” or others like that. I want the llm to conjure up ideas.

  So as the prompt increases in complexity, with different sections that could be categorised as kinds of human thought, will it get better? Can we find parallels between how the human minds works that can be translated into a prompt. A key area that developed was using the future as well as the past in the prompt. Having memories is an obvious inclusion, as is including what happened in the past runs of the program, but what wasn’t obvious was including predictions for the future. Including the ability to make long and short term predictions, and have it be able to change these, and record the outcomes, saw big improvements in the directions it would take over time. It also seemed to ‘ground it’. Without the predictions space, it became overly concerned with its performance metrics as a proxy for improvement and it began over-optimising.

  Defining ‘better’ or ‘improving’ is very difficult. Right now, i’m using token input size growth whilst maintaining clarity of thought as the rough measure.

How i’m build­ing my ai sys­tem, how it’s go­ing so far, and my thoughts on it