Lecture Series on Tiling Agents
For my AISC, I’ll[1] be presenting more details about the research every Thursday for approximately the next three months. If you are interested in listening in, here is a calendar link.
EDIT:
The calendar link apparently doesn’t invite people to the recurring event; I’m not sure I can do that with google calendar unfortunately. The subsequent meetings will be at the same time-slot each week and can be attended via this link:
https://meet.google.com/bwp-nkck-ros
- ^
Maybe there will be guest speakers at some point, EG, the AISC mentees.
Mod note: I’ve put this on Personal rather than Frontpage. I imagine the content of these talks will be frontpage content, but event announcements in general are not.
Are there any plans to have writtten materials in parallel ?
I’m hopeful about it, but preparing the lectures alone will be a lot of work (although the first one will be a repeat of some material presented at ILIAD).
Mmm. You are entering the Cyborg Era. The only ideas you may take to the next epoch are those that can be uploaded to the machine intelligence.
Entering, but not entered. The machines do not yet understand the prompts I write them. (Seriously, it’s total garbage still, even with lots of high quality background material in the context.)
skills issue.
prep for the model that is coming tomorrow not the model of today
I’m seeing some agreement-upvotes of Alexander here so I am curious for people to explain the skill issue I am having.
Those people will probably not see this so wont reply.
What I can tell you is that in the last three months I went through a phase transition in my AI use and I regret not doing this ~1 year earlier.
It’s not that I didnt use AI daily before for mundane tasks or writing emails, it’s not that I didnt try a couple times to get it to solve my thesis problem (it doesn’t get it) - it’s that I failed to refrain my thinking from asking “can AI do X?” to “how can I reengineer and refactor my own workflow, even the questions I am working on so as to maximally leverage AI?”
About 6 months ago you strongly recommended that I make use of the integrated AI plugin for Overleaf (Writefull). I did try it. Its recommended edits seem quite useless to me; they always seem to flow from a desire to make the wording more normal/standard/expected in contrast to more correct (which makes some sense given the way generatie pre-training works). This is obviously useful to people with worse English, but for me, the tails come apart massively between “better” and “more normal/standard/expected”, such that all the AI suggestions are either worse or totally neutral rephrasing.
It also was surprisingly bad at helping me write LaTeX; I had a much better time asking Claude instead.
I haven’t found AI at all useful for writing emails, because the AI doesn’t know what I want to say, and taking the time to tell the AI isn’t any easier than writing it myself. AI can only help me write the boring boilerplate stuff that email recipients would skim over anyway (which I don’t want to add to my emails). AI can’t help me get info out of my head this way—it can only help me in so far as emails have a lot of low-entropy cruft. I can see how this could be useful for someone who has to write a lot of low-entropy emails, but I’m not in that situation. To some degree this could be mitigated if the LLMs had a ton of context (EG recording everything that happens on my computer), but again, only the more boring cases I think.
I’d love to restore the Abram2010 ability to crank out several multi-page emails a day on intellectual topics, but I don’t think AI is helpful towards that end yet. I haven’t tried fine-tuning on my own writing, however. (I haven’t tried fine-tuning at all.)
Similarly, LLMs can be very useful for well-established mathematics which had many examples in the training data, but get worse the more esoteric the mathematics becomes. The moment I ask for something innovative, the math becomes phony.
Across the board, LLMs seem very useful for helping people who are at the lower end of a skill ladder, but not yet very useful for people at the upper end.
So I’m curious, how did you refactor your workflow to make better use of AI?
Fair enough.
I’m still quite curious what you have found useful and how you’ve refactored your workflow to leverage AI more (such that you wish you did it a year ago).
I do use Perplexity, exa.ai and elicit as parts of my search strategy.
@abramdemski I think I’m the biggest agree vote for alexander (without me alexander would have −2 agree), and I do see this because I follow both of you on my subscribe tab.
I basically endorse Alexander’s elaboration.
On the “prep for the model that is coming tomorrow not the model of today” front, I will say that LLMs are not always going to be as dumb as they are today. Even if you can’t get them to understand or help with your work now, their rate of learning still makes them in some sense your most promising mentee, and that means trying to get as much of the tacit knowledge you have into their training data as possible (if you want them to be able to more easily & sooner build on your work). Or (if you don’t want to do that for whatever reason) just generally not being caught flat-footed once they are smart enough to help you, as all your ideas are in videos or otherwise in high context understandable-only-to-abram notes.
In the words of gwern,
Right, I strongly agree with this part.
I disagree in the sense that they’re no mentee of mine, ie, me trying to get today’s models to understand me doesn’t directly help tomorrow’s models to understand. (With the exception of the limited forms of feedback in the interface, like thumbs up/down, the impact of which I’m unsure of so it doesn’t feel like something I should deliberately spend a lot of time on.)
I also disagree in the sense that engaging with LLMs right now seems liable to produce a lot less fruits downstream, even as measured by “content that can usefully prompt an LLM later”. IE, if mentees are viewed as machines that convert time-spent-dialoging-with-me to text that is useful later, I don’t think LLMs are currently my most promising mentees.
So although I strongly agree with continuing to occasionally poke at LLMs to prep for the models that are coming soon & notice when things get better, to the extent that “most promising mentee” is supposed to imply that significant chunks of my time could be usefully spent with LLMs in the present, I disagree based on my (fairly extensive) experience.
Barring special relationships with frontier labs, this sounds functionally equivalent to trying to get my work out there for humans to understand, for now at least.
I did talk to Anthropic last year about the possibility of me providing detailed feedback on Claude’s responses (wrt my research questions), but it didn’t end up happening. The big problems I identified seemed to be things they thought would definitely get addressed in another way, so there wasn’t a mutually agreed-on value proposition (I didn’t understand what they hoped to gain, & they didn’t endorse the sorts of things I hoped to train). I got busy and moved on to other things.
I feel like this is speaking from a model I don’t understand. Are videos so bad? Video transcriptions are already a thing, and future models should be better at watching video and getting info from it. Are personal notes so bad? What sorts of actions are you recommending? I already want to write as many text posts as I can.
Don’t get me wrong, I’ve kept trying and plan to keep trying.