Cole Wyeth

Karma: 702

I am a PhD student in computer science at the University of Waterloo, supervised by Professor Ming Li and advised by Professor Marcus Hutter.

My current research is related to applications of algorithmic probability to sequential decision theory (universal artificial intelligence). Recently I have been trying to start a dialogue between the computational cognitive science and UAI communities (if this includes you, consider contacting me about the reading group). Sometimes I build robots, professionally or otherwise. Another hobby (and a personal favorite of my posts here) is the Sherlockian abduction master list, which is a crowdsourced project seeking to make “Sherlock Holmes” style inference feasible by compiling observational cues. Give it a read and see if you can contribute!

See my personal website colewyeth.com for an overview of my interests and work.

Cole Wyeth 20 Nov 2024 19:53 UTC
1 point
0
on: Six Plausible Meta-Ethical Alternatives
I believe 3 is about right in principle but 5 describes humans today.

Cole Wyeth 20 Nov 2024 19:45 UTC
1 point
0
in reply to: Thomas Kwa’s comment on: Probability is Real, and Value is Complex
I don’t think this proves probability and utility are inextricable. I prefer Jaynes’ approach of motivating probabilities by coherence conditions on beliefs—later, he notes that utility and probability are on equal footing in decision theory as explained in this post, but (as far as I remember) ultimately decides that he can’t carry this through to a meaningful philosophy that stands on its own. By choosing to introduce probabilities as conceptually prior, he “extricates” the two in a way that seems perfectly sensible to me.

Cole Wyeth 20 Nov 2024 17:17 UTC
3 points
0
on: The Obliqueness Thesis
I think that at least the weak orthogonality thesis survives these arguments in the sense that any coherent utility function over an ontology “closely matching” reality should in principle be reachable for arbitrarily intelligent agents, along some path of optimization/learning. Your only point that seems to contradict this is the existence of optimization daemons, but I’m confident that an anti-daemon immune system can be designed, so any agent that chooses to design itself in a way where it can be overtaken by daemons must do this with the knowledge that something close to its values will still be optimized for—so this shouldn’t cause much observable shift in values.
It’s unclear how much measure is assigned to various “final/limiting” utility functions by various agent construction schemes—I think this is far beyond our current technical ability to answer.
Personally, I suspect that the angle is more like 60 degrees, not 3.

Cole Wyeth 19 Nov 2024 23:01 UTC
8 points
0
in reply to: brambleboy’s comment on: Social events with plausible deniability
“Cancel culture is good actually” needs to go in the hat ;)

Cole Wyeth 19 Nov 2024 17:32 UTC
0 points
−4
in reply to: jenn’s comment on: Social events with plausible deniability
You may be right that the benefits are worth the costs for some people, but I think if you have access to a group interested in doing social events with plausible deniability, that group is probably already a place where you should be able to be honest about your beliefs without fear of “cancellation.” Then it is preferable to practice (and expect) the moral courage / accountability / honesty of saying what you actually believe and defending it within that group. If you don’t have a group of people interested in doing social events with plausible deniability, you probably can’t do them and this point is mute. So I’m not sure I understand the use case—you have a friend group that is a little cancel-ish but still interested in expressing controversial beliefs? That sounds like something that is not a rationalist group (or maybe I am spoiled by the culture of Jenn’s meetups).

Cole Wyeth 19 Nov 2024 2:30 UTC
2 points
−9
on: Social events with plausible deniability
This kind of thing does justified harm to our community’s reputation. If you have fun arguing that only white people can save us while deliberately obfuscating whether you actually believe that, it is in fact a concerning sign about your intentions/seriousness/integrity/trustworthiness.

Cole Wyeth 18 Nov 2024 1:41 UTC
3 points
6
on: “The Solomonoff Prior is Malign” is a special case of a simpler argument
I don’t believe that these anthropic considerations actually apply, either to us, to oracles, or to Solomonoff induction. The arguments are too informal, it’s very easy to miscalculate Kolmogorov complexities and the measures assigned by the universal distribution using intuitive gestures like this. However I do think that this is a correct generalization of the idea of a malign prior, and I actually appreciate that you wrote it up this way because it makes clear that none of the load-bearing parts of the argument actually rely on reliable calculations (invocations of algorithmic information theory concepts have no been reduced to rigorous math, so the original argument is not stronger than this one).

Cole Wyeth 15 Nov 2024 15:37 UTC
1 point
0
in reply to: Viliam’s comment on: Heresies in the Shadow of the Sequences
My impression is that e.g. the Catholic church has a pretty deeply thought out moral philosophy that has persisted across generations. That doesn’t mean that every individual Catholic understands and executes it properly.

Cole Wyeth 14 Nov 2024 22:44 UTC
2 points
0
in reply to: Milan W’s comment on: Heresies in the Shadow of the Sequences
- Perhaps Legg-Hutter intelligence.
- I’m not sure how much the goal matters—probably the details depend on the utility function you want to optimize. I think you can do about as well as possible by carving out a utility function module and designing the rest uniformly to pursue the objectives of that module. But perhaps this comes at a fairly significant cost (i.e. you’d need a somewhat larger computer to get the same performance if you insist on doing it this way).
- ...And yes, there does exist a computer program which is remarkably good at just chess and nothing else, but that’s not the kind of thing I’m talking about here.
- Yes, the I/O channels should be fixed along with the hardware.

Cole Wyeth 14 Nov 2024 20:49 UTC
2 points
0
in reply to: jbash’s comment on: Heresies in the Shadow of the Sequences
The standard method for training LLM’s is next token prediction with teacher-forcing, penalized by the negative log-loss. This is exactly the right setup to elicit calibrated conditional probabilities, and exactly the “prequential problem” that Solomonoff induction was designed for. I don’t think this was motivated by decision theory, but it definitely makes perfect sense as an approximation to Bayesian inductive inference—the only missing ingredient is acting to optimize a utility function based on this belief distribution. So I think it’s too early to suppose that decision theory won’t play a role.

Cole Wyeth 11 Nov 2024 19:36 UTC
1 point
0
AF
on: Radical Probabilism
What would you have to see proven about Solomonoff induction to conclude it does not have convergence/calibration problems? My friend Aram Ebtekar has worked on showing it converges to 50% on adversarial sequences.

Cole Wyeth 11 Nov 2024 16:37 UTC
8 points
8
on: Cole Wyeth’s Shortform
Perhaps LLM’s are starting to approach the intelligence of today’s average human: capable of only limited original thought, unable to select and autonomously pursue a nontrivial coherent goal across time, learned almost everything they know from reading the internet ;)

Cole Wyeth 11 Nov 2024 12:33 UTC
1 point
0
in reply to: cubefox’s comment on: Cole Wyeth’s Shortform
No that seems paywalled, curious though?

Cole Wyeth 9 Nov 2024 19:46 UTC
2 points
0
on: Inverse Problems In Everyday Life
An example I’ve been studying obsessively: https://www.lesswrong.com/posts/Yz33koDN5uhSEaB6c/sherlockian-abduction-master-list

Cole Wyeth 9 Nov 2024 19:32 UTC
1 point
0
on: Could randomly choosing people to serve as representatives lead to better government?
How do you suggest advocating for this effectively?

Cole Wyeth 9 Nov 2024 19:19 UTC
1 point
0
in reply to: habryka’s comment on: Cole Wyeth’s Shortform
I’m in Canada so can’t access the latest Claude, so my experience with these things does tend to be a couple months out of date. But I’m not really impressed with models spitting out slightly wrong code that tells me what functions to call. I think this is essentially a more useful search engine.

Cole Wyeth 9 Nov 2024 18:34 UTC
1 point
0
in reply to: eigen’s comment on: Cole Wyeth’s Shortform
I’ve noticed occasional surprises in that direction, but none of them seem to shake out into utility for me.

Cole Wyeth 9 Nov 2024 18:31 UTC
4 points
0
on: Graceful Degradation
Semi-interestingly, my MMA school taught that it’s best for the punch to arrive before the leading foot lands so that the punch carries your full weight. Many people at advanced levels weren’t aware of this because we did not introduce it right away—if you try to do this before learning a few other details (and building strength), you run a risk of hurting your wrist by punching too hard.

Cole Wyeth 9 Nov 2024 17:45 UTC
1 point
0
in reply to: Cole Wyeth’s comment on: Cole Wyeth’s Shortform
I’ve been waiting to say this until OpenAI’s next larger model dropped, but this has now failed to happen for so long that it’s become it’s own update, and I’d like to state my prediction before it becomes obvious.

Cole Wyeth 9 Nov 2024 17:42 UTC
8 points
−2
on: Cole Wyeth’s Shortform
This doesn’t seem to be reflected in the general opinion here, but it seems to me that LLM’s are plateauing and possibly have already plateaued a year or so ago. Scores on various metrics continue to go up, but this tends to provide weak evidence because they’re heavily gained and sometimes leak into the training data. Still, those numbers overall would tend to update me towards short timelines, even with their unreliability taken into account—however, this is outweighed by my personal experience with LLM’s. I just don’t find them useful for practically anything. I have a pretty consistently correct model of the problems they will be able to help me with and it’s not a lot—maybe a broad introduction to a library I’m not familiar with or detecting simple bugs. That model has worked for a year or two without expanding the set much. Also, I don’t see any applications to anything economically productive except for fluffy chatbot apps.