Old man 1: Life is one trouble after another. I’d be better off dead, better yet, I wish I was never born
Old man 2: True, true, but who has such luck ?.. maybe one in a thousand.
My blog: https://cerebralab.com
I’m also building an open source generic ML library: https://github.com/mindsdb/mindsdb & https://github.com/mindsdb/lightwood …. which I guess might be of interest to some people here
I’m not arguing you should take any position on those axi, I am just suggesting them as potential axi.
I think that falling on one extreme of the spectrum is equivalent to thinking the spectrum doesn’t exist—so yes, I guess people that are very aligned with a MIRI style position on AI wouldn’t even find the spectrum valid or useful. Much like, say, an atheist wouldn’t find a “how much you believe in the power of prayer” spectrum insightful or useful. This was not something I considered while originally writing this, but even with it in mind now, I can’t think of any way I could address it.
In-so-far as your object level arguments against the spectrums I present being valid and/or of one extreme being nonsensical, I can’t say that, right now, I could say anything of much value on those topics that you haven’t probably already considered yourself.
To address your later point, I doubt I fall into that particular fallacy. Rather, I’d say, I’m on the opposite spectrum where I’d consider most people and institutions to be beyond incompetent.
Hence why I’ve reached the conclusion that improving on rationally legible metrics seems low ROI, because otherwise rationlandia would have arisen and ushered prosperity and unimaginable power in a seemingly dumb world.
But I think that’s neither here nor there, as I said, I’m really not trying to argue my view here is correct, I’m trying to figure out why wide differences in view in both directions exist.
A negative utilitarian could easily judge that something that had the side effect of making people infertile would cause far less suffering than not doing it, causing immense real world suffering amongst the people who wanted to have kids, and ending civilizations. If they were competent enough, or the problem slightly easier than expected, they could use a disease that did that without obvious symptoms, and end humanity.
But you’re thinking of people completely dedicated to an ideology.
That’s why I’m saying a “negative utilitarian charter” rather than “a government formed of people autistically following a philosophy”… much like, e.g. the US government has a “liberal democratic” charter, or the USSR had a “communist” charter of sorts.
In practice these things don’t come about because member in the organization disagree, secret leak, conspiracies are throttled by lack of consensus, politicians voted out, engineered solutions imperfect (and good engineers and scinetists are aware of as much)
For gwern’s specific story, I agree it’s somewhat implausible that one engineer (tho with access to corporate compute) trains Clippy and there’s not lots of specialized models;
I think the broader argument of “can language models become gods” is a separate one.
My sole objective there was to point out flaws in this particular narrative (which hopefully I stated clearly in the beginning).
I think the “can language models become gods” debate is broader and I didn’t care much to engage with it, superficially it seems that some of the same wrong abstractions that lead to this kind of narrative also back up that premise, but I’m in no position to make a hands-down argument for that.
The rest of your points I will try to answer later, I don’t particularly disagree with any stated that way except on the margins (e.g. GLUE is a meaningless benchmark that everyone should stop using—a weak and readable take on this would be—https://www.techtarget.com/searchenterpriseai/feature/What-do-NLP-benchmarks-like-GLUE-and-SQuAD-mean-for-developers ), but I don’t think the disagreements are particularly relevant(?)
Last time I checked that wouldn’t work for a sizeable amount. Maybe I’m wrong? I claim no expertise in crypto and, as I said, I think that’s my weakest point. In principle, I can see smart-contract-based swapping with a large liquidity pool + ETH VM sidechain being sufficient to do this.
Wouldn’t fit the exact description in the story but would server roughly the same purpose and be sufficient if you assume the ETH VM-optimized sidechain has enough volume (or a similar thing, with whatever would overthrone ETH in 20xx)
See cases above, even if you assume asymmetry (how does using banks square with that belief?), you still are left with the adversarial problem that all easy to claim exploits are taken and new exploits are usually found on the same (insecure, old) software and hardware.
So all exploitable niches are close to saturation at any given time if an incentive exists (and it does) to find them.
I would like to see actual benchmarks on that not like… a PR blog.
My general take I that T5 models that were trained on data did better than adding a head-on open source GPTs (I think it was neoX at the time, whatever the 2.7B param one was) and training that, and obviously training a GPT as a whole… this is specifically for what I looked into at the time which was language-to-code- and code-to-code translation. And the best GPT-3 of the time, with prompts, even for tasks it was specifically trained for (nl to SQL) was horrible
More broadly I’ve not seen any large language model outperform small models at specific tasks, their embedding might be better in certain cases but mainly to reduce training time.
But to keep the argument simpler: there’s little proof that out-of-sample training (MLM style or otherwise) improves in-sample performance (note, out-of-sample trained models could be better than random, which is why using pre-loaded weights is a thing)
However, the strong version of the argument is closer to:
Clippy-sized model with a specific task of finding exploits will perform better than clippy-sized model trained with a broad objective.
I think I could make that claim as, 1/1000 Clippy sized but at that point you’re haggling over benchmarks.
Cases like code completion and creation (e.g. gpt copilot) are fuzzy and have no clear way to benchmark, but even their custom-build models (e.g. the recent one from google) seem to lead on benchmarks.
That being said it’s been like ~1 year since I had an applied interest in the issue so maybe I’m wrong. For more concrete examples I’d take things like protein folding and self driving, where nobody does any OOS training even though relevant datasets could be found outside the specific field they are applied to, and billions (trillions) of dollars in investment lead me to think the research there is close-to-optimla.
With text it’s a bit fuzzier to call it since the goals are fuzzier and text benchmarks start to breakdown when you are evaluating small differences.
I guess the main reason for that bit of the article is not to make a prescriptive claim about the “amount one should aim for” but rather to say that the non-linear nature does mean that investing in it linearly is non-optimal. So it’s essentially conceding to the point that there might be caps, much like your point out. I don’t think I disagree with anything you said above.
Agree, and I’d say the traveling and time buy-back bit is very much gradual, which I enjoy.
I.e. 4-hour work week lifestyles are kind of a pipe dream, but 20-hour work week digital nomading is pretty run of the mill.
This is the burden of literally anything worth having in the world though.
You’re a famous person, you’re afraid people want to befriend you just because you’re kinda famous, you have to refuse to come to your friend’s parties just because they want you there to attract people.
You’re a politician, you’re afraid people want to befriend you just to influence policy, you have to refuse your friend’s requests for having an unethical word with x or y to help them.
Fill in for professor, doctor, and so on.
Money is actually very anonymous compared to almost all other “things worth having”. Maybe past a point it does “get out” but you’re talking billionaire, and that’s a very niche state that’s not worth concerning ourselves with.
In the “small and medium time millionaire” club I expect hiding wealth to be both very easy and very “profitable”, it just involves not buying status goods, and maybe one or two white lies around inheriting this or that or this or that costing less than they did.
Heck, even if word “got out” that you were rich -- you can go for 2 months to Bali—coming back with a story about falling in love, getting married to <this girl>, and having it all taken away—or some such. You can’t white lie your way out of being seen as a dentist if you are a dentist, your cabinet is a physical thing people can see, so good luck explaining to your friends that you won’t do their expensive procedures for free.
I agree, and in so far as avoiding “sales” goes I think it’s been very useful for me to have the knowledge just for avoiding people with knowledge of salesmanship.
Then again, I’m pretty sure I’ve avoided “good” things people were trying to “sell” to me in the past just because they seemed to have little knowledge and a lot of knowledge-of-salesmanship.
Eh, I think I came in a bit too strong trying to argue that even if he was 20% goofing around and thinking oddly, that 20% might me more than current academic slack allows.
I did try to clarify that in a comment in the original blog-post but I do feel like I’ve heard takes of this argument made before.
The biggest update for me are people claiming he actually was a top mathematician in every sense of the word, which might be true and would disprove a big part of my hypothesis, which is that he was a good mathematician, but good in the “top 10k people in the world” sense not in the “top 10 people in the world” sense.
I’m one of those odd “eat anything people”, I mean like, chug galons of milk, stop eating for 10 days, eat 5000+ kcal a day, go for months eating street food in poor parts of SEA and India, raw meet of various kinds, any random fruit peal on etc.. with two food poisonings in my life (both raw oysters) and feeling nausea once or twice a year tops, if even.
Granted FMT is about more than food tolerance, but I assume that’s a good proxy, and given the fact that I’ve never been to athletic nor health conscious food wise I have to think there’s some “hidden variable” influencing this and it’s probably more subtle.
That being said the Danish study looks promising and it’s better than nothing, but I’d love to have some stronger theoretical backing for what the secret sauce of microboime actually is, besides poorly fitting but non zero effect proxies.
Agree on that point, though I think the “crazy things are happening” scenario is likely one where money is less legible so countries bother about taxing such things less.
There are several things at the extreme of non-quantifiable:
There’s “data” which can be examined in so much detail by human senses (which are intertwined with our thinking) that it would be inefficient to extract even with SF-level machinery. I gave as an example being able to feel another persons muscles and the tension within (hence the massage chair, but I agree smart-massage-chairs aren’t that advanced so it’s a poor analogy). Maybe a better example is “what you can tell from looking into someone’s eyes”
There’s data that is interwound with our internal experience. So, for example, I can’t tell you the complex matrix of muscular tension I feel, but I can analyze my body and almost subconsciously decide “I need to stretch my left leg”. Similarly, I might not be able to tell you what the perfect sauce is for me or what patterns of activity it triggers in my brain, or how its molecules bind to my taste buds, but I can keep tasting the sauce and adding stuff and conclude “voila, this is perfect”
There are things beyond data that one can never quantify, like revelations from god or querying the global consciousness or whatever
I myself am pretty convinced there are a lot of things falling under <1> and <2> that are practically impossible to quantify (not fundamentally or theoretically impossible), even provided 1000x better camera, piezo, etc sensors and even provided 0.x nm transistors making perfect use of all 3 dimensions in their packing (so, something like 1000x better GPUs).
I think <3> is false and mainly make fun of the people that believe in it (I’ve taken enough psychedelics not to be able to say this conclusively, but still). However, I still think it will be a generator of disagreement with AI alignment for the vast majority of people.
I can see very good arguments that both 1 and 2 are uncritical and not that hard to quantify, and obviously that 3 is a giant hoax. Alas, my positions have remained unchanged on those, hence why I said a discussion around them may be unproductive.