Houshalter

Karma: 1,109

Houshalter 26 Mar 2016 3:48 UTC
57 points
in reply to: Huluk’s comment on: Lesswrong 2016 Survey
I took the survey 2 days ago. It was fun. I think I was well calibrated for those calibration questions, but sadly there was no “results” section.

Houshalter 22 Feb 2016 14:57 UTC
40 points
on: Open Thread Feb 22 - Feb 28, 2016
I found this paper which is interesting. But at the start he tells an interesting anecdote about existential risk:

The first occurred at Los Alamos during WWII when we were designing atomic bombs. Shortly before the first field test (you realize that no small scale experiment can be done—either you have the critical mass or you do not), a man asked me to check some arithmetic he had done, and I agreed, thinking to fob it off on some subordinate. When I asked what it was, he said, “it is the probability that the test bomb will ignite the whole atmosphere,” I decided I would check it myself! The next day when he came for the answers I remarked to him, “The arithmetic was apparently correct but I do not know about the formulas for the capture cross sections for oxygen and nitrogen—after all, there could be no experiments at the needed energy levels.” He replied, like a physicist talking to a mathematician, that he wanted me to check the arithmetic not the physics, and left. I said to myself, “what have you done, Hamming, you are involved in risking all life that is known in the Universe, and you do not know much of an essential part?” I was pacing up and down the corridor when a friend asked me what was bothering me. I told him. His reply was, “Never mind, Hamming, no one will ever blame you.” Yes, we risked all the life we knew of in the known universe on some mathematics. Mathematics is not merely an idle art form, it is an essential part of our society.

Houshalter 21 Sep 2013 21:13 UTC
28 points
0
in reply to: Z._M._Davis’s comment on: Sorting Pebbles Into Correct Heaps
There is a pattern to what kinds of heaps the Pebblesorters find “right” and “wrong”, but they haven’t figured it out yet. They have always just used their intuition to decide if a heap was right or wrong, but their intuition got less precise in extreme cases like very large heaps. The Pebblesorters would have been better off if only they could have figured out the pattern and applied it to extreme heaps, rather than fighting over differences of intuition.

Also if they had just figured out the pattern, they could have programmed it into the AI rather than hoping that the AI’s intuition would be exactly the same as their own, or manually programming the AI with every special case.

I think this was the main point of the essay but it went right over my head at first.

Houshalter 15 Sep 2015 2:04 UTC
21 points
on: [Link] Marek Rosa: Announcing GoodAI
I talk to a guy on a private AGI IRC server sometimes. He now works for them. He does some really impressive AI work.

He can’t talk about most of the stuff he is working on now due to NDAs. But he did mention he is working on (and has worked with in the past) evolving learning rules for AIs instead of hand coding them.

I discussed AI risk with him, but he doesn’t particularly care about it. He thinks an intelligence explosion is possible, but that an unfriendly AI wouldn’t be so bad. It would just be the next step of evolution. I see the same view in some of the comments on that blog post, though I’m not sure if they are from members of that organization.

I see similar kinds of views about AI risk in even well respected and accomplished AI researchers like Jürgen Schmidhuber.

The other thing different about this company is they come from the game industry. They appear to have written their own NN code from scratch in CUDA. It works on windows, and has a good user interface.

Houshalter 9 Feb 2015 23:13 UTC
19 points
in reply to: CellBioGuy’s comment on: Open thread, Feb. 9 - Feb. 15, 2015
Artificial lighting (mainly blue light) inhibits melatonin and affects sleeping. Try turning off the lights, dimming screens, using f.lux, or wearing yellow tinted goggles.

Houshalter 10 Aug 2015 13:12 UTC
18 points
on: Open thread, Aug. 10 - Aug. 16, 2015
Do Artificial Reinforcement-Learning Agents Matter Morally?

I’ve read this paper and find it fascinating. I think it’s very relevant to Lesswrong’s interests. Not just that it’s about AI,but also that it asks hard moral and philosophical questions.

There are many interesting excerpts. For example:

The drug midazolam (also known as ‘versed,’ short for ‘versatile sedative’) is often used in procedures like endoscopy and colonoscopy… surveyed doctors in Germany who indicated that during endoscopies using midazolam, patients would ‘moan aloud because of pain’ and sometimes scream. Most of the endoscopists reported ‘fierce defense movements with midazolam or the need to hold the patient down on the examination couch.’ And yet, because midazolam blocks memory formation, most patients didn’t remember this: ‘the potent amnestic effect of midazolam conceals pain actually suffered during the endoscopic procedure’. While midazolam does prevent the hippocampus from forming memories, the patient remains conscious, and dopaminergic reinforcement-learning continues to function as normal.

Houshalter 17 Jul 2015 4:47 UTC
17 points
on: Open Thread, Jul. 13 - Jul. 19, 2015
I found this paper: Adults Can Be Trained to Acquire Synesthetic Experiences.

The goal of the study was to see if they could induce synesthesia artificially by forcing people to associate letters with colors. But the interesting part is that after 9 weeks of training, the participants gained 12 IQ points. I have read that increasing IQ is really difficult, and effect sizes this large are unheard of. So I found this really surprising, especially since it doesn’t seem to have gotten a lot of attention.

EDIT: This is a Cattell Culture Fair IQ which uses 24 points as a standard deviation instead of 15. So it’s more like 7.5 IQ points.

They made each participant do 30 minutes of training every day of 9 weeks, which involved a few different tasks to try to form associations between colors and letters. They also assigned colored reading material to read at home.

They took IQ tests before and after and gained 12 IQ points after the training. A control group also took the tests before and after but did not receive training, and did not improve. The sample sizes are small, but the effect sizes might be large enough to justify it. They give a p value of 0.008.

In the paper there are some quotes from subjects, and they describe thinking about words visually. E.g. ‘‘I see the colors like on a monitor in my head and its very automatic’’ or ‘‘The color immediately pops into my head… When I look at a sign the whole word appears colored according to the training colors… it is just as automatic for single letters’’.

I speculate that this might be the cause of the effect, something about using more of the visual system when thinking. That’s just weak speculation though.

I tried to do some more research to see if there was any correlation between synesthesia and IQ. I did not expect there to be, but perhaps it does correlate. This paper suggests it might:

In addition, a neuropsychological test battery was employed, in which all subjects performed superior on tests of general intelligence (mean IQ = 120 ± 17) [out of 9 subjects]

The data from this study shows 10 synesthetes had the same average IQ scores as the controls (but greater standard deviation if that means anything.)

Same story with this study of 10 female synesthetes:

The two subject groups were matched for… IQ, as assessed by the Multiple Choice Vocabulary Test version B… synesthetes = 117 ± 10.2, controls = 116 ± 13.2.

But on second look, it looks like the last two studies intentionally selected the control group to have the same IQs to avoid confounders. If that’s the case then it does support the hypothesis as the reported IQ is greater than average.

Here is another study with more of the same:

Control subjects (n = 14) and synesthesia subjects (n = 14) were matched for age..., gender..., and general intelligence (IQ values for synesthetes: 119 ± 13 and controls: 112 ± 17) as assessed by the MWT-B – “Mehrfach–Wortschatz Test” (Lehrl et al., 1995).

So now I want to try the experiment on myself. I’m considering how to do this. I want to make some kind of tool or browser extension that could color text to match the desired associations. I want to know if it would be better to try letter level associations or word level ones.

I think that word level coloring would be more semantically meaningful and therefore likely to help. But the paper used letter coloring. Most of the subjects in those papers reportedly had grapheme–color synesthesia. They weren’t very specific on the details, or I didn’t look too closely.

Second whether to just use random colors, or try to assign them meaningfully. Like grouping nouns together, or using something like word2vec to find semantically similar words and optimize them to be close in color space if possible. If I do that it’s more complicated and there are a lot of technical decisions to make.

And then how to actually color text in a readable way. Perhaps limiting the color space to what can be read on a white background, or somehow outlining the letters.

EDIT: I found a chrome extension that has some of these features. Only does letter level associations. And the source is available!

Houshalter 1 Jun 2010 20:29 UTC
17 points
on: Rationality quotes: June 2010

It’s no wonder that truth is stranger than fiction. Fiction has to make sense.

Another Twain quote.

Houshalter 9 Mar 2016 22:10 UTC
14 points
in reply to: cousin_it’s comment on: AlphaGo versus Lee Sedol
EY was influenced by E.T. Jaynes, who was really against neural networks, in favor of bayesian networks. He thought NNs were unprincipled and not mathematically elegant, and bayes nets were. I see the same opinions in some of EY’s writings, like the one you link. And the general attitude that “non-elegant = bad” is basically MIRI’s mission statement.

I don’t agree with this at all. I wrote a thing here about how NNs can be elegant, and derived from first principles. But more generally, AI should use whatever works. If that happens to be “scruffy” methods, then so be it.

Approximating Solomonoff Induction

Houshalter29 May 2015 12:23 UTC

13 points

45 comments3 min readLW link

Houshalter 9 Jun 2015 9:23 UTC
13 points
on: Lesswrong, Effective Altruism Forum and Slate Star Codex: Harm Reduction
People seem to be complaining about community fracturing, and good writers going off onto their own blogs. Why not just accept that and encourage people to post links to the good content from these places?

Hacker News is successful mainly because they encourage people to post their own blog posts there, to get a wider audience and discussion. As opposed to reddit where self promotion is heavily discouraged.

Lesswrong is based on reddit’s code. You could add a lesswrong.com/r/links, and just tell people it’s ok to publish links to whatever they want there. This could be quite successful, given lesswrong already has a decent community to seed it with. As opposed to going off and starting another subreddit, where it’s very hard to attract an initial user base (and you run into the self promotion problem I mentioned.)
What links here?
- Gondolinian's comment on [Informal/Colloquial] Open Survey: Ideas for Improving Less Wrong by Gondolinian (10 Jun 2015 14:18 UTC; 1 point)

Houshalter 9 Feb 2015 2:03 UTC
13 points
on: Discussion of concrete near-to-middle term trends in AI
All sorts of really cool stuff in the next few years. Deepmind had amazing results on reinforcement learning beating Atari games with just raw video data (video.) Google bought them a month later for half a billion dollars.

Reinforcement learning is of interest because it’s not just machine learning, predicting outputs given inputs. It is AI, it’s very general.

Some other interesting work is neural turing machines. The NNs can learn to take advantage of an infinite tape. As opposed to learning individual memory and i/o cells, they can operate on arrays. So you can theoretically learn arbitrary programs with gradient descent.

Deep neural networks have shown a huge amount of progress in a lot of AI domains, from vision to natural language, speech. Recently a paper showed they could predict the move an expert Go player would make 44% of the time.

Machine vision has consistently been decreasing the error rate by half every year for the past few years, and just today surpassed human performance.

Stuart Russel said that there has been more investment in AI in the last five years, than since the field was founded. And it’s increasing exponentially.

I think it’s well within the realm of possibility we could get strong AI within 10-20 years.

Houshalter 7 Jun 2015 9:33 UTC
12 points
on: Agency is bugs and uncertainty
Ancient people, who didn’t know any better, saw agency in everything around them. There was a weather god controlling the weather, and a sea god controlling the sea, etc. I wonder to what extent this explains that.

EDIT: This could explain the AI effect too. Once we understand how something works, we don’t see it as AI.

Houshalter 28 Sep 2016 16:24 UTC
11 points
on: Linkposts now live!
This is really awesome and could change the fate of lesswrong. I really think this will bring people back (at least more than any other easy to implement change.) I personally expect to spend more time here now, at least.

One thing to take note of is that lesswrong, by default, sorts by /new. As the volume of posts increases, it may be necessary to change the default sort to /hot or /top/?t=week. Especially if you want it to be presentable to newcomers or even old timers coming back to the site, you want them to see the best links first.

Houshalter 7 Mar 2016 3:08 UTC
11 points
in reply to: Mark_Friedenbach’s comment on: AIFoom Debate—conclusion?

learning cycles are measured in hours or days, not milliseconds.

Yeah but that’s still orders of magnitude faster than humans. AI may not be able to do everything humans can, but what they can do, they can do much faster. No human can learn to be an expert at Go in a week, babies take longer than a day to learn to see, and much longer than a day to learn the basics of language.

Houshalter 16 Jul 2015 19:28 UTC
11 points
on: Examples of AI’s behaving badly
There’s an old story about a neural network that was trained to recognize pictures of tanks for the military. It worked almost perfectly. When someone else tried the program, they found it didn’t work at all.

It turned out that they had taken all the pictures of tanks on cloudy days, and all the other pictures on different days. The neural network simply learned to recognize the weather.

This actually sort of happens in modern deep neural networks. It thinks that roads are parts of cars, or that people’s lips are part of bubbles, or that arms are parts of dumbbells.

Houshalter 9 Feb 2015 13:41 UTC
11 points
in reply to: IlyaShpitser’s comment on: Open thread, Feb. 9 - Feb. 15, 2015
I made a list of AI developments in 2014.

Metaknowledge—Improving on the Wisdom of Crowds

Houshalter1 Jan 2017 21:25 UTC

10 points

0 comments1 min readLW link

(aeon.co)

Houshalter 4 Nov 2016 4:42 UTC
10 points
on: Is Global Reinforcement Learning (RL) a Fantasy?

...add a primary supergoal which imposes a restriction on the degree to which “instrumental goals” are allowed to supercede the power of other goals. At a stroke, every problem he describes in the paper disappears, with the single addition of a goal that governs the use of instrumental goals—the system cannot say “If I want to achieve goal X I could do that more efficiently if I boosted my power, so therefore I should boost my power to cosmic levels first, and then get back to goal X.”

This is not so simple. “Power” and “instrumental goals” are abstractions. Not things that actually can be programmed into an AI. The AI has no concept of “power” and will do whatever leads to it’s goal.

Imagine for instance, a chess playing AI. You tell it to limit it’s “power”. How do you do this? Is using the queen too powerful? Is taking opposing pieces too powerful? How do you define “power” precisely, in a way that can be coded into an actual algorithm?

Of course the issues of AI risk go well beyond just figuring out how to build an AI that doesn’t want to take over the world. Even if your proposed solution could actually work, you can’t stop other people from making AIs that don’t use it, or use a bugged version of it, etc.

The rest of your essay is just a misunderstanding of what reinforcement learning is. Yes it does have origins in old psychology research. But the field has moved on an awful lot since then.

There are many different ideas on how to implement RL algorithms. But the simplest is, to use an algorithm that can predict the future reward. And then take an action which leads to the highest reward.

This procedure is totally independent of what method is used to predict the future reward. There is absolutely nothing that says it has to be an algorithm that can only make short term predictions. Sure, it’s a lot easier to make algorithms that just predict the short term. But that doesn’t mean it’s impossible to do otherwise. Humans sure seem capable of predicting the long term future.

The RL theorist would say that you somehow did a search through all the quintillions of possible actions you could take, sitting there in front of an equation that requires L’Hôpital’s Rule, and in spite of the fact that the list of possible actions included such possibilities as jumping-on-the-table-and-singing-I-am-the-walrus, and driving-home-to-get-a-marmite-sandwich, and asking-the-librarian-to-go-for-some-cheeky-nandos, you decide instead that the thing that would give you the best dopamine hit right now would be applying L’Hôpital’s Rule to the equation.

This just demonstrates how badly you misunderstand RL. Real AIs like AlphaGo don’t need to search through the entire search space. In fact the reason they work over other methods is because they avoid that. It uses machine learning to eliminate large chunks of the search space almost instantly. And so it only needs to consider a few candidate paths.

People started to point out that the extra machinery was where all the action was happening. And that extra machinery was most emphatically not designed as a kind of RL mechanism, itself. In theory, there was still a tiny bit of reinforcement learning somewhere deep down inside all the extra machinery, but eventually people just said “What’s the point?” Why even bother to use the RL language anymore? The RL, if it is there at all, is pointless. A lot of parameter values get changed in complex ways, inside all the extra machinery, so why even bother to mention the one parameter among thousands, that is supposed to be RL, when it is obvious that the structure of that extra machinery is what matters.

RL is a useful concept, because it lets you get useful work out of other, more limited algorithms. A neural net, on it’s own, can’t do anything but supervised learning. You give it some inputs, and it makes a prediction of what the output should be. You can’t use this to play a video game. You need RL to build on top of it to do anything interesting.

You go on and on about how the tasks that AI researchers achieve with AI are “too simple”. This is just typical AI Effect. Problems in AI don’t seem as hard as they actually are, so significant progress never seems like progress at all, from the outside.

But whatever. You are right that currently NNs are restricted to “simple” tasks that don’t require long term prediction or planning. Because long term prediction is hard. But again, I don’t see any reason to believe it’s impossible. It would certainly be much more complex than today’s feed-forward NNs, but it would still be RL. It would still be just doing predictions about what action leads to the most reward, and taking that action.

There is some recent work in this regard. Researcher are combining “new” NN methods with “old” planning algorithms and symbolic methods, so they can get the best of both worlds.

You keep making this assertion that RL has exponential resource requirements. It doesn’t. It’s a simple loop; predict the action that leads to the highest reward, and take it. With a number of variations, but they are all similar.

The current machine learning algorithms that RL methods use, might have exponential requirements. But so what? No one is claiming that future AIs will be just like today’s machine learning algorithms.

Let’s say you are right about everything. RL doesn’t scale, and future AIs will be based on something entirely different that we can’t even imagine.

So what? The same problems that affect RL apply to every AI architecture. That is the control problem. Making AIs do what we want. The problem that most AI goals lead to them seeking more power to better maximize their goals. The problem is that most utility functions are not aligned with human values.

Unless you have an alternative AI method that isn’t subject to this, you aren’t adding anything. And I’m pretty sure you don’t.

Houshalter 7 Mar 2016 2:28 UTC
10 points
on: AIFoom Debate—conclusion?
I don’t know if I’m saying anything that hasn’t been said before elsewhere, but looking at the massive difference in intelligence between humans seems like a strong argument for FOOM to me. Humans are basically all the same. We have 99.99% the same DNA, the same brain structure, size, etc. And yet some humans have exceptional abilities.

I was just reading about Paul Erdos. He could hold 3 conversations at the same time, with mathematicians on highly technical subjects. He was constantly having insights into mathematical research left and right. He produced more papers than any other mathematician.

I don’t think it’s a matter of culture. I don’t think an average person could “learn” to have a higher IQ, let alone be Erdos. And yet he very likely had the same brain structure as everyone else. Who knows what would be possible if you are allowed to move far outside the space of humans.

But this isn’t the (main) argument Yudkowsky uses. He relies on this intuition that I don’t think was explicitly stated or argued strongly enough. This one intuition is central to all the points about recursive self improvement.

It’s that humans kind of suck. At least at engineering and solving complicated technical problems. We weren’t evolved to be good at it. There are many cases where simple genetic algorithms outperform humans. Humans outperform GAs in other cases of course, but it shows we are far from perfect. Even in the areas where we do well, we have trouble keeping track of many different things in our heads. We are very bad at prediction and pattern matching compared to small machine learning algorithms much of the time.

I think this intuition that “humans kind of suck” and “there are a lot of places we could make big improvements” is at the core of the FOOM debate and most these AI risk debates. If you really believe this, then it seems almost obvious that AI will very rapidly become much smarter than humans. People that don’t have this seem to believe that AI is going to be very slow. Perhaps with steep diminishing returns.

Houshalter

Ap­prox­i­mat­ing Solomonoff Induction

Me­ta­knowl­edge—Im­prov­ing on the Wis­dom of Crowds

Approximating Solomonoff Induction

Metaknowledge—Improving on the Wisdom of Crowds