Definitely on the order of “tens of hours”, but it’d be hard to say more specifically. Also, almost all of that time (at least for me) went into learning stuff that didn’t go into this post. Partly that’s because the project is broader than this post, and partly because I have my own research priority of understanding systems theory pretty well.
Alex_Altair
Karma: 4,765
For what it’s worth, I think you’re getting downvoted in part because what you write seems to indicate that you didn’t read the post.
Huh, interesting! So the way I’m thinking about this is, your loss landscape determines the attractor/repellor structure of your phase space (= network parameter space). For a (reasonable) optimization algorithm to have chaotic behavior on that landscape, it seems like the landscape would either have to have 1) a positive-measure flat region, on which the dynamics were ergodic, or 2) a strange attractor, which seems more plausible.
I’m not sure how that relates to the above link; it mentions the parameters “diverging”, but it’s not clear to me how neural network weights can diverge; aren’t they bounded?
I’m curious about this part;
even though the motion of the trebuchet with sling isn’t chaotic during the throw, it can be made chaotic by just varying the initial conditions, which rules out a simple closed form solution for non-chaotic initial conditions
Do you know what theorems/whatever this is from? It seems to me that if you know that “throws” constitute a subset of phase space that isn’t chaotic, then you should be able to have a closed-form solution for those trajectories.
It turns out I have the ESR version of firefox on this particular computer: Firefox
115.14.0esr (64-bit). Also tried it in incognito, and with all browser extensions turned off, and checked multiple posts that used sections.
My overall review is, seems fine, some pros and some cons, mostly looks/feels the same to me. Some details;
I had also started feeling like the stuff between the title and the start of the post content was cluttered.
I think my biggest current annoyance is the TOC on the left sidebar. This has actually disappeared for me, and I don’t see it on hover-over, which I assume is maybe just a firefox bug or something. But even before this update, I didn’t like the TOC. Specifically, you guys had made it so that there was spacing between the sections that was supposed to be proportional to the length of each section. This never felt like it worked for me (I could speculate on why if you’re interested). I’d much prefer if the TOC was just a normal outline-type thing (which it was in a previous iteration).
I think I’ll also miss the word count. I use it quite frequently (usually after going onto the post page itself, so the preview card wouldn’t help much). Having the TOC progress bar thing never felt like it worked either. I agree with Neel that it’d be fine to have the word count in the date hover-over, if you want to have less stuff on the page.
The tags at the top right are now just bare words, which I think looks funny. Over the years you guys have often seemed to prefer really naked minimalist stuff. In this case I think the tags kinda look like they might be site-wide menus, or something. I think it’s better to have the tiny box drawn around each tag as a visual cue.
The author name is now in a sans-serif font, which looks pretty off to me in between the title and the text as serif fonts. It looks like when the browser failed to load the site font and falls back onto the default font, or something. (I do see that it matches the fact that usernames in the comments are sans serif, though.)
I initially disliked the karma section being so suppressed, but then I read one of your comments in this thread explaining your reasoning behind that, and now I agree it’s good.
I also use the comment count/link to jump to comments fairly often, and agree that having that in the lower left is fine.
It does not! At least, not anywhere that I’ve tried hovering.
Is it just me, or did the table of contents for posts disappear? The left sidebar just has lines and dots now.
Quick look: applications of chaos theory
[Talk transcript] What “structure” is and why it matters
There is a little crackpot voice in my head that says something like, “the real numbers are dumb and bad and we don’t need them!” I don’t give it a lot of time, but I do let that voice exist in the back of my mind trying to work out other possible foundations. A related issue here is that it seems to me that one should be able to have a uniform probability distribution over a countable set of numbers. Perhaps one could do that by introducing infinitesimals.
A simple model of math skill
Empirical vs. Mathematical Joints of Nature
Agreed the title is confusing. I assumed it meant that some metric was 5% for last year’s course, and 37% for this year’s course. I think I would just nix numbers from the title altogether.
One model I have is that when things are exponentials (or S-curves), it’s pretty hard to tell when you’re about to leave the “early” game, because exponentials look the same when scaled. If every year has 2x as much activity as the previous year, then every year feels like the one that was the big transition.
For example, it’s easy to think that AI has “gone mainstream” now. Which is true according to some order of magnitude. But even though a lot of politicians are talking about AI stuff more often, it’s nowhere near the top of the list for most of them. It’s more like just one more special interest to sometimes give lip service too, nowhere near issues like US polarization, China, healthcare and climate change.
Of course, AI isn’t necessarily well-modelled by an S-curve. Depending on what you’re measuring, it could be non-monotonic (with winters and summers). It could also be a hyperbola. And if we all dropped dead in the same minute from nanobots, then there wouldn’t really be a mid- or end-game at all. But I currently hold a decent amount of humility around ideas like “we’re in midgame now”.
(Tiny bug report, I got an email for this comment reply, but I don’t see it anywhere in my notifications.)
Done
I propose that this tag be merged into the tag called Infinities In Ethics.
3.
3b.*?
The paper Gleick was referring to is this one, but it would be a lot of work to discern whether it was causal in getting telephone companies to do anything different. It sounds to me like the paper is saying that the particular telephone error data they were looking at could not be well-modeled as IID, nor could it be well-modeled as a standard Markov chain; instead, it was best modeled as a statistical fractal, which corresponds to a heavy-tailed distribution somehow.