dkirmani

• dkirmaniMar 1, 2024, 3:51 PM

  2 points

  0

  in reply to: Viliam’s comment on: On the abo­li­tion of man

  an all-knowing God could predict which humans would sin and which would not

  And how would God predict (with perfect fidelity) what humans would do without simulating them flawlessly? A truly flawless physical simulation has no less moral weight than “reality”—indeed, the religious argument could very well be that our world exists as a figment of this God’s imagination.

• dkirmaniFeb 19, 2024, 5:56 AM

  LW: 2 AF: 1

  AF

  on: The ground of optimization

  systems that have a tendency to evolve towards a narrow target configuration set when started from any point within a broader basin of attraction, and continue to do so despite perturbations.

  
  When determining whether a system “optimizes” in practice, the heavy lifting is done by the degree to which the set of states that the system evolves toward—the suspected “target set”—feels like it forms a natural class to the observer.
  
  The issue here is that what the observer considers “a natural class” is informed by the data-distribution that the observer has previously been exposed to.

• dkirmaniFeb 14, 2024, 6:22 PM

  2 points

  −1

  in reply to: Veedrac’s comment on: Op­ti­mal­ity is the tiger, and agents are its teeth

  Whether or not an axis is “useful” depends on your utility function.

  If you only care about compressing certain books from The Library of Babel, then “general optimality” is real — but if you value them all equally, then “general optimality” is fake.

  When real, the meaning of “general optimality” depends on which books you deem worthy of consideration.

  Within the scope of an analysis whose consideration is restricted to the cluster of sequences typical to the Internet, the term “general optimality” may be usefully applied to a predictive model. Such analysis is unfit to reason about search over a design-space — unless that design-space excludes all out-of-scope sequences.

• dkirmaniJan 22, 2024, 11:37 AM

  4 points

  2

  in reply to: tangerine’s comment on: There is way too much serendipity

  Yeah. Here’s an excerpt from Antifragile by Taleb:

  One can make a list of medications that came Black Swan–style from serendipity and compare it to the list of medications that came from design. I was about to embark on such a list until I realized that the notable exceptions, that is, drugs that were discovered in a teleological manner, are too few—mostly AZT, AIDS drugs.

• dkirmaniJan 11, 2024, 7:29 PM

  3 points

  −1

  on: Op­ti­mal­ity is the tiger, and agents are its teeth

  Backpropagation designed it to be good on mostly-randomly selected texts, and for that it bequeathed a small sliver of general optimality.

  “General optimality” is a fake concept; there is no compressor that reduces the filesize of every book in The Library of Babel.

• dkirmaniDec 1, 2023, 4:14 PM

  4 points

  0

  in reply to: TurnTrout’s comment on: TurnTrout’s short­form feed

  This was kinda a “holy shit” moment

  Publicly noting that I had a similar moment recently; perhaps we listened to the same podcast.

  

• dkirmaniOct 27, 2023, 12:55 PM

  6 points

  0

  in reply to: Algon’s comment on: AI #35: Re­spon­si­ble Scal­ing Policies

  For what it’s worth, he has shared (confidential) AI predictions with me, and I was impressed by just how well he nailed (certain unspecified things) in advance—both in absolute terms & relative to the impression one gets by following him on twitter.

• dkirmaniOct 9, 2023, 7:27 PM

  7 points

  0

  in reply to: Dan H’s comment on: how 2 tell if ur in­put is out of dis­tri­bu­tion given only model weights

  I resent the implication that I need to “read the literature” or “do my homework” before I can meaningfully contribute to a problem of this sort.

  The title of my post is “how 2 tell if ur input is out of distribution given only model weights”. That is, given just the model, how can you tell which inputs the model “expects” more? I don’t think any of the resources you refer to are particularly helpful there.

  Your paper list consists of six arXiv papers (1, 2, 3, 4, 5, 6).

  Paper 1 requires you to bring a dataset.

  We propose leveraging [diverse image and text] data to improve deep anomaly detection by training anomaly detectors against an auxiliary dataset of outliers, an approach we call Outlier Exposure (OE). This enables anomaly detectors to generalize and detect unseen anomalies.

  Paper 2 just says “softmax classifers tend to make more certain predictions on in-distribution inputs”. I should certainly hope so. (Of course, not every model is a softmax classifer.)

  We present a simple baseline that utilizes probabilities from softmax distributions. Correctly classified examples tend to have greater maximum softmax probabilities than erroneously classified and out-of-distribution examples, allowing for their detection.

  Paper 3 requires you to know the training set, and also it only works on models that happen to be softmax classifiers.

  Paper 4 requires a dataset of in-distribution data, it requires you to train a classifier for every model you want to use their methods with, and it looks like it requires the data to be separated into various classes.

  

  Paper 5 is basically the same as Paper 2, except it says “logits” instead of “probabilities”, and includes more benchmarks.

  We [...] find that a surprisingly simple detector based on the maximum logit outperforms prior methods in all the large-scale multi-class, multi-label, and segmentation tasks.

  Paper 6 only works for classifiers and it also requires you to provide an in-distribution dataset.

  We obtain the class conditional Gaussian distributions with respect to (low- and upper-level) features of the deep models under Gaussian discriminant analysis, which result in a confidence score based on the Mahalanobis distance.

  It seems that all of the six methods you referred me to either (1) require you to bring a dataset, or (2) reduce to “Hey guys, classifiers make less confident predictions OOD!”. Therefore, I feel perfectly fine about failing to acknowledge the extant academic literature here.

  (Additionally, the methods in my post were also replicated in language models by @voooooogel:

  

  )

• dkirmaniSep 25, 2023, 1:24 PM

  18 points

  12

  on: How have you be­come more hard-work­ing?

  Obtaining an Adderall prescription.

  I use Done, and can recommend messaging their support to switch you to RxOutreach (a service that mails you your medication) if you live in an area with Adderall shortages, like, say, the Bay Area.

• dkirmaniSep 12, 2023, 9:12 PM

  10 points

  0

  on: Startup Roundup #1: Happy Demo Day

  I recently met several YC founders and OpenAI enterprise clients — a salient theme was the use of LLMs to ease the crushing burden of various forms of regulatory compliance.

• dkirmaniAug 31, 2023, 9:19 PM

  2 points

  0

  in reply to: jefftk’s comment on: The Eco­nomics of the As­teroid Deflec­tion Prob­lem (Dom­i­nant As­surance Con­tracts)

  Does “Kickstarter but they refund you if the project doesn’t meet its goal” exist? I think not — that might get you some of the gains of a DAC platform.

  Also, what does PayPal’s TOS say about this? I seriously considered building something similar, and Stripe only approves “crowdfunding” if you have their written approval.

• dkirmaniAug 16, 2023, 8:35 PM

  2 points

  0

  in reply to: Zvi’s comment on: Sum­mary of and Thoughts on the Hotz/​Yud­kowsky Debate

  4 occurrences of “Holz”

• dkirmaniAug 13, 2023, 3:43 AM

  2 points

  on: dkir­mani’s Shortform

  If Guyenet is right that olfactory/​taste signals are critical to the maintenance of obesity, then we should expect people who take their meals exclusively through feeding tubes to be obese at rates well below baseline.

• dkirmaniAug 6, 2023, 4:34 PM

  11 points

  5

  in reply to: Chris_Leong’s comment on: how 2 tell if ur in­put is out of dis­tri­bu­tion given only model weights

  I’m also somewhat confused about why this works

  Abstraction is about what information you throw away. For a ReLU activation function, all negative inputs are mapped to zero—you lose information there, in a way that you don’t when applying a linear transformation.

  Imagine your model (or a submodule thereof) as a mapping from one vector space to another. In order to focus on the features relevant to the questions you care about (is the image a truck, is it a lizard, …) you throw away information that is not relevant to these questions—you give it less real-estate in your representation-space. We can expect more out-of-distribution regions of input-space to be “pinched” by the model—they’re not represented as expressively as are the more in-distribution regions of input-space.

  So if your cosine-similarity decreases less when you nudge your input, you’re in a more “pinched” region of input-space, and if it decreases more, you’re in a more “expanded” region of input space—which means the model was tuned to focus on that region, which means it’s more in-distribution.

• dkirmaniAug 6, 2023, 4:21 PM

  4 points

  0

  in reply to: Chris_Leong’s comment on: how 2 tell if ur in­put is out of dis­tri­bu­tion given only model weights

  Is there idea that pictures with texta drawn over them are out of distribution?

  Yes, the idea is that images that have been taken with a camera were present in the training set, whereas images that were taken with a camera and then scribbled on in GIMP were not.

  If you refer to section 4.2 in the paper that leogao linked, those authors also use “corrupted input detection” to benchmark their method. You’re also welcome to try it on your own images—to run the code you just have to install the pip dependencies and then use paths to your own files. (If you uncomment the block at the bottom, you can run it off your webcam in real-time!)

• dkirmaniAug 6, 2023, 2:44 AM

  4 points

  0

  in reply to: leogao’s comment on: how 2 tell if ur in­put is out of dis­tri­bu­tion given only model weights

  We introduced the concept of the space of models in terms of optimization and motivated the utility of gradients as a distance measure in the space of model that corresponds to the required amount of adjustment to model parameters to properly represent given inputs.

  Looks kinda similar, I guess. But their methods require you to know what the labels are, they require you to do backprop, they require you to know the loss function of your model, and it looks like their methods wouldn’t work on arbitrarily-specified submodules of a given model, only the model as a whole.

  The approach in my post is dirt-cheap, straightforward, and it Just Works™. In my experiments (as you can see in the code) I draw my “output” from the third-last convolutional state. Why? Because it doesn’t matter—grab inscrutable vectors from the middle of the model, and it still works as you’d expect it to.

how 2 tell if ur in­put is out of dis­tri­bu­tion given only model weights

• dkirmaniAug 5, 2023, 6:21 PM

  3 points

  0

  in reply to: Gerald Monroe’s comment on: how 2 tell if ur in­put is out of dis­tri­bu­tion given only model weights

  Yep, but:

  given only model weights

  Also, the cossim-based approach should work on arbitrary submodules of a given model! Also it’s fast!

• dkirmaniAug 5, 2023, 5:31 PM

  13 points

  8

  in reply to: gbear605’s comment on: video games > IQ tests

  often rely on skills that aren’t generally included in “intelligence”, like how fast and precise you can move your fingers

  That’s a funny example considering that (negative one times a type of) reaction time is correlated with measures of g-factor at about $r=0.5$.

• dkirmaniAug 5, 2023, 4:54 PM

  3 points

  0

  in reply to: Dagon’s comment on: Pri­vate notes on LW?

  Yes, of course you could. The concern is user friction.