Danielle Ensign

Karma: 110

Danielle Ensign Mar 16, 2025, 6:35 AM
1 point
0
in reply to: gwern’s comment on: gwern’s Shortform
Seems like this could be addressed by filtering out comments that use evidence or personal examples from your dataset.

If that’s too intense, filtering responses to remove personal examples and checking sources shouldn’t be too bad? But maybe you’d just end up with a model that tries to subvert the filter/draw misleading conclusions from sources instead of actually being helpful…

Danielle Ensign Feb 10, 2025, 10:26 PM
1 point
0
in reply to: tailcalled’s comment on: Attribution-based parameter decomposition

There are certain cases where pure gradient-based attributions predictably don’t work (most notably when a softmax is saturated)

Do you have a source or writeup somewhere on this? (or do you mind explaining more/have some examples where this is true?) Is this issue actually something that comes up for modern day LLMs?

In my observations it works fine for the toy tasks people have tried it on. The challenge seems to be in interpreting the attributions, not issues with the attributions themselves.

Danielle Ensign Feb 10, 2025, 5:34 PM
3 points
0
in reply to: tailcalled’s comment on: Attribution-based parameter decomposition
In my experience gradient-based attributions (especially if you use integrated gradients) are almost identical to the attributions you get from ablating away each component. It’s kinda crazy but is the reason ppl use edge-attribution patching over older approaches like ACDC.

Look at page 15 of https://openreview.net/forum?id=lq7ZaYuwub (left is gradient attributions, right is attributions from ablating each component). This is for Mamba but I’ve observed similar things for transformers.

Danielle Ensign Apr 22, 2024, 9:11 PM
1 point
0
in reply to: Chakshu Mira’s comment on: Ophiology (or, how the Mamba architecture works)
fixed :)

Danielle Ensign Apr 22, 2024, 4:50 PM
1 point
0
in reply to: Chakshu Mira’s comment on: Ophiology (or, how the Mamba architecture works)
Fixed, thank you!

Ophiology (or, how the Mamba architecture works)

Danielle Ensign, SrGonao and Adrià Garriga-alonso

Apr 9, 2024, 7:31 PM

67 points

8 comments10 min readLW link

Danielle Ensign Mar 16, 2023, 2:36 PM
4 points
0
in reply to: Charlie Steiner’s comment on: What’s the Least Impressive Thing GPT-4 Won’t be Able to Do
GPT-4 generated the following:

Review 2:

Sophia L.

B.R., United States

4.5 stars

A cozy, eco-friendly gem in the city! Green Street Cafe is my new go-to spot for catching up with friends or enjoying a quiet afternoon with a good book. Their plant-based menu offers a delightful variety of options—I absolutely loved their vegan BLT and lavender latte. The only downside was the slightly slow service, but the staff was so warm and attentive that it hardly mattered. Can’t wait to visit again!

Review 3:

Michael N.

T.S., United States

4 stars

Stopped by Green Street Cafe for a quick bite and was pleasantly surprised by the charming atmosphere and delicious food. The spinach and feta quiche was to die for, and the iced chai latte had just the right amount of spice. The place was buzzing with bees, which added a unique touch to the experience. Although the Wi-Fi was a bit spotty, I’d still recommend this spot for a relaxing break or casual meeting.

Review 4:

Emily P.

D.W., United States

5 stars

Green Street Cafe is my new favorite spot for brunch! The cafe’s bright, inviting interior, complete with lush greenery and an eco-conscious design, makes for the perfect weekend retreat. I can’t get enough of their avocado toast and freshly squeezed orange juice. The bees add a quirky touch, and their presence speaks to the cafe’s commitment to sustainability. Fantastic service, too—the staff is always smiling and eager to help. Highly recommended!

Review 5:

David T.

G.J., United States

3.5 stars

While the Green Street Cafe has a charming ambiance and friendly staff, the food and drinks were a bit hit-or-miss. The honey lemonade was fantastic, but the cappuccino I ordered was lukewarm and lacked flavor. My wife enjoyed her quinoa salad, but my vegetable panini was soggy. The bees are an interesting touch, but they may not be for everyone. I might give this place another shot, but I’ll temper my expectations next time.

Danielle Ensign Nov 27, 2022, 10:29 PM
1 point
0
on: Socialist firms as a cause area
I highly recommend this video for those wanting a more detailed analysis of the pros and cons of worker co-ops

Danielle Ensign Apr 8, 2021, 3:39 AM
6 points
on: What will GPT-4 be incapable of?
Things that it can probably do sometimes, but will fail on some inputs:
- Factor numbers
- Solve NP-Complete or harder problems
- Execute code
There are other “tail end” tasks like this that should eventually become the hardest bits that optimization spends the most time on, once it manages to figure everything else out.

Danielle Ensign Mar 24, 2021, 2:14 AM
3 points
on: Chaos Induces Abstractions
If we knew (for some reason) that a system had no useful abstractions (or at least, no small ones), what could we say about that system? Does it reduce to some vacuous thing? Or does it require it to be adversarial in nature?

Human Biases that Obscure AI Progress

Danielle EnsignSep 25, 2020, 12:24 AM

42 points

2 comments4 min readLW link

Danielle Ensign

Ophiol­ogy (or, how the Mamba ar­chi­tec­ture works)

Hu­man Bi­ases that Ob­scure AI Progress

Ophiology (or, how the Mamba architecture works)

Human Biases that Obscure AI Progress