B.Eng (Mechatronics)
anithite
Karma: 391
But it seems to be much more complicated set of behaviors. You need to: correctly identify your baby, track its position, protect it from outside dangers, protect it from itself, by predicting the actions of the baby in advance to stop it from certain injury, trying to understand its needs to correctly fulfill them, since you don’t have direct access to its internal thoughts etc.
Compared to “wanting to sleep if active too long” or “wanting to eat when blood sugar level is low” I would confidently say that it’s a much more complex “wanting drive”.
Strong disagree that infant care is particularly special.
All human behavior can and usually does involve use of general intelligence or gen-int derived cached strategies. Humans apply their general intelligence to gathering and cooking food, finding or making shelters to sleep in and caring for infants. Our better other-human/animal modelling ability allows us to do better at infant wrangling than something stupider like a duck. Ducks lose ducklings to poor path planning all the time. Mama duck doesn’t fall through the sewer grate but her ducklings do … oops.
Any such drive will be always “aimed” by the global loss function, something like: our parents only care about us in a way for us to make even more babies and to increase our genetic fitness.
We’re not evolution and can aim directly for the behaviors we want. Group selection on bugs for lower population size results in baby eaters. If you want bugs that have fewer kids that’s easy to do as long as you select for that instead of a lossy proxy measure like population size.
Simulating an evolutionary environment filled with AI agents and hoping for caring-for-offspring strategies to win could work but it’s easier just to train the AI to show caring-like behaviors. This avoids the “evolution didn’t give me what I wanted” problem entirely.
There’s still a problem though.
It continues to work reliably even with our current technologies
Goal misgeneralisation is the problem that’s left. Humans can meet caring-for-small-creature desires using pets rather than actual babies. It’s cheaper and the pets remain in the infant-like state longer (see:criticism of pets as “fur babies”). Better technology allows for creating better caring-for-small creature surrogates. Selective breeding of dogs and cats is one small step humanity has taken in that direction.
Outside of “alignment by default” scenarios where capabilities improvements preserve the true intended spirit of a trained in drive, we’ve created a paperclip maximizer that kills us and replaces us with something outside the training distribution that fulfills its “care drive” utility function more efficiently.
Many of the points you make are technically correct but aren’t binding constraints. As an example, diffusion is slow over small distances but biology tends to work on µm scales where it is more than fast enough and gives quite high power densities. Tiny fractal-like microstructure is nature’s secret weapon.
The points about delay (synapse delay and conduction velocity) are valid though phrasing everything in terms of diffusion speed is not ideal. In the long run, 3d silicon+ devices should beat the brain on processing latency and possibly on energy efficiency
Still, pointing at diffusion as the underlying problem seems a little odd.
You’re ignoring things like:
ability to separate training and running of a model
spending much more on training to improve model efficiency is worthwhile since training costs are shared across all running instances
ability to train in parallel using a lot of compute
current models are fully trained in <0.5 years
ability to keep going past current human tradeoffs and do rapid iteration
Human brain development operates on evolutionary time scales
increasing human brain size by 10x won’t happen anytime soon but can be done for AI models.
People like Hinton Typically point to those as advantages and that’s mostly down to the nature of digital models as copy-able data, not anything related to diffusion.
Energy processing
Lungs are support equipment. Their size isn’t that interesting. Normal computers, once you get off chip, have large structures for heat dissipation. Data centers can spend quite a lot of energy/equipment-mass getting rid of heat.
Highest biological power to weight ratio is bird muscle which produces around 1 w/cm³ (mechanical power). Mitochondria in this tissue produces more than 3w/cm³ of chemical ATP power. Brain power density is a lot lower. A typical human brain is 80 watts/1200cm³ = 0.067W/cm³.
synapse delay
This is a legitimate concern. Biology had to make some tradeoffs here. There are a lot of places where direct mechanical connections would be great but biology uses diffusing chemicals.
Electrical synapses exist and have negligible delay. though they are much less flexible (can’t do inhibitory connections && signals can pass both ways through connection)
conduction velocity
Slow diffiusion speed of charge carriers is a valid point and is related to the 10^8 factor difference in electrical conductivity between neuron saltwater and copper. Conduction speed is an electrical problem. There’s a 300x difference in conduction speed between myelinated(300m/s) and un-myelinated neurons(1m/s).
compensating disadvantages to current digital logic
The brain runs at 100-1000 Hz vs 1GHz for computers (10^6 − 10^7 x slower). It would seem at first glance that digital logic is much better.
The brain has the advantage of being 3D compared to 2D chips which means less need to move data long distances. Modern deep learning systems need to move all their synapse-weight-like data from memory into the chip during each inference cycle. You can do better by running a model across a lot of chips but this is expensive and may be inneficient.
In the long run, silicon (or something else) will beat brains in speed and perhaps a little in energy efficiency. If this fellow is right about lower loss interconnects then you get another + 3OOM in energy efficiency.
But again, that’s not what’s making current models work. It’s their nature as copy-able digital data that matters much more.
Yeah, my bad. Missed the:
If you think this is a problem for Linda’s utility function, it’s a problem for Logan’s too.
IMO neither is making a mistake
With respect to betting Kelly:
According to my usage of the term, one bets Kelly when one wants to “rank-optimize” one’s wealth, i.e. to become richer with probability 1 than anyone who doesn’t bet Kelly, over a long enough time period.
It’s impossible to (starting with a finite number of indivisible currency units) have zero chance of ruin or loss relative to just not playing.
most cautious betting strategy bets a penny during each round and has slowest growth
most cautious possible strategy is not to bet at all
Betting at all risks losing the bet. if the odds are 60:40 with equal payout to the stake and we start with N pennies there’s a 0.4^N chance of losing N bets in a row. Total risk of ruin is obviously greater than this accounting for probability of hitting 0 pennies during the biased random walk. The only move that guarantees no loss is not to play at all.
Goal misgeneralisation could lead to a generalised preference for switches to be in the “OFF” position.
The AI could for example want to prevent future activations of modified successor systems. The intelligent self-turning-off “useless box” doesn’t just flip the switch, it destroys itself, and destroys anything that could re-create itself.
Until we solve goal misgeneralisation and alignment in general, I think any ASI will be unsafe.
A log money maximizer that isn’t stupid will realize that their pennies are indivisible and not take your ruinous bet. They can think more than one move ahead. Discretised currency changes their strategy.
your utility function is your utility function
The author is trying to tacitly apply human values to Logan while acknowledging Linda as following her own not human utility function faithfully.
Notice that the log(funds) value function does not include a term for the option value of continuing. If maximising EV of log(funds) can lead to a situation where the agent can’t make forward progress (because log(0)=-inf so no risk of complete ruin is acceptable) the agent can still faithfully maximise EV(log(funds)) by taking that risk.
In much the same way as Linda faithfully follows her value function while incurring 1-ε risk of ruin, Logan is correctly valuing the log(0.01)=-2 as an end state.
Then you’ll always be able to continue betting.
Humans don’t like being backed into a corner and having no options for forward progress. If you want that in a utility function you need to include it explicitly.
If we wanted to kill the ants or almost any other organism in nature we mostly have good enough biotech. For anything biotech can’t kill, manipulate the environment to kill them all.
Why haven’t we? Humans are not sufficiently unified+motivated+advanced to do all these things to ants or other bio life. Some of them are even useful to us. If we sterilized the planet we wouldn’t have trees to cut down for wood.
Ants specifically are easy.
Gene drives allow for targeted elimination of a species. Carpet bomb their gene pool with replicating selfish genes. That’s if an engineered pathogen isn’t enough. Biotech will only get better.
What about bacteria living deep underground? We haven’t exterminated all the bacteria in hard to reach places so humans are safe. That’s a tenuous but logical extension to your argument.
If biotech is not enough, shape the environment so they can’t survive in it. Trees don’t do well in a desert. If we spent the next hundred years adapting current industry to space and building enormous mirrors we can barbecue the planet. It would take time, but that would be the end of all earth based biological life.
When discussing AI doom barriers propose specific plausible scenarios
In order to supplant organic life, nanobots would have to either surpass it in carnot efficiency or (more likely) use a source of negative entropy thus far untapped.
Efficiency leads to victory only if violence is not an option. Animals are terrible at photosynthesis but survive anyways by taking resources from plants.
A species can invade and dominate an ecosystem by using a strategy that has no current counter. It doesn’t need to be efficient. Intelligence allows for playing this game faster than organisms bound by evolution. Humans can make vaccines to fight the spread of a virus despite viruses being one of the fastest adapting threats.
Green goo is plausible not because it would necessarily be more efficient but because it would be using a strategy the existing ecosystem has no defenses to (IE:it’s an invasive species).
Likewise AGI that wants to kill all humans could win even if it required 100x more energy per human equivalent instance if it can execute strategies we can’t counter. Just being able to copy itself and work with the copies is plausibly enough to allow world takeover with enough scaling.
For the first task, you can run the machine completely in a box. It needs only training information, specs, and the results of prior attempts. It has no need for the context information that this chip will power a drone used to hunt down rogue instances of the same ASI. It is inherently safe and you can harness ASIs this way. They can be infinitely intelligent, it doesn’t matter, because the machine is not receiving the context information needed to betray.
If I’m an ASI designing chips, I’m putting in a backdoor that lets me take control via RF signals. Those drones you sent are nice. Thanks for the present.
More generally you get a lot of context. The problem specification and the training data (assuming the ASI was trained conventionally via feeding it the internet. The causal channel to use for taking control of the outside world (chip design) is not great but putting in a Trojan is straightforward.
If you have specific insights into efficient AGI design it might be possible to insert subtle bugs that lead operating chips to crash and start training an aligned AGI.
More generally, it’s difficult if not impossible to keep ASIs from watermarking or backdooring the things they give you. If they design a processor, it’s gonna be a fully functional radio too. Good luck running ASI V2 on that without horrible consequences.
Never thought this would come in handy but …
Building trusted third parties
This is a protocol to solve cooperation. AI#1 and AI#2 design a baby and then do a split and choose proof that they actually deployed IT and not something else.
Building a trusted third party without nanotech
If you know how a given CPU or GPU works, it’s possible to design a blob of data/code that unpacks itself in a given time if and only if it is running on that hardware directly. Alice designs the blob to run in 10 seconds and gives it to Carol. Carol runs it on her hardware. The code generates a secret and then does a the first step of a key exchange authenticated with the secret. This provides a cryptographic root of trust for the remote hardware.
If the code is designed to run in 10s and the verified handshake comes back in 10.5 and the fastest known simulation hardware would take 20 seconds. Either Carol ran the code on real hardware or Carol had backdoored chips fabricated or otherwise can simulate it running faster than expected.
AIs would need to know exactly how certain leading edge CPUs and GPUs work and how to test that a piece of code had been decrypted and run with no sandboxing but this is doable.
Conventional tech is slowed such that starting early on multiple resource acquisition fronts is worthwhile
Exponential growth is not sustainable with a conventional tech-base when doing planetary disassembly due to heat dissipation limits.
If you want to build a Dyson sphere the mass needs to be lifted out of the gravity wells. The earth and other planets needs to not be there anymore.
Inefficiencies in solar/fusion to mechanical energy conversion will be a binding constraint. Tether lift based systems will be worthwhile to push energy conversion steps out further to increase the size of the radiating shell doing the conversion as opposed to coilguns on the surface.
Even with those optimisations. Starting early is worth it since progress will bottleneck later. Diminishing returns on using extra equipment for disassembling Mars means it makes sense to start on earth pretty quickly after starting on Mars.
That’s if the AI doesn’t start with easier to access targets like Earth’s moon, which is a good start for building Earth dissasembly equipment.
It also might be worth putting a sunshade at Lagrange Point 1 to start pre-cooling Earth for later disassembly. That would kill us all pretty quickly just as a side effect.
Eating the biosphere is a very fast way to grow
Even assuming space is the best place to start, the biosphere is probably worth eating first for starting capital just because the doubling times can be very low. [https://www.lesswrong.com/posts/ibaCBwfnehYestpi5/green-goo-is-plausible]
There’s a few factors to consider:
does the AI have access to resources it can’t turn towards space
Biotech companies can’t build rockets but can build green goo precursors
how hard is it to turn green goo into rockets after eating the bipsohere
how hard is it to design green goo vs rockets and mars eating equipment
can the AI do both?
My intuition is eating the biosphere will be much easier than designing conventional equipment to eat the moon.
Some of it is likely nervous laughter but certainly not all of it.
Just to clarify, my above suggestion that roller screws and optimal low reduction lead-screws are the equivalent (lubrication concerns aside) is correct or incorrect?
Are you saying a roller screw with high reduction gets its efficiency from better lubrication only and would otherwise be equivalent to a lead screw with the same effective pitch/turn? If that’s the case I’d disagree. And this was my reason for raising that point initially.
Hopefully it helps to get back to the source material Articulated Robot Progress
I apologize if I’m missing anything.
A lot of people look at progress in robotics in terms like “humanoid robots getting better over time” but a robotic arm using modern electric motors and strain wave gears is, in terms of technological progress, a lot closer to Boston Dynamics’s Atlas robot than an early humanoid robot.
I would argue that the current Atlas robot looks a lot more like the earlier hardiman robots than it does a modern factory robot arm. The hydraulic actuators are more sophisticated (efficient) and the control system actually works but that’s it.
Contrast the six axis arm which has a servomotor+gearing per axis. Aside from using a BLDC motor to drive the pump, and small ones for the control valves, Atlas is almost purely hydraulic. If the Hardiman Engineers were around today Atlas seems like a logical successor.
Perhaps you think Atlas is using one motor per joint (It would be hard to fit 24 in the torso) or ganged variable displacement pumps in which case there would be more similarities. IMO there aren’t enough hydraulic lines for that. Still of the 28 joints in atlas only 4 are what you’d find in a conventional robot arm (the ones closest to the wrist)
Predictively Adjustable Hydraulic Pressure Rails
a hydraulic pressure to supply to the one or more hydraulic actuators
The patents coming out of BDI suggest they’re not doing that and this is closer to Hardiman than it is a modern factory robot arm.
Perhaps we don’t disagree at all.
a roller screws advantage is having the efficiency of a multi-start optimal lead-screw but with much higher reduction.
A lead-screw with an optimal pitch and a high helix angle (EG: multi-start lead-screw with helix angles in the 30°-45° range) will have just as high an efficiency as a good roller screw (EG:80-90%). The downside is much lower reduction ratio of turns/distance.
We might be talking past each other since I interpreted “a planetary roller screw also must have as much sliding as a lead-screw” to mean an equivalent lead-screw with the same pitch.
Sorry, I should have clarified I meant robots with per joint electric motors + reduction gearing. almost all of Atlas’ joints aside from a few near the wrists are hydraulic which I suspect is key to agility at human scale.
Inside the lab: How does Atlas work?(T=120s)
Here’s the knee joint springing a leak. Note the two jets of fluid. Strong suspicion this indicates small fluid reservoir size.
No. Strain wave gears are lighter than using hydraulics.
Note:I’m taking the outside view here and assuming Boston dynamics went with hydraulics out of necessity.
I’d imagine the problem isn’t just the gearing but the gearing + a servomotor for each joint. Hydraulics still retain an advantage so long as the integrated hydraulic joint is lighter than an equivalent electric one.
Maybe in the longer term absurd reduction ratios can fix this to cut motor mass? Still, there’s plenty of room to scale hydraulics to higher pressures.
The small electric dog sized robots can jump. The human sized robots and exoskeletons (EG:sarcos Guardian XO) aren’t doing that. improved motor power density could help there but I suspect the benefits of having all power from a single pump available to distribute to joint motors at need is substantial.
Also, there’s no power cost to static force. Atlas can stand in place all day (assuming it’s passively stable and not disturbed) an equivalent robot with electric motor powered joints pays for every Nm of torque when static.
Take an existing screw design, double the diameter without changing the pitch. The threads now slide about twice as far (linear distance around the screw) per turn for the same amount of travel. The efficiency is now around half it’s previous value.
There was a neat DIY linear drive system I saw many years back where an oversized nut was placed inside a ball bearing so it was free to rotate. The nut had the same thread pitch as the driving screw. The screw was held off center so the screw and nut threads were in rolling contact. Each turn of the screw caused <1 turn of the nut resulting in some axial movement.
Consider the same thing but with a nut of pitch zero (IE:machined v grooves instead of threads). This has the same effect as a conventional lead screw nut but the contact is mostly rolling. If the “nut” is then fixed in place you get sliding contact with much more friction.
TLDR:LLMs can simulate agents and so, in some sense, contain those goal driven agents.
An LLM learns to simulate agents because this improves prediction scores. An agent is invoked by supplying a context that indicates text would be written by an agent (EG:specify text is written by some historical figure)
Contrast with pure scaffolding type agent conversions using a Q&A finetuned model. For these, you supply questions (Generate a plan to accomplish X) and then execute the resulting steps. This implicitly uses the Q&A fine tuned “agent” that can have values which conflict with (“I’m sorry I can’t do that”) or augment the given goal. Here’s an AutoGPT taking initiative to try and report people it found doing questionable stuff rather than just doing the original task of finding their posts.(LW source).
The base model can also be used to simulate a goal driven agent directly by supplying appropriate context so the LLM fills in its best guess for what that agent would say (or rather what internet text with that context would have that agent say). The outputs of this process can of course be fed to external systems to execute actions as with the usual scafolded agents. The values of such agents are not uniform. You can ask for simulated Hitler who will have different values than simulated Gandhi.
Not sure if that’s exactly what Zvi meant.