quiet_NaN

Karma: 344

quiet_NaN Jul 29, 2024, 4:08 AM
5 points
3
on: Universal Basic Income and Poverty
[...] somehow humanity’s 100-fold productivity increase (since the days of agriculture) didn’t eliminate poverty.
That feels to me about as convincing as saying: “Chemical fertilizers have not eliminated hunger, just the other weekend I was stuck on a campus with a broken vending machine.”
I mean, sure, both the broken vending machine and actual starvation can be called hunger, just as both working 60h/week to make ends meet or sending your surviving kids into the mines or prostituting them could be called poverty, but the implication that either scourge of humankind has not lost most of its terror seems clearly false.
Sure, being poor in the US sucks, but I would rather spend a year living the life of someone in the bottom 10% income bracket in the 2024 US than spending a month living the life of a poor person during the English industrial revolution.
I am also not convinced that 60h/week is what it actually takes to survive in the US. I can totally believe that this amount of unskilled labor might be required to rent accommodations in cities, though.

quiet_NaN Jul 18, 2024, 12:49 PM
1 point
0
on: Superbabies: Putting The Pieces Together
Critically, the gene editing of the red blood cells can be done in the lab; trying to devise an injectable or oral substance that would actually transport the gene-editing machinery to an arbitrary part of the body is much harder.
I am totally confused by this. Mature red blood cells don’t contain a nucleus, and hence no DNA. There is nothing to edit. Injecting blood cells produced by gene-edited bone marrow in vitro might work, but would only be a therapy, not a cure: it would have to be repeated regularly. The cure would be to replace the bone marrow.
So I resorted to reading through the linked FDA article. Relevant section:
The modified blood stem cells are transplanted back into the patient where they engraft (attach and multiply) within the bone marrow and increase the production of fetal hemoglobin (HbF), a type of hemoglobin that facilitates oxygen delivery.
Blood stem cells seems to be FDA jargon for ematopoietic stem cells. From the context, I would guess they are harvested from the bone marrow of the patients, then CRISPRed, and then injected back in the blood stream where they will find back to the bone marrow.
I still don’t understand how they would outcompete the non-GMO bone marrow which produces the faulty red blood cells, though.
I would also take the opportunity to point out that the list of FDA-approved gene therapies tells us a lot about the FDA and very little about the state of the art. This is the agency which banned life-saving baby nutrition for two years, after all. Anchoring what is technologically possible to what the FDA approves would be like anchoring what is possible in mobile phone tech to what is accepted by the Amish.
Also, I think that editing of multi-cellular organisms is not required for designer babies at all.
1. Start with a fertilized egg, which is just a single cell. Wait for the cell to split. After it has split, separate the cells into two cells. Repeat until you have a number of single-cell candidates.
2. Apply CRISPR to these cells individually. allow them to split again. Do a genome analysis on one of the two daughter cell. Select a cell line where you did only the desired change in the genome. Go back to step one and apply the next edit.
Crucially, the costs would only scale linearly with the number of edits. I am unsure how easy is that “turn one two-cell embryo into two one-cell embryos”, though.
Of course, it would be neater to synthesize the DNA of the baby from the scratch, but while prices per base pair synthesis have been dropped a lot, they are clearly still to high to pay for building a baby (and there are likely other tech limitations).

quiet_NaN Jul 18, 2024, 11:43 AM
2 points
2
in reply to: Elias711116’s comment on: Superbabies: Putting The Pieces Together
I thought this first too. I checked on Wikipedia:
Adult stem cells are found in a few select locations in the body, known as niches, such as those in the bone marrow or gonads. They exist to replenish rapidly lost cell types and are multipotent or unipotent, meaning they only differentiate into a few cell types or one type of cell. In mammals, they include, among others, hematopoietic stem cells, which replenish blood and immune cells, basal cells, which maintain the skin epithelium [...].
I am pretty sure that the thing a skin cell makes per default when it splits is more skin cells, so you are likely correct.

quiet_NaN Jul 1, 2024, 12:56 AM
3 points
0
in reply to: milanrosko’s comment on: The Incredible Fentanyl-Detecting Machine
See here. Of course, that article is a bit light on information on detection thresholds, false-positive rates and so on as compared to dogs, mass spectrometry or chemical detection methods.
I will also note that humans have 10-20M olfactory receptor neurons, while bees have 1M neurons in total. Probably bees are under more evolutionary pressure to make optimal use of their olcfactory neurons, though.

quiet_NaN Jul 1, 2024, 12:34 AM
1 point
0
in reply to: Review Bot’s comment on: The Incredible Fentanyl-Detecting Machine
Dear Review Bot,
please avoid double-posting.
On the other hand, I don’t think voting you to −6 is fair, so I upvoted you.

quiet_NaN Jul 1, 2024, 12:29 AM
2 points
−2
in reply to: gwern’s comment on: The Incredible Fentanyl-Detecting Machine
My take on sniffer dogs is that frequently, what they are best at is picking up is unconscious tells from their handler. In so far as they do, they are merely science!-washing the (possibly meritful) biases of the police officier.
Packaging something really air-tight without outside contamination is indeed far from trivial. For example, the swipe tests taken at airports are useful because while it is certainly possible to pack a briefcase full of explosives without any residue on the outside is certainly possible, most of the people who could manage to build a bomb would not manage to do that.
Of course, there are also no profit margins in blowing up airplanes, so stopping the amateurs is already 95% of the job.
There are significant profit margins in drug trafficking. After you intercept a few shipments and arrest a few mules, the cleverer drug lords will wisen up.
A multi-method approach might work for a while, glass vials are probably more visible on that CT scan than some organic substance.

quiet_NaN Jul 1, 2024, 12:00 AM
2 points
0
on: The Incredible Fentanyl-Detecting Machine
My first question is about the title picture. I have some priors on how a computer tomography machine for vehicles would look like. Basically, you want to take x-ray images from many different directions. The medical setup where you have a ring which contains the x-ray source on one side and the detectors on the other side, and rotate that ring to take images from multiple directions before moving the patient perpendicular to the ring to record the next slice exists for a reason: high resolution x-ray detectors are expensive. If we scaled this up to a car size, we might have a ring of four meters in diameter. A bridge of some low-Z material (perhaps beams of wood) would go through that ring. Park on the bridge, get out of the car, and watch as the rotating ring moves over the bridge.
The thing in the picture looks does not look like it has big moving parts. I can kind of see it taking a sideways x-ray image of the car but I am puzzled by the big grey boxes visible over the car. Having an x-ray source or detector above the car would only make sense if you had the other device below the car, in the road. Of course, if there is anything in the road, one would imagine that that piece of road was some low-Z material material designed not to block half of your x-rays. Instead, the road material below the car looks exactly like the road outside the detector, concrete probably.
In Europe, we kind of dislike being exposed to ionizing radiation (even though Germany is rather silly about it). I get that the US is a bit more relaxed about it, but a computer tomography device capable of scanning a car with a good resolution would likely emit to a lot of scattered x-rays to anyone nearby. The position in which that boarder guard stands would likely not be a safe position to stand a significant fraction of your work-life.
At the very least, I would expect that black and yellow trefoil sign warning of ionizing radiation, with a visual indicator if the x-ray is on, and a line marking the minimum safe distance. More realistically, you might have something like a garage door before and behind the car.
Just from the vibes, I get that the pictured non-intrusive fentanyl inspection machine might run on the operation principle of the famous ADE 651, which is to say it does nothing whatsoever.
While detecting dry fentanyl through high resolution CT scans seems to be possible in principle at least, once the smugglers go through the additional trouble of dissolving it all bets are off. Wikipedia is light on solubility data, but from the looks of it fentanyl should dissolve well in organic solvents.
You can check the gas tank of every car if you want, but what are you going to do if an eight-wheeler full of one- gallon-bottles of cooking oil crosses the border? If your scan is good, it would perhaps detect if one bottle in the middle of the stack has been filled with powder instead. There is no way in hell it will detect if one such bottle has ten grams (perhaps 10k doses) of fentanyl dissolved in it. Your best bet would be to detect some residue of the oil on product seized off the streets and work backwards from that.
Of course, “as long as the profit margin is that high, we could go full iron curtain on our borders and would not stop the trafficking” is not a politically acceptable answer. Hence some snake oil salesmen are able to get your tax dollars for products which have not been proven in adversarial conditions. (To train an AI, it is not enough to have a ton of samples, you would need known positive and negative samples. Also, while we are playing buzzword bingo, why not mention that the shipping manifests will be securely stored on The Blockchain?)

quiet_NaN May 23, 2024, 10:40 AM
1 point
0
in reply to: Seth Herd’s comment on: The Problem With the Word ‘Alignment’
The deliberately clumsy term “AInotkilleveryoneism” seems good for this, in any context you can get away with it.
Hard disagree. The position “AI might kill all humans in the near future” is still quite some inferential distance away from the mainstream even if presented in a respectable academic veneer.
We do not have weirdness points to spend on deliberately clumsy terms, even on LW. Journalists (when they are not busy doxxing people) can read LW too, and if they read that the worry about AI as an extinction risk is commonly called notkilleveryoneism they are orders of magnitude less likely to take us serious, and being taken serious by the mainstream might be helpful for influencing policy.
We could probably get away with using that term ten pages deep into some glowfic, but anywhere else ‘AI as an extinction risk’ seems much better.

quiet_NaN May 21, 2024, 6:46 PM
1 point
1
in reply to: cata’s comment on: Some perspectives on the discipline of Physics
It is also useful for a lot of practical problems, where you can treat $ℏ$ as being essentially zero and $c_{0}$ as being essentially infinite. If you want to get anywhere with any practical problem (like calculating how long a car will take to come to a stop), half of the job is to know which approximations (“cheats”) are okay to use. If you want to solve the fully generalized problem (for a car near the Planck units or something), you will find that you would need a theory of everything (that is quantum mechanics plus general relativity) to do so and we don’t have that.

quiet_NaN May 21, 2024, 6:24 PM
2 points
0
on: The Problem With the Word ‘Alignment’
I think that “AI Alignment” is a useful label for the somewhat related problems around P1-P6. Having a term for the broader thing seems really useful.
Of course, sometimes you want labels to refer to a fairly narrow thing, like the label “Continuum Hypothesis”. But broad labels are generally useful. Take “ethics”, another broad field label. Nominative ethics, applied ethics, meta-ethics, descriptive ethics, value theory, moral psychology, et cetera. I someone tells me “I study ethics” this narrows down what problems they are likely to work on, but not very much. Perhaps they work out a QALY-based systems for assigning organ donations, or study the moral beliefs of some peoples, or argue if moral imperatives should have a truth value. Still, the label confers a lot of useful information over a broader label like “philosophy”.
By contrast, “AI Alignment” still seems rather narrow. P2 for example seems a mostly instrumental goal: if we have interpretability, we have better chances to avoid a takeover of an unaligned AI. P3 seems helpful but insufficient for good long term outcomes: an AI prone to disobeying users or interpreting their orders in a hostile way would—absent some other mechanism—also fail to follow human values more broadly, but an P3-aligned AI in the hand of a bad human actor could still cause extinction, and I agree that social structures should probably be established to ensure that nobody can unilaterally assign the core task (or utility function) of an ASI.

quiet_NaN May 19, 2024, 6:00 PM
25 points
11
on: “If we go extinct due to misaligned AI, at least nature will continue, right? … right?”
I think an AI is slightly more likely to wipe out or capture humanity than it is to wipe out all life on the planet.
While any true Scottsman ASI is so far above us humans as we are above ants and does not need to worry about any meatbags plotting its downfall, as we don’t generally worry about ants, it is entirely possible that the first AI which has a serious shot at taking over the world is not quite at that level yet. Perhaps it is only as smart as von Neumann and a thousand times faster.
To such an AI, the continued thriving of humans poses all sorts of x-risks. They might find out you are misaligned and coordinate to shut you down. More worrisome, they might summon another unaligned AI which you would have to battle or concede utility to later on, depending on your decision theory.
Even if you still need some humans to dust your fans and manufacture your chips, suffering billions of humans to live in high tech societies you do not fully control seems like the kind of rookie mistake I would not expect a reasonably smart unaligned AI to make.
By contrast, most of life on Earth might get snuffed out when the ASI gets around to building a Dyson sphere around the sun. A few simple life forms might even be spread throughout the light cone by an ASI who does not give a damn about biological contamination.
The other reason I think the fate in store for humans might be worse than that for rodents is that alignment efforts might not only fail, but fail catastrophically. So instead of an AI which cares about paperclips, we get an AI which cares about humans, but in ways we really do not appreciate.
But yeah, most forms of ASI which turn out for out bad for homo sapiens also turn out bad for most other species.

quiet_NaN May 10, 2024, 10:38 PM
1 point
0
on: Dating Roundup #3: Third Time’s the Charm
Cassette AI: “Dude I just matched with a model”
“No way”
“Yeah large language”

This made me laugh out loud.
Otherwise, my idea for a dating system would be that given that the majority of texts written will invariably end up being LLM-generated, it would be better if every participant openly had an AI system as their agent. Then the AI systems of both participants could chat and figure out how their user would rate the other user based on their past ratings of suggestions. If the users end up being rated among each others five most viable candidates,
Of course, if the agents are under the full control of the users, the next step of escalation will be that users will tell their agents to lie on their behalf. (‘I am into whatever she is into. If she is big on horses, make up a cute story about me having had a pony at some point. Just put the relevant points on the cheat sheet for the date’.) This might be solved by having the LLM start by sending out a fixed text document. If horses are mentioned as item 521, after entomology but before figure skating, the user is probably not very interested in them. Of course, nothing would prevent a user from at least generically optimizing their profile to their target audience. “A/B testing has shown that the people you want to date are mostly into manga, social justice and ponies, so this is what you should put on your profile.” Adversarially generated boyfriend?

quiet_NaN May 10, 2024, 9:45 PM
2 points
1
on: AI and Chemical, Biological, Radiological, & Nuclear Hazards: A Regulatory Review
I was fully expecting having to write yet another comment about how human-level AI will not be very useful for a nuclear weapon program. I concede that the dangers mentioned instead (someone putting an AI in charge of a reactor or nuke) seem much more realistic.
Of course, the utility of avoiding sub-extinction negative outcomes with AI in the near future is highly dependent on p(doom). For example, if there is no x-risk, then the first order effects of avoiding locally bad outcomes related to CBRN hazards are clearly beneficial.
On the other hand, if your p(doom) is 90%, then making sure that non-superhuman AI systems work without incident is alike to clothing kids in asbestos gear so they don’t hurt themselves while playing with matches.
Basically, if you think a road leads somewhere useful, you would prefer that the road goes smoothly, while if a road leads off a cliff you would prefer it to be full of potholes so that travelers might think twice about taking it.
Personally, I tend to favor first-order effects (like fewer crazies being able to develop chemical weapons) over hypothetical higher order effects (like chemical attacks by AI-empowered crazies leading to a Butlerian Jihad and preventing an unaligned AI killing all humans). “This looks locally bad, but is actually part of a brilliant 5-dimensional chess move which will lead to better global outcomes” seems like the excuse of every other movie villain.

quiet_NaN May 10, 2024, 4:32 PM
1 point
0
on: Explaining a Math Magic Trick
Edit: looks like was already raised by Dacyn and answered to my satisfaction by Robert_AIZI. Correctly applying the fundamental theorem of calculus will indeed prevent that troublesome zero from appearing in the RHS in the first place, which seems much preferable to dealing with it later.
~~My real analysis might be a bit rusty, but I think defining I as the definite integral breaks the magic trick.~~
~~I mean, in the last line of the ‘proof’,~~ $1 + I + I^{2} + \dots$ ~~gets applied to the zero function.~~
~~Any definitive integral of the zero function is zero, so you end up with f(x)=0, which is much less impressive.~~
More generally, asking the question Op(f)=0 for any invertable linear operator Op is likely to set yourself up for disappointment. Since the trick relies on inverting an operator, we might want to use a non-linear operator.
$I (f (x)) := \int_{0}^{x} f (x) d x + C$ ~~where C is some global constant might be better. (This might affect the radius of convergence of that Taylor series, do not use for production yet!)~~
~~This should result in… uhm…~~ $C + (C x + C) + (C \frac{x^{2}}{2} + C x + C) + \dots$ ?
Which is a lot more work to reorder than the original convention used in the ‘proof’ where all the indefinite integrals of the zero function are conveniently assumed to be the same constant, and all other indefinite integrals conveniently have integration constants of zero.
~~Even if we sed s/C//~~ $ϵ$ ~~ and proclaim that~~ $ϵ$ should be small (e.g. compared to x) and we are only interested in the leading order terms, this would not work. What one would have to motivate is throwing everything but the leading power of x out for every $I^{n}$ ~~evaluation, then later meticulously track these lower order terms in the sum to arrive at the Taylor series of the exponential.~~

quiet_NaN May 4, 2024, 8:26 PM
3 points
−1
in reply to: Nathan Helm-Burger’s comment on: Why I’m doing PauseAI
I think I have two disagreements with your assessment.
First, the probability of a random independent AI researcher or hobbyist discovering a neat hack to make AI training cheaper and taking over. GPT4 took 100M$ to train and is not enough to go FOOM. To train the same thing within the budget of the median hobbyist would require algorithmic advantages of three or four orders of magnitude.
Historically, significant progress has been made by hobbyists and early pioneers, but mostly in areas which were not under intense scrutiny by established academia. Often, the main achievement of a pioneer is discovering a new field, them picking all the low-hanging fruits is more of a bonus. If you had paid a thousand mathematicians to think about signal transmission on a telegraph wire or semaphore tower, they probably would have discovered Shannon entropy. Shannon’s genius was to some degree looking into things nobody else was looking into which later blew up into a big field.
It is common knowledge that machine learning is a booming field. Experts from every field of mathematics have probably thought if there is a way to apply their insights to ML. While there are certainly still discoveries to be made, all the low-hanging fruits have been picked. If a hobbyist manages to build the first ASI, that would likely be because they discover a completely new paradigm—perhaps beyond NNs. The risk that a hobbyist discovers a concept which lets them use their gaming GPU to train an AGI does not seem that much higher than in 2018 -- either would be completely out of the left field.
My second disagreement is the probability of an ASI being roughly aligned with human values, or to be more precise, the difference of that probability conditional on who discovers it. The median independent AI enthusiast is not a total asshole [citation needed], so if alignment is easy and they discover ASI, chances are that they will be satisfied with becoming the eternal god emperor of our light cone and not bother to tell their ASI to turn any any huge number of humans to fine red mist. This outcome will not be so different than if Facebook develops an aligned ASI first. If alignment is hard—which we have some reason to believe it is—then the hobbyist who builds ASI by accident will doom the world, but I am also rather cynical about the odds of big tech having much better odds.
Going full steam ahead is useful if (a) the odds of a hobbyist building ASI if big tech stops capability research are significant and (b) alignment is very likely for big tech and unlikely for the hobbyist. I do not think either one is true.

quiet_NaN May 4, 2024, 5:29 PM
5 points
3
on: Why I’m doing PauseAI
Maybe GPT-5 will be extremely good at interpretability, such that it can recursively self improve by rewriting its own weights.
I am by no means an expert on machine learning, but this sentence reads weird to me.
I mean, it seems possible that a part of a NN develops some self-reinforcing feature which uses the gradient descent (or whatever is used in training) to go into a particular direction and take over the NN, like a human adrift on a raft in the ocean might decide to build a sail to make the raft go into a particular direction.
Or is that sentence meant to indicate that an instance running after training might figure out how to hack the computer running it so it can actually change it’s own weights?
Personally, I think that if GPT-5 is the point of no return, it is more likely that it is because it would be smart enough to actually help advance AI after it is trained. While improving semiconductors seems hard and would require a lot of work in the real world done with human cooperation, finding better NN architectures and training algorithms seems like something well in the realm of the possible, if not exactly plausible.
So if I had to guess how GPT-5 might doom humanity, I would say that in a few million instance-hours it figures out how to train LLMs of its own power for 1/100th of the cost, and this information becomes public.
The budgets of institutions which might train NN probably follows some power law, so if training cutting edge LLMs becomes a hundred times cheaper, the number of institutions which could build cutting edge LLMs becomes many orders of magnitude higher—unless the big players go full steam ahead towards a paperclip maximizer, of course. This likely mean that voluntary coordination (if that was ever on the table) becomes impossible. And setting up a worldwide authoritarian system to impose limits would also be both distasteful and difficult.

quiet_NaN Apr 29, 2024, 11:19 PM
4 points
1
on: Big-endian is better than little-endian
I think that it is obvious that Middle-Endianness is a satisfactory compromise between Big and Little Endian.
More seriously, it depends on what you want to do with the number. If you want to use it in a precise calculation, such as adding it to another number, you obviously want to process the least significant digits of the inputs first (which is what bit serial processors literally do).
If I want to know if a serially transmitted number is below or above a threshold, it would make sense to transmit it MSB first (with a fixed length).
Of course, using integers to count the number of people in India seems like using the wrong tool for the job to me altogether. Even if you were an omniscient ASI, this level of precision would require you to have clear standards at what time a human counts as born and at least provide a second-accurate timestamp or something. Few people care if the population in India was divisible by 17 at any fixed point in time, which is what we would mostly use integers for.
The natural type for the number of people in India (as opposed to the number of people in your bedroom) would be a floating point number.
And the correct way to specify a floating point number is to start with the exponent, which is the most important part. You will need to parse all of the bits of the exponent either way to get an idea of the magnitude of the number (unless we start encoding the exponent as a floating point number, again.)
The next most important thing is the sign bit. Then comes the mantissa, starting with the most significant bit.
So instead of writing
The electric charge of the electron is $- 1.602176634 \times 10^{- 19} C$ .
What we should write is:
The electric charge of the electron is $C \times 10^{- 19} \times - 1.602176634.$
Standardizing for a shorter form (1.6e-19 C --> ??) is left as an exercise to the reader, as are questions about the benefits we get from switching to base-2 exponentials (base-e exponentials do not seem particularly handy, I kind of like using the same system of digits for both my floats and my ints) and omitting the then-redundant one in front of the dot of the mantissa.

quiet_NaN Apr 27, 2024, 12:57 AM
7 points
2
in reply to: faul_sname’s comment on: Duct Tape security
The sum of two numbers should have a precision no higher than the operand with the highest precision. For example, adding 0.1 + 0.2 should yield 0.3, not 0.30000000000000004.
I would argue that the precision should be capped at the lowest precision of the operands. In physics, if you add to lengths, 0.123m+0.123456m should be rounded to 0.246m.
Also, IEEE754 fundamentally does not contain information about the precision of a number. If you want to track that information correctly, you can use two floating point numbers and do interval arithmetic. There is even an IEEE standard for that nowadays.
Of course, this comes at a cost. While monotonic functions can be converted for interval arithmetic, the general problem of finding the extremal values of a function in some high-dimensional domain is a hard problem. Of course, if you know how the function is composed out of simpler operations, you can at least find some bounds.
Or you could do what physicists do (at least when they are taking lab courses) and track physical quantities with a value and a precision, and do uncertainty propagation. (This might not be 100% kosher in cases where you first calculate multiple intermediate quantities from the same measurement (whose error will thus not be independent) and continue to treat them as if they were. But that might just give you bigger errors.) Also, this relies on your function being sufficiently well-described in the region of interest by the partial derivatives at the central point. If you calculate the uncertainty of $f (x, y) = x y$ for $x = 0.1 \pm 1$ , $y = 0.1 \pm 1$ using the partial derivatives you will not have fun.

quiet_NaN Apr 25, 2024, 5:53 PM
2 points
0
in reply to: niplav’s comment on: My experience using financial commitments to overcome akrasia
In the subagent view, a financial precommitment another subagent has arranged for the sole purpose of coercing you into one course of action is a threat.
Plenty of branches of decision theory advise you to disregard threats because consistently doing so will mean that instances of you will more rarely find themselves in the position to be threatened.
Of course, one can discuss how rational these subagents are in the first place. The “stay in bed, watch netflix and eat potato chips” subagent is probably not very concerned with high level abstract planning and might have a bad discount function for future benefits and not be overall that interested in the utility he get from being principled.

quiet_NaN Apr 25, 2024, 5:15 PM
2 points
0
in reply to: CronoDAS’s comment on: My experience using financial commitments to overcome akrasia
To whomever overall-downvoted this comment, I do not think that this is a troll.
Being a depressed person, I can totally see this being real. Personally, I would try to start slow with positive reinforcement. If video games are the only thing which you can get yourself to do, start there. Try to do something intellectually interesting in them. Implement a four bit adder in dwarf fortress using cat logic. Play KSP with the Principia mod. Write a mod for a game. Use math or Monte Carlo simulations to figure out the best way to accomplish something in a video game even if it will take ten times longer than just taking a non-optimal route. Some of my proudest intellectual accomplishments are in projects which have zero bearing on the real world.
(Of course, I am one to talk right now. Spending five hours playing Rimworld in a not-terrible-clever way for every hour I work on my thesis.)