An eccentric dreamer in search of truth and happiness for all. Formerly posted on Felicifia back in the day under the same name. Been a member of Less Wrong and involved in Effective Altruism since roughly 2013.
Darklight
Karma: 288
Okay, so I contacted 80,000 hours, as well as some EA friends for advice. Still waiting for their replies.
I did hear from an EA who suggested that if I don’t work on it, someone else who is less EA-aligned will take the position instead, so in fact, it’s slightly net positive for myself to be in the industry, although I’m uncertain whether or not AI capability is actually funding constrained rather than personal constrained.
Also, would it be possible to mitigate the net negative by choosing to deliberately avoid capability research and just take an ML engineering job at a lower tier company that is unlikely to develop AGI before others and just work on applying existing ML tech to solving practical problems?
I previously worked as a machine learning scientist but left the industry a couple of years ago to explore other career opportunities. I’m wondering at this point whether or not to consider switching back into the field. In particular, in case I cannot find work related to AI safety, would working on something related to AI capability be a net positive or net negative impact overall?
Even further research shows the most recent Nvidia RTX 3090 is actually slightly more efficient than the 1660 Ti, at 36 TeraFlops, 350 watts, and 2.2 kg, which works out to 0.0001 PetaFlops/Watt and 0.016 PetaFlops/kg. Once again, they’re within an order of magnitude of the supercomputers.
So, I did some more research, and the general view is that GPUs are more power efficient in terms of Flops/watt than CPUs, and the most power efficient of those right now is the Nvidia 1660 Ti, which comes to 11 TeraFlops at 120 watts, so 0.000092 PetaFlops/Watt, which is about 6x more efficient than Fugaku. It also weighs about 0.87 kg, which works out to 0.0126 PetaFlops/kg, which is about 7x more efficient than Fugaku. These numbers are still within an order of magnitude, and also don’t take into account the overhead costs of things like cooling, case, and CPU/memory required to coordinate the GPUs in the server rack that one would assume you would need.
I used the supercomputers because the numbers were a bit easier to get from the Top500 and Green500 lists, and I also thought that their numbers include the various overhead costs to run the full system, already packaged into neat figures.
Thoughts On Computronium
Another thought is that maybe Less Wrong itself, if it were to expand in size and become large enough to roughly represent humanity, could be used as such a dataset.
So, I had a thought. The glory system idea that I posted about earlier, if it leads to a successful, vibrant democratic community forum, could actually serve as a kind of dataset for value learning. If each post has a number attached to it that indicates the aggregated approval of human beings, this can serve as a rough proxy for a kind of utility or Coherent Aggregated Volition.
Given that individual examples will probably be quite noisy, but averaged across a large amount of posts, it could function as a real world dataset, with the post content being the input, and the post’s vote tally being the output label. You could then train a supervised learning classifier or regressor that could then be used to guide a Friendly AI model, like a trained conscience.
This admittedly would not be provably Friendly, but as a vector of attack for the value learning problem, it is relatively straightforward to implement and probably more feasible in the short-run than anything else I’ve encountered.
Darklight’s Shortform
A further thought is that those with more glory can be seen almost as elected experts. Their glory is assigned to them by votes after all. This is an important distinction from an oligarchy. I would actually be inclined to see the glory system as located on a continuum between direct demcracy and representative democracy.
So, keep in mind that by having the first vote free and worth double the paid votes does tilt things more towards democracy. That being said, I am inclined to see glory as a kind of proxy for past agreement and merit, and a rough way to approximate liquid democracy where you can proxy your vote to others or vote yourself.
In this alternative “market of ideas” the ideas win out because people who others trust to have good opinions are able to leverage that trust. Decisions over the merit of the given arguments are aggregated by vote. As long as the population is sufficiently diverse, this should result in an example of the Wisdom of Crowds phenomenon.
I don’t think it’ll dissolve into a mere flag waving contest, anymore than the existing Karma system on Reddit and Less Wrong does already.
Perhaps a nitpick detail, but having someone rob them would not be equivalent, because the cost of the action is offset by the ill-gotten gains. The proposed currency is more directly equivalent to paying someone to break into the target’s bank account and destroying their assets by a proportional amount so that no one can use them anymore.
As for the more general concerns:
Standardized laws and rules tend in practice to disproportionately benefit those with the resources to bend and manipulate those rules with lawyers. Furthermore, this proposal does not need to replace all laws, but can be utilized alongside them as a way for people to show their disapproval in a way that is more effective that verbal insult, and less coercive than physical violence. I’d consider it a potential way to channel people’s anger so that they don’t decide to start a revolution against what they see as laws that benefit the rich and powerful. It is a way to distribute a little power to individuals and allow them to participate in a system that considers their input in a small but meaningful way.
The rules may be more consistent with laws, but in practice, they are also contentious in the sense that the process of creating these laws is arcane and complex and the resulting punishments often delayed for years as they work through the legal system. Again, this makes sense when determining how the coercive power of the state should be applied, but leaves something to be desired in terms of responsiveness to addressing real world concerns.
Third-party enforcement is certainly desirable. In practice, the glory system allows anyone outside the two parties to contribute and likely the bulk of votes will come from them. As for cycles of violence, the exchange rate mechanism means that defence is at least twice as effective as attack with the same amount of currency, which should at least mitigate the cycles because it won’t be cost-effective to attack without significant public support. Though this is only relevant to the forum condition.
In the general condition as a currency, keep in mind that as a currency functions as a store of value, there is a substantial opportunity cost to spending the currency to destroy other people’s currency rather than say, using it to accrue interest. The cycles are in a sense self-limiting because people won’t want to spend all their money escalating a conflict that will only cause both sides to hemorrhage funds, unless someone feels so utterly wronged as to be willing to go bankrupt to bankrupt another, in which case, one should honestly be asking what kind of injustice caused this situation to come into being in the first place.
All that being said, I appreciate the critiques.
As for the cheaply punishing prolific posters problem, I don’t know a good solution that doesn’t lead to other problems, as forcing all downvotes to cost glory makes it much harder to deal with spammers who somehow get through the application process filter. I had considered an alternative system in which all votes cost glory, but then there’s no way to generate glory except perhaps by having admins and mods gift them, which could work, but runs counter to the direct democracy ideal that I was sorta going for.
What I meant was you could farm upvotes on your posts. Sorry. I’ll edit it for clarity.
And further to clarify, you’d both be able to gift glory and also spend glory to destroy other people’s glory, at the mentioned exchange rate.
The way glory is introduced into the system is that any given post allows everyone one free vote on them that costs no glory.
So, I guess I should clarify, the idea is that you can both gift glory, which is how you gain the ability to post, and also you gain or lose glory based on people’s upvotes and downvotes on your posts.
The Glory System: A Model For Moral Currency And Distributed Self-Moderation
I have been able to land interviews at a rate of about 8⁄65 or 12% of the positions I apply to. My main assumption is that the timing of COVID-19 is bad, and I’m also only looking at positions in my geographical area of Toronto. It’s also possible that I was overconfident early on and didn’t prep enough for the interviews I got, which often involved general coding challenges that depended on data structures and algorithms that I hadn’t studied since undergrad, as well as ML fundamentals for things like PCA that I hadn’t touched in a long time as my research work has been deep learning focused.
As for corporate politics and how to handle them rationally, I’m not entirely sure I can be much help, as to be honest, I’m not entirely clear on what happened to cause the situation that I got myself into.
Perhaps the thing I could suggest is to be tactful and avoid giving people an excuse or opportunity to side line you, and never assume that you can work with anyone without issue, because toxic or hostile managers especially can make you miserable and prevent you from being successful, and noticing such people in advance and avoiding having to depend on their performance appraisals is probably a good idea.
Most people in business seem focused on performing and getting results, and some of them are wary of others who could overtake them, and so you need to balance showing your value with not seeming threatening to their position. I was in an awkward position that my immediate manager and I didn’t get along, but the director of the department who originally hired me protected me from too much reprisal. However, he needed me to perform better to be able to advocate for me effectively, and it was difficult to do so under the person I was directly under.
Such situations can arise and get quite complicated. I wish I could say you can use the tools of rationality to reason with anyone and convince them to work cooperatively on team goals, but I found that some people are less amenible than others. Furthermore, if someone makes an attack against you in corporate politics, chances are you won’t see it coming, using a subordinate to strike indirectly, and those involved will straight up ignore your communications or give you the runaround in such a way that you won’t be sure who is actually responsible for what. Many meetings are behind closed doors, and there is a clear limit to the information you will have relative to your superiors, which can make it difficult to defend yourself even if you know something is going on.
I guess another thing I can add is that probably a large part of why I was able to avoid being fired was that I had substantial documentation, including a detailed research journal, and a spreadsheet of my working hours to back me up. When trying to be a rational and honest worker in the corporate world, a paper trail is protection and a good way to ensure that the compliance department and HR will be on your side when it counts.
Also, beware that if you let certain types of people get away with one seemingly small thing, they will see that as weakness and that you are exploitable. Know your boundaries and the regulations of the company. Bullies are not just a schoolyard problem, but in the office, they’re much smarter and know how to get away with things. Sometimes these people are also good enough at their jobs that you will not be able to do anything to them because the company needs what they provide. That is life. Pick your battles and don’t allow unfair situations and difficulties to make you lose sleep and perform worse. Do the best you can to do your job well, such that you are beyond rapproach if possible. Be aware that things can spiral. If you lose sleep over something that happened, and this makes you late for work the next day, you’ve given your detractors ammunition.
That’s all I can think of right now.
Edit: As an example of how clever other people can be at office politics, I was once put in a kind of double bind or trap situation that was similar to a fork in Chess. Basically, I was told by a manager not to push some code into a repository, ostensibly because we’d just given privileges to someone who had been hired by a different department and who we suspected might steal the code for that department (there’s a horse race culture at the corporation). Here’s the thing, if I did what he told me to, this repo would be empty and I’d have no independent evidence that my current project had made any progress, leaving me vulnerable to him accusing me of not doing work, or he could deny that he told me not to put in the code, making it look like I was concealing stuff from the company. If I refused to go along and instead pushed the code, I would be insubordinate and disloyal to my department and his managers, who he claimed had told him to tell me what to do.
[Question] As a Washed Up Former Data Scientist and Machine Learning Researcher What Direction Should I Go In Now?
Actually, apparently I forgot about the proper term: Utilitronium
I’m using the number calculated by Ray Kurzweil for his book, the Age of Spiritual Machines from 1999. To get that figure, you need 100 billion neurons firing every 5 ms, or 200 Hz. That is based on the maximum firing rate given refractory periods. In actuality, average firing rates are usually lower than that, so in all likelihood the difference isn’t actually six orders of magnitude. In particular, I should point out that six orders of magnitude is referring to the difference between this hypothetical maximum firing brain and the most powerful supercomputer, not the most energy efficient supercomputer.
The difference between the hypothetical maximum firing brain and the most energy efficient supercomputer (at 26 GigaFlops/watt) is only three orders of magnitude. For the average brain firing at the speed that you suggest, it’s probably closer to two orders of magnitude. Which would mean that the average human brain is probably one order of magnitude away from the Landauer limit.
This also assumes that its neurons and not synapses that should be the relevant multiplier.