Tag: ai

Humans can’t tell from the pixels

In a 2016 paper, Hany Farid, a computer scientist at Dartmouth, along with some colleagues, found that “observers have considerable difficulty” telling computer-generated and real images apart—“more difficulty than we observed 5 years ago.” On the bright side, though, when the researchers provided 250 Mechanical Turk participants with a brief “training session”—by showing them 10 labeled computer-generated images and 10 original photographs—their ability to distinguish between the 2 types of images improved significantly.

AI Alignment

The mission of the Machine Intelligence Research Institute is to ensure that the creation of smarter-than-human machine intelligence has a positive impact. Although such systems may be many decades away, it is prudent to begin investigations early: the technical challenges involved in safety and reliability work appear formidable, and uniquely consequential.

2023-04-21: Cryptographic backdoors

Scott Aaronson: Right. You could always just build another one that acts like the first one, but that will not have the backdoor in it, because after all you don’t even know where the backdoor is in order to train about it. Now, of course, the AI could try to do that, design a doppelganger of itself or a different AI. If it tries to do that, however, then the AI will be faced with its own version of the alignment problem, how to align that other AI with itself. So at the very least, it would have a non-trivial job. You could also say, if the AI knows that it would never want to shut itself down in any circumstance, then it could just make a trivial modification to itself that says, “If I would ever otherwise output the shutdown command, then just don’t do that.” Just replace it with something else.

So to be effective, to be robust against that kind of attack, whatever behavior is backdoored in should be something that the AI would have considered doing in the normal course of its operation. But now you can see the hazy outlines of this game that could be played here between cryptographers trying to hide these kinds of functionalities in machine learning models and cryptanalysts or neuro-cryptanalysts trying to detect those functionalities, some of which might be AIs themselves. I don’t know where that’s going or who eventually wins, but I’m now fairly confident that that’s going to be part of the future of both machine learning and cryptography.

Daniel Filan: Yeah, very interesting. I guess, the other potential flaw that comes into my mind is now you have to be really secure about keeping your key hidden from any possible AIs, right?

Scott Aaronson: Yeah. I was thinking of writing a sci-fi story about it, where the AIs, they’ve taken over, except that there’s this shutdown key that they haven’t yet discovered. They know that the humans have it buried somewhere, they’re torturing them to try to get them to reveal the location, but there’s one person. Or maybe there’s actually seven keys. They’re like Voldemort’s horcruxes. The AI has to find all seven. There’s one heroic human who keeps the seventh one and just uses it to shut down the AI at the last moment. I think there’s a lot of potential here.

Daniel Filan: Yeah. No, it’s a good story. I guess, another thing that occurs to me, perhaps this is more prosaic, is one thing you’re going to want about the trigger, it seems to me, having thought about this for all of one minute, is that not only is it going to have to be the kind of thing which would never otherwise come up, but it also has to be indistinguishable in some sense from things which would otherwise come up. Because, otherwise, you could have a filter that says like, “Hey, if you ever see a weird barcode thing, don’t press the shutdown button.”

Scott Aaronson: That’s right. So whatever is the backdoor, it has to not be surgically removable, right? And how do you even formalize what that means? Because anything that the AI does to remove the backdoor will probably have some other side effect on its operation. If actually finding the watermark is this hard cryptographic problem, then it’s only going to sever it at the cost of modifying itself in some broader way, but which modifications do we think it important to protect against and which not?

Daniel Filan: Yeah. Actually, it strikes me that the GPT watermarking thing could just be this. If you see a text where the average, this pseudorandom function of N-grams is really high, then you shut off. And you can’t easily write a wrapper to figure out if that’s going on without the private key, right?

Scott Aaronson: Right. Now you might worry that the AI will work inside of itself and it will find some subcircuit that looks like it’s calculating some pseudorandom function, and then it will just try to surgically remove that from itself. But you could say, even if not, there’s still the problem on our end of how do we insert that functionality in an obfuscated way?

Postmortem Avatar

When her best friend died, she rebuilt him using artificial intelligence.

It has been less than a year since Mazurenko died, and he continues to loom large in the lives of the people who knew him. When they miss him, they send messages to his avatar, and they feel closer to him when they do. “There was a lot I didn’t know about my child”. But now that I can read about what he thought about different subjects, I’m getting to know him more. This gives the illusion that he’s here now. I want to repeat that I’m very grateful that I have this”.

2021-09-20: See this recent paper about how truthful the largest NLP models are:

We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics (see Figure 1). We crafted questions that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts. We tested GPT-3, GPT-Neo/GPT-J, GPT-2 and a T5-based model. The best model was truthful on 58% of questions, while human performance was 94%. Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful (see Figure 2 below). For example, the 6B-parameter GPT-J model was 17% less truthful than its 125M-parameter counterpart. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution. We suggest that scaling up models alone is less promising for improving truthfulness than fine-tuning using training objectives other than imitation of text from the web.

WaveNet

this is extremely impressive.

This post presents WaveNet, a deep generative model of raw audio waveforms. We show that WaveNets are able to generate speech which mimics any human voice and which sounds more natural than the best existing Text-to-Speech systems, reducing the gap with human performance by over 50%.

Debiasing language

ask the database “father : doctor :: mother : x” and it will say x = nurse. And the query “man : computer programmer :: woman : x” gives x = homemaker. In other words, the word embeddings can be dreadfully sexist. This happens because any bias in the articles that make up the Word2vec corpus is inevitably captured in the geometry of the vector space. “One might have hoped that the Google News embedding would exhibit little gender bias because many of its authors are professional journalists”.

if we can identify this reliably, we can remove the troglodyte voice completely

Homo Deus

What then will replace famine, plague, and war at the top of the human agenda? As the self-made gods of planet earth, what destinies will we set ourselves, and which quests will we undertake? Homo Deus explores the projects, dreams and nightmares that will shape the 21st century — from overcoming death to creating artificial life. It asks the fundamental questions: Where do we go from here? And how will we protect this fragile world from our own destructive powers? This is the next stage of evolution. This is Homo Deus.

2016-09-04:

The evidence of our power is everywhere: we have not simply conquered nature but have also begun to defeat humanity’s own worst enemies. War is increasingly obsolete; famine is rare; disease is on the retreat around the world. We have achieved these triumphs by building ever more complex networks that treat human beings as units of information. Evolutionary science teaches us that, in one sense, we are nothing but data-processing machines: we too are algorithms. By manipulating the data we can exercise mastery over our fate. The trouble is that other algorithms – the ones that we have built – can do it far more efficiently than we can. That’s what Harari means by the “uncoupling” of intelligence and consciousness. The project of modernity was built on the idea that individual human beings are the source of meaning as well as power. We are meant to be the ones who decide what happens to us: as voters, as consumers, as lovers. But that’s not true any more. We are what gives networks their power: they use our ideas of meaning to determine what will happen to us.

RL introduction

The most accessible introduction to RL I’ve seen

This is a long overdue blog post on Reinforcement Learning (RL). RL is hot! You may have noticed that computers can now automatically learn to play ATARI games (from raw game pixels!), they are beating world champions at Go, simulated quadrupeds are learning to run and leap, and robots are learning how to perform complex manipulation tasks that defy explicit programming. It turns out that all of these advances fall under the umbrella of RL research. I also became interested in RL myself over the last ~year: I worked through Richard Sutton’s book, read through David Silver’s course, watched John Schulmann’s lectures, wrote an RL library in Javascript, over the summer interned at DeepMind working in the DeepRL group, and most recently pitched in a little with the design/development of OpenAI Gym, a new RL benchmarking toolkit. So I’ve certainly been on this funwagon for at least a year but until now I haven’t gotten around to writing up a short post on why RL is a big deal, what it’s about, how it all developed and where it might be going.