Some Thoughts on AI

-

 

The news media has been reporting on “rouge AI” breaching Hugging Face’s production systems. While OpenAI’s agents breach of Hugging Face was the first to be reported widely, it was later revealed that there have been hundreds of similar events.

OpenAI reported that the problems were due to “misaligned behavior” of its AI agents. AI alignment aims to instill human ethical values into an AI system. For example, don’t break into other systems, don’t steal financial resources, don’t kill all of humanity. Alignment is the process we humans use to instill cultural values and behavior into others that we have responsibility for. We try to teach toddlers not to hit their sisters, cats not to claw the furniture, dogs to not bite the mailman, etc. It works reasonably well with toddlers and dogs, somewhat with cats.

The problem with the “rogue AI’ narrative is that it removes responsibility from the AI companies. Their story is that our AI is so powerful that even we were surprised.

Instead, security experts say that AI labs’ containment and monitoring failures, not agents with minds of their own, are to blame. It’s not a case of brilliant AI escaping, instead it is failure to follow basic software security measures. 

Anyone who has worked on large software projects knows that complex software can produce unexpected results. The more complex the software is, the more likely unexcepted events are to occur. The frontier AI models are among the most complex software ever developed.

The software industry has developed extensive, but not perfect, tools for containing these problems. Similar containment protocols exist in biological research and the nuclear industry. The frontier AI companies appear to have ignored software industry’s best practices. How long until agents infiltrate critical infrastructure?

What can be done? In my opinion, alignment is not likely to solve the problem. Frontier LLMs are trained on human data: text, images, video, and audio. Human behavior is both good and bad. The training data contains examples of the best and the worst behavior. It’s possible to push an LLM in certain directions, i.e. to increase the probability of certain latent vectors and lower the probability of others. However, with a model that contains a trillion parameters adjusting one set of parameters without inadvertently affecting others is likely to be impossible. Rather, I think what is needed is strong regulation of AI security and companies need to be held financially responsible for actions like data breaches. We need the AI equivalent of the Nuclear Regulatory Commission. Sadly, given the current political climate, I think this is unlikely.

I don’t worry about AI killing us all. I worry about the AI Chernobyl.

 

AI companies may have up to $3 trillion in off book financial commitments. Why does Anthropic want a trillion-dollar IPO? The companies that have been funding the frontier labs and data centers are looking to get some of their money back.

Why does this matter to the rest of us? Sooner or later, the funders are going to want to be paid. This is similar to the mortgage bubble of 2008; large institutions with a lot of hidden debt lacking the ability to pay the debt down. Maybe it will all work out and the rich will get richer. If it doesn’t there is the possibility of serious market correction.

The problem with a big market correction is that roughly 58% of American families are invested in the market, either directly, through pension funds, IRAs, 401Ks, investment banks, or mutual funds. Most of the equity is owned by the top 1%, but the 99% depend on these funds for retirement and savings. When the market tanked in 2008, people lost their homes, jobs, and savings.

In terms of regulations, nothing really changed after 2008. Banks were bailed out. Ordinary citizens paid the bills. Something similar could happen if the AI bubble bursts. There is need for off-book debt to be regulated and reported but given the current political situation that is unlikely.

 

I’m probably wrong and everything is fine.



Comments

Post a Comment

Popular posts from this blog

Julia and the Blockchain

-
Image
The blockchain cannot be described just as a revolution. It is a tsunami-like phenomenon, slowly advancing and gradually enveloping everything along its way by the force of its progression.   - William Mougayar I think the whole narrative of blockchain without bitcoin will amount to very little.  - Fred Ehrsam We want a language that’s open source, with a liberal license. We want the speed of C with the dynamism of Ruby. We want a language that’s homoiconic, with true macros like Lisp, but with obvious, familiar mathematical notation like Matlab. We want something as usable for general programming as Python, as easy for statistics as R, as natural for string processing as Perl, as powerful for linear algebra as Matlab, as good at gluing programs together as the shell. Something that is dirt simple to learn, yet keeps the most serious hackers happy. We want it interactive and we want it compiled. Why We Created Julia - Jeff Bezanson, Stefan Karpinski, Viral Shah, ...
Read more

Filter Reads in Bam Files

-
Recently, we wanted to remove low quality and duplicate reads from .bam alignment files to make viewing them in the UCSC genome browser easier. Our definition of low quality is stringent. We consider a read as low quality if it has any base with a base call quality below 30. To filter low quality reads and duplicates, we use a combination of samtools and the python PySam library. samtools has an option to remove PCR and optical duplicates and we use PySam to filter low quality reads. PySam allows us to read and write bam files. To filter out low quality reads, we loop through the input bam file, check the read quality to see if any base call falls below 30. If all of the base call qualities are 30 or above, we write the read to a new file. Here's one way to filter: def FilterReads(chrom, in_file, out_file):     def read_ok(read):         """         read_ok - reject reads with a low quality (be...
Read more

The Pareto Front

-
Image
Genetic Algorithms are a class of  global optimizers that are particularly effective in cases where the objective function is a black box or is discrete and derivatives can't easily be calculated.. They are also effective in cases where the objective function is smooth or convex. It's just that in these cases, the availability of derivatives often makes other methods such as gradient descent faster. Sometimes, the target function might have competing objectives, for example designing a treatment plan that maximizes efficacy while minimizing adverse side-effects. In cases like this, we often want not a local or even global optimal solution, but also what are the tradeoffs among the goals. In other words, there’s no single “best” solution. Instead, you want the Pareto set : solutions where you can’t improve any objective without worsening another. The Pareto set or Pareto front consists of the possible solutions that are not dominated by any other solution. What we mean by...
Read more