Posts

Showing posts with the label language model

Hallucinations

Image
Created by Dall-e https://chatgpt.com/g/g-2fkFE8rbu-dall-e/c/6772fc7f-2754-800d-ab66-c2271a4fbde9 I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful. It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does. — Andrej Karpathy I hate the term "hallucination" when applied to Large Language Models (LLMs). Hallucination implies that there is some sort of entity that is mis-perceiving reality, like a teenager in a bad 60s LSD movie. In one sense LLMs can't hallucinate, there is no entity to mis-perceive anything. When an LLM pro...

Programming With ChatGPT

Image
You may have read stories like this on Reddit or other sites: "Two weeks ago I couldn't program at all, now, thanks to ChatGPT , I'm a senior software engineer at Google." That is a bit of an exaggeration, but sites like r/ChatGPT  often contain quite a bit of AI hype. I was recently working on using PyTorch to detect change points in time series sequences.  I have written about change point detection before: here , here , here , and here . There are numerous methods for detecting regions of change in series data depending on the characteristics of the data. I was interested to see what ChatGPT could contribute. Change point detection algorithms are traditionally classified as online or offline. Offline methods have the entire data sequence available. Online algorithms process each data point as it arrives. I'm interested in offline change point detection. This paper gives a good overview of some of the common detection methods. Generating Data My first step wa...