The Analytic Garden

In a previous post , I wrote about Hyperdimensional computing (HDC) and showed some simple code illustrating the use of high dimensional vectors (HDVs). There is a large technical literature about HDC, but it's spread over several different disciplines and under several names, including Hyperdimensional Computing, Vector Symbolic Architectures (VSA),  Holographic Reduced Representations (HRR), and others. There are variations in implementation among the different models, but all use some form of HDVs. A two part survey, part 1 and part 2 , provide a comprehensive description of the field and the various approaches. Despite the number of academic articles, I was only able to find two working software packages implementing HDC, VSA  for R, implementing the HRR model, and torchhd for Python.  VSA VSA implements the HRR model . I was able to install VSA on R 4.2.3, but when I tried to install it with the latest version of R, 4.3.0, it fails to install due to an update in ...

Hyperdimensional  computing recently made the news . This article gives some examples of hyper vector databases. In a previous post , we looked at hyperdimensional computing (HDC) using high dimensional vectors. As we saw, orthogonality is an important property of high-d vectors. We said that two random (-1, 1) high-d vectors were orthogonal. What do we really mean by that? In these lecture notes , Kothari and Arora show the following result, \[P\left( {\left| {\cos ({\theta _{x,y}})} \right| > \sqrt {\frac{{\log (c)}}{N}} } \right) < \frac{1}{c}\]. What this is saying is that probability that the absolute value of the cosine of the angle between randomly generated high-d vectors is greater that a simple function of N and c . N is the dimension of the random vectors. c is an arbitrary constant. We can choose c to adjust the probability. What is a good choice for c ?  If we choose $c = {e^{0.01N}}$, then $\sqrt {\frac{{\log (c)}}{N}}  = 0.1$. In other ...

I write because I don't know what I think until I read what I say. - Flannery O'Connor I program because I don't understand a subject until I see the code that I have written. - me Artificial neural networks (ANNs) are sometimes described by analogy to biological neurons. However, their mode of operation is not similar to biological brains. It's impossible to deny the impressive success of NN models like chatGPT-3 and 4 or DALL-E-2. Unfortunately, training these models requires very large computing infrastructure available to only a few institutions. chatGPT was trained using Microsoft's Azure cloud system. OpenAI has not disclosed the length of time required to train GPT-3, complicating the researchers’ estimations². However, Microsoft has built supercomputers for AI training and says that its latest supercomputer contains 10,000 graphics cards and over 285,000 processor cores². The process of training ChatGPT involves a combination of data preparation, model des...

 It's been a while since I looked at this data. I was updating my version of R and decided this would be a good exercise to test it and also to see what Sars-COV-2 variants were on the rise. I downloaded metadata from GISAID on April 20, 2023. The file contained records for 15,398,189 sequences. I took Pango lineages for the top four emerging and used the method described here to plot the accumulation of sequences for the variants. system.time(df_variants_usa <- get_selected_variants_ch( "data/metadata.tsv" , selected_variants = selected_variants, title = "Selected Variants USA 2023-04-20" , start_date = as.Date( "2022-09-01" ))) Here's the plot. You can find the code to produce the plot here .

chatGPT and its friends have been all over the news. One of its uses is to generate program code. In fact, this article even predicts it will replace programmers. Is your job in danger? I think, you're probably safe for a little while. I have seen chatGPT tools promoted as a way for non-coders to generate code without the pain of learning Python, JavaScript, or some other language. Like it's use for text generation, it does some thing reasonably well, However, I think programming noobs should be careful. I tried to generate some very simple code using the code-davinci-002 model.  prompt = """write a python function that accepts a string and returns a dictionary with each letter in the text as key and the letter frequency as value""" Here's the code that makes the request. #!/usr/bin/env python # -*- coding: utf-8 -*- """ test_GPTcode.py - test GP# code generation author: Bill Thompson license: GPL 3 copyright:...

chatGPT and its cousins have been making news lately. The media seems enthralled with attempts to hack GPT based systems into producing outrageous statements. It's somewhat like ten year olds saying dirty words into a recorder and giggling when the recorder repeats them. It doesn't seem like much of  a trick. What follows isn't much of a trick either. It should come as no surprise that large language models (LLMs) trained on the internet might produce output that is less than factual. They are trained to produce plausible not factual text.  I have read about the supposed dangers of students using LLMs to submit essays in academic classes. I thought I might try to get GPT to produce a short paper on a medical subject. I signed up for an account with OpenAI  and obtained an API key so I could connect with one of the LLMs without going through the public website. There are a large number of  GPT models, 66 in total, to choose from when selecting a text generator. I c...