The Analytic Garden

Hyperdimensional  computing recently made the news . This article gives some examples of hyper vector databases. In a previous post , we looked at hyperdimensional computing (HDC) using high dimensional vectors. As we saw, orthogonality is an important property of high-d vectors. We said that two random (-1, 1) high-d vectors were orthogonal. What do we really mean by that? In these lecture notes , Kothari and Arora show the following result, \[P\left( {\left| {\cos ({\theta _{x,y}})} \right| > \sqrt {\frac{{\log (c)}}{N}} } \right) < \frac{1}{c}\]. What this is saying is that probability that the absolute value of the cosine of the angle between randomly generated high-d vectors is greater that a simple function of N and c . N is the dimension of the random vectors. c is an arbitrary constant. We can choose c to adjust the probability. What is a good choice for c ?  If we choose $c = {e^{0.01N}}$, then $\sqrt {\frac{{\log (c)}}{N}}  = 0.1$. In other ...

I write because I don't know what I think until I read what I say. - Flannery O'Connor I program because I don't understand a subject until I see the code that I have written. - me Artificial neural networks (ANNs) are sometimes described by analogy to biological neurons. However, their mode of operation is not similar to biological brains. It's impossible to deny the impressive success of NN models like chatGPT-3 and 4 or DALL-E-2. Unfortunately, training these models requires very large computing infrastructure available to only a few institutions. chatGPT was trained using Microsoft's Azure cloud system. OpenAI has not disclosed the length of time required to train GPT-3, complicating the researchers’ estimations². However, Microsoft has built supercomputers for AI training and says that its latest supercomputer contains 10,000 graphics cards and over 285,000 processor cores². The process of training ChatGPT involves a combination of data preparation, model des...

 It's been a while since I looked at this data. I was updating my version of R and decided this would be a good exercise to test it and also to see what Sars-COV-2 variants were on the rise. I downloaded metadata from GISAID on April 20, 2023. The file contained records for 15,398,189 sequences. I took Pango lineages for the top four emerging and used the method described here to plot the accumulation of sequences for the variants. system.time(df_variants_usa <- get_selected_variants_ch( "data/metadata.tsv" , selected_variants = selected_variants, title = "Selected Variants USA 2023-04-20" , start_date = as.Date( "2022-09-01" ))) Here's the plot. You can find the code to produce the plot here .

chatGPT and its friends have been all over the news. One of its uses is to generate program code. In fact, this article even predicts it will replace programmers. Is your job in danger? I think, you're probably safe for a little while. I have seen chatGPT tools promoted as a way for non-coders to generate code without the pain of learning Python, JavaScript, or some other language. Like it's use for text generation, it does some thing reasonably well, However, I think programming noobs should be careful. I tried to generate some very simple code using the code-davinci-002 model.  prompt = """write a python function that accepts a string and returns a dictionary with each letter in the text as key and the letter frequency as value""" Here's the code that makes the request. #!/usr/bin/env python # -*- coding: utf-8 -*- """ test_GPTcode.py - test GP# code generation author: Bill Thompson license: GPL 3 copyright:...

chatGPT and its cousins have been making news lately. The media seems enthralled with attempts to hack GPT based systems into producing outrageous statements. It's somewhat like ten year olds saying dirty words into a recorder and giggling when the recorder repeats them. It doesn't seem like much of  a trick. What follows isn't much of a trick either. It should come as no surprise that large language models (LLMs) trained on the internet might produce output that is less than factual. They are trained to produce plausible not factual text.  I have read about the supposed dangers of students using LLMs to submit essays in academic classes. I thought I might try to get GPT to produce a short paper on a medical subject. I signed up for an account with OpenAI  and obtained an API key so I could connect with one of the LLMs without going through the public website. There are a large number of  GPT models, 66 in total, to choose from when selecting a text generator. I c...

This post is an addendum to a previous post . On January 9 2023, I downloaded a SAR-CoV-2 metadata file from GISAID. The file contained 14,499,984 observations of 24 variables. I was interested in the growth of the XBB.1.5 variant. XBB.1.5, also know as the Kraken subvarian t, has been in the news lately because it appears to have greater transmissibility than the other Omicron variants. Here's the plot of the top 5 emerging variants from GISAID: BQ.1.1.22, CH.1.1, XBB.1.5, BQ.1.1, BQ.1.23. XBB.1.5 isn't the major variant yet, but it is growing in the US. The plot above was created with the program  get_selected_variants.R from GitHub . The code was described in the previous post . When I ran  get_selected_variants.R, on Linux R version 4.1.2  it produced a warning. > system.time(df_variants_usa <- get_selected_variants( 'data/metadata.tsv' , selected_variants = selected_variants, title = 'Selected Variants USA 2023-01-09' , start_date = as.Date...

 On November 22, 2022, I downloaded a metadata file containing 13,932,236 records from GISAID  in order to view the growth of several emerging SARS-CoV-2 variants of interest. First, the results. I plotted the growth of the variants BA.2.75, BQ.1, BQ.1.1, BQ.1.18, CL.1, and XBB in the USA from June 1 to the present.  BQ.1. and BQ.1.1 are growing in the data. It's unclear yet whether they will have a large impact. We'll see after the Thanksgiving holidays. Memory Issues The metadata file is large, 9.8 GB on the disk. Loading into R on Windows takes a little over two minutes and used about 8 GB for the dataframe. For the purposes of demonstration, I'm using Windows 10 22H2, RStudio 2022.07.2, and R 4.2.2. > system.time(meta_data <- read_tsv( 'data/metadata.tsv' , name_repair = 'universal' )) user system elapsed 197.84 9.44 146.04 > object.size(meta_data) 8087686552 bytes > RStudio uses 9.6 GB once the data is loaded. PS...