GA Vibes Part 2

-

 As a final exercise for the change point Genetic Algorithm (GA), I'll apply it to global temperature anomaly data from climate.gov.


I applied the GA to this data.


$ python src/ga_cp_bic.py /mnt/c/Users/bbth/OneDrive/analytic_garden/GA/data/Temperature_1880_2024.csv -R 5 -m 7

Num Breakpoints: 0
Restart: 1 MSE: 0.10286155700349382 BIC: -319.83037125244255
Restart: 2 MSE: 0.10780974084485254 BIC: -313.01768583106514
Restart: 3 MSE: 0.08734095963729796 BIC: -343.5472153386782
Restart: 4 MSE: 0.10132137776725991 BIC: -322.0179216010041
Restart: 5 MSE: 0.10279682698913807 BIC: -319.9216473989269

Train MSE: 0.08734095963729796
Random number seed: 1757604284
Breaks: []
Slopes: [0.00282703]
Initial slope: -5.444485953515934
BIC: -343.5472153386782

Num Breakpoints: 1
Restart: 1 MSE: 0.017966374806006904 BIC: -562.8847985227927
Restart: 2 MSE: 0.02173895666295018 BIC: -535.2472265839913
Restart: 3 MSE: 0.019616340777016433 BIC: -550.1449553421734
Restart: 4 MSE: 0.019261114065503186 BIC: -552.7947844964565
Restart: 5 MSE: 0.01750368343862006 BIC: -566.6679360394593

Train MSE: 0.01750368343862006
Random number seed: 1757604284
Breaks: [1964.67750704]
Slopes: [0.00082202 0.01918589]
Initial slope: -1.757743468295864
BIC: -566.6679360394593

Num Breakpoints: 2
Restart: 1 MSE: 0.012558768692663756 BIC: -604.853340277318
Restart: 2 MSE: 0.01243195194604011 BIC: -606.3249733982208
Restart: 3 MSE: 0.012256993676719138 BIC: -608.3800934600993
Restart: 4 MSE: 0.012404091892174283 BIC: -606.6502836232555
Restart: 5 MSE: 0.012360294729070118 BIC: -607.1631649126923

Train MSE: 0.012256993676719138
Random number seed: 1757604284
Breaks: [1910.99769156 1978.23065214]
Slopes: [-0.00083913  0.00547718  0.02034855]
Initial slope: 1.3201818192165882
BIC: -608.3800934600993

Num Breakpoints: 3
Restart: 1 MSE: 0.012045151521528737 BIC: -600.9546242498568
Restart: 2 MSE: 0.012163941059597838 BIC: -599.531636485958
Restart: 3 MSE: 0.011603136914732033 BIC: -606.3757001535032
Restart: 4 MSE: 0.01190177761619636 BIC: -602.6909190843479
Restart: 5 MSE: 0.011670814364981009 BIC: -605.5324176504653

Train MSE: 0.011603136914732033
Random number seed: 1757604284
Breaks: [1911.64976897 1975.72513814 2011.81844148]
Slopes: [-0.00048812  0.00554298  0.01736572  0.03586234]
Initial slope: 0.6524353229392093
BIC: -606.3757001535032

Num Breakpoints: 4
Restart: 1 MSE: 0.011243692142443392 BIC: -600.985123450339
Restart: 2 MSE: 0.010865804227730874 BIC: -605.9421865319424
Restart: 3 MSE: 0.012045515153927539 BIC: -590.996779410205
Restart: 4 MSE: 0.010284096448293435 BIC: -613.9203711913805
Restart: 5 MSE: 0.010369721033699245 BIC: -612.7181105416433

Train MSE: 0.010284096448293435
Random number seed: 1757604284
Breaks: [1919.53490314 1938.10421616 1972.5892532  2021.97580519]
Slopes: [1.30486670e-05 1.31599883e-02 9.79989807e-04 1.94837933e-02
 1.52292353e-01]
Initial slope: -0.31905209887440406
BIC: -613.9203711913805

Num Breakpoints: 5
Restart: 1 MSE: 0.010563575952060386 BIC: -600.0789918903789
Restart: 2 MSE: 0.01090483758234514 BIC: -595.4687672208581
Restart: 3 MSE: 0.012853939170602405 BIC: -571.6244149255311
Restart: 4 MSE: 0.011618067543810444 BIC: -586.2823027185707
Restart: 5 MSE: 0.013196558413601666 BIC: -567.8100804868834

Train MSE: 0.010563575952060386
Random number seed: 1757604284
Breaks: [1898.04502854 1909.02150721 1911.94399846 1939.8396544  1976.88305957]
Slopes: [-0.00011573 -0.00953683  0.00783257  0.00878155  0.00372577  0.02041029]
Initial slope: -0.027933942052242734
BIC: -600.0789918903789

Num Breakpoints: 6
Restart: 1 MSE: 0.019633770680921862 BIC: -500.2488368195941
Restart: 2 MSE: 0.011184138493904447 BIC: -581.8482406068417
Restart: 3 MSE: 0.010112379981382221 BIC: -596.454983032689
Restart: 4 MSE: 0.012608653893720186 BIC: -564.4646507590862
Restart: 5 MSE: 0.011814209204126873 BIC: -573.9013114106497

Train MSE: 0.010112379981382221
Random number seed: 1757604284
Breaks: [1917.28923842 1926.43721336 1941.79939384 1950.4680748  1972.97878809
 2020.78135042]
Slopes: [-0.00037242  0.0045773   0.02257268 -0.0176755   0.00322486  0.02033635
  0.07903338]
Initial slope: 0.38934946340024545
BIC: -596.454983032689

Num Breakpoints: 7
Restart: 1 MSE: 0.018475132314107433 BIC: -499.11505942658897
Restart: 2 MSE: 0.010834309939876826 BIC: -576.5026735630659
Restart: 3 MSE: 0.013694116102149087 BIC: -542.5366680207851
Restart: 4 MSE: 0.009965352036705595 BIC: -588.6252049219258
Restart: 5 MSE: 0.011517972724760742 BIC: -567.630019724583

Train MSE: 0.009965352036705595
Random number seed: 1757604284
Breaks: [1898.87137158 1911.03762661 1936.98854563 1942.96865935 1966.4471147
 1986.32893246 2018.65593223]
Slopes: [-4.00679079e-04 -1.34579498e-02  1.21892843e-02  1.03684266e-02
  4.70006514e-05  8.84254578e-03  2.27507740e-02  4.16041057e-02]
Initial slope: 0.5241383104699019
BIC: -588.6252049219258

====================================
Best Model
Train MSE: 0.010284096448293435
Random number seed: 1757604284
Breaks: [1919.53490314 1938.10421616 1972.5892532  2021.97580519]
Slopes: [1.30486670e-05 1.31599883e-02 9.79989807e-04 1.94837933e-02
 1.52292353e-01]
Initial slope: -0.31905209887440406
BIC: -613.9203711913805
$

The predicted change points match the changes in the global temperature anomaly reasonably well.


The code and data for the GA can be found on GitHub.

Comments

Post a Comment

Popular posts from this blog

Julia and the Blockchain

-
Image
The blockchain cannot be described just as a revolution. It is a tsunami-like phenomenon, slowly advancing and gradually enveloping everything along its way by the force of its progression.   - William Mougayar I think the whole narrative of blockchain without bitcoin will amount to very little.  - Fred Ehrsam We want a language that’s open source, with a liberal license. We want the speed of C with the dynamism of Ruby. We want a language that’s homoiconic, with true macros like Lisp, but with obvious, familiar mathematical notation like Matlab. We want something as usable for general programming as Python, as easy for statistics as R, as natural for string processing as Perl, as powerful for linear algebra as Matlab, as good at gluing programs together as the shell. Something that is dirt simple to learn, yet keeps the most serious hackers happy. We want it interactive and we want it compiled. Why We Created Julia - Jeff Bezanson, Stefan Karpinski, Viral Shah, ...
Read more

Filter Reads in Bam Files

-
Recently, we wanted to remove low quality and duplicate reads from .bam alignment files to make viewing them in the UCSC genome browser easier. Our definition of low quality is stringent. We consider a read as low quality if it has any base with a base call quality below 30. To filter low quality reads and duplicates, we use a combination of samtools and the python PySam library. samtools has an option to remove PCR and optical duplicates and we use PySam to filter low quality reads. PySam allows us to read and write bam files. To filter out low quality reads, we loop through the input bam file, check the read quality to see if any base call falls below 30. If all of the base call qualities are 30 or above, we write the read to a new file. Here's one way to filter: def FilterReads(chrom, in_file, out_file):     def read_ok(read):         """         read_ok - reject reads with a low quality (be...
Read more

School Shootings Are Political Violence

-
Image
It’s worth it to have a cost of, unfortunately, some gun deaths every single year so that we can have the Second Amendment to protect our other God given rights. ~ Charlie Kirk Charlie Kirk was killed on September 10, 2025. On the same day in Evergreen CO, two students were critically wounded during a shooting at school . Kirk's death resulted in politicians and pundits either declaring him a martyr or a victim of his own rhetoric.  Before September 10, I knew nothing about Charlie Kirk. From what I can gather since then, he shared views similar to those of many on the US right: contempt for LGBQ folks, trans people, black and brown people, and just about anyone not white, male, and christian. Flags were ordered to be flown at half-mast in honor of Kirk. The students shot in Colorado barely received thoughts and prayers . The news coverage of the Kirk shooting prompted me to look at shooting incidents, in particular school shootings. 2025 School Shootings in the USA I decided to l...
Read more