Monday, October 7, 2019

Axis aligned artifacts

Left: Original data distribution. Right: Learned co-displacement, darker is lower.
Notice the echoes around (10,-10) and (-10, 10)

There are minor artifacts created by choosing axis aligned cuts in RRCF, similar to what was noted with IsoForest. 

Friday, October 4, 2019

State of Astro-informatics

I had a glance through "Realizing the potential of astrostatistics and astroinformatics" by Eadie et al. (2019). While I do not feel qualified or informed to comment on the suggestions, I can summarize them quickly. There are three problems:

  1. Education: Most astronomers are not trained in code development resulting in maybe good but fragile code. Similarly, most computer scientists don't have the astronomy background or connections. 
  2. Funding: Grants for methodology improvement are scarce. I wonder if these things can be funded from the computer science side of things in collaborations. 
  3. Quality: Astro-informatics lacks support of state-of-the-art methodology as it stands. 
I was much more interested in the final section about potential themes in research:
  1. Nonlinear dimensionality reduction.
  2. Sparsity.
  3. Deep learning.
I find the last theme incredibly broad and am unclear exactly how they mean it. It seems they're most interested in hierarchical representations of data. I would also claim that anomaly detection/clustering is important for reducing the volume of data.