SDSS Data Access

How to get hold of data from the SDSS astronomical catalogue.

When it comes to astronomical catalogues, SDSS was without peer for many years. We used it routinely for our work on differential photometry, and developed ways of accessing SDSS data directly to R. This post is really just a reminder-to-self, so that next time I need to get my SDSS data I won’t need to scrabble around looking for old scripts.

The packages needed are tidyverse (of course), RCurl, and glue.

N <- 5 # makes a table of 5 star targets
# we'll just get the spectrum from the first one
# delta <- 0.1 # 
bands_min <- 15 # this places an upper limit on the brightness of the star
bands_max <- 20 # this places an lower limit on the brightness of the star
# fainter stars have noisy spectra

# SQL that downloads some info on the chosen target from SDSS.
# ObjID from SDSS specifies the target

master_target_SqlQuery <- glue("SELECT top {N} p.ra, p.dec, ",
                       "p.u, p.g, p.r, p.i, p.z, p.objid, ", 
                       "s.specobjid, s.class, s.subclass, s.survey, ", 
                       "s.plate, s.mjd, s.fiberid ", 
                       "FROM photoObj AS p ", 
                       "JOIN SpecObj AS s ON s.bestobjid = p.objid ",
                       "WHERE p.g BETWEEN {bands_min} AND {bands_max} ",
                       "AND p.r BETWEEN {bands_min} AND {bands_max} ", 
                       "AND p.i BETWEEN {bands_min} AND {bands_max} ", 
                       "AND s.class = 'STAR' ",
                       "AND s.survey != 'eboss'" )

# downloads target data
# dataframe master_targets has necessary info
master_target_SqlQuery <- str_squish(master_target_SqlQuery)
urlBase <- "http://skyserver.sdss.org/dr17/SkyserverWS/SearchTools/SqlSearch?"
X <- getForm(urlBase, cmd = master_target_SqlQuery, format = "csv")
master_targets <- read.table(text = X, header = TRUE, sep = ",", dec = ".", comment.char = "#")

ra	dec	u	g	r	i	z	objid	specobjid	subclass	plate	mjd	fiberid
167.0870	26.15828	16.51	15.64	15.38	15.28	15.27	1237667430104694784	7220539695150946304	F0IV (81937)	6413	56336	522
167.0042	26.44916	19.31	18.40	18.14	18.04	18.02	1237667323249950976	7220550140511410176	F3/F5V (30743)	6413	56336	560
167.3888	25.94575	20.15	19.57	19.37	19.28	19.25	1237667429567955200	7220550690267224064	CV	6413	56336	562
167.4996	26.31292	15.61	15.67	15.96	16.22	16.47	1237667430104891392	7220551240023037952	O8e (188001)	6413	56336	564
167.7413	26.13576	19.60	18.68	18.35	18.28	18.21	1237667322713341952	7220564434162571264	F3/F5V (30743)	6413	56336	612

There are times when the SDSS data server is down. In this case, expect to see an error message like - “Error: InternalServerError”.

Now that we have the plate, mjd, and fiberid for some stars, we can go ahead and download their spectra.

index <- 1 # uses first star from list
get_spectrum <- function(object, wavelength_lower_limit = 5500, wavelength_upper_limit = 7000){
  plate <- object$plate
  mjd <- object$mjd
  fiber <- object$fiberid
  url_spect <- glue("http://dr12.sdss.org/csvSpectrum?plateid={plate}", 
                    "&mjd={mjd}&fiber={fiber}&reduction2d=v5_7_0")
  spectrum <- read_csv(file = url_spect)
  spectrum %>% 
    filter(between(Wavelength, wavelength_lower_limit, wavelength_upper_limit)) %>% 
    select(Wavelength, BestFit)
}
spect1 <- get_spectrum(master_targets[index,], 
                       wavelength_lower_limit = 3500, 
                       wavelength_upper_limit = 8000)

This gives a dataframe with wavelength and intensity. Let’s plot this

The spectrum from sdss is shown below, and can be seen here

SDSS Spectrum RA = 167.0869741, Dec = 26.1582751

The SDSS image of the star itself is given here

Full code is available from github.

Note, for some reason the SDSS objid returned from the SQL query is incorrect. I’m still trying to figure out why and to fix this.

Update, the following is an ugly workaround on the objid issue. Haven’t verified that it always works yet.

radial_url_root <- "http://skyserver.sdss.org/dr17/SkyServerWS/SearchTools/SqlSearch?cmd="
radial_url_core <- glue("SELECT top {N} p.ra, p.dec, ",
                        "p.u, p.g, p.r, p.i, p.z, p.objid, ", 
                        "s.specobjid, s.class, s.subclass, s.survey, ", 
                        "s.plate, s.mjd, s.fiberid ", 
                        "FROM photoObj AS p ", 
                        "JOIN SpecObj AS s ON s.bestobjid = p.objid ",
                        "WHERE p.g BETWEEN {bands_min} AND {bands_max} ",
                        "AND p.r BETWEEN {bands_min} AND {bands_max} ", 
                        "AND p.i BETWEEN {bands_min} AND {bands_max} ", 
                        "AND s.class = 'STAR' ",
                        "AND s.survey != 'eboss'" ) %>% 
  str_replace_all(" ", "%20") %>% 
  str_replace_all("\n", "")
w <- rvest::read_html(glue::glue(radial_url_root, radial_url_core, "&format=csv"))
X <- as_list(w)$html$body$p[[1]] %>% 
  as.character() %>% 
  str_remove("#Table1\n")
master_targets <- read.table(text = X, header = TRUE, sep = ",", dec = ".", comment.char = "#") %>% 
  mutate(across(where(is.numeric), round, 2),
         objid = as.character(objid),
         specobjid = as.character(specobjid))
master_targets$objid # check the top one to see if this works