The interstellar space in the Milky Way contains diffuse gas and solid dust grains. There is constant recycling of matter and energy between this interstellar medium (ISM) and the stars. Likewise, there is constant recycling between galaxies and the circumgalactic medium (CGM). I will describe two applications of atomic physics and condensed matter physics relevant to our studies of the ISM and the CGM. First, I will describe our ongoing efforts to improve the atomic physics needed for interpretation of absorption-line spectroscopy of the ISM and the CGM. This work is motivated by the fact that the inferred properties of the gas depend sensitively on the atomic parameters used for the spectral features, e.g. the oscillator strengths used for the relevant transitions, which remain uncertain for a significant fraction of the transitions. Our work includes calculations and compilations of oscillator strengths for the lines of greatest interest, and examination of their implications for studies of distant galaxies. Second, I will describe our ongoing efforts to determine the nature of the solid particles (dust grains) in the ISM and the CGM. We have made the first detection of silicate dust grains in distant galaxies out to redshifts z~1.5. The dust grains in these distant galaxies appear to be more silicate-rich than the grains in the Milky Way ISM. Furthermore, in some cases, these grains appear to be crystalline, unlike the amorphous grains observed in the ISM of the Milky Way. These differences may be caused by differences in grain properties (e.g. the grain composition or size distribution), or differences in their environment (e.g. cosmic ray fluxes). Improvements to both the atomic physics of the relevant transitions in the ISM/CGM, and the condensed matter physics of dust grains mixed in with the diffuse gas will be crucial for improving our understanding of the evolution of stars and galaxies.