In recent decades, a large scientific effort has focused on harnessing spin transport for providing insights into novel materials and low-dissipation information processing. We introduce single spin magnetometry based on nitrogen-vacancy (NV) centers in diamond as a new and generic platform to locally probe spin chemical potentials which essentially determine the flow of spin currents. We use this platform to investigate magnons in a magnetic insulator yttrium iron garnet (YIG) on a 100 nanometer length scale. We demonstrate that the local magnon chemical potential can be systematically controlled through both ferromagnetic resonance and electrical spin excitation, which agrees well with the theoretical analysis of the underlying multi-magnon processes. Our results open up new possibilities for nanoscale imaging and manipulation of spin-related phenomena in condensed-matter systems.