How can I play a sine/square wave using Pygame?

You can load the numpy array directly to a pygame.mixer.Sound object and play it. The jarring sound occurs because you're probably sending 32bit float samples to the mixer when it's expecting 16bit integer samples. If you want to use an array of 32bit floats, you must set the size parameter of the mixer's init function to 32:

import pygame
import numpy as np

pygame.mixer.init(size=32)

buffer = np.sin(2 * np.pi * np.arange(44100) * 440 / 44100).astype(np.float32)
sound = pygame.mixer.Sound(buffer)

sound.play(0)
pygame.time.wait(int(sound.get_length() * 1000))