If you have issues with installing numpy related packages on Big sur, these steps helped me: 1- Uninstall pyhon installed with homebrew and use python-installer. This resolves numpy issue. And it's better to install 1.18 or just anything bellow 1.19.
brew uninstall python
And install it from: https://www.python.org/downloads/
Then:
pip install numpy==1.18