Installing Apache Superset on Windows 10

apache-superset-on-windows10.md

Installing Apache Superset on Windows 10

⚠️ WARN: This doc might be outdated. Use with caution. Only tested with Python v3.7

🙋‍♂️ INFO: If you have fixes/suggestions to for this doc, please comment below.

🌟 STAR: This doc if you found this document helpful.

---

Pre-Requisites

1. Install Microsoft Visual C++ 14.x standalone: Build Tools for Visual Studio 2019 (x86, x64, ARM, ARM64)

   1. Select latest version of MSVCv142 - VS 2019 C++ x64/x86 build tools

   2. Select Windows 10 SDK

2. Install Python 3.7.x

   1. Install PIP within the installer

   2. Add Python 3.7 to PATH

3. Use CMD to execute below commands (Recommended)

Installation

Ideally run these commands sequentially ...

:: Create directory to host the files
mkdir D:\superset
cd /d D:\superset

:: Check Versions
python --version
pip --version
systeminfo | findstr /C:"OS"

:: Upgrade Setuptools & PIP
pip install --upgrade setuptools pip

:: Create Virtual Environment named venv
python -m venv venv

:: Activate Virtual Environment
venv\Scripts\activate

:: Workaround - Install PIP within Virtual Environment
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py --ssl-no-revoke
python get-pip.py

:: Workaround - Upgrade Setuptools & PIP within venv
pip install --upgrade setuptools pip

:: (OUTDATED) Install Superset Requirements - Not required towards end of 2020
:: -- ref: https://gist.github.com/mark05e/d9cccae129dd11a21d7219eddd7d9923#gistcomment-3614048
:: curl https://raw.githubusercontent.com/apache/incubator-superset/master/requirements.txt -o requirements.txt --ssl-no-revoke
:: pip install -r requirements.txt

:: Install Superset
pip install apache-superset

:: (OUTDATED) Workaround - Install specific versions for compatibility (USE WITH CAUTION - ONLY IF NEEDED)
:: pip uninstall pandas
:: pip install pandas==0.23.4
:: pip install Flask==1.0
:: pip install SQLAlchemy==1.2.18

:: Install DB Drivers - Postgres & MS SQL
pip install psycopg2
pip install pymssql

:: Open Scripts folder to do superset related stuff
cd venv\Scripts

:: Create application database
python superset db upgrade

:: Create admin user
set FLASK_APP=superset
flask fab create-admin

:: Load some data to play with (optional)
python superset load_examples

:: Create default roles and permissions
python superset init

:: Start web server on port 8088
python superset run -p 8088 --with-threads --reload --debugger

(OUTDATED) Sample requirements.txt

Click to expand!

This sample can be used as a reference if something does not work correctly or incase of any installation issues.

alembic==1.0.11
amqp==2.5.0
apispec==1.3.3
asn1crypto==0.24.0
attrs==19.1.0
Babel==2.7.0
billiard==3.6.0.0
bleach==3.1.0
boto3==1.4.7
botocore==1.7.48
cchardet==2.1.4
celery==4.3.0
certifi==2019.6.16
cffi==1.12.3
chardet==3.0.4
click==6.7
colorama==0.3.9
contextlib2==0.5.5
cryptography==2.7
defusedxml==0.6.0
docutils==0.15.2
et-xmlfile==1.0.1
Flask==1.0
Flask-AppBuilder==2.1.10
Flask-Babel==0.12.2
Flask-Caching==1.7.2
Flask-Compress==1.4.0
Flask-JWT-Extended==3.21.0
Flask-Login==0.4.1
Flask-Migrate==2.5.2
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.0
Flask-WTF==0.14.2
flower==0.9.3
future==0.16.0
geographiclib==1.49
geopy==1.20.0
gunicorn==19.9.0
humanize==0.5.1
idna==2.8
ijson==2.4
isodate==0.6.0
itsdangerous==1.1.0
jdcal==1.4.1
Jinja2==2.10.1
jmespath==0.9.4
jsonlines==1.2.0
jsonschema==3.0.2
kombu==4.6.3
linear-tsv==1.1.0
Mako==1.1.0
Markdown==3.1.1
MarkupSafe==1.1.1
marshmallow==2.19.5
marshmallow-enum==1.4.1
marshmallow-sqlalchemy==0.17.0
numpy==1.17.0
openpyxl==2.4.11
pandas==0.23.4
parsedatetime==2.4
pathlib2==2.3.4
polyline==1.4.0
prison==0.1.2
psycopg2==2.8.3
pure-sasl==0.6.1
pycparser==2.19
pydruid==0.5.6
PyHive==0.6.1
PyJWT==1.7.1
pymssql==2.1.4
pyrsistent==0.15.4
python-dateutil==2.8.0
python-editor==1.0.4
python-geohash==0.8.5
python3-openid==3.1.0
pytz==2019.2
PyYAML==5.1.2
requests==2.22.0
rfc3986==1.3.2
s3transfer==0.1.13
simplejson==3.16.0
six==1.12.0
SQLAlchemy==1.2.18
SQLAlchemy-Utils==0.34.1
sqlparse==0.3.0
superset==0.28.1
tableschema==1.6.0
tabulator==1.23.0
thrift==0.11.0
thrift-sasl==0.3.0
tornado==5.1.1
unicodecsv==0.14.1
Unidecode==1.1.1
urllib3==1.25.3
vine==1.3.0
webencodings==0.5.1
Werkzeug==0.15.5
WTForms==2.2.1
xlrd==1.2.0

This output can be obtained by running this command against your venv

pip freeze > requirements.txt

🔎 References

1. Superset Installation Guide

2. Windows Compilers for Python

3. Direct Link to download Visual Studio Build Tools 2019 (might be broken)

4. Stackoverflow - Remove PIP installed packages

5. Workaround - Pandas package downgrade (Ref 1, Ref 2)

6. Workaround - SQLAlchaemy package downgrade (Ref 1)

7. @philip-sparks suggestion from superset issue #5788

8. @hainv suggestion

9. @ronna update command - ref 1, ref 2

---

mark05e / apache-superset-on-windows10.md

Hi @mark05e , thank you for sharing this solution. I have successfully installed apache superset on windows. I would like to know how to secure superset on windows 10 using Lets Encrypt

Hey @bhekimaggs glad the article is still relevant. I do not have anything readymade for your request but I can point out some resources that can help. The general idea is to keep superset config as-is and setup a reverse proxy webserver with HTTPS enabled.

Caddy is a web server that can help setup reverse proxy and also has support for Lets Encrypt.

• Reverse Proxy example
• Setting up HTTPS

thanks @mark05e , I will have a look at it.

@mark05e thank you for this amazing work

I am stuck at db upgrade, below is the error i get

(venv) C:\Users\DELL\venv\Scripts>python superset db upgrade
C:\Users\DELL\AppData\Local\Programs\Python\Python310\python.exe: can't open file 'C:\Users\DELL\venv\Scripts\superset': [Errno 2] No such file or directory

basically i only have one drive in my laptop, that's C drive, so in the start I am creating CD superset inside C drive instead of D drive as mentioned in steps.

And python also has gotten installed in C drive, so when i do db upgrade inside venv it says its not able to open a file in superset as it doesnt exist at all.

how do i go about it ? Thanks in advance

@ubersant - from what you have typed here, I think you are running the venv and superset db upgrade command in the wrong directory.

(venv) C:\Users\DELL\venv\Scripts>python superset db upgrade
C:\Users\DELL\AppData\Local\Programs\Python\Python310\python.exe: can't open file 'C:\Users\DELL\venv\Scripts\superset': [Errno 2] No such file or directory

basically i only have one drive in my laptop, that's C drive, so in the start I am creating CD superset inside C drive instead of D drive as mentioned in steps.

If you have prepared the folder C:\superset, you should open command prompt into this directory first and then follow the steps in the guide. In your case, it would be

:: Create directory to host the files
mkdir C:\superset
cd /d C:\superset

thanks for the response @mark05e

as I had already done this superset creation in C as you suggested but couldn't continue in the same command prompt as I was getting this error highlighted in yellow

thats when i understood that these commands should be executed in default cmd prompt of system

So it looks like am doing it wrong, since am executing all this in wrong directory what should be done ? Thanks much in advance

@mark05e so basically if you see the above message with screenshots, am thinking these set of commands cannot be executed in the same command prompt window, right ? since this cmd should be run from superset folder, python will come as alien in superset folder, cannot detect python from superset folder

@ubersant - I think you have your python installation wrong. You need to reinstall python and maybe restart your computer to update environment variables. Once it is updated, you should be able to run the python --version command and others from any folder in command prompt window.

Hi,
I installed it on Winodws11 and Superset is working fine, but I have two problems.
・ Deck.gl Demo does not display the background map
I put superset_config.py in the root directory of superset and wrote MAPBOX_API_KEY, but the system doesn't seem to read superset_config.py.
· I tried to add a sqlite3 database and put user.db in the same folder as ./superset/superset.db.
How do you describe the SQLALCHEMY URI *?
I hope you can help me.

Hey @REIZER - I haven't faced this issue before. I'm sorry I'm not sure how I can help out here. You could probably try to ask directly on the superset issue tracker with a lot more details?

@mark05e Thank you for introducing the URL.
I haven't solved it yet, but I was able to find a description like that.

Hi, I installed it on Winodws11 and Superset is working fine, but I have two problems. ・ Deck.gl Demo does not display the background map I put superset_config.py in the root directory of superset and wrote MAPBOX_API_KEY, but the system doesn't seem to read superset_config.py. · I tried to add a sqlite3 database and put user.db in the same folder as ./superset/superset.db. How do you describe the SQLALCHEMY URI *? I hope you can help me.

have a look https://github.com/alitrack/superset_app, just add superset 1.5 support

or you can run it in this way,

set SUPERSET_CONFIG_PATH=superset_config.py 
set FLASK_APP=superset 
superset   version

Hi all,

I have followed the instructions but keep getting stuck at the python superset db upgrade point with the error [Errno 2] No such file or directory

I also tried the suggestion of changing superset to superset-script.py but it still brings the error

@primarchsolutions - can you try superset db upgrade without python keyword

ref: https://superset.apache.org/docs/installation/installing-superset-from-scratch#installing-and-initializing-superset

Hi @mark05e, thanks for the response. The command seemed to work though got a different error

I tried looking into the error but found a reference that TypedDict is not available on python 3.7. Has anyone else come across this?

@primarchsolutions looks like TypedDict was taken up on python 3.8. You will have to change your default python setup to the latest version of python.

Quote from the installation guide

You should install a recent version of Python

Hi @mark05e, can ths work with python 3.9.+ and 3.10.+? There was a point during one of my earlier attempts at installation with 3.10 that I got an error about one of the requirements needing a python version less than 3.9

@primarchsolutions I have not used superset in a while so I am unsure if it works with 3.9 or 3.10. On the installation guide they mention that that they are using 3.8.12 in their docker editions - probably a safe bet to try this version.

• Quote from the installation guide
• Docker Hub Superset Image details

Hi @mark05e I successfully setup superset on a windows server and wanted to migrate it to another server. I am having challenges moving the superset database to the new server. When I run superset init I get an "invalid decryption key error" and on the superset front end the database list does not load. how do I overcome this error. I have tried using the old SECRET KEY , but still does not work.

I have a better solution

and also offer portable version

https://github.com/alitrack/superset_app

Hi @alitrack - I'm trying to follow steps mentioned in https://github.com/alitrack/superset_app#readme
However, it looks like the link to download 3rd file is not working. Can you please share the correct location of this file?
python_geohash‑0.8.5‑cp38‑cp38‑win_amd64.whl
Thanks!!

@misbadigitalai
download link fixed,
you can download from here, https://www.lfd.uci.edu/~gohlke/pythonlibs/

Hi @mark05e I successfully setup superset on a windows server and wanted to migrate it to another server. I am having challenges moving the superset database to the new server. When I run superset init I get an "invalid decryption key error" and on the superset front end the database list does not load. how do I overcome this error. I have tried using the old SECRET KEY , but still does not work.

Go through these steps to fix "Invalid decryption key" error

and when upgrade from lower version to 1.4 or newer version , also meets this kind error, and I have a solution talk about it, feel free to have a look(Chinese).

set SUPERSET_CONFIG_PATH=superset_config.py 
set FLASK_APP=superset 
superset   version

you need tell where to find your config file, for example,

set SUPERSET_CONFIG_PATH=superset_config.py 
set FLASK_APP=superset 
superset   version

Thank you for the assist @alitrack

Hello guys, can you help me fixed this problem.

Little update for 2023: I needed to install C++ build tools as shown in the following screenshot:

Otherwise the build fails at package geohash.

Another note: I had so set FLASK_APP variable before running anythin in superset. I also had many issues while trying to use a local SQLite database for testing. Using a MySQL database running on a server, everythin works perfectly.

Thank you anyway for this great step-by-step gist for setting up a local test instance of superset! Thats what we need :-)

Thank you for sharing @sebastianknopf !

First of all THANK YOU for getting it to work on Windows. I have tried and failed several times to get it to work on Windows, and gave up after several hours each time. I did get it to work on Windows Virtual Box Ubuntu, but that was a hassle to keep running locally.

Thank you for providing detailed instructions on how to use a compiler to update specific Python wheels or other packages that were not portable from Linux to Windows. I will try this, but first I have some questions.

### HAS ANYONE GOTTEN THE INSTALLATION PROCEDURE ABOVE TO WORK INSIDE A CONDA VIRTUAL ENVIRONMENT?

What is required to make this happen? Which packages are not available in conda format? What would it take to do this?

Frankly I am scared of using pip outside of conda environments, because I have corrupted my Anaconda "base" installation a couple of times by "pipping" packages into a specific virtual environment and it somehow affects my base environment.

I successfully install and use conda and pip installed in all my new conda environments. This keeps the environment packages isolated and all located in one place where conda can located and manage all conda and pip packages: C:\ProgramData\Anaconda3\envs<>" )

I have a beef about the Superset development team philosophy and have a rant / kvetch about how it has never been supported on Windows... The Superset Development Team was no help with Superset on Windows over the past several years. I asked 2, 3, and 4 years ago, and each time I tried and failed to get it to work on Windows, wasting many hours in the process, before giving up and giving in to Linux hacking. I got the same party line statement everywhere from the Superset development team leader(s), "Superset is not officially supported on Windows, but go ahead and try... try Docker on Linux". When maintenance was taken over by Preset Consulting, Preset Consulting continued with that same party line "not officially supported on Windows, try Docker on Linux". Nobody would even answer technical questions, nor provide any insight on specific errors, nor recommended anything to attempt when trying fix the common errors. Too busy with their consulting startup, so they could charge expensive pay-by-the-hour IT consulting services to answer the same questions. That attitude is not helpful at all and holds back adoption by the masses; relegating it to expensive pay-by-the-hour IT consulting services. You had to be a paying customer to get answers on how to install Free Open Source Software? And on Micro$oft-Windows, the world's largest computer user base operating system? Frankly it did not make sense for open source software from Apache Foundation. OFF SOAP-BOX !!

This is more proper steps below.

mkdir D:\superset
cd /d D:\superset

pip install --upgrade setuptools pip

python -m venv venv

venv\Scripts\activate

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py --ssl-no-revoke
python get-pip.py

pip install --upgrade setuptools pip

pip install apache-superset
pip install psycopg2
pip install pymssql
pip install mysqlclient

set FLASK_APP=superset
set SUPERSET_SECRET_KEY="oh-so-secret"

cd venv\Scripts

superset db upgrade
superset fab create-admin
superset init
superset run -p 8088 --with-threads --reload --debugger

If your app (superset) to be accessible from other machines other than 'localhost', you may want to run the app with host 0.0.0.0 as below.
superset run --host=0.0.0.0 -p 8088 --with-threads --reload --debugger