Installing Apache Superset on Windows 10

apache-superset-on-windows10.md

Installing Apache Superset on Windows 10

⚠️ WARN: This doc might be outdated. Use with caution. Only tested with Python v3.7

🙋‍♂️ INFO: If you have fixes/suggestions to for this doc, please comment below.

🌟 STAR: This doc if you found this document helpful.

---

Pre-Requisites

1. Install Microsoft Visual C++ 14.x standalone: Build Tools for Visual Studio 2019 (x86, x64, ARM, ARM64)

   1. Select latest version of MSVCv142 - VS 2019 C++ x64/x86 build tools

   2. Select Windows 10 SDK

2. Install Python 3.7.x

   1. Install PIP within the installer

   2. Add Python 3.7 to PATH

3. Use CMD to execute below commands (Recommended)

Installation

Ideally run these commands sequentially ...

:: Create directory to host the files
mkdir D:\superset
cd /d D:\superset

:: Check Versions
python --version
pip --version
systeminfo | findstr /C:"OS"

:: Upgrade Setuptools & PIP
pip install --upgrade setuptools pip

:: Create Virtual Environment named venv
python -m venv venv

:: Activate Virtual Environment
venv\Scripts\activate

:: Workaround - Install PIP within Virtual Environment
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py --ssl-no-revoke
python get-pip.py

:: Workaround - Upgrade Setuptools & PIP within venv
pip install --upgrade setuptools pip

:: (OUTDATED) Install Superset Requirements - Not required towards end of 2020
:: -- ref: https://gist.github.com/mark05e/d9cccae129dd11a21d7219eddd7d9923#gistcomment-3614048
:: curl https://raw.githubusercontent.com/apache/incubator-superset/master/requirements.txt -o requirements.txt --ssl-no-revoke
:: pip install -r requirements.txt

:: Install Superset
pip install apache-superset

:: (OUTDATED) Workaround - Install specific versions for compatibility (USE WITH CAUTION - ONLY IF NEEDED)
:: pip uninstall pandas
:: pip install pandas==0.23.4
:: pip install Flask==1.0
:: pip install SQLAlchemy==1.2.18

:: Install DB Drivers - Postgres & MS SQL
pip install psycopg2
pip install pymssql

:: Open Scripts folder to do superset related stuff
cd venv\Scripts

:: Create application database
python superset db upgrade

:: Create admin user
set FLASK_APP=superset
flask fab create-admin

:: Load some data to play with (optional)
python superset load_examples

:: Create default roles and permissions
python superset init

:: Start web server on port 8088
python superset run -p 8088 --with-threads --reload --debugger

(OUTDATED) Sample requirements.txt

Click to expand!

This sample can be used as a reference if something does not work correctly or incase of any installation issues.

alembic==1.0.11
amqp==2.5.0
apispec==1.3.3
asn1crypto==0.24.0
attrs==19.1.0
Babel==2.7.0
billiard==3.6.0.0
bleach==3.1.0
boto3==1.4.7
botocore==1.7.48
cchardet==2.1.4
celery==4.3.0
certifi==2019.6.16
cffi==1.12.3
chardet==3.0.4
click==6.7
colorama==0.3.9
contextlib2==0.5.5
cryptography==2.7
defusedxml==0.6.0
docutils==0.15.2
et-xmlfile==1.0.1
Flask==1.0
Flask-AppBuilder==2.1.10
Flask-Babel==0.12.2
Flask-Caching==1.7.2
Flask-Compress==1.4.0
Flask-JWT-Extended==3.21.0
Flask-Login==0.4.1
Flask-Migrate==2.5.2
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.0
Flask-WTF==0.14.2
flower==0.9.3
future==0.16.0
geographiclib==1.49
geopy==1.20.0
gunicorn==19.9.0
humanize==0.5.1
idna==2.8
ijson==2.4
isodate==0.6.0
itsdangerous==1.1.0
jdcal==1.4.1
Jinja2==2.10.1
jmespath==0.9.4
jsonlines==1.2.0
jsonschema==3.0.2
kombu==4.6.3
linear-tsv==1.1.0
Mako==1.1.0
Markdown==3.1.1
MarkupSafe==1.1.1
marshmallow==2.19.5
marshmallow-enum==1.4.1
marshmallow-sqlalchemy==0.17.0
numpy==1.17.0
openpyxl==2.4.11
pandas==0.23.4
parsedatetime==2.4
pathlib2==2.3.4
polyline==1.4.0
prison==0.1.2
psycopg2==2.8.3
pure-sasl==0.6.1
pycparser==2.19
pydruid==0.5.6
PyHive==0.6.1
PyJWT==1.7.1
pymssql==2.1.4
pyrsistent==0.15.4
python-dateutil==2.8.0
python-editor==1.0.4
python-geohash==0.8.5
python3-openid==3.1.0
pytz==2019.2
PyYAML==5.1.2
requests==2.22.0
rfc3986==1.3.2
s3transfer==0.1.13
simplejson==3.16.0
six==1.12.0
SQLAlchemy==1.2.18
SQLAlchemy-Utils==0.34.1
sqlparse==0.3.0
superset==0.28.1
tableschema==1.6.0
tabulator==1.23.0
thrift==0.11.0
thrift-sasl==0.3.0
tornado==5.1.1
unicodecsv==0.14.1
Unidecode==1.1.1
urllib3==1.25.3
vine==1.3.0
webencodings==0.5.1
Werkzeug==0.15.5
WTForms==2.2.1
xlrd==1.2.0

This output can be obtained by running this command against your venv

pip freeze > requirements.txt

🔎 References

1. Superset Installation Guide

2. Windows Compilers for Python

3. Direct Link to download Visual Studio Build Tools 2019 (might be broken)

4. Stackoverflow - Remove PIP installed packages

5. Workaround - Pandas package downgrade (Ref 1, Ref 2)

6. Workaround - SQLAlchaemy package downgrade (Ref 1)

7. @philip-sparks suggestion from superset issue #5788

8. @hainv suggestion

9. @ronna update command - ref 1, ref 2

---

mark05e / apache-superset-on-windows10.md

Hi @mark05e, thanks for the response. The command seemed to work though got a different error

I tried looking into the error but found a reference that TypedDict is not available on python 3.7. Has anyone else come across this?

@primarchsolutions looks like TypedDict was taken up on python 3.8. You will have to change your default python setup to the latest version of python.

Quote from the installation guide

You should install a recent version of Python

Hi @mark05e, can ths work with python 3.9.+ and 3.10.+? There was a point during one of my earlier attempts at installation with 3.10 that I got an error about one of the requirements needing a python version less than 3.9

@primarchsolutions I have not used superset in a while so I am unsure if it works with 3.9 or 3.10. On the installation guide they mention that that they are using 3.8.12 in their docker editions - probably a safe bet to try this version.

• Quote from the installation guide
• Docker Hub Superset Image details

Hi @mark05e I successfully setup superset on a windows server and wanted to migrate it to another server. I am having challenges moving the superset database to the new server. When I run superset init I get an "invalid decryption key error" and on the superset front end the database list does not load. how do I overcome this error. I have tried using the old SECRET KEY , but still does not work.

I have a better solution

and also offer portable version

https://github.com/alitrack/superset_app

Hi @alitrack - I'm trying to follow steps mentioned in https://github.com/alitrack/superset_app#readme
However, it looks like the link to download 3rd file is not working. Can you please share the correct location of this file?
python_geohash‑0.8.5‑cp38‑cp38‑win_amd64.whl
Thanks!!

@misbadigitalai
download link fixed,
you can download from here, https://www.lfd.uci.edu/~gohlke/pythonlibs/

Hi @mark05e I successfully setup superset on a windows server and wanted to migrate it to another server. I am having challenges moving the superset database to the new server. When I run superset init I get an "invalid decryption key error" and on the superset front end the database list does not load. how do I overcome this error. I have tried using the old SECRET KEY , but still does not work.

Go through these steps to fix "Invalid decryption key" error

and when upgrade from lower version to 1.4 or newer version , also meets this kind error, and I have a solution talk about it, feel free to have a look(Chinese).

set SUPERSET_CONFIG_PATH=superset_config.py 
set FLASK_APP=superset 
superset   version

you need tell where to find your config file, for example,

set SUPERSET_CONFIG_PATH=superset_config.py 
set FLASK_APP=superset 
superset   version

Thank you for the assist @alitrack

Hello guys, can you help me fixed this problem.

Little update for 2023: I needed to install C++ build tools as shown in the following screenshot:

Otherwise the build fails at package geohash.

Another note: I had so set FLASK_APP variable before running anythin in superset. I also had many issues while trying to use a local SQLite database for testing. Using a MySQL database running on a server, everythin works perfectly.

Thank you anyway for this great step-by-step gist for setting up a local test instance of superset! Thats what we need :-)

Thank you for sharing @sebastianknopf !

First of all THANK YOU for getting it to work on Windows. I have tried and failed several times to get it to work on Windows, and gave up after several hours each time. I did get it to work on Windows Virtual Box Ubuntu, but that was a hassle to keep running locally.

Thank you for providing detailed instructions on how to use a compiler to update specific Python wheels or other packages that were not portable from Linux to Windows. I will try this, but first I have some questions.

### HAS ANYONE GOTTEN THE INSTALLATION PROCEDURE ABOVE TO WORK INSIDE A CONDA VIRTUAL ENVIRONMENT?

What is required to make this happen? Which packages are not available in conda format? What would it take to do this?

Frankly I am scared of using pip outside of conda environments, because I have corrupted my Anaconda "base" installation a couple of times by "pipping" packages into a specific virtual environment and it somehow affects my base environment.

I successfully install and use conda and pip installed in all my new conda environments. This keeps the environment packages isolated and all located in one place where conda can located and manage all conda and pip packages: C:\ProgramData\Anaconda3\envs<>" )

I have a beef about the Superset development team philosophy and have a rant / kvetch about how it has never been supported on Windows... The Superset Development Team was no help with Superset on Windows over the past several years. I asked 2, 3, and 4 years ago, and each time I tried and failed to get it to work on Windows, wasting many hours in the process, before giving up and giving in to Linux hacking. I got the same party line statement everywhere from the Superset development team leader(s), "Superset is not officially supported on Windows, but go ahead and try... try Docker on Linux". When maintenance was taken over by Preset Consulting, Preset Consulting continued with that same party line "not officially supported on Windows, try Docker on Linux". Nobody would even answer technical questions, nor provide any insight on specific errors, nor recommended anything to attempt when trying fix the common errors. Too busy with their consulting startup, so they could charge expensive pay-by-the-hour IT consulting services to answer the same questions. That attitude is not helpful at all and holds back adoption by the masses; relegating it to expensive pay-by-the-hour IT consulting services. You had to be a paying customer to get answers on how to install Free Open Source Software? And on Micro$oft-Windows, the world's largest computer user base operating system? Frankly it did not make sense for open source software from Apache Foundation. OFF SOAP-BOX !!

This is more proper steps below.

mkdir D:\superset
cd /d D:\superset

pip install --upgrade setuptools pip

python -m venv venv

venv\Scripts\activate

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py --ssl-no-revoke
python get-pip.py

pip install --upgrade setuptools pip

pip install apache-superset
pip install psycopg2
pip install pymssql
pip install mysqlclient

set FLASK_APP=superset
set SUPERSET_SECRET_KEY="oh-so-secret"

cd venv\Scripts

superset db upgrade
superset fab create-admin
superset init
superset run -p 8088 --with-threads --reload --debugger

If your app (superset) to be accessible from other machines other than 'localhost', you may want to run the app with host 0.0.0.0 as below.
superset run --host=0.0.0.0 -p 8088 --with-threads --reload --debugger