Skip to content

Instantly share code, notes, and snippets.

@mark05e

mark05e/apache-superset-on-windows10.md

Last active April 24, 2024 23:32
Show Gist options
  • Star 58 You must be signed in to star a gist
  • Fork 11 You must be signed in to fork a gist
  • Save mark05e/d9cccae129dd11a21d7219eddd7d9923 to your computer and use it in GitHub Desktop.
Save mark05e/d9cccae129dd11a21d7219eddd7d9923 to your computer and use it in GitHub Desktop.
Download ZIP
Installing Apache Superset on Windows 10
Raw
apache-superset-on-windows10.md

Installing Apache Superset on Windows 10

⚠️ WARN: This doc might be outdated. Use with caution. Only tested with Python v3.7

🙋‍♂️ INFO: If you have fixes/suggestions to for this doc, please comment below.

🌟 STAR: This doc if you found this document helpful.

Pre-Requisites

  1. Install Microsoft Visual C++ 14.x standalone: Build Tools for Visual Studio 2019 (x86, x64, ARM, ARM64)

    1. Select latest version of MSVCv142 - VS 2019 C++ x64/x86 build tools

    2. Select Windows 10 SDK

  2. Install Python 3.7.x

    1. Install PIP within the installer

    2. Add Python 3.7 to PATH

  3. Use CMD to execute below commands (Recommended)

Installation

Ideally run these commands sequentially ...

:: Create directory to host the files
mkdir D:\superset
cd /d D:\superset

:: Check Versions
python --version
pip --version
systeminfo | findstr /C:"OS"

:: Upgrade Setuptools & PIP
pip install --upgrade setuptools pip

:: Create Virtual Environment named venv
python -m venv venv

:: Activate Virtual Environment
venv\Scripts\activate

:: Workaround - Install PIP within Virtual Environment
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py --ssl-no-revoke
python get-pip.py

:: Workaround - Upgrade Setuptools & PIP within venv
pip install --upgrade setuptools pip

:: (OUTDATED) Install Superset Requirements - Not required towards end of 2020
:: -- ref: https://gist.github.com/mark05e/d9cccae129dd11a21d7219eddd7d9923#gistcomment-3614048
:: curl https://raw.githubusercontent.com/apache/incubator-superset/master/requirements.txt -o requirements.txt --ssl-no-revoke
:: pip install -r requirements.txt

:: Install Superset
pip install apache-superset

:: (OUTDATED) Workaround - Install specific versions for compatibility (USE WITH CAUTION - ONLY IF NEEDED)
:: pip uninstall pandas
:: pip install pandas==0.23.4
:: pip install Flask==1.0
:: pip install SQLAlchemy==1.2.18

:: Install DB Drivers - Postgres & MS SQL
pip install psycopg2
pip install pymssql

:: Open Scripts folder to do superset related stuff
cd venv\Scripts

:: Create application database
python superset db upgrade

:: Create admin user
set FLASK_APP=superset
flask fab create-admin

:: Load some data to play with (optional)
python superset load_examples

:: Create default roles and permissions
python superset init

:: Start web server on port 8088
python superset run -p 8088 --with-threads --reload --debugger

(OUTDATED) Sample requirements.txt

Click to expand!

This sample can be used as a reference if something does not work correctly or incase of any installation issues.

alembic==1.0.11
amqp==2.5.0
apispec==1.3.3
asn1crypto==0.24.0
attrs==19.1.0
Babel==2.7.0
billiard==3.6.0.0
bleach==3.1.0
boto3==1.4.7
botocore==1.7.48
cchardet==2.1.4
celery==4.3.0
certifi==2019.6.16
cffi==1.12.3
chardet==3.0.4
click==6.7
colorama==0.3.9
contextlib2==0.5.5
cryptography==2.7
defusedxml==0.6.0
docutils==0.15.2
et-xmlfile==1.0.1
Flask==1.0
Flask-AppBuilder==2.1.10
Flask-Babel==0.12.2
Flask-Caching==1.7.2
Flask-Compress==1.4.0
Flask-JWT-Extended==3.21.0
Flask-Login==0.4.1
Flask-Migrate==2.5.2
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.0
Flask-WTF==0.14.2
flower==0.9.3
future==0.16.0
geographiclib==1.49
geopy==1.20.0
gunicorn==19.9.0
humanize==0.5.1
idna==2.8
ijson==2.4
isodate==0.6.0
itsdangerous==1.1.0
jdcal==1.4.1
Jinja2==2.10.1
jmespath==0.9.4
jsonlines==1.2.0
jsonschema==3.0.2
kombu==4.6.3
linear-tsv==1.1.0
Mako==1.1.0
Markdown==3.1.1
MarkupSafe==1.1.1
marshmallow==2.19.5
marshmallow-enum==1.4.1
marshmallow-sqlalchemy==0.17.0
numpy==1.17.0
openpyxl==2.4.11
pandas==0.23.4
parsedatetime==2.4
pathlib2==2.3.4
polyline==1.4.0
prison==0.1.2
psycopg2==2.8.3
pure-sasl==0.6.1
pycparser==2.19
pydruid==0.5.6
PyHive==0.6.1
PyJWT==1.7.1
pymssql==2.1.4
pyrsistent==0.15.4
python-dateutil==2.8.0
python-editor==1.0.4
python-geohash==0.8.5
python3-openid==3.1.0
pytz==2019.2
PyYAML==5.1.2
requests==2.22.0
rfc3986==1.3.2
s3transfer==0.1.13
simplejson==3.16.0
six==1.12.0
SQLAlchemy==1.2.18
SQLAlchemy-Utils==0.34.1
sqlparse==0.3.0
superset==0.28.1
tableschema==1.6.0
tabulator==1.23.0
thrift==0.11.0
thrift-sasl==0.3.0
tornado==5.1.1
unicodecsv==0.14.1
Unidecode==1.1.1
urllib3==1.25.3
vine==1.3.0
webencodings==0.5.1
Werkzeug==0.15.5
WTForms==2.2.1
xlrd==1.2.0

This output can be obtained by running this command against your venv

pip freeze > requirements.txt

🔎 References

  1. Superset Installation Guide

  2. Windows Compilers for Python

  3. Direct Link to download Visual Studio Build Tools 2019 (might be broken)

  4. Stackoverflow - Remove PIP installed packages

  5. Workaround - Pandas package downgrade (Ref 1, Ref 2)

  6. Workaround - SQLAlchaemy package downgrade (Ref 1)

  7. @philip-sparks suggestion from superset issue #5788

  8. @hainv suggestion

  9. @ronna update command - ref 1, ref 2

mark05e / apache-superset-on-windows10.md

@mark05e
Copy link
Author

mark05e commented Sep 25, 2023

Thank you for sharing @sebastianknopf !

@richlysakowski
Copy link

First of all THANK YOU for getting it to work on Windows. I have tried and failed several times to get it to work on Windows, and gave up after several hours each time. I did get it to work on Windows Virtual Box Ubuntu, but that was a hassle to keep running locally.

Thank you for providing detailed instructions on how to use a compiler to update specific Python wheels or other packages that were not portable from Linux to Windows. I will try this, but first I have some questions.

### HAS ANYONE GOTTEN THE INSTALLATION PROCEDURE ABOVE TO WORK INSIDE A CONDA VIRTUAL ENVIRONMENT?

What is required to make this happen? Which packages are not available in conda format? What would it take to do this?

Frankly I am scared of using pip outside of conda environments, because I have corrupted my Anaconda "base" installation a couple of times by "pipping" packages into a specific virtual environment and it somehow affects my base environment.

I successfully install and use conda and pip installed in all my new conda environments. This keeps the environment packages isolated and all located in one place where conda can located and manage all conda and pip packages: C:\ProgramData\Anaconda3\envs<>" )

I have a beef about the Superset development team philosophy and have a rant / kvetch about how it has never been supported on Windows... The Superset Development Team was no help with Superset on Windows over the past several years. I asked 2, 3, and 4 years ago, and each time I tried and failed to get it to work on Windows, wasting many hours in the process, before giving up and giving in to Linux hacking. I got the same party line statement everywhere from the Superset development team leader(s), "Superset is not officially supported on Windows, but go ahead and try... try Docker on Linux". When maintenance was taken over by Preset Consulting, Preset Consulting continued with that same party line "not officially supported on Windows, try Docker on Linux". Nobody would even answer technical questions, nor provide any insight on specific errors, nor recommended anything to attempt when trying fix the common errors. Too busy with their consulting startup, so they could charge expensive pay-by-the-hour IT consulting services to answer the same questions. That attitude is not helpful at all and holds back adoption by the masses; relegating it to expensive pay-by-the-hour IT consulting services. You had to be a paying customer to get answers on how to install Free Open Source Software? And on Micro$oft-Windows, the world's largest computer user base operating system? Frankly it did not make sense for open source software from Apache Foundation. OFF SOAP-BOX !!

@3supower
Copy link

This is more proper steps below.

mkdir D:\superset
cd /d D:\superset

pip install --upgrade setuptools pip

python -m venv venv

venv\Scripts\activate

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py --ssl-no-revoke
python get-pip.py

pip install --upgrade setuptools pip

pip install apache-superset
pip install psycopg2
pip install pymssql
pip install mysqlclient

set FLASK_APP=superset
set SUPERSET_SECRET_KEY="oh-so-secret"

cd venv\Scripts

superset db upgrade
superset fab create-admin
superset init
superset run -p 8088 --with-threads --reload --debugger

@3supower
Copy link

If your app (superset) to be accessible from other machines other than 'localhost', you may want to run the app with host 0.0.0.0 as below.
superset run --host=0.0.0.0 -p 8088 --with-threads --reload --debugger

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment