Last active
February 7, 2024 10:20
-
-
Save vrtmrz/b437a539af25ef191bd452aae369242f to your computer and use it in GitHub Desktop.
deploy_couchdb_to_flyio_v2.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "view-in-github", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<a href=\"https://colab.research.google.com/gist/vrtmrz/b437a539af25ef191bd452aae369242f/deploy_couchdb_to_flyio_v2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "source": [ | |
| "History:\n", | |
| "- 18, May, 2023: Initial.\n", | |
| "- 5 Feb. 2024 : Refined version is here: https://gist.github.com/vrtmrz/9402b101746e08e969b1a4f5f0deb465" | |
| ], | |
| "metadata": { | |
| "id": "HiRV7G8Gk1Rs" | |
| } | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "2Vh0mEQEZuAK" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Configurations\n", | |
| "import os\n", | |
| "os.environ['region']=\"nrt\"\n", | |
| "os.environ['couchUser']=\"alkcsa93\"\n", | |
| "os.environ['couchPwd']=\"c349usdfnv48fsasd\"" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "SPmbB0jZauQ1" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Delete once\n", | |
| "!rm ./fly.toml" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "Nze7QoxLZ7Yx" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Installation\n", | |
| "# You have to set up your account in here.\n", | |
| "!curl -L https://fly.io/install.sh | sh\n", | |
| "!/root/.fly/bin/flyctl auth signup" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "MVJwsIYrbgtx" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Generate server\n", | |
| "!/root/.fly/bin/flyctl launch --auto-confirm --generate-name --detach --no-deploy --region ${region}\n", | |
| "!/root/.fly/bin/fly volumes create --region ${region} couchdata --size 2 --yes" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "source": [ | |
| "# Check the toml once.\n", | |
| "!cat fly.toml" | |
| ], | |
| "metadata": { | |
| "id": "2RSoO9o-i2TT" | |
| }, | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "zUtPZLVnbvdQ" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Modify fly.toml\n", | |
| "## Port modification\n", | |
| "!sed -i 's/8080/5984/g' fly.toml\n", | |
| "## Add user into.\n", | |
| "!echo -e \"\\n[env]\\n COUCHDB_USER = \\\"${couchUser}\\\"\" >> ./fly.toml\n", | |
| "## Set the location of an ini file which to save configurations persistently via erlang flags.\n", | |
| "!echo -e \"\\nERL_FLAGS=\\\"-couch_ini /opt/couchdb/etc/default.ini /opt/couchdb/etc/default.d/ /opt/couchdb/etc/local.d /opt/couchdb/etc/local.ini /opt/couchdb/data/persistence.ini\\\"\" >> ./fly.toml\n", | |
| "## Mounting volumes to store data and ini file.\n", | |
| "!echo -e \"\\n[mounts]\\n source=\\\"couchdata\\\"\\n destination=\\\"/opt/couchdb/data\\\"\" >> ./fly.toml\n", | |
| "!cat fly.toml" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "source": [ | |
| "# Make the Dockerfile to modify the permission of the ini file. If you want to use a specific version, you should change `latest` here.\n", | |
| "!echo -e \"\\n[build]\\n dockerfile = \\\"./Dockerfile\\\"\" >> ./fly.toml\n", | |
| "!echo -e \"FROM couchdb:latest\\nRUN sed -i '2itouch /opt/couchdb/data/persistence.ini && chmod +w /opt/couchdb/data/persistence.ini' /docker-entrypoint.sh\" > ./Dockerfile" | |
| ], | |
| "metadata": { | |
| "id": "LQPsZ_dYxkTu" | |
| }, | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "source": [ | |
| "# Check dockerfile\n", | |
| "!cat ./Dockerfile" | |
| ], | |
| "metadata": { | |
| "id": "ai2R3BbpxRSe" | |
| }, | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "xWdsTCI6bzk2" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Configure password\n", | |
| "!/root/.fly/bin/flyctl secrets set COUCHDB_PASSWORD=${couchPwd}" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "k0WIQlShcXGa" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Deploy server\n", | |
| "# Be sure to shutdown after the test.\n", | |
| "!/root/.fly/bin/flyctl deploy --detach --remote-only\n", | |
| "!/root/.fly/bin/flyctl status" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "0ySggkdlfq7M" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "import subprocess, json\n", | |
| "result = subprocess.run([\"/root/.fly/bin/flyctl\",\"status\",\"-j\"], capture_output=True, text=True)\n", | |
| "if result.returncode==0:\n", | |
| " hostname = json.loads(result.stdout)[\"Hostname\"]\n", | |
| " os.environ['couchHost']=\"https://%s\" % (hostname)\n", | |
| " print(\"Your couchDB server is https://%s/\" % (hostname))\n", | |
| "else:\n", | |
| " print(\"Something occured.\")\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "source": [ | |
| "# Finish setting up the CouchDB\n", | |
| "!curl -X POST \"${couchHost}/_cluster_setup\" -H \"Content-Type: application/json\" -d \"{\\\"action\\\":\\\"enable_single_node\\\",\\\"username\\\":\\\"${couchUser}\\\",\\\"password\\\":\\\"${couchPwd}\\\",\\\"bind_address\\\":\\\"0.0.0.0\\\",\\\"port\\\":5984,\\\"singlenode\\\":true}\" --user \"${couchUser}:${couchPwd}\"" | |
| ], | |
| "metadata": { | |
| "id": "cGlSzVqlQG_z" | |
| }, | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "source": [ | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate\" -H \"Content-Type: application/json\" -d '\"Basic realm=\\\"couchdb\\\"\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size\" -H \"Content-Type: application/json\" -d '\"4294967296\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size\" -H \"Content-Type: application/json\" -d '\"50000000\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/cors/credentials\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n", | |
| "!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/cors/origins\" -H \"Content-Type: application/json\" -d '\"app://obsidian.md,capacitor://localhost,http://localhost\"' --user \"${couchUser}:${couchPwd}\"" | |
| ], | |
| "metadata": { | |
| "id": "JePzrsHypY18" | |
| }, | |
| "execution_count": null, | |
| "outputs": [] | |
| } | |
| ], | |
| "metadata": { | |
| "colab": { | |
| "provenance": [], | |
| "private_outputs": true, | |
| "include_colab_link": true | |
| }, | |
| "kernelspec": { | |
| "display_name": "Python 3", | |
| "name": "python3" | |
| }, | |
| "language_info": { | |
| "name": "python" | |
| }, | |
| "gpuClass": "standard" | |
| }, | |
| "nbformat": 4, | |
| "nbformat_minor": 0 | |
| } |
Do you mind explaining a bit on what "swap enabled" here means? Would appreciate some pointers. Thank you!
Thank you for asking me! It is not clear when, but at some point, fly.io started to actively evacuate machines that ran out of memory. So we have to enable using the swap memory if we are using small machines.
Thanks for the reply. Unfortunately I need you to dumb it down even more for noobs like me 😄. To be more specific, I want to know when and why I need to run the swap-enabled instead of this version. I got here from Setup CouchDB on fly.io and I don't know if I should proceed using this version or the swap-enabled version.
You mention that "we have to enable using the swap memory if we are using small machines", are "we" the users of your plugin and small machines here mean weak PCs?
Sorry for being so vague! Yes, you have to enable the swap if you are using the free machine. (Small is not about your local device, but about the machine hosted on fly.io).
We means that just I am using fly.io for testing, and also I am always one of the users! Cheers!
Oh okay. So if I'm using the free machine (mine has 2GB storage I believe), I should use the swap-enabled version. Got it. Thanks a lot for your help!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Swap enabled version is here: https://gist.github.com/vrtmrz/37c3efd7842e49947aaaa7f665e5020a