Skip to content

Instantly share code, notes, and snippets.

@RickCogley

RickCogley/1. Problem.md

Last active December 9, 2024 02:51
Show Gist options
  • Save RickCogley/58ac2531e473b7ae15791e68ab1031f5 to your computer and use it in GitHub Desktop.
Save RickCogley/58ac2531e473b7ae15791e68ab1031f5 to your computer and use it in GitHub Desktop.
Download ZIP
Deno fetch gives error (TLS related, and origin is rust library)
Raw
1. Problem.md

Deno fetch gives error (TLS related, and origin is rust library)

Using fetch in Lume v2.4.2 (a Deno Typescript static site generator app) on Deno 2.1.2, I get this error when fetching to our ops db REST API:

fetch TypeError: "client error connection reset by peer"

I see a lot of people have had this same problem, in various GH issues, and it appears that it is because of the underlying Rust library being used. For example:

When I test the connection using curl -v you can see the below, and https://www.ssllabs.com/ssltest/analyze.html?d=pro.dbflex.net shows the site gets an A rating, yet Deno rejects the connection.

I naively tried an alternative to fetch, ky, which I thought might do something magical, but no, since I tried it in Deno, it's using Deno's fetch underneath, in the end. Pasted test code below for posterity. (I must say, ky really simplifies this type of code; nice library!)

Raw
2. Resolution in Lume.md

Resolution: curl-based workaround in Lume beforeBuild script

Lume runs on Deno and since Deno is probably not going to easily be able to change the library their fetch uses easily (to, say, openssl) then how can we work around it?

Lume allows you to run scripts before and after the site build, in _config.ts, so the venerable curl comes to our rescue:

...
// Prepare script to get holidays from dbflex
site.script("getholidays", "cd src/_data && curl -H \"Authorization: Bearer ${API_KEY_01}\" https://pro.dbflex.net/secure/api/v2/15331/Work%20Holiday/API%20Holidays%20Today%20or%20Later/select.json -o futureholidays.json");

// Execute scripts before build
site.addEventListener("beforeBuild", "getholidays");
...
export default site;

The ${API_KEY_01} is the key we need to access the database's REST i/f, and I have it set locally and in Github environment variables. Note, in the context of setting a secret for Github Actions, I found that this must be set in Settings, Security, Secrets and Variables, Actions, Repository Secrets, not "environment" Secrets.

Assuming the secret name is MYKEY, pull it into your build step in the Github workflow like so:

- name: Build site
        # Run the build script from deno.json
        env:
          API_KEY_01: ${{ secrets.MYKEY }}
        run: deno task build

This curl script pulls the data into a file prior to build, src/_data/futureholidays.json, which Lume then sets up as an object that can be accessed in Vento templates, during the build. Full workflow action below, and if the secret is being evaluated correctly, you can see the value as *** (as expected) in the GH action run log:

...
##[debug]Evaluating: secrets.MYKEY
##[debug]Evaluating Index:
##[debug]..Evaluating secrets:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'MYKEY'
##[debug]=> '***'
##[debug]Result: '***'
##[debug]Evaluating condition for step: 'Build site'
##[debug]Evaluating: success()
##[debug]Evaluating success:
...

The json looks like:

[
...
  {
    "@row.id": 477,
    Date: "2024-12-29T00:00:00+00:00",
    "Date in YYYYMMDD": "20241229",
    "Date in YYYY-MM-DD": "2024-12-29",
    "Day of Week": "Sunday",
    "Date with Month Name": "29 Dec 2024",
    Name: "Year End Holiday",
    "Name Jp": "年末休み"
  },
  {
    "@row.id": 476,
    Date: "2024-12-28T00:00:00+00:00",
    "Date in YYYYMMDD": "20241228",
    "Date in YYYY-MM-DD": "2024-12-28",
    "Day of Week": "Saturday",
    "Date with Month Name": "28 Dec 2024",
    Name: "Year End Holiday",
    "Name Jp": "\t年末休み"
  }
...
]

In our Vento template, since Vento supports js, we want to match on today's date, find the node with that date (we assume one entry per date), and use javascript filter and map in the following way:

{{ set matchdate = todaysDateYYYYMMDD }}
{{ set matchholi = futureholidays.filter(item => item['Date in YYYY-MM-DD'] === matchdate ) }}
{{ if matchholi.length > 0 }}**It's a holiday in Japan:** 
{{ set resultholi = matchholi.map(item => item['Name']) + " / " + matchholi.map(item => item['Name Jp']) }}
{{ resultholi }}
{{ /if }}

The Vento template code is doing:

  1. set a variable to today's date, in the right format to match in the json
  2. using filter to find the json node where today's date matches the date in the node
  3. if there was a match, return some markdown text "It's a holiday..."
  4. prepare the string as a concatenation of the json node's English and Japanese names, using map to find the sibling to the matched date
  5. return the string to the page

The todaysDateYYYYMMDD constant is prepared by Lume from /src/_data.ts as follows:

export const todaysDateYYYYMMDD = `${
  new Date().toISOString("ja-JP", { timeZone: "Asia/Tokyo" }).split("T")[0]
}`;
Raw
2b. Resolution in Lume via deno flag.md

Per this issue I tried running deno with the --unsafely-ignore-certificate-errors flag, but after trying to fetch directly to my PaaS DB REST i/f, I still get the same error.

Raw
3. curl -v.sh
❯ curl -v https://pro.dbflex.net/secure
* Host pro.dbflex.net:443 was resolved.
* IPv6: (none)
* IPv4: 208.100.33.100
* Trying 208.100.33.100:443...
* Connected to pro.dbflex.net (208.100.33.100) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* CAfile: /etc/ssl/cert.pem
* CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384 / [blank] / UNDEF
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
* subject: CN=dbflex.net
* start date: Feb 15 00:00:00 2024 GMT
* expire date: Mar 17 23:59:59 2025 GMT
* subjectAltName: host "pro.dbflex.net" matched cert's "*.dbflex.net"
* issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
* SSL certificate verify ok.
* using HTTP/1.x
> GET /secure HTTP/1.1
> Host: pro.dbflex.net
> User-Agent: curl/8.7.1
> Accept: */*
Raw
4. ky test.js
// Ky test
import ky from 'https://esm.sh/ky';
ky.extend({
hooks: {
beforeRequest: [
request => {
const token = Deno.env.get("API_KEY_01");
request.headers.set('Authorization', `Bearer ${token}`);
}
]
}
});
const json = await ky.get('https://pro.dbflex.net/secure/api/v2/15331/Work%20Holiday/API%20Holidays%20Today%20or%20Later/select.json', { retry: 2 }).json();
console.log(json);
Raw
5. Alternative script curl with embedded bearer token.ts
// The db we use has a way to put the bearer token inside the URL
// This is what that looks like
site.script("getholidays", "cd src/_data && curl https://pro.dbflex.net/secure/api/v2/15331/${API_KEY_01}/Work%20Holiday/API%20Holidays%20Today%20or%20Later/select.json -o futureholidays.json");
Raw
6. gh-action.yml
name: Update Profile README
on:
push:
# Run on main branch pushes or PRs
branches: [main]
pull_request:
branches: [main]
# Allow to manually trigger the workflow
workflow_dispatch:
schedule:
# Rebuild every day at 16:30 PM UTC
- cron: "30 16 * * *"
jobs:
build:
# What OS to use
runs-on: ubuntu-latest
# Allow commits to the repository
permissions:
id-token: write
contents: write
pages: write
# What steps to run
steps:
- name: Clone repository
# Git clone the repository
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
fetch-depth: '0' # Fetch all history for git
- name: Setup Deno environment
# Downloads deno and caches it
uses: denoland/setup-deno@v2
with:
# Latest Deno 2.x version
deno-version: v2.x
- name: Build site
# Run the build script from deno.json
env:
API_KEY_01: ${{ secrets.MYKEY }}
run: deno task build
- name: Commit changes
# Commit the changes to the repository
uses: stefanzweifel/git-auto-commit-action@v5
with:
commit_message: Apply changes during gh actions build
status_options: "--untracked-files=no"
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: "_site"
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment