Skip to content

Instantly share code, notes, and snippets.

@Telomelonia
Created October 15, 2024 16:20
Show Gist options
  • Save Telomelonia/055671870f647e465aa354744aa7b70e to your computer and use it in GitHub Desktop.
Save Telomelonia/055671870f647e465aa354744aa7b70e to your computer and use it in GitHub Desktop.
Sync templates to apify/actor-templates on update
name: Sync Templates
on:
push:
branches:
- main
paths:
- 'templates/**'
jobs:
sync-templates:
runs-on: ubuntu-latest
steps:
- name: Checkout crawlee-python
uses: actions/checkout@v2
with:
path: crawlee-python
- name: Checkout actor-templates
uses: actions/checkout@v2
with:
repository: apify/actor-templates
path: actor-templates
token: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: '14'
- name: Install zip
run: sudo apt-get install zip
- name: Zip templates
run: |
cd crawlee-python/templates
for dir in */; do
zip -r "../../actor-templates/dist/templates/${dir%/}.zip" "$dir"
done
- name: Update manifest
run: |
cd actor-templates
npm install @octokit/core
node -e "
const { Octokit } = require('@octokit/core');
const fs = require('fs');
const path = require('path');
const octokit = new Octokit({ auth: '${{ secrets.GITHUB_TOKEN }}' });
const manifest = JSON.parse(fs.readFileSync('manifest.json', 'utf8'));
const templateDirs = fs.readdirSync('../crawlee-python/templates');
templateDirs.forEach(dir => {
const templatePath = path.join('..', 'crawlee-python', 'templates', dir);
if (fs.statSync(templatePath).isDirectory()) {
const templateInfoPath = path.join(templatePath, 'template-info.json');
if (fs.existsSync(templateInfoPath)) {
const templateInfo = JSON.parse(fs.readFileSync(templateInfoPath, 'utf8'));
const existingTemplate = manifest.templates.find(t => t.id === dir);
if (existingTemplate) {
//update existing template
Object.assign(existingTemplate, templateInfo);
existingTemplate.archiveUrl = `https://github.com/apify/actor-templates/blob/master/dist/templates/${dir}.zip?raw=true`;
} else {
//add new template
manifest.templates.push({
id: dir,
name: dir,
...templateInfo,
archiveUrl: `https://github.com/apify/actor-templates/blob/master/dist/templates/${dir}.zip?raw=true`
});
}
}
}
});
// Remove templates that no longer exist
manifest.templates = manifest.templates.filter(template =>
templateDirs.includes(template.id)
);
fs.writeFileSync('manifest.json', JSON.stringify(manifest, null, 2));
"
- name: Create Pull Request
uses: peter-evans/create-pull-request@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
path: actor-templates
commit-message: Update template zips and manifest
title: 'Update template zips and manifest'
body: 'This PR updates the template zip files and manifest based on changes in the crawlee-python repository.'
branch: update-templates
base: main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment