Coworking today.
I want to get a couple of ideas for a config file structure to configure sqleton. I started by asking the ai straight, but then realized I would like to have couple more takes on the whole thing.
In order to more easily generate multiple opinions, I decided to use the noweb capability.
I want to create a YAML configuration file to configure how a webserver serves its files and commands. I want to configure url routes, so that I can for each route specify:
- the path or wildcard path (along with capture argument)
- a potential repository path
- an index template path
- a type that can be "command", "static", "template"
- default flag values
- additional templating data
- a potential command template file
- override flag valuesThis second suggestion seems pretty decent, and it will end up being the one I use. But before moving on, let’s try again
I don’t like this one as much.
Now that we have a decent example for the config file, let’s start asking for the go implementation. First create a markdown block that contains the yaml, and retouch it a bit.
Here's an example YAML configuration file that covers the specified cases:
```yaml
routes:
- path: /api/users/
type: commandDirectory
repository: commands/users
commandTemplateFile: user_command.sh.tmpl
additionalData:
foobar: bla
defaults:
layers:
sqleton-connection:
dbt-profile: foobar.localhost
flags:
limit: 10
sort: asc
arguments:
foobar: bla
overrides:
layers:
glazed:
fields: [foobar, bla]
flags:
admin_user: "admin"
- path: /api/blop
type: command
commandPath: commands/blop.yaml
commandTemplateFile: single_command.md.tmpl
additionalData:
foobar: bla
defaults:
layers:
sqleton-connection:
dbt-profile: foobar.localhost
flags:
limit: 10
sort: asc
arguments:
foobar: bla
overrides:
layers:
glazed:
fields: [foobar, bla]
flags:
admin_user: "admin"
- path: /static/*filepath
type: static
localPath: public/static
- path: /blog
type: templateDirectory
localDirectory:
indexTemplateFile: templates/blog_post.html
additionalData:
author: "John Doe"
publication: "2021-01-01"
- path: /home
type: template
templateFile: templates/home.html
```I actually don’t like the fact that all the different types are merged in the same struct. Instead, I am going to ask the AI to reformat the yaml.
Here's an example YAML config file:
```yaml
routes:
- path: /api/users/
commandDirectory:
repository: commands/users
templateFile: user_command.sh.tmpl
indexTemplateFile: index.md.tmpl
additionalData:
foobar: bla
defaults:
layers:
sqleton-connection:
dbt-profile: foobar.localhost
flags:
limit: 10
sort: asc
arguments:
foobar: bla
overrides:
layers:
glazed:
fields: [foobar, bla]
flags:
admin_user: "admin"
- path: /api/blop
command:
file: commands/blop.yaml
templateFile: single_command.md.tmpl
additionalData:
foobar: bla
defaults:
layers:
sqleton-connection:
dbt-profile: foobar.localhost
flags:
limit: 10
sort: asc
arguments:
foobar: bla
overrides:
layers:
glazed:
fields: [foobar, bla]
flags:
admin_user: "admin"
- path: /static/*filepath
static:
localPath: public/static
- path: /blog
templateDirectory:
localDirectory: templates/
indexTemplateFile: blog_post.html
additionalData:
author: "John Doe"
publication: "2021-01-01"
- path: /home
template:
templateFile: templates/home.html
```
I don’t think there’s anything to amend to that generated code, so let’s store it as markdown block.
Here are the structs used to parse the config file.
type Route struct {
Path string `yaml:"path"`
CommandDirectory *CommandDir `yaml:"commandDirectory,omitempty"`
Command *Command `yaml:"command,omitempty"`
Static *Static `yaml:"static,omitempty"`
TemplateDirectory *TemplateDir `yaml:"templateDirectory,omitempty"`
Template *Template `yaml:"template,omitempty"`
}
type CommandDir struct {
Repository string `yaml:"repository"`
TemplateFile string `yaml:"templateFile"`
IndexTemplateFile string `yaml:"indexTemplateFile"`
AdditionalData map[string]string `yaml:"additionalData,omitempty"`
Defaults LayerParams `yaml:"defaults"`
Overrides LayerParams `yaml:"overrides"`
}
type Command struct {
File string `yaml:"file"`
TemplateFile string `yaml:"templateFile"`
AdditionalData map[string]string `yaml:"additionalData,omitempty"`
Defaults LayerParams `yaml:"defaults"`
Overrides LayerParams `yaml:"overrides"`
}
type Static struct {
LocalPath string `yaml:"localPath"`
}
type TemplateDir struct {
LocalDirectory string `yaml:"localDirectory"`
IndexTemplateFile string `yaml:"indexTemplateFile"`
AdditionalData map[string]string `yaml:"additionalData,omitempty"`
}
type Template struct {
TemplateFile string `yaml:"templateFile"`
}
type LayerParams struct {
Layers map[string]map[string]string `yaml:"layers,omitempty"`
Flags map[string]interface{} `yaml:"flags,omitempty"`
Arguments map[string]string `yaml:"arguments,omitempty"`
}
type Config struct {
Routes []Route `yaml:"routes"`
}
I want now to generate some kind of register code, but I wonder if I should just write it out by hand. Let’s first take a look at what we have in our sqleton file. I wonder if I should work in goland now, and copy paste across, or if should try to open the project in emacs and get more comfortable in emacs. TBH I think this is all better done in goland for now, until I get a bit more comfortable with the keyboard.:
Here’s the current code used to serve a commandDir, let’s see if we can use that to generate something sensible. This is a wild try.
server.Router.GET("/sqleton/*CommandPath", func(c *gin.Context) {
commandPath := c.Param("CommandPath")
commandPath = strings.TrimPrefix(commandPath, "/")
sqlCommand, ok := getRepositoryCommand(c, r, commandPath)
if !ok {
c.JSON(404, gin.H{"error": "command not found"})
return
}
type Link struct {
Href string
Text string
Class string
}
name := sqlCommand.Description().Name
dateTime := time.Now().Format("2006-01-02--15-04-05")
links := []Link{
{
Href: fmt.Sprintf("/download/%s/%s-%s.csv", commandPath, dateTime, name),
Text: "Download CSV",
Class: "download",
},
{
Href: fmt.Sprintf("/download/%s/%s-%s.json", commandPath, dateTime, name),
Text: "Download JSON",
Class: "download",
},
{
Href: fmt.Sprintf("/download/%s/%s-%s.xlsx", commandPath, dateTime, name),
Text: "Download Excel",
Class: "download",
},
{
Href: fmt.Sprintf("/download/%s/%s-%s.md", commandPath, dateTime, name),
Text: "Download Markdown",
Class: "download",
},
{
Href: fmt.Sprintf("/download/%s/%s-%s.html", commandPath, dateTime, name),
Text: "Download HTML",
Class: "download",
},
{
Href: fmt.Sprintf("/download/%s/%s-%s.txt", commandPath, dateTime, name),
Text: "Download Text",
Class: "download",
},
}
dev, _ := ps["dev"].(bool)
var dataTablesProcessorFunc glazed.CreateProcessorFunc
var localTemplateLookup render.TemplateLookup
if dev {
// NOTE(2023-04-19, manuel): This would lookup a precomputed handlerFunc that is computed by the repository watcher
// See note in WithUpdateCallback above.
// let's make our own template lookup from a local directory, with blackjack and footers
localTemplateLookup, err = render.LookupTemplateFromFSReloadable(
os.DirFS("."),
"cmd/sqleton/cmds/templates",
"cmd/sqleton/cmds/templates/**.tmpl.html",
)
if err != nil {
c.JSON(500, gin.H{"error": "could not create template lookup"})
return
}
} else {
localTemplateLookup, err = render.LookupTemplateFromFSReloadable(embeddedFiles, "templates/", "templates/**/*.tmpl.html")
if err != nil {
c.JSON(500, gin.H{"error": "could not create template lookup"})
return
}
}
dataTablesProcessorFunc = render.NewHTMLTemplateLookupCreateProcessorFunc(
localTemplateLookup,
"data-tables.tmpl.html",
render.WithHTMLTemplateOutputFormatterData(
map[string]interface{}{
"Links": links,
},
),
render.WithJavascriptRendering(),
)
handle := server.HandleSimpleQueryCommand(
sqlCommand,
glazed.WithCreateProcessor(
dataTablesProcessorFunc,
),
glazed.WithParserOptions(
glazed.WithStaticLayer("sqleton-connection", sqletonConnectionLayer.Parameters),
glazed.WithStaticLayer("dbt", dbtConnectionLayer.Parameters),
),
)
handle(c)
})