caddy variables.md

Variables in the Caddyfile

Caddy is a super webserver that has many useful features. Caddy can enable very very powerful scenarios and many of them are documented in this Wiki. As these scenarios become more elaborate (some might say complex!) writing a caddy config file starts to feel more like programming than basic configuration.

When that starts to happen I find myself reaching out for variables to enable multiple scenarios in a single configuration file by manipulating those variables.

For those situations caddy has a few types of variables to consider. What I hope to do here is illuminate how to use these variable types in the Caddyfile.

A note about examples

I will use examples to illustrate the concepts. I use the respond directive to verify how caddy works.

All the caddy example configurations are in a file named Caddyfile.

Open a command window and you can run caddy with caddy run.

In another command window you will be able to execute the caddy reload and curl commands.

Placeholders

You can read about placeholders in the Conventions#placeholders section of the docs and more infortion in the Concepts#placeholders section of the docs.

Note: Placeholders are generally usable anywhere there is a text field you would like to substitute. But... not every field can substitute placeholders today. The team has worked hard enabling placeholders in as many places as they thought they would be super useful, but not all fields. If you run into a field that you think should be enabled for placeholders simply add an issue to caddy.

OK, let's see them in action!

Simple reference

Let's just respond to a query with a string that has substitutions.

# My Caddyfile...
:2022 {
  respond "{time.now}:{system.os}:{system.arch}"
}

Remember to caddy run in it's own command window 😸.

Now that we have changed the Caddyfile we can reload it with the caddy reload command and execute the curl command.

$ caddy reload && curl localhost:2022

Expected output (something like):

2022-03-20 12:26:34.55429 -0700 PDT m=+59.288837501:linux:amd64

Great, I can see the time, my OS & my architecture separated by ":".

This just showed how you can reference placeholders within a Caddyfile.

Setting a custom placeholder with map

There are a couple of ways to set your own custom placeholders. Lets start with a map.

# My Caddyfile...
:2022 {
  map 1 {my_customvar} {
    default "this_is_my_custom_var"
  }
  respond "{my_customvar}"
}

outputs: "this_is_my_custom_var" Great we have set our own custom variable.

Using custom placeholders inside a snippet

Snippets can process arguments that are passed to the import directive so this is a form of setting variables also.

# My Caddyfile...
# declare mycustomargs snippet
(mycustomargs) {
  respond "arg0: {args.0} arg1: {args.1}"
}

:2022 {
  import mycustomargs my_argument1 my_argument2
}

outputs: "arg0: my_argument1 arg1: my_argument2" Slick! The arguments can only be referenced inside the Snippet but this turns out great for some customizations.

Custom placeholder limits

OK, we can set custom placeholders and we can reference them. But hold on, they cannot be referenced everywhere! Lets try to reference a placeholder within a map directive.

# My Caddyfile...
:2022, :2023 {
  map 1 {my_customvar} {
    default "custom1"
  }
  
  map {port} {mynewvar} {
    2022 "custom2022"
    2023 {my_customvar}
  }
  respond "{my_customvar} {mynewvar}"
}

I am trying to create a new custom variable mynewvar in the 2nd map directive. For port 2022 I set it to a string. For port 2023 I set it to the reference of my other custom variable my_customvar.

output from curl localhost:2022 : custom1 custom2022

so far so good, but...

output from curl localhost:2023 : custom1 {my_customvar}

Note: Directive ordering could be an issue here because the order of the 2 map directives is not deterministic. In this case it works in file order. But as you start to work on your Caddyfile it is important to know that the order of directives is not deterministic for multiple instances of the same directive.

To be more explicit we could have ordered the map directives in a route directive.

hang on, the custom variable my_customvar is not evaluated in the 2nd map configuration. It just set mynewvar to "{my_customvar}".

In Version v2.4.6 This is one of those places where placeholder substitution is not supported (yet). As part of writing this article I pointed out that substitution is not happening here and the Caddy team added that capability for later releases. (The team responded very quickly to my questions and suggestions which is a big reason I love working with Caddy).

But when doing this work, I had the thought it would be great if Caddy had a simple substitution method prior to parsing the Caddyfile. So I would not have to completely depend on placeholders.

How could I do that?

Environment Variables

Caddy Environment variables are a little different in that they only work in a Caddfyfile (not a JSON config file) and they perform the substitution before the file is parsed by caddy to create the webserver.

Using environment variables

# My Caddyfile...
:2022, :2023 {
  map {port} {mynewvar} {
    2022 "custom2022"
    2023 {$MYENVVAR}
  }
  respond "{$MYENVVAR} {mynewvar}"
}

$ MYENVVAR=custom1 caddy reload && curl localhost:2023
custom1 custom1
$ curl localhost:2022
custom1 custom2022

When I hit port 2023, "{$MYENVVAR}" is substituted with the value of your environment variable MYENVVAR and it even works on the parameters of the map directive. So now when I hit localhost:2023 map will set {mynewvar} equal to the subtitution for {$MYENVVAR}. Pretty nice, now I could paramaterize the parameters in a map, yay!

BTW! Setting the environment variable with a reload works. That is what I call a "surprise & delight", it just kind of works like you hope it might. (I was not sure it would work... thinking I might have to restart the first command window.)

Unintialized environment variables & defaults

We can set a default value for any environment variable substitution, but it is not a global substitution for all variable references.

# My Caddyfile...
:2022, :2023 {
  map {port} {mynewvar} {
    2022 "custom2022"
    2023 {$MYENVVAR:defaultValue}
  }
  respond "{$MYENVVAR} {mynewvar}"
}

$ MYENVVAR=custom1 caddy reload && curl localhost:2023
custom1 custom1
$ curl localhost:2022
custom1 custom2022

Still works the same, but what if I do not set the environment variable?

$ caddy reload && curl localhost:2023
 defaultValue
$ curl localhost:2022
 custom2022

This makes sense... MYENVVAR is not a defined environment variable. The value for {$MYENVVAR} is substituted with the value "defaultvalue" specified in the map parameters & {mynewvar} gets set with that default substitution. But the {$MYENVVAR} has no default in the response, so the empty string (value of the MYENVVAR environment variable) is substituted there.

That's a wrap for now!

Other topics for possible later post(s)

• What about variables in Caddy JSON configuration files?
• How does an expression directive interact with variables?
• How is a Named Matcher like a variable?
• What is this vars directive, how does it relate to variables?

@mholt, @francislavoie,

Hmm, another question for you.

When can I use placeholders? Here was a surprise (to me):

    file_server {
      index "index.{host}.html"
    }

{host} placeholder was not substituted in the string. e.g. my host: example.com:

    file_server {
      index "index.example.com.html"
    }

works, so I am thinking that the placeholder substitution does not work here.

What defines a place where substitution would work? I kind of expect it to work anywhere, but maybe with some exceptions. This is what is I took from the documentation:

Not all config fields support placeholders, but most do where you would expect it.

Thanks!

It's hard to properly document everywhere whether a field supports placeholders or not. We'd probably need to set up this little marker/widget that we'd plop throughout the docs to say whether something does or doesn't support placeholders, but that's a lot of work to go through the code and check each thing to see if they do or not.

But yeah, file_server's index that is one spot that likely doesn't right now, but probably not for any particular reason. It's just that there needs to be a line of code that actually triggers the placeholder replacement to happen at runtime, and it didn't get done for that one, because nobody's had a need for it there yet.

If you want to open an issue requesting support for it there to remind us, we can add it when we have a few minutes.

(Just seeing this again)

@francislavoie is right about the differences between vars and map. Although, I want to clarify something that I think I'm remembering correctly:

Of note, this means the map will be evaluated each time you use one of the destination outputs, not just once ahead of time.

This means (correct me if I'm wrong @mholt) if you need to use an output multiple times (and you have a huge map with like 10k+ items), it might not be a good idea to use map directly and it might be a good idea to use vars to evaluate the map to "cache" the output values instead (because map has higher order than vars so you can chain map -> vars but not the other way). But that's pretty edgecase-y.

Yes, map calls a function each time a custom placeholder is looked up. However, no (or only very few) allocations are made unless a match is found, in which case only the output value is allocated. See this PR (linked above in a previous comment as well): caddyserver/caddy#2674

So the map function being called is very cheap even for large maps, assuming the callback function is implemented efficiently (usually a switch statement which of course is mostly just a JMP or JE in assembly so it can be very fast like that).

I am not so sure about the performance of vars, it might be worse, or might be about the same. I haven't benchmarked it. 🤷‍♂️ Could be a good exercise left to the reader 😄

@brucejo75

When can I use placeholders?

The unfortunate answer is, anywhere the developer (often me, oops) added support for them. In general, placeholders can be used in most fields where you might expect them, but every so often we find places where users are putting placeholders that we didn't anticipate, so just open an issue to request it. When I wrote the index feature of the file server, I didn't anticipate people using dynamic values there; like it wasn't obvious that someone would want a dynamic value for a file index. Usually it's a pretty static index.html. So things like that we just wait for feature requests.

And yeah, it's not really worth documenting all the fields that do or don't support placeholders. Mainly because most fields can support placeholders even if they don't right now, so no need to codify one way or the other. And if we're going to cover the whole code base and document thusly, we might as well just add placeholder support while we go along... so. It's just a time savings thing.

Basically if you need placeholders somewhere they aren't expanded already, just request it in an issue.

Thanks @mholt, @francislavoie!

I dropped an issue and I updated the wiki page with a little note:

Note: Placeholders are generally usable anywhere there is a text field you would like to substitute. But... not every field can substitute placeholders today. The team has worked hard enabling placeholders in as many places as they thought they would be super useful, but not all fields. If you run into a field that you think should be enabled for placeholders simply add an issue to caddy.

Sounds good. Any plans to publish the wiki sometime?

Hi @mholt,

Quite a few changes have happened in v2.5. And this section will be wrong, because you enabled placeholders for maps. I was thinking I would wait until v2.5 released?

Would you rather I post what I have now? And not wait?

Ah yeah, good point. The post would need to be updated for 2.5, I think that's best, rather than publishing information that will go out of date very soon. When you have a chance to update the post for 2.5, I'll review it happily!