hybrid-cluster.md

Hybrid Cluster

There are two server components, the cloud server and the internal or on-prem cluster.

On-Prem Server

The on-prem server, defines:

• an account X with a single user x.
• a leaf node configuration that maps a remote user leaf (and by extension leaf's account in the cloud) to account X

Users from the X account will be able to see all traffic through the specified remote. Note that the remotes is an array, you can specify and map multiple remote credentials (presumably to different accounts), into the same local account.

port: 5222
server_name: S1

cluster {
  name: internal
  listen: "127.0.0.1:5001"
}
host: "127.0.0.1"
http: "127.0.0.1:5002"

leafnodes: {
  remotes = [
    { url: "nats://leaf:leaf@localhost:7422", account: "X" },
  ]
}

accounts: {
  X: {
    users: [
      { user: "x", password: "x" }
    ],
  },
}

The Cloud Server

The cloud server (think of it as NGS) has multiple accounts. The user for account S is the credentials used by the on-prem service. Note that it exports a single service. Which is imported by account U.

When users of U request to q, the subject is mapped to q.u, and delivered to clients of account S. In this case the clients for S are behind the leaf node.

port: 4222
server_name: S1

cluster {
  name: internal
  listen: "127.0.0.1:4001"
}
host: "127.0.0.1"
http: "127.0.0.1:4002"

leafnodes: {
  port: 7422
}

accounts: {
  U: {
    users: [
      { user: "u", password: "u" }
    ],
    imports: [
      { service: { account: "S", subject: "q.u"}, to: "q"}
    ]
  },
  S: {
    users: [
      { user: "leaf", password: "leaf" }
    ],
    exports: [
      { service: "q.*", accounts: ["U"] }
    ]
  }
}

Start the cloud server: nats-server -c cloud.conf. Start the service server: nats-server -c internal.conf.

A subscriber on the service side (connected via account X:

nats sub -s nats://x:x@localhost:5222 ">"

To publish a message from account U:

nats pub -s nats://u:u@localhost:4222 q hello

A couple more hints here. If you have an internal cluster and you want to leaf-node access all servers must be leafnode remotes. Traffic that is sent via the leafnode doesn't travel through the routes (the normal clustering), otherwise messages would be delivered multiple times.

JWT is similar to all of this, with the exception that it makes the configuration harder to inspect.

The on prem server can also be a cluster of server.

We do support multiple remotes, but when you want to map all accounts directly, you need to reload (and add a new remote) every time you add a new user account. This is here we connect accounts X and S and from there connect S and and the user account U via exports/imports. For JWT this makes it easier to add more user accounts Un as exports/imports are part of the JWT and can be added during account creation AND DO NOT require modifying the server config .