Skip to content

Instantly share code, notes, and snippets.

@FelixMcFelix

FelixMcFelix/2024-01-12-floating-ip-strawman.md

Last active January 14, 2024 16:57
Show Gist options
  • Save FelixMcFelix/18d20262a918ccf691a325a8d948379d to your computer and use it in GitHub Desktop.
Save FelixMcFelix/18d20262a918ccf691a325a8d948379d to your computer and use it in GitHub Desktop.
Download ZIP
Strawman for moving forward on floating IP APIs
Raw
2024-01-12-floating-ip-strawman.md

CLI project selectors omitted for clarity.

I'm writing this mainly to figure out what we want for the initial floating IP attach/detach API from our discussion earlier in the week. We need to balance usability (API, CLI) with space for future features tthat will build directly on this.

Having written this out now, I'm leaning towards Prop.3 for floating IP atach/detach, and /v1/instances/{instance}/external-ips/ephemeral for ephemeral IP add/delete.

Refresher on today's state

  • oxide floating-ip {list,create,view,delete}
    • GET /v1/floating-ips
    • POST /v1/floating-ips
    • GET /v1/floating-ips/{floating_ip}
    • DELETE /v1/floating-ips/{floating_ip}
  • oxide instance external-ip list --instance xxx will list all bound to a target instance
    • GET /v1/instances/{instance}/external-ips
      • Today: [Ephemeral(IpAddr) | Floating(IpAddr)]
      • Omicron#4694: [Ephemeral(IpAddr) | Floating(FloatingIp)]
  • FloatingIp contains all identity metadata, and refers to attached instance via parent_id: Option<Uuid>.
  • /v1/instances/{instance}/external-ips/[attach,detach] is determined as the wrong expression for FIPs, and leads to a deeply nested CLI that needs manual command specification.
  • We know that Floating IPs will have several targets: instances today, silo services in future, and other project-scoped resources like load balancers and Internet gateways.

Attach/detach FIP

Proposal 1 -- move attach/detach to FIP resource

/v1/floating-ips/{floating_ip}/attach

  • POST { "instance": "<name_or_id>" } -> FloatingIp
  • POST { "service": "<name>" } -> FloatingIp
  • POST { "load-balancer": "<name_or_id>" } -> FloatingIp
  • POST { "gateway": "<name_or_id>" } -> FloatingIp

The above is captured with a body type like the below:

enum FloatingIpAttach {
    Instance(NameOrId),
    Service(Name), // won't be present today.
    LoadBalancer(NameOrId), // won't be present today.
    InternetGateway(NameOrId), // won't be present today.
}

/v1/floating-ips/{floating_ip}/detach

  • POST <empty> -> FloatingIp

Ideal CLI expression

  • oxide floating-ip attach --floating-ip xxx --instance yyy

  • oxide floating-ip attach --floating-ip xxx --service dns

  • oxide floating-ip attach --floating-ip xxx --service nexus

  • oxide floating-ip detach --floating-ip xxx

However, using the above TypedBody/FloatingIpAttach does not autogenerate a useful API. We instead get:

Usage: oxide floating-ip attach [OPTIONS] --floating-ip <floating-ip> --json-body <JSON-FILE>

Options:
      --floating-ip <floating-ip>  Name or ID of the Floating IP
      --project <project>          Name or ID of the project
      --json-body <JSON-FILE>      Path to a file that contains the full json body.
      --json-body-template         XXX
  -h, --help                       Print help

Detach works nicely though.

Pros/cons:

  • (+) One endpoint for each 'verb', so REST API feels tidier to me.
  • (-) Most users should not be assigning FIPs to services, so maybe it shouldn't have the same 'visibility'.
  • (-) CLI needs manual endpoint insertion/maintenance.

Proposal 2 -- Split Prop1. endpoint by parent class.

/v1/floating-ips/{floating_ip}/attach

  • POST { "instance": "<name_or_id>" } -> FloatingIp

/v1/floating-ips/{floating_ip}/attach-service

  • POST { "service": "<name>" } -> FloatingIp

/v1/floating-ips/{floating_ip}/detach

  • POST <empty> -> FloatingIp

CLI expression

  • oxide floating-ip attach --floating-ip xxx --instance yyy

  • oxide floating-ip attach service --floating-ip xxx --service dns

  • oxide floating-ip attach service --floating-ip xxx --service nexus

  • oxide floating-ip detach --floating-ip xxx

Pros/cons:

  • (+) These autogenerate nicely with the CLI, including the attach service subcommand.
  • (+) Service attach will require silo admin access I'd imagine, so maybe splitting this out from 'regular' calls makes that more explicit.
  • (+) Gives us space to include more parameters for Services -- we don't know exactly what service attach will need yet (and this may vary on a per-service basis).
  • ( ) The 'happy path' has the simpler syntax for both CLI/API.
  • (-) Attachment to other project-scoped objects like load balancers will also be common, and delegating to another command feels unsatisfying.
  • (-) This feels a bit more untidy as an API.

Proposal 3 -- Prop.1 with separate type

/v1/floating-ips/{floating_ip}/attach

  • POST { "type": "instance", "parent": "<name_or_id>" } -> FloatingIp
  • POST { "type": "service", "parent": "<name>" } -> FloatingIp
  • POST { "type": "load-balancer", "parent": "<name_or_id>" } -> FloatingIp
  • POST { "type": "gateway", "parent": "<name_or_id>" } -> FloatingIp

via:

struct FloatingIpAttach {
    parent: NameOrId,
    type: FloatingIpAttachKind,
}

enum FloatingIpAttachKind {
    Instance,
    Service, // won't be present today.
    LoadBalancer, // won't be present today.
    InternetGateway, // won't be present today.
}

/v1/floating-ips/{floating_ip}/detach

  • POST <empty> -> FloatingIp

CLI expression

  • oxide floating-ip attach --floating-ip xxx --type instance --parent xxx

  • oxide floating-ip attach --floating-ip xxx --type service --parent dns

  • oxide floating-ip attach --floating-ip xxx --type service --parent nexus

  • oxide floating-ip detach --floating-ip xxx

Pros/cons:

  • (+) These autogenerate okay.
  • (+) API itself has fewer endpoints for the same operation.
  • (+) Puts the attach operations which reference objects likely to be regularly created in the user-facing API on equal footing (instances, loadbalancers, gateways).
  • ( ) If instances are the most common target, we could make type optional in the CLI. It should always be specified in the API.
  • ( ) Nothing stopping us from breaking out services into a separate command if extra flags etc needed, or we feel the separation of access rights should be highlighted.
  • (-) Autogenerated CLI is noisy. We could put in custom parse logic via OxideOverride to attempt to simplify.

Proposal 4 -- PUT/PATCH on parent/parent_id field

/v1/floating-ips/{floating_ip}/parent

  • PUT { "instance": "<name_or_id>" } -> FloatingIp (attach instance)
  • PUT { "service": "<name?>" } -> FloatingIp (attach service)
  • PUT <empty> -> FloatingIp (detach)

Alternatively, we could PUT/PATCH on the floating-ip resource itself (i.e., via the parent field). I don't think this would be at all ergonomic in CLI.

Attach/detach ephemeral

I'm a little bit fuzzier on what this should look like:

/v1/instances/{instance}/external-ips/ephemeral

  • GET -> Option<ExternalIp>
  • POST {"pool": "<name_or_id>"} -> ExternalIp
  • DELETE -> Ok()

We could have these as separate endpoints using POST, but I think that would be awful:

  • /v1/instances/{instance}/external-ips/add-ephemeral
  • /v1/instances/{instance}/external-ips/delete-ephemeral

CLI expression

  • oxide instance external-ip add-ephemeral --instance xxx --pool yyy
  • oxide instance external-ip delete-ephemeral --instance xxx --pool yyy

Pros/cons:

  • (+) Default behaviour is that users will receive an ephemeral IP, and this API is not a necessity to receive one.

Future Scope

  • Ephemeral should be dealloc'd + realloc's on instance start stop
    • API consideration: /v1/instances/{instance}/external-ips and related should change Ephemeral(IpAddr)->Ephemeral(Option<IpAddr>)
  • Effective A->B attach without detach for, e.g., blue-green deployment.
    • will likely require extra OPTE work to guarantee there's always an instance listening to traffic.
    • should active-passive use be left as a consideration for load balancers and/or DNS? This suggests multiple parents which is a bit of a shift from where we are today.
  • Non-instance parents of floating IPs:
    • API consideration: FloatingIp::parent_id should change Option<Uuid>->Option<Parent>, with enum Parent {Instance(Uuid), Service(...), ...}. This would make Proposal 4 more infeasible, but I'm not sure that's even in the running.
  • Ephemeral upgrade to float:
    • /v1/instances/{instance}/external-ips/upgrade-ephemeral feels bad -- it could possibly be specified as a source parameter on the existing FIP create API?
  • GET /v1/floating-ips/available to list unused FIPs in a project?
  • Resource limit for 'IPs in use by a project'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment