-
-
Save nikosmeds/3738f24853c85d27548645a75807b973 to your computer and use it in GitHub Desktop.
## Applying the following Keystone policy overrides with openstack-ansible. | |
keystone_policy_overrides: | |
cloud_admin: "role:admin and (is_admin_project:True or domain_id:238eb8a7ec424998b439d716c423dbde)" | |
admin_required: "role:admin" | |
admin_and_matching_user_domain_id: "rule:admin_required and domain_id:%(user.domain_id)s" | |
identity:create_user: "rule:cloud_admin or rule:admin_and_matching_user_domain_id" | |
## Confirm the cloud_admin alias' domain_id matches the `cloud_admin` domain. | |
$ openstack domain show cloud_admin | |
+-------------+----------------------------------+ | |
| Field | Value | | |
+-------------+----------------------------------+ | |
| description | Cloud admin domain | | |
| enabled | True | | |
| id | 238eb8a7ec424998b439d716c423dbde | | |
| name | cloud_admin | | |
| tags | [] | | |
+-------------+----------------------------------+ | |
$ openstack domain show product_test | |
+-------------+----------------------------------+ | |
| Field | Value | | |
+-------------+----------------------------------+ | |
| description | Niko product test domain | | |
| enabled | True | | |
| id | 5abad140eb3349c2b55ba9cf419f493c | | |
| name | product_test | | |
| tags | [] | | |
+-------------+----------------------------------+ | |
## Create project and user within the product_test domain. | |
$ openstack project create --domain product_test product_test2 | |
+-------------+----------------------------------+ | |
| Field | Value | | |
+-------------+----------------------------------+ | |
| description | | | |
| domain_id | 5abad140eb3349c2b55ba9cf419f493c | | |
| enabled | True | | |
| id | 2b3dda2239374ba98240dfd476af487d | | |
| is_domain | False | | |
| name | product_test2 | | |
| parent_id | 5abad140eb3349c2b55ba9cf419f493c | | |
| tags | [] | | |
+-------------+----------------------------------+ | |
$ openstack user create --project product_test2 --password product_test2 --domain product_test product_test2 | |
+---------------------+----------------------------------+ | |
| Field | Value | | |
+---------------------+----------------------------------+ | |
| default_project_id | 2b3dda2239374ba98240dfd476af487d | | |
| domain_id | 5abad140eb3349c2b55ba9cf419f493c | | |
| enabled | True | | |
| id | d6f0bc2a452c4775be2e22d61fd10993 | | |
| name | product_test2 | | |
| options | {} | | |
| password_expires_at | None | | |
+---------------------+----------------------------------+ | |
$ openstack role add admin --user product_test2 --user-domain product_test --project product_test2 | |
## Load the environment variable configuration file. | |
$ cat openrc-product-test2 | |
export LC_ALL=C | |
# COMMON CINDER ENVS | |
export CINDER_ENDPOINT_TYPE=internalURL | |
# COMMON NOVA ENVS | |
export NOVA_ENDPOINT_TYPE=internalURL | |
# COMMON OPENSTACK ENVS | |
export OS_ENDPOINT_TYPE=internalURL | |
export OS_INTERFACE=internalURL | |
export OS_USERNAME=product_test2 | |
export OS_PASSWORD=product_test2 | |
export OS_PROJECT_NAME=product_test2 | |
export OS_TENANT_NAME=product_test2 | |
export OS_AUTH_TYPE=password | |
export OS_AUTH_URL=http://10.103.0.8:5000/v3 | |
export OS_NO_CACHE=1 | |
export OS_USER_DOMAIN_NAME=product_test | |
export OS_PROJECT_DOMAIN_NAME=product_test | |
export OS_REGION_NAME=RegionOne | |
# For openstackclient | |
export OS_IDENTITY_API_VERSION=3 | |
export OS_AUTH_VERSION=3 | |
$ source openrc-product-test2 | |
$ openstack user create product_test3 --password product_test3 | |
+---------------------+----------------------------------+ | |
| Field | Value | | |
+---------------------+----------------------------------+ | |
| domain_id | default | | |
| enabled | True | | |
| id | d7eaab191f8544c1b86cf01520d3b065 | | |
| name | product_test3 | | |
| options | {} | | |
| password_expires_at | None | | |
+---------------------+----------------------------------+ |
Throwing darts in the dark here: let's allow any of the domain_id variations (shown in previous comment) just to see if ANY of them return True
.
$ git diff HEAD~1
diff --git a/roles/system-osa/templates/user_variables.yml.j2 b/roles/system-osa/templates/user_variables.yml.j2
index dd7a995..9b688c8 100644
--- a/roles/system-osa/templates/user_variables.yml.j2
+++ b/roles/system-osa/templates/user_variables.yml.j2
@@ -261,7 +261,7 @@ keystone_policy_overrides:
admin_and_matching_target_user_domain_id: "rule:admin_required and domain_id:%(target.user.domain_id)s"
admin_and_matching_user_domain_id: "rule:admin_required and domain_id:%(user.domain_id)s"
identity:get_user: "rule:cloud_admin or rule:admin_and_matching_target_user_domain_id or rule:owner"
- identity:list_users: "rule:cloud_admin or rule:admin_and_matching_domain_id"
+ identity:list_users: "rule:cloud_admin or rule:admin_and_matching_domain_id or rule:admin_and_matching_target_user_domain_id or rule:admin_and_matching_user_domain_id or rule:admin_or_owner"
Applied changes, and...
$ openstack user list
You are not authorized to perform the requested action: identity:list_users. (HTTP 403) (Request-ID: req-6dc1b47a-a74a-4780-9864-a4a71d694c84)
$ openstack user list --domain product_test
You are not authorized to perform the requested action: identity:list_users. (HTTP 403) (Request-ID: req-8f8b48a2-a257-4216-aea3-1f2ef5115442)
Ok... not good. How about we test deleting of existing users, since this applies policies to an actual target resource. Start by creating a new user (using cloud admin credentials).
$ openstack user create nsmeds3 --password nsmeds3 --domain product_test
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | 5abad140eb3349c2b55ba9cf419f493c |
| enabled | True |
| id | 47c27fc0576041de8eff4ca792b7a4ed |
| name | nsmeds3 |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
Switch to product_test
credentials, with domain-scoped token.
$ openstack user delete nsmeds3 --domain product_test
Failed to delete user with name or ID 'nsmeds3': You are not authorized to perform the requested action: identity:list_users. (HTTP 403) (Request-ID: req-7311227c-acb2-4603-8e26-de721d35f18a)
1 of 1 users failed to delete.
Oh common. Okay, we're opening the list_users
action to any admins.
$ git diff HEAD~1
diff --git a/roles/system-osa/templates/user_variables.yml.j2 b/roles/system-osa/templates/user_variables.yml.j2
index 9b688c8..e57edd8 100644
--- a/roles/system-osa/templates/user_variables.yml.j2
+++ b/roles/system-osa/templates/user_variables.yml.j2
@@ -261,7 +261,7 @@ keystone_policy_overrides:
- identity:list_users: "rule:cloud_admin or rule:admin_and_matching_domain_id or rule:admin_and_matching_target_user_domain_id or rule:admin_and_matching_user_domain_id or rule:admin_or_owner"
+ identity:list_users: "rule:admin_required"
Applied above changes. Again, using the product_test
user credentials with domain-scoped token.
$ openstack user list
+----------------------------------+---------------+
| ID | Name |
+----------------------------------+---------------+
| 1bbd32cd2371468fab764ea28586c235 | product_test |
| 47c27fc0576041de8eff4ca792b7a4ed | nsmeds3 |
| d6f0bc2a452c4775be2e22d61fd10993 | product_test2 |
+----------------------------------+---------------+
$ openstack user list
+----------------------------------+---------------+
| ID | Name |
+----------------------------------+---------------+
| 1bbd32cd2371468fab764ea28586c235 | product_test |
| 47c27fc0576041de8eff4ca792b7a4ed | nsmeds3 |
| d6f0bc2a452c4775be2e22d61fd10993 | product_test2 |
+----------------------------------+---------------+
$ openstack user delete nsmeds3 --domain product_test
Failed to delete user with name or ID 'nsmeds3': No user with a name or ID of 'nsmeds3' exists.
1 of 1 users failed to delete.
$ openstack user delete nsmeds3
$ openstack user list
+----------------------------------+---------------+
| ID | Name |
+----------------------------------+---------------+
| 1bbd32cd2371468fab764ea28586c235 | product_test |
| d6f0bc2a452c4775be2e22d61fd10993 | product_test2 |
+----------------------------------+---------------+
Apparently when you specify your default domain with --domain <domain>
things break. Good times. But happy to see we can in fact delete a user created within the same domain! Let's test a bit more functionality with the same credentials.
Using cloud_admin
credentials, creating user in cloud_admin
domain.
$ openstack user create nsmeds5 --password nsmeds5 --domain cloud_admin
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | 238eb8a7ec424998b439d716c423dbde |
| enabled | True |
| id | 52816efd1a374339b274820124210841 |
| name | nsmeds5 |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
Switch back to product_test
credentials and attempt to delete the previously created user.
$ openstack user delete nsmeds5 --domain cloud_admin
Failed to delete user with name or ID 'nsmeds5': No user with a name or ID of 'nsmeds5' exists.
1 of 1 users failed to delete.
Interesting - not permission denied, but simply states cannot find the user. Switching back to cloud_admin
credentials to delete with same command.
$ openstack user delete nsmeds5 --domain cloud_admin
$
How about user creation? Switching to product_test
credentials again.
$ openstack user create product_test5 --password product_test5 --domain default
You are not authorized to perform the requested action: identity:create_user. (HTTP 403) (Request-ID: req-605db68a-522d-4cd9-8111-c6ba2ce7d9b4)
$ openstack user create product_test5 --password product_test5
You are not authorized to perform the requested action: identity:create_user. (HTTP 403) (Request-ID: req-e5024d7e-30ee-4c06-8178-d3c436ab033f)
Yep, create_user
suffers from same issue as list_users
(since they target a domain and not a specific user resource). Will need to figure that out.
Reminder to self: add --debug
to any CLI command.
It isn't helping with my above problem, but hey - it's good to remember.
Alright... I don't want to admit to this, but I will: decided to directly test user creation with HTTP API (using curl). Was curious if issue somehow related to Python client.
- Request domain-scoped token (response not shown below, but contains token).
curl -i \
-H "Content-Type: application/json" \
-d '
{ "auth": {
"identity": {
"methods": ["password"],
"password": {
"user": {
"name": "product_test",
"domain": { "name": "product_test" },
"password": "product_test"
}
}
},
"scope": {
"domain": {
"name": "product_test"
}
}
}
}' \
"https://cloud.auto-h.net:5000/v3/auth/tokens" ; echo
- Add token as header, send user creation request.
curl -i \
-X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: gAAAAABcBcAq72OtOU7GE5RAogLhEMvIQtbY5CdLk0Hvq4n0frBN_EPe2sWhI03PeEmYtHSylZQ6yL4hJZv1UKr9YvqVz1AvcioogUORwybiwuvJLxZbEsO5eqGe7Us2vu5WRTy8NqV5TlNkKMXkW3D4WUFOymZfi99eHxqWRwxM6jvj1mfVh0Q" \
-d '
{ "user": {
"default_project_id": "cc91843aee474f3295eb3fc46961bab3",
"domain_id": "5abad140eb3349c2b55ba9cf419f493c",
"enabled": true,
"name": "Product REST",
"password": "productrest"
}
}' \
"https://cloud.auto-h.net:5000/v3/users" ; echo
HTTP/1.1 201 Created
Server: nginx/1.10.3 (Ubuntu)
Date: Mon, 03 Dec 2018 23:49:15 GMT
Content-Type: application/json
Content-Length: 338
Vary: X-Auth-Token
x-openstack-request-id: req-cd8ab7e9-37fa-4bb0-815b-ed09b653c9f8
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'self' https: wss:;
X-Frame-Options: DENY
{"user": {"name": "Product REST", "links": {"self": "https://cloud.auto-h.net:5000/v3/users/940f01d8819f4700a9fd017fea474e37"}, "domain_id": "5abad140eb3349c2b55ba9cf419f493c", "enabled": true, "options": {}, "default_project_id": "cc91843aee474f3295eb3fc46961bab3", "id": "940f01d8819f4700a9fd017fea474e37", "password_expires_at": null}}
Yeah I've left passwords in this and previous comments - will destroy these users once testing complete.
... but more importantly - it worked.
$ openstack user list --domain product_test
+----------------------------------+---------------+
| ID | Name |
+----------------------------------+---------------+
| 1bbd32cd2371468fab764ea28586c235 | product_test |
| 940f01d8819f4700a9fd017fea474e37 | Product REST |
| d6f0bc2a452c4775be2e22d61fd10993 | product_test2 |
+----------------------------------+---------------+
And ensuring the product_test
token cannot create a user in cloud_admin
domain.
curl -i \
-X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: gAAAAABcBcAq72OtOU7GE5RAogLhEMvIQtbY5CdLk0Hvq4n0frBN_EPe2sWhI03PeEmYtHSylZQ6yL4hJZv1UKr9YvqVz1AvcioogUORwybiwuvJLxZbEsO5eqGe7Us2vu5WRTy8NqV5TlNkKMXkW3D4WUFOymZfi99eHxqWRwxM6jvj1mfVh0Q" \
-d '
{ "user": {
"default_project_id": "ec1bd3dd5e4949b4907fed4cf7c95569",
"domain_id": "238eb8a7ec424998b439d716c423dbde",
"enabled": true,
"name": "Product REST",
"password": "productrest"
}
}' \
"https://cloud.auto-h.net:5000/v3/users" ; echo
HTTP/1.1 403 Forbidden
Server: nginx/1.10.3 (Ubuntu)
Date: Mon, 03 Dec 2018 23:57:10 GMT
Content-Type: application/json
Content-Length: 138
Vary: X-Auth-Token
x-openstack-request-id: req-f54c1b7e-7f27-4e1b-bd2c-25db860b9f63
{"error": {"message": "You are not authorized to perform the requested action: identity:create_user.", "code": 403, "title": "Forbidden"}}
Amazing - RBAC is working as expected. Either the Python client has an issue, or I'm configuring the environment variables wrong.
This has been rough.
It came to me, like a vision of embarrassment and shame, that most of my confusion stems from one possibility.
Ready?
$ source openrc-product-test
$ openstack token issue
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| domain_id | 5abad140eb3349c2b55ba9cf419f493c |
| expires | 2018-12-05T04:04:59+0000 |
| id | gAAAAABcBqWrOKQGJOoSy72gF59ieqQ_KNUGVMlzZdLNN_UAlhxZkqAZvWSI7NAxilBYm5YfCiZUNaC7_lRk7eaexP2e2gmaYJRXAy7GrCaGHgbpw4ESvDgGCWFMtP4cY5bz4AnYlEtP08cMBXRCyIjEAzQusmSh4tpueDQ38Ca6WmWvP4yzpbU |
| user_id | 1bbd32cd2371468fab764ea28586c235 |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack user list --domain product_test
You are not authorized to perform the requested action: identity:list_users. (HTTP 403) (Request-ID: req-eb49060b-7b46-4e0a-94d6-5c502f1e909c)
$ openstack user list --domain 5abad140eb3349c2b55ba9cf419f493c
+----------------------------------+---------------+
| ID | Name |
+----------------------------------+---------------+
| 1bbd32cd2371468fab764ea28586c235 | product_test |
| d2fba554a133468f9567453be72ee0b1 | product-test3 |
| d6f0bc2a452c4775be2e22d61fd10993 | product_test2 |
+----------------------------------+---------------+
The Keystone policy files compare domain ID, and therefore we must provide the API with a domain ID. Even though names and IDs have a one-to-one relationship, Keystone doesn't make that conversion and will deny actions when provided with a domain name.
This also explains why my cURL request in the previous comment was successful - I provided a domain ID.
It's time to get all Keystone policy changes (https://gist.github.com/nikosmeds/74881a4de5c777b8e808cdee442cdcdb) applied and working - however, once again there are issues to investigate and resolve.
My
nsmeds
user in thecloud_admin
domain can successfully create and delete users in any domain, great.The
product_test
user inproduct_test
domain is unable to list users in any domain, includingproduct_test
domain. Not so great.We see the following in Keystone container's log file.
Am I having issues again with correctly scoping the token?
Commenting out the
OS_DOMAIN_NAME
variable allows us to grab a system-scoped token.Nope. Alright, here we go again. So below we can see the policy rules for listing users.
Going to troubleshoot by removing requirements until
product_test
user can list users in their own domain. Because we're comparing the token's domain ID to the API request's target domain, we need to ensure that we use a domain-scoped token.Nope, same permission denied error.
Let's check the reverse. Also, personal note: this is tedious as hell to troubleshoot.
Apply changes and confirm
product_test
user can list users now.Great, so the issue is with
"domain_id:%(domain_id)s"
, which should evaluateTrue
when the token's domain ID matches the API calls domain ID. There are a few variations of this in policy.v3cloudsample, and either they have a mistake in their example or I'm once again misunderstanding something. Here's my current understanding of these variations.domain_id:%(target.user.domain_id)s
domain_id:%(domain_id)s
domain_id:%(target.project.domain_id)s
domain_id:%(project.domain_id)s
domain_id:%(target.token.user.domain.id)s
check_token
,validate_token
, andrevoke_token
.The best related documentation I've been able to find is https://docs.openstack.org/keystone/queens/admin/identity-service-api-protection.html. Also, reviewing what rules are applied to what actions in https://github.com/openstack/keystone/blob/master/etc/policy.v3cloudsample.json gave me a better understanding of the above
domain_id
examples.... except it isn't working.