-
-
Save SolomonHD/b55be40146b7a53b8f26fe244f5be52e to your computer and use it in GitHub Desktop.
| ## Prerequisites: | |
| # 1) Create a Docker IPv4 bridge network | |
| # 2) Pick an IPv4 Address for Traefik, use the end of the network range | |
| # 3) Pick a domain, recommend real DNS but faking is possible by editing the /etc/hosts file to point to Traefik's IPv4 address. | |
| # 4) Replace the ${DOMAIN} place holder in the config-ldap.yaml, and env-config.js and files with the domain. | |
| # 5) Get a valid certificate via Let's Encrypt or another method, recommend a wildcard cert | |
| # 6) Copy cert/private key as /etc/letsencrypt/live/${DOMAIN}/{privkey.pem, fullchain.pem}, Uncomment lines 233-234 in the docker.compose.yml file | |
| # Also uncomment the lines in traefik_dynamic.yaml after cert and key are in place | |
| # 7) Copy this file as '.env', fill out with the appropriate values | |
| ### Domain Names | |
| ## https://tk.${DOMAIN} # Main website, terrakube-ui | |
| ## https://tk-api.${DOMAIN} # API, terrakube-api | |
| ## https://tk-registry.${DOMAIN} # Registry, terrakube-registry | |
| ## https://tk-executor.${DOMAIN} # Executor, terrakube-executor | |
| ## https://tk-dex.${DOMAIN} # Dex, terrakube-dex | |
| # Required Variables | |
| DOMAIN= # Set domain | |
| EXTERNAL_NETWORK_NAME= # External network is required | |
| HOST_GATEWAY=host-gateway # Should be okay leaving as is, may have to change to Traefik's IPv4 address | |
| TK_OUTPUT_ACCESS_KEY=minioadmin | |
| TK_OUTPUT_ENDPOINT=http://terrakube-minio:9000 | |
| TK_OUTPUT_SECRET_KEY=minioadmin | |
| TK_OUTPUT_STORAGE_REGION=us-east-1 | |
| TK_OUTPUT_BUCKET_NAME=sample | |
| TK_VERSION=2.19.2 | |
| TRAEFIK_IPV4_ADDRESS= # Give Traefik a reserved IPv4 Address in your external network, pick something towards the end of the network to avoid conflicts | |
| TRAEFIK_HTTP_PORT=80 | |
| TRAEFIK_HTTPS_PORT=443 | |
| # Optional Variables | |
| DNS_IP_PUBLIC= |
| .env | |
| .test |
| dn: dc=example,dc=org | |
| dc: example | |
| objectClass: dcObject | |
| objectClass: organization | |
| o: Example, Inc | |
| dn: ou=users,dc=example,dc=org | |
| ou: users | |
| objectClass: organizationalunit | |
| dn: cn=lester,ou=users,dc=example,dc=org | |
| objectClass: inetOrgPerson | |
| sn: Parkinson | |
| cn: Lester | |
| mail: admin@example.com | |
| userpassword: admin | |
| dn: cn=grady,ou=users,dc=example,dc=org | |
| objectClass: inetOrgPerson | |
| sn: Chambers | |
| cn: Grady | |
| mail: aws@example.com | |
| userpassword: azure | |
| dn: cn=saarah,ou=users,dc=example,dc=org | |
| objectClass: inetOrgPerson | |
| sn: Lott | |
| cn: Saarah | |
| mail: azure@example.com | |
| userpassword: aws | |
| dn: cn=eugene,ou=users,dc=example,dc=org | |
| objectClass: inetOrgPerson | |
| sn: Monaghan | |
| cn: Eugene | |
| mail: gcp@example.com | |
| userpassword: gcp | |
| # Group definitions. | |
| dn: ou=Groups,dc=example,dc=org | |
| objectClass: organizationalUnit | |
| ou: Groups | |
| dn: cn=TERRAKUBE_ADMIN,ou=Groups,dc=example,dc=org | |
| objectClass: groupOfNames | |
| cn: TERRAKUBE_ADMIN | |
| member: cn=lester,ou=users,dc=example,dc=org | |
| dn: cn=TERRAKUBE_DEVELOPERS,ou=Groups,dc=example,dc=org | |
| objectClass: groupOfNames | |
| cn: TERRAKUBE_DEVELOPERS | |
| member: cn=lester,ou=users,dc=example,dc=org | |
| dn: cn=AZURE_DEVELOPERS,ou=Groups,dc=example,dc=org | |
| objectClass: groupOfNames | |
| cn: AZURE_DEVELOPERS | |
| member: cn=saarah,ou=users,dc=example,dc=org | |
| dn: cn=AWS_DEVELOPERS,ou=Groups,dc=example,dc=org | |
| objectClass: groupOfNames | |
| cn: AWS_DEVELOPERS | |
| member: cn=grady,ou=users,dc=example,dc=org | |
| dn: cn=GCP_DEVELOPERS,ou=Groups,dc=example,dc=org | |
| objectClass: groupOfNames | |
| cn: GCP_DEVELOPERS | |
| member: cn=eugene,ou=users,dc=example,dc=org |
| # REPLACE ${DOMAIN} with the base domain! | |
| issuer: https://tk-dex.${DOMAIN}/dex | |
| storage: | |
| type: memory | |
| web: | |
| http: 0.0.0.0:5556 | |
| allowedOrigins: ["*"] | |
| oauth2: | |
| responseTypes: ["code", "token", "id_token"] | |
| connectors: | |
| - type: ldap | |
| name: OpenLDAP | |
| id: ldap | |
| config: | |
| # The following configurations seem to work with OpenLDAP: | |
| # | |
| # 1) Plain LDAP, without TLS: | |
| host: terrakube-ldap-service:1389 | |
| insecureNoSSL: true | |
| insecureSkipVerify: true | |
| # | |
| # 2) LDAPS without certificate validation: | |
| #host: localhost:636 | |
| #insecureNoSSL: false | |
| #insecureSkipVerify: true | |
| # | |
| # 3) LDAPS with certificate validation: | |
| #host: YOUR-HOSTNAME:636 | |
| #insecureNoSSL: false | |
| #insecureSkipVerify: false | |
| #rootCAData: 'CERT' | |
| # ...where CERT="$( base64 -w 0 your-cert.crt )" | |
| # This would normally be a read-only user. | |
| bindDN: cn=admin,dc=example,dc=org | |
| bindPW: admin | |
| usernamePrompt: Email Address | |
| userSearch: | |
| baseDN: ou=users,dc=example,dc=org | |
| filter: "(objectClass=person)" | |
| username: mail | |
| # "DN" (case sensitive) is a special attribute name. It indicates that | |
| # this value should be taken from the entity's DN not an attribute on | |
| # the entity. | |
| idAttr: DN | |
| emailAttr: mail | |
| nameAttr: cn | |
| groupSearch: | |
| baseDN: ou=Groups,dc=example,dc=org | |
| filter: "(objectClass=groupOfNames)" | |
| userMatchers: | |
| # A user is a member of a group when their DN matches | |
| # the value of a "member" attribute on the group entity. | |
| - userAttr: DN | |
| groupAttr: member | |
| # The group name should be the "cn" value. | |
| nameAttr: cn | |
| staticClients: | |
| - id: example-app | |
| redirectURIs: | |
| - "https://tk.${DOMAIN}" | |
| - "https://tk-api.${DOMAIN}" | |
| - "https://tk-dex.${DOMAIN}" | |
| - "/device/callback" | |
| - "http://localhost:3000/login" | |
| - "http://localhost:10001/login" | |
| name: "Example App" | |
| #secret: ZXhhbXBsZS1hcHAtc2VjcmV0 | |
| public: true |
| version: "3.8" | |
| ### DNS #### | |
| x-terrakube_dns: &terrakube_dns | |
| - ${DNS_IP_PUBLIC:-1.1.1.1} | |
| #### Extra Hosts ##### | |
| x-terrakube_hosts: &terrakube_hosts | |
| - "tk-api.${DOMAIN}:${HOST_GATEWAY}" | |
| - "tk-dex.${DOMAIN}:${HOST_GATEWAY}" | |
| - "tk-executor.${DOMAIN}:${HOST_GATEWAY}" | |
| - "tk.${DOMAIN}:${HOST_GATEWAY}" | |
| - "tk-registry.${DOMAIN}:${HOST_GATEWAY}" | |
| #### Container Environment Variables ##### | |
| x-api: &api_env | |
| ApiDataSourceType: POSTGRESQL | |
| DatasourceHostname: postgresql-service | |
| DatasourceDatabase: terrakubedb | |
| DatasourceUser: terrakube | |
| DatasourcePassword: terrakubepassword | |
| GroupValidationType: DEX | |
| UserValidationType: DEX | |
| AuthenticationValidationType: DEX | |
| TerrakubeHostname: https://tk-api.${DOMAIN} | |
| AzBuilderExecutorUrl: https://tk-executor.${DOMAIN}/api/v1/terraform-rs | |
| PatSecret: ejZRSFgheUBOZXAyUURUITUzdmdINDNeUGpSWHlDM1g= | |
| InternalSecret: S2JeOGNNZXJQTlpWNmhTITkha2NEKkt1VVBVQmFeQjM= | |
| DexIssuerUri: https://tk-dex.${DOMAIN}/dex | |
| StorageType: AWS | |
| AwsStorageAccessKey: $TK_OUTPUT_ACCESS_KEY | |
| AwsStorageSecretKey: $TK_OUTPUT_SECRET_KEY | |
| AwsStorageBucketName: $TK_OUTPUT_BUCKET_NAME | |
| AwsStorageRegion: $TK_OUTPUT_STORAGE_REGION | |
| AwsEndpoint: $TK_OUTPUT_ENDPOINT | |
| TerrakubeUiURL: https://tk.${DOMAIN} | |
| spring_profiles_active: demo | |
| DexClientId: example-app | |
| CustomTerraformReleasesUrl: "https://releases.hashicorp.com/terraform/index.json" | |
| TerrakubeRedisHostname: terrakube-redis | |
| TerrakubeRedisPort: 6379 | |
| TerrakubeRedisPassword: password123456 | |
| JAVA_TOOL_OPTIONS: -Dcom.sun.security.enableAIAcaIssuers=true | |
| x-executor: &executor_env | |
| TerrakubeEnableSecurity: true | |
| InternalSecret: S2JeOGNNZXJQTlpWNmhTITkha2NEKkt1VVBVQmFeQjM= | |
| TerraformStateType: AwsTerraformStateImpl | |
| AwsTerraformStateAccessKey: $TK_OUTPUT_ACCESS_KEY | |
| AwsTerraformStateSecretKey: $TK_OUTPUT_SECRET_KEY | |
| AwsTerraformStateBucketName: $TK_OUTPUT_BUCKET_NAME | |
| AwsTerraformStateRegion: us-east-1 | |
| AwsEndpoint: $TK_OUTPUT_ENDPOINT | |
| TerraformOutputType: AwsTerraformOutputImpl | |
| AwsTerraformOutputAccessKey: $TK_OUTPUT_ACCESS_KEY | |
| AwsTerraformOutputSecretKey: $TK_OUTPUT_SECRET_KEY | |
| AwsTerraformOutputBucketName: $TK_OUTPUT_BUCKET_NAME | |
| AwsTerraformOutputRegion: us-east-1 | |
| AzBuilderApiUrl: https://tk-api.${DOMAIN} | |
| ExecutorFlagBatch: false | |
| ExecutorFlagDisableAcknowledge: false | |
| TerrakubeToolsRepository: https://github.com/AzBuilder/terrakube-extensions.git | |
| TerrakubeToolsBranch: main | |
| TerrakubeRegistryDomain: tk-registry.${DOMAIN} | |
| TerrakubeApiUrl: https://tk-api.${DOMAIN} | |
| CustomTerraformReleasesUrl: "https://releases.hashicorp.com/terraform/index.json" | |
| TerrakubeRedisHostname: terrakube-redis | |
| TerrakubeRedisPort: 6379 | |
| TerrakubeRedisPassword: password123456 | |
| JAVA_TOOL_OPTIONS: > | |
| -Xmx512m -Xms256m -Dcom.sun.security.enableAIAcaIssuers=true | |
| x-registry: ®istry_env | |
| AzBuilderRegistry: https://tk-registry.${DOMAIN} | |
| AzBuilderApiUrl: https://tk-api.${DOMAIN} | |
| AuthenticationValidationTypeRegistry: DEX | |
| TerrakubeEnableSecurity: true | |
| DexIssuerUri: https://tk-dex.${DOMAIN}/dex | |
| TerrakubeUiURL: https://tk.${DOMAIN} | |
| PatSecret: ejZRSFgheUBOZXAyUURUITUzdmdINDNeUGpSWHlDM1g= | |
| InternalSecret: S2JeOGNNZXJQTlpWNmhTITkha2NEKkt1VVBVQmFeQjM= | |
| RegistryStorageType: AwsStorageImpl | |
| AwsStorageAccessKey: $TK_OUTPUT_ACCESS_KEY | |
| AwsStorageSecretKey: $TK_OUTPUT_SECRET_KEY | |
| AwsStorageBucketName: $TK_OUTPUT_BUCKET_NAME | |
| AwsStorageRegion: $TK_OUTPUT_STORAGE_REGION | |
| AwsEndpoint: $TK_OUTPUT_ENDPOINT | |
| AppClientId: example-app | |
| AppIssuerUri: https://tk-dex.${DOMAIN}/dex | |
| JAVA_TOOL_OPTIONS: -Dcom.sun.security.enableAIAcaIssuers=true | |
| x-ldap: &ldap_env | |
| LDAP_TLS_VERIFY_CLIENT: try | |
| LDAP_ADMIN_USERNAME: "admin" | |
| LDAP_ADMIN_PASSWORD: "admin" | |
| LDAP_ROOT: "dc=example,dc=org" | |
| LDAP_CUSTOM_LDIF_DIR: "/ldifs" | |
| x-minio: &minio_env | |
| MINIO_ROOT_USER: minioadmin | |
| MINIO_ROOT_PASSWORD: minioadmin | |
| MINIO_DEFAULT_BUCKETS: 'sample' | |
| x-ui: &ui_env | |
| REACT_APP_TERRAKUBE_API_URL: https://tk-api.${DOMAIN}/api/v1/ | |
| REACT_APP_CLIENT_ID: example-app | |
| REACT_APP_AUTHORITY: https://tk-dex.${DOMAIN}/dex | |
| REACT_APP_REDIRECT_URI: https://tk.${DOMAIN} | |
| REACT_APP_REGISTRY_URI: https://tk-registry.${DOMAIN} | |
| REACT_APP_SCOPE: email openid profile offline_access groups | |
| JAVA_TOOL_OPTIONS: -Dcom.sun.security.enableAIAcaIssuers=true | |
| x-traefik_env: &traefik_env | |
| TRAEFIK_API_DASHBOARD: false | |
| TRAEFIK_ENTRYPOINTS_WEB: true | |
| TRAEFIK_ENTRYPOINTS_WEB_ADDRESS: ":${TRAEFIK_HTTP_PORT}" | |
| TRAEFIK_ENTRYPOINTS_WEB_HTTP_ENCODEQUERYSEMICOLONS: true | |
| TRAEFIK_ENTRYPOINTS_WEBSECURE_HTTP_ENCODEQUERYSEMICOLONS: true | |
| TRAEFIK_ENTRYPOINTS_WEBSECURE: true | |
| TRAEFIK_ENTRYPOINTS_WEBSECURE_ADDRESS: ":${TRAEFIK_HTTPS_PORT}" | |
| TRAEFIK_PROVIDERS_DOCKER: true | |
| TRAEFIK_PROVIDERS_DOCKER_EXPOSEDBYDEFAULT: false | |
| TRAEFIK_PROVIDERS_FILE_FILENAME: /etc/traefik_dynamic.yml | |
| ## Terrakube API | |
| x-traefik_api_labels: &traefik_api_labels | |
| traefik.enable: true | |
| ## Terrakube API HTTPS | |
| traefik.http.routers.terrakube-api-https.rule: Host(`tk-api.${DOMAIN}`) | |
| traefik.http.routers.terrakube-api-https.entrypoints: websecure | |
| traefik.http.routers.terrakube-api-https.tls: true | |
| traefik.http.routers.terrakube-api-https.middlewares: terrakube-dex-allow-origin | |
| traefik.http.services.terrakube-api-https.loadbalancer.server.port: 8080 | |
| ## Redirect to HTTPS | |
| traefik.http.routers.terrakube-api-http.rule: Host(`tk-api.${DOMAIN}`) | |
| traefik.http.routers.terrakube-api-http.entrypoints: web | |
| traefik.http.routers.terrakube-api-http.middlewares: terrakube-api-redirect-https | |
| traefik.http.middlewares.terrakube-api-redirect-https.redirectscheme.scheme: https | |
| ## Traefik labels | |
| ## Terrakube Executor | |
| x-traefik_executor_labels: &traefik_executor_labels | |
| traefik.enable: true | |
| ## Terrakube executor HTTPS | |
| traefik.http.routers.terrakube-executor-https.rule: Host(`tk-executor.${DOMAIN}`) | |
| traefik.http.routers.terrakube-executor-https.entrypoints: websecure | |
| traefik.http.routers.terrakube-executor-https.tls: true | |
| traefik.http.routers.terrakube-executor-https.middlewares: terrakube-dex-allow-origin | |
| traefik.http.services.terrakube-executor-https.loadbalancer.server.port: 8090 | |
| ## Redirect to HTTPS | |
| traefik.http.routers.terrakube-executor-http.rule: Host(`tk-executor.${DOMAIN}`) | |
| traefik.http.routers.terrakube-executor-http.entrypoints: web | |
| traefik.http.routers.terrakube-executor-http.middlewares: terrakube-executor-redirect-https | |
| traefik.http.middlewares.terrakube-executor-redirect-https.redirectscheme.scheme: https | |
| ## Terrakube UI | |
| x-traefik_ui_labels: &traefik_ui_labels | |
| traefik.enable: true | |
| ## Terrakube UI HTTPS | |
| traefik.http.routers.terrakube-ui-https.rule: Host(`tk.${DOMAIN}`) | |
| traefik.http.routers.terrakube-ui-https.entrypoints: websecure | |
| traefik.http.routers.terrakube-ui-https.middlewares: terrakube-dex-allow-origin | |
| traefik.http.routers.terrakube-ui-https.tls: true | |
| traefik.http.services.terrakube-ui-https.loadbalancer.server.port: 8080 | |
| ## Redirect to HTTPS | |
| traefik.http.routers.terrakube-ui-http.rule: Host(`tk.${DOMAIN}`) | |
| traefik.http.routers.terrakube-ui-http.entrypoints: web | |
| traefik.http.routers.terrakube-ui-http.middlewares: terrakube-ui-redirect-https | |
| traefik.http.middlewares.terrakube-ui-redirect-https.redirectscheme.scheme: https | |
| x-traefik_dex_labels: &traefik_dex_labels | |
| traefik.enable: true | |
| ## Terrakube DEX HTTPS | |
| traefik.http.routers.terrakube-dex-https.rule: Host(`tk-dex.${DOMAIN}`) | |
| traefik.http.routers.terrakube-dex-https.entrypoints: websecure | |
| traefik.http.routers.terrakube-dex-https.middlewares: terrakube-dex-allow-origin | |
| traefik.http.routers.terrakube-dex-https.tls: true | |
| traefik.http.services.terrakube-dex-https.loadbalancer.server.port: 5556 | |
| ## Redirect to HTTPS | |
| traefik.http.routers.terrakube-dex-http.rule: Host(`tk-dex.${DOMAIN}`) | |
| traefik.http.routers.terrakube-dex-http.entrypoints: web | |
| traefik.http.routers.terrakube-dex-http.middlewares: terrakube-dex-redirect-https | |
| traefik.http.middlewares.terrakube-dex-redirect-https.redirectscheme.scheme: https | |
| ## Allow CORs from Terrakube UI | |
| traefik.http.middlewares.terrakube-dex-allow-origin.headers.accesscontrolallowmethods: GET, PATCH, PUT, POST, DELETE, HEAD, OPTIONS | |
| traefik.http.middlewares.terrakube-dex-allow-origin.headers.accesscontrolallowheaders: > | |
| Content-Type, Accept, Authorization, X-Requested-With, Origin, * | |
| traefik.http.middlewares.terrakube-dex-allow-origin.headers.accesscontrolalloworiginlist: https://tk.${DOMAIN} | |
| traefik.http.middlewares.terrakube-dex-allow-origin.headers.accesscontrolallowcredentials: true | |
| traefik.http.middlewares.terrakube-dex-allow-origin.headers.accessControlExposeHeaders: > | |
| Cache-Control, Content-Language, Content-Length, Content-Type, Expires, Last-Modified, | |
| Pragma, x-amz-server-side-encryption, x-amz-request-id, x-amz-id-2, ETag | |
| traefik.http.middlewares.terrakube-dex-allow-origin.headers.accesscontrolmaxage: 86400 | |
| traefik.http.middlewares.terrakube-dex-allow-origin.headers.addvaryheader: true | |
| ## Terrakube registry | |
| x-traefik_registry_labels: &traefik_registry_labels | |
| traefik.enable: true | |
| ## Terrakube registry HTTPS | |
| traefik.http.routers.terrakube-registry-https.rule: Host(`tk-registry.${DOMAIN}`) | |
| traefik.http.routers.terrakube-registry-https.entrypoints: websecure | |
| traefik.http.routers.terrakube-registry-https.tls: true | |
| traefik.http.routers.terrakube-registry-https.middlewares: terrakube-dex-allow-origin | |
| traefik.http.services.terrakube-registry-https.loadbalancer.server.port: 8075 | |
| ## Redirect to HTTPS | |
| traefik.http.routers.terrakube-registry-http.rule: Host(`tk-registry.${DOMAIN}`) | |
| traefik.http.routers.terrakube-registry-http.entrypoints: web | |
| traefik.http.routers.terrakube-registry-http.middlewares: terrakube-registry-redirect-https | |
| traefik.http.middlewares.terrakube-registry-redirect-https.redirectscheme.scheme: https | |
| ### Containers | |
| services: | |
| traefik: | |
| image: traefik:latest | |
| container_name: terrakube-traefik | |
| # Give Traefik a reserved IP address in your external network, pick something towards the end of the network to avoid conflicts | |
| networks: | |
| default: | |
| ipv4_address: $TRAEFIK_IPV4_ADDRESS | |
| environment: *traefik_env | |
| dns: *terrakube_dns | |
| ports: | |
| - $TRAEFIK_HTTP_PORT:80 | |
| - $TRAEFIK_HTTPS_PORT:443 | |
| volumes: | |
| - /var/run/docker.sock:/var/run/docker.sock:ro | |
| - ./traefik_dynamic.yml:/etc/traefik_dynamic.yml | |
| ## Uncomment for let's encrypt cert based off Domain | |
| # - /etc/letsencrypt/live/${DOMAIN}/privkey.pem:/letsencrypt/privkey.pem:ro | |
| # - /etc/letsencrypt/live/${DOMAIN}/fullchain.pem:/letsencrypt/fullchain.pem:ro | |
| restart: unless-stopped | |
| terrakube-api: | |
| image: azbuilder/api-server:${TK_VERSION} | |
| container_name: terrakube-api | |
| environment: | |
| <<: [*api_env] | |
| labels: *traefik_api_labels | |
| extra_hosts: *terrakube_hosts | |
| depends_on: | |
| - postgresql-service | |
| terrakube-ui: | |
| image: azbuilder/terrakube-ui:${TK_VERSION} | |
| container_name: terrakube-ui | |
| extra_hosts: *terrakube_hosts | |
| labels: *traefik_ui_labels | |
| volumes: | |
| - ./env-config.js:/app/env-config.js | |
| environment: | |
| <<: [*ui_env] | |
| terrakube-executor: | |
| dns: *terrakube_dns | |
| image: azbuilder/executor:${TK_VERSION} | |
| labels: *traefik_executor_labels | |
| extra_hosts: *terrakube_hosts | |
| container_name: terrakube-executor | |
| environment: | |
| <<: [*executor_env] | |
| terrakube-registry: | |
| dns: *terrakube_dns | |
| image: azbuilder/open-registry:${TK_VERSION} | |
| labels: *traefik_registry_labels | |
| extra_hosts: *terrakube_hosts | |
| container_name: terrakube-registry | |
| environment: | |
| <<: [*registry_env] | |
| terrakube-dex: | |
| image: ghcr.io/dexidp/dex:v2.37.0 | |
| extra_hosts: *terrakube_hosts | |
| labels: *traefik_dex_labels | |
| container_name: terrakube-dex | |
| volumes: | |
| - ./config-ldap.yaml:/etc/dex/config.docker.yaml | |
| ldap-service: | |
| image: bitnami/openldap:2.6.4-debian-11-r4 | |
| container_name: terrakube-ldap-service | |
| environment: *ldap_env | |
| volumes: | |
| - ./config-ldap.ldif:/ldifs/config-ldap.ldif | |
| minio: | |
| container_name: terrakube-minio | |
| image: docker.io/bitnami/minio:2022 | |
| environment: *minio_env | |
| volumes: | |
| - 'minio_data:/data' | |
| redis-service: | |
| image: bitnami/redis:7.0.10 | |
| container_name: terrakube-redis | |
| environment: | |
| - REDIS_REPLICATION_MODE=master | |
| - REDIS_PASSWORD=password123456 | |
| - REDIS_MASTER_PASSWORD=password123456 | |
| - REDIS_DISABLE_COMMANDS=FLUSHDB,FLUSHALL | |
| volumes: | |
| - 'redis_data:/bitnami/redis/data' | |
| postgresql-service: | |
| image: docker.io/bitnami/postgresql:15 | |
| container_name: postgresql-service | |
| environment: | |
| - POSTGRESQL_USERNAME=terrakube | |
| - POSTGRESQL_PASSWORD=terrakubepassword | |
| - POSTGRESQL_DATABASE=terrakubedb | |
| volumes: | |
| - postgresql_data:/bitnami/postgresql | |
| volumes: | |
| minio_data: | |
| redis_data: | |
| postgresql_data: | |
| driver: local | |
| # External network is required, import its name below | |
| networks: | |
| default: | |
| name: $EXTERNAL_NETWORK_NAME | |
| external: true |
| window._env_ = { | |
| REACT_APP_TERRAKUBE_API_URL: "https://tk-api.${DOMAIN}/api/v1/", | |
| REACT_APP_CLIENT_ID: "example-app", | |
| REACT_APP_AUTHORITY: "https://tk-dex.${DOMAIN}/dex", | |
| REACT_APP_REDIRECT_URI: "https://tk.${DOMAIN}", | |
| REACT_APP_REGISTRY_URI: "https://tk-registry.${DOMAIN}", | |
| REACT_APP_SCOPE: "email openid profile offline_access groups", | |
| } |
| # tls: | |
| # stores: | |
| # default: | |
| # defaultCertificate: | |
| # certFile: /letsencrypt/fullchain.pem | |
| # keyFile: /letsencrypt/privkey.pem |
You're hitting traefik but it's not serving the website, Looks like you're getting the default traefik cert?
Are you using a self signed cert or a real one? Start chrome with those flags in my earlier post if you're using a self signed. Those flags are for Ubuntu, not sure if they're right for MacOS if you're running that.
I was getting this result at first but I thought I fixed it, will test again fresh from scratch.
docker logs terrakube-traefik, will show any traefik errors.
docker inspect terrakube-ui, confirm the labels section has the host names and not the Domain placeholder
ETA: Your network doesn't have a gateway. Delete it and remake it with a smaller network
docker network create my-bridge-network -d bridge \
--subnet 10.0.0.0/24 --gateway 10.0.0.254
Then set Traefik to 10.0.0.150
I didn't add any kind of certificate I just run "docker-compose up -d" and replaced the "domain" in the docker-compose.yaml
I got the following:
user@pop-os:~/git/poc$ docker logs terrakube-traefik
time="2024-03-27T21:21:22Z" level=info msg="Configuration loaded from environment variables."
time="2024-03-27T21:21:22Z" level=error msg="middleware \"terrakube-dex-allow-origin@docker\" does not exist" entryPointName=websecure routerName=terrakube-executor-https@docker
time="2024-03-27T21:21:22Z" level=error msg="middleware \"terrakube-dex-allow-origin@docker\" does not exist" entryPointName=websecure routerName=terrakube-registry-https@docker
time="2024-03-27T21:21:22Z" level=error msg="middleware \"terrakube-dex-allow-origin@docker\" does not exist" entryPointName=websecure routerName=terrakube-ui-https@docker
time="2024-03-27T21:21:24Z" level=error msg="middleware \"terrakube-dex-allow-origin@docker\" does not exist" entryPointName=websecure routerName=terrakube-ui-https@docker
time="2024-03-27T21:21:24Z" level=error msg="middleware \"terrakube-dex-allow-origin@docker\" does not exist" entryPointName=websecure routerName=terrakube-api-https@docker
time="2024-03-27T21:21:24Z" level=error msg="middleware \"terrakube-dex-allow-origin@docker\" does not exist" entryPointName=websecure routerName=terrakube-executor-https@docker
time="2024-03-27T21:21:24Z" level=error msg="middleware \"terrakube-dex-allow-origin@docker\" does not exist" entryPointName=websecure routerName=terrakube-registry-https@docker
UI
user@pop-os:~/git/poc$ docker inspect terrakube-ui
[
{
"Id": "79980f25ffe97484a7e0ded35219d243dce273277a6b9d2d967d0ae703a7bdea",
"Created": "2024-03-27T21:21:20.613277947Z",
"Path": "/opt/bitnami/scripts/nginx/entrypoint.sh",
"Args": [
"/opt/bitnami/scripts/nginx/run.sh"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 63098,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-03-27T21:21:21.47056938Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:949614c9d34ff22b79c25b30d6ea3e21547eb5244efe6634975c61e43726db70",
"ResolvConfPath": "/var/lib/docker/containers/79980f25ffe97484a7e0ded35219d243dce273277a6b9d2d967d0ae703a7bdea/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/79980f25ffe97484a7e0ded35219d243dce273277a6b9d2d967d0ae703a7bdea/hostname",
"HostsPath": "/var/lib/docker/containers/79980f25ffe97484a7e0ded35219d243dce273277a6b9d2d967d0ae703a7bdea/hosts",
"LogPath": "/var/lib/docker/containers/79980f25ffe97484a7e0ded35219d243dce273277a6b9d2d967d0ae703a7bdea/79980f25ffe97484a7e0ded35219d243dce273277a6b9d2d967d0ae703a7bdea-json.log",
"Name": "/terrakube-ui",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "docker-default",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/home/user/git/poc/env-config.js:/app/env-config.js:rw"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "my-bridge-network",
"PortBindings": {},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": [],
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": [
"tk-api.terrakubepoc.com:host-gateway",
"tk-dex.terrakubepoc.com:host-gateway",
"tk-executor.terrakubepoc.com:host-gateway",
"tk.terrakubepoc.com:host-gateway",
"tk-registry.terrakubepoc.com:host-gateway"
],
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware",
"/sys/devices/virtual/powercap"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/32c7b13938194f3fafb3999e958ae36fb147109d1f89f2ec5d1fc32bc1713cf1-init/diff:/var/lib/docker/overlay2/18a407390b121268adf859fea935cf4465de438a7fac6c9928507395aeed8c70/diff:/var/lib/docker/overlay2/50001f30412dc480eb8b94df73f8674bd0183ae275ccbae95f07f613c7127c7e/diff:/var/lib/docker/overlay2/63f519678a267bd4afd312d05a25b8076487d8906dfa152fc70e5e8f791b8d5a/diff",
"MergedDir": "/var/lib/docker/overlay2/32c7b13938194f3fafb3999e958ae36fb147109d1f89f2ec5d1fc32bc1713cf1/merged",
"UpperDir": "/var/lib/docker/overlay2/32c7b13938194f3fafb3999e958ae36fb147109d1f89f2ec5d1fc32bc1713cf1/diff",
"WorkDir": "/var/lib/docker/overlay2/32c7b13938194f3fafb3999e958ae36fb147109d1f89f2ec5d1fc32bc1713cf1/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/home/user/git/poc/env-config.js",
"Destination": "/app/env-config.js",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "79980f25ffe9",
"Domainname": "",
"User": "1001",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"8080/tcp": {},
"8443/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"REACT_APP_TERRAKUBE_API_URL=https://tk-api.terrakubepoc.com/api/v1/",
"REACT_APP_CLIENT_ID=example-app",
"REACT_APP_AUTHORITY=https://tk-dex.terrakubepoc.com/dex",
"REACT_APP_REDIRECT_URI=https://tk.terrakubepoc.com",
"REACT_APP_REGISTRY_URI=https://tk-registry.terrakubepoc.com",
"REACT_APP_SCOPE=email openid profile offline_access groups",
"JAVA_TOOL_OPTIONS=-Dcom.sun.security.enableAIAcaIssuers=true",
"PATH=/opt/bitnami/common/bin:/opt/bitnami/nginx/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"HOME=/",
"OS_ARCH=amd64",
"OS_FLAVOUR=debian-11",
"OS_NAME=linux",
"APP_VERSION=1.25.4",
"BITNAMI_APP_NAME=nginx",
"NGINX_HTTPS_PORT_NUMBER=",
"NGINX_HTTP_PORT_NUMBER="
],
"Cmd": [
"/opt/bitnami/scripts/nginx/run.sh"
],
"Image": "azbuilder/terrakube-ui:2.19.2",
"Volumes": {
"/app/env-config.js": {}
},
"WorkingDir": "/app",
"Entrypoint": [
"/opt/bitnami/scripts/nginx/entrypoint.sh"
],
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "540e567d83858ec819d31b286eae3f7e2fa42f0a58792f61e9f129d0860512cd",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "poc",
"com.docker.compose.project.config_files": "docker-compose.yml",
"com.docker.compose.project.working_dir": "/home/user/git/poc",
"com.docker.compose.service": "terrakube-ui",
"com.docker.compose.version": "1.29.2",
"com.vmware.cp.artifact.flavor": "sha256:1e1b4657a77f0d47e9220f0c37b9bf7802581b93214fff7d1bd2364c8bf22e8e",
"org.opencontainers.image.base.name": "docker.io/bitnami/minideb:bullseye",
"org.opencontainers.image.created": "2024-02-16T05:14:04Z",
"org.opencontainers.image.description": "Application packaged by VMware, Inc",
"org.opencontainers.image.licenses": "Apache-2.0",
"org.opencontainers.image.ref.name": "1.25.4-debian-11-r4",
"org.opencontainers.image.title": "nginx",
"org.opencontainers.image.vendor": "VMware, Inc.",
"org.opencontainers.image.version": "1.25.4",
"traefik.enable": "true",
"traefik.http.middlewares.terrakube-ui-redirect-https.redirectscheme.scheme": "https",
"traefik.http.routers.terrakube-ui-http.entrypoints": "web",
"traefik.http.routers.terrakube-ui-http.middlewares": "terrakube-ui-redirect-https",
"traefik.http.routers.terrakube-ui-http.rule": "Host(`tk.terrakubepoc.com`)",
"traefik.http.routers.terrakube-ui-https.entrypoints": "websecure",
"traefik.http.routers.terrakube-ui-https.middlewares": "terrakube-dex-allow-origin",
"traefik.http.routers.terrakube-ui-https.rule": "Host(`tk.terrakubepoc.com`)",
"traefik.http.routers.terrakube-ui-https.tls": "true",
"traefik.http.services.terrakube-ui-https.loadbalancer.server.port": "8080"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "99d0751a65012ff956fc3749bd915c4132be298655bd2365f8972cff40c73c66",
"SandboxKey": "/var/run/docker/netns/99d0751a6501",
"Ports": {
"8080/tcp": null,
"8443/tcp": null
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"my-bridge-network": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"terrakube-ui",
"79980f25ffe9"
],
"MacAddress": "02:42:0a:00:00:04",
"NetworkID": "dc4ee06bf70ec0febb35d8ec2f84b7a8ee29b35e8037dfb9e934b87a201af042",
"EndpointID": "55c1c651e36a3a3c54192a78344864b5f718c2e58e0313bb169f74acc56a94c9",
"Gateway": "10.0.0.1",
"IPAddress": "10.0.0.4",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"terrakube-ui",
"79980f25ffe9"
]
}
}
}
}
]
The labels look good, I edited my previous post, it's the network gateway,
ETA: And please use docker compose up -d --force-recreate for more consistent results.
Hello @SolomonHD
I think I almost got this working, thank you for your suggestion.
The UI is working:
The registry is working:
The executor is working I guess because it is returning some spring response:
The api is working:
By the way your are missing the tk-api in the domain setup it should be:
### Domain Names
## https://tk.${DOMAIN} # Main website, terrakube-ui
## https://tk-api.${DOMAIN} # Terrakube API
## https://tk-registry.${DOMAIN} # Registry, terrakube-registry
## https://tk-executor.${DOMAIN} # Executor, terrakube-executor
## https://tk-dex.${DOMAIN} # Dex, terrakube-dex
Now I just need to fix the certificates and it should work.
Hello @SolomonHD
I did one small adjustment, for testing I generated a wildcard certificate using mkcert, once you have installed you can just run:
mkcert -install
mkcert -key-file key.pem -cert-file cert.pem terrakubepoc.com *.terrakubepoc.comThe above command will generate the certificate information like the following:
user@pop-os:~/certs/terrakubepoc.com$ pwd
/home/user/certs/terrakubepoc.com
user@pop-os:~/certs/terrakubepoc.com$ ls
cert.pem key.pemThe certificate will be valid for your browser and for the terraform CLI at least for testing
I remove the comments form this part:
traefik_dynamic.yml
tls:
stores:
default:
defaultCertificate:
certFile: /letsencrypt/fullchain.pem
keyFile: /letsencrypt/privkey.pemI also did a small adjustment here with my development certificates:
### Containers
services:
traefik:
image: traefik:latest
container_name: terrakube-traefik
# Give Traefik a reserved IP address in your external network, pick something towards the end of the network to avoid conflicts
networks:
default:
ipv4_address: $TRAEFIK_IPV4_ADDRESS
environment: *traefik_env
dns: *terrakube_dns
ports:
- $TRAEFIK_HTTP_PORT:80
- $TRAEFIK_HTTPS_PORT:443
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik_dynamic.yml:/etc/traefik_dynamic.yml
## Uncomment for let's encrypt cert based off Domain
- /home/user/certs/terrakubepoc.com/key.pem:/letsencrypt/privkey.pem:ro
- /home/user/certs/terrakubepoc.com/cert.pem:/letsencrypt/fullchain.pem:roI still need to do some small adjustment to load my development CA inside the containers but I think is a good starting point to check your issue.
user@pop-os:~/certs/terrakubepoc.com$ mkcert -CAROOT
/home/user/.local/share/mkcert
Example:
I finally got it working with my development CA certifciates.
It just need some small adjustment that I will explain here:
Step 1: Create a file called "type" in the docker compose folder with the following text "ca-certificates":
user@pop-os:~/git/poc$ cat type
ca-certificatesStep 2: Add the following environment variable to the api, registry and executor container:
SERVICE_BINDING_ROOT: /mnt/platform/bindingsStep 3: Mount the file called "type" and my development CA to the containers like the following:
terrakube-api:
image: azbuilder/api-server:${TK_VERSION}
container_name: terrakube-api
environment:
<<: [*api_env]
labels: *traefik_api_labels
extra_hosts: *terrakube_hosts
volumes:
## CUSTOM CA CERTS
- ./type:/mnt/platform/bindings/ca-certificates/type:ro
- /home/user/.local/share/mkcert/rootCA.pem:/mnt/platform/bindings/ca-certificates/rootCA.pem:ro
depends_on:
- postgresql-service
terrakube-ui:
image: azbuilder/terrakube-ui:${TK_VERSION}
container_name: terrakube-ui
extra_hosts: *terrakube_hosts
labels: *traefik_ui_labels
volumes:
- ./env-config.js:/app/env-config.js
environment:
<<: [*ui_env]
terrakube-executor:
dns: *terrakube_dns
image: azbuilder/executor:${TK_VERSION}
labels: *traefik_executor_labels
extra_hosts: *terrakube_hosts
volumes:
## CUSTOM CA CERTS
- ./type:/mnt/platform/bindings/ca-certificates/type:ro
- /home/user/.local/share/mkcert/rootCA.pem:/mnt/platform/bindings/ca-certificates/rootCA.pem:ro
container_name: terrakube-executor
environment:
<<: [*executor_env]
terrakube-registry:
dns: *terrakube_dns
image: azbuilder/open-registry:${TK_VERSION}
labels: *traefik_registry_labels
extra_hosts: *terrakube_hosts
volumes:
## CUSTOM CA CERTS
- ./type:/mnt/platform/bindings/ca-certificates/type:ro
- /home/user/.local/share/mkcert/rootCA.pem:/mnt/platform/bindings/ca-certificates/rootCA.pem:ro
container_name: terrakube-registry
environment:
<<: [*registry_env]The important part is this section:
volumes:
## CUSTOM CA CERTS
- ./type:/mnt/platform/bindings/ca-certificates/type:ro
- /home/user/.local/share/mkcert/rootCA.pem:/mnt/platform/bindings/ca-certificates/rootCA.pem:ro
Now I am able to see your issue:
feign.FeignException$BadRequest: [400 ] during [GET] to [https://tk-api.terrakubepoc.com/api/v1/organization/3a130a1c-d96f-4f99-83b8-58d472567e3a/module?filter%5Bmodule%5D=name==iam;provider==aws] [TerrakubeClient#getModuleByNameAndProvider(String,String,String)]: [{"errors":[{"detail":"Found undefined keys in request: provider"}]}]
Hello @SolomonHD I found the following:
The request that is failing is this one:
https://tk-api.terrakubepoc.com/api/v1/organization/3a130a1c-d96f-4f99-83b8-58d472567e3a/module?filter[module]=name==iam;provider==aws
{
"errors": [
{
"detail": "Found undefined keys in request: provider"
}
]
}
The funny part is that if I exposed the API port and I do a direct request I don't see the error, I think it could be related to some traefik configuration that maybe change the request when going through "https://tk-api.terrakubepoc.com"
http://localhost:8080/api/v1/organization/3a130a1c-d96f-4f99-83b8-58d472567e3a/module?filter[module]=name==iam;provider==aws
{
"data": [
{
"type": "module",
"id": "74196ed8-63ff-4521-805d-621251bf1f84",
"attributes": {
"createdBy": null,
"createdDate": null,
"description": null,
"downloadQuantity": 30,
"name": "iam",
"provider": "aws",
"registryPath": "aws/iam/aws",
"source": "https://github.com/terraform-aws-modules/terraform-aws-iam.git",
"updatedBy": null,
"updatedDate": null,
"versions": [
"v4.16.0",
"v2.24.0",
"v5.1.0",
"v5.23.1",
"v5.27.0"
]
},
"relationships": {
"organization": {
"data": {
"type": "organization",
"id": "3a130a1c-d96f-4f99-83b8-58d472567e3a"
}
},
"ssh": {
"data": null
},
"vcs": {
"data": null
}
}
}
]
}
So I fixed this just by changing the registry environment variables like the following "AzBuilderApiUrl: http://terrakube-api:8080" instead of using "AzBuilderApiUrl: https://tk-api.terrakubepoc.com", in this way the request will go directly to the container and not passing through traefik gateway
x-registry: ®istry_env
AzBuilderRegistry: https://tk-registry.terrakubepoc.com
AzBuilderApiUrl: http://terrakube-api:8080
AuthenticationValidationTypeRegistry: DEX
TerrakubeEnableSecurity: "true"
DexIssuerUri: https://tk-dex.terrakubepoc.com/dex
TerrakubeUiURL: https://tk.terrakubepoc.com
PatSecret: ejZRSFgheUBOZXAyUURUITUzdmdINDNeUGpSWHlDM1g=
InternalSecret: S2JeOGNNZXJQTlpWNmhTITkha2NEKkt1VVBVQmFeQjM=
RegistryStorageType: AwsStorageImpl
AwsStorageAccessKey: $TK_OUTPUT_ACCESS_KEY
AwsStorageSecretKey: $TK_OUTPUT_SECRET_KEY
AwsStorageBucketName: $TK_OUTPUT_BUCKET_NAME
AwsStorageRegion: us-east-1
AwsEndpoint: http://terrakube-minio:9000
AppClientId: example-app
AppIssuerUri: https://tk-dex.terrakubepoc.com/dex
JAVA_TOOL_OPTIONS: -Dcom.sun.security.enableAIAcaIssuers=true
SERVICE_BINDING_ROOT: /mnt/platform/bindings
By the way you did an amazing job doing the Traefik setup and I think this setup could help people that would like to use Terrakube with docker-compose instead of using Kubernetes option.
I could create a page in our documentation something like "Docker Compose + Traefik" if you would like to share your setup information.
You could also send a pull request to the main repository and adding a new folder "docker-compose-traefik" with some small readme with the steps to run terrakube using traefik too.
Yeah, I'm happy to share my config. I'm glad you like my changes.
I'll recreate on my local and test out bypassing traefik via open port for the registry.
Hello @SolomonHD
I have created the page in our documentation, the link is the following:
https://docs.terrakube.io/getting-started/docker-compose-+-traefik
Feel free to fork our documentation repository and send the pull request with the information using this page when you have some free time.
https://github.com/AzBuilder/docs/blob/main/getting-started/docker-compose-%2B-traefik.md
Will do, I am also going to create a ticket in the Traefik repo, hopefully they'll have some insight on this situation.
ETA: Provider is a special term in Traefik, for example we're using the Docker provider in this compose file. Perhaps there is some sort of conflict?
Will do, I am also going to create a ticket in the Traefik repo, hopefully they'll have some insight on this situation.
ETA: Provider is a special term in Traefik, for example we're using the Docker provider in this compose file. Perhaps there is some sort of conflict?
I don't think so because I tried using just the one parameter like the following and It works.
-
Using filter[module]=name==iam
-
filter[module]=provider==aws
But if I switch the order I got an error in the name parameter with "filter[module]=provider==aws;name==iam"
Okay...so is your instinct is that it's a Traefik problem or a Terrakube problem or some combination of both?
Also how should I phase the problem to Traefik? Problems with multiple parameters in the query string?
Maybe the issue is with the semicolon?
traefik/traefik#9164
ETA: They have a setting for it:
https://doc.traefik.io/traefik/routing/entrypoints/#encodequerysemicolons
Okay...so is your instinct is that it's a Traefik problem or a Terrakube problem or some combination of both?
Also how should I phase the problem to Traefik? Problems with multiple parameters in the query string?
Maybe the issue is with the semicolon? traefik/traefik#9164
ETA: They have a setting for it: https://doc.traefik.io/traefik/routing/entrypoints/#encodequerysemicolons
I think it could be some configuration option in traefik to allow doing request like the above, maybe you could try the suggested option and check if that works. I will try to see if I can get the request that is reaching the API after passing through traefik.
Progress! 🚀Adding
TRAEFIK_ENTRYPOINTS_WEB_HTTP_ENCODEQUERYSEMICOLONS: true
TRAEFIK_ENTRYPOINTS_WEBSECURE_HTTP_ENCODEQUERYSEMICOLONS: true
To the Traefik ENV Vars seems to have done something, I can now load the Registry's Readme page, but the Input, Output and Resources Tabs are still stuck in loading.
Oddly I don't see any more errors in the Registry or any other container after the change. The web console of my browser has an error:
Uncaught (in promise) TypeError: t is undefined
I've updated this Gist with the new config, pull it and recreate, see if the same happens for you
ETA: Tested by adding the lambda module on github.com and it loads the Readme; will not load other tabs.
Any time I refresh it takes me back to the Readme tab, which does load.
On the Readme tab switching versions does bring up the Readme again after a few seconds.
logs:
Copy Input-stream to file
2024-03-29 15:32:57.428 INFO 1 --- [nio-8075-exec-5] o.t.r.service.module.ModuleServiceImpl : Registry Path: https://tk-registry.domain.com/terraform/modules/v1/download/aws/lambda/aws/v7.2.5/module.zip
2024-03-29 15:32:57.453 INFO 1 --- [nio-8075-exec-6] o.t.registry.service.ReadMeServiceImpl : Checking README.md
2024-03-29 15:32:57.460 INFO 1 --- [nio-8075-exec-6] o.t.registry.service.ReadMeServiceImpl : Temp folder deleted...
2024-03-29 15:33:02.679 INFO 1 --- [nio-8075-exec-7] o.t.r.p.s.aws.AwsStorageServiceImpl : Checking Aws S3 Object exist registry/aws/lambda/aws/v6.4.0/module.zip
2024-03-29 15:33:02.707 INFO 1 --- [nio-8075-exec-8] o.t.r.p.s.aws.AwsStorageServiceImpl : Checking Aws S3 Object exist registry/aws/lambda/aws/v6.4.0/module.zip
2024-03-29 15:33:03.577 INFO 1 --- [nio-8075-exec-7] o.t.r.p.s.aws.AwsStorageServiceImpl : Upload Aws S3 Object completed
2024-03-29 15:33:03.584 INFO 1 --- [nio-8075-exec-7] o.t.r.p.s.aws.AwsStorageServiceImpl : Successfully delete folder
2024-03-29 15:33:03.584 INFO 1 --- [nio-8075-exec-7] o.t.r.service.module.ModuleServiceImpl : Registry Path: https://tk-registry.domain.com/terraform/modules/v1/download/aws/lambda/aws/v6.4.0/module.zip
2024-03-29 15:33:03.584 INFO 1 --- [nio-8075-exec-7] o.t.r.p.s.aws.AwsStorageServiceImpl : Searching: /registry/aws/lambda/aws/v6.4.0/module.zip
2024-03-29 15:33:03.590 INFO 1 --- [nio-8075-exec-7] o.t.registry.service.ReadMeServiceImpl : Get content README.md
2024-03-29 15:33:03.590 INFO 1 --- [nio-8075-exec-7] o.t.registry.service.ReadMeServiceImpl : Creating temp folder
2024-03-29 15:33:03.591 INFO 1 --- [nio-8075-exec-7] o.t.registry.service.ReadMeServiceImpl : Copy Input-stream to file
2024-03-29 15:33:03.605 INFO 1 --- [nio-8075-exec-7] o.t.registry.service.ReadMeServiceImpl : Checking README.md
2024-03-29 15:33:03.611 INFO 1 --- [nio-8075-exec-7] o.t.registry.service.ReadMeServiceImpl : Temp folder deleted...
2024-03-29 15:33:03.703 INFO 1 --- [nio-8075-exec-8] o.t.r.p.s.aws.AwsStorageServiceImpl : Upload Aws S3 Object completed
2024-03-29 15:33:03.715 INFO 1 --- [nio-8075-exec-8] o.t.r.p.s.aws.AwsStorageServiceImpl : Successfully delete folder
2024-03-29 15:33:03.715 INFO 1 --- [nio-8075-exec-8] o.t.r.service.module.ModuleServiceImpl : Update module download count
2024-03-29 15:33:03.736 INFO 1 --- [nio-8075-exec-8] o.t.r.service.module.ModuleServiceImpl : Registry Path: https://tk-registry.domain.com/terraform/modules/v1/download/aws/lambda/aws/v6.4.0/module.zip
ETA2: Tried again on 2.20 release, slightly different error message:
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'buffer')
It is failing when is trying to download the module zip file to parse the inputs, outputs, etc but I have no idea why because the URL is the correct.
The UI is doing a request in this part of the code:
The response headers are the correct because they include the "x-terraform-get" with the URL.
And it is failing when trying to download that URL here
But the URL is correct.
I can do a curl directly to download the module and it works with curl.
user@pop-os:~/git/poc$ curl https://tk-registry.terrakubepoc.com/terraform/modules/v1/download/aws/iam/aws/v5.37.1/module.zip --output module.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 885k 100 885k 0 0 10.2M 0 --:--:-- --:--:-- --:--:-- 10.2M
Maybe you could build a custom UI image and add some console.log(url) just to check the value is the correct.
docker build -t terrakube-ui:latest .
I have no idea why the download method in this line is failing, adding some log information can help debugging.
So, the API is failing to download a file from the Registry, or is it the UI?
I am running this test on some really weak hardware that had difficulties downloading in other situations.
I'm going to take what we have and attempt to deploy it on my institution's hardware, see if we get the same errors.
So, the API is failing to download a file from the Registry, or is it the UI?
I am running this test on some really weak hardware that had difficulties downloading in other situations.
I'm going to take what we have and attempt to deploy it on my institution's hardware, see if we get the same errors.
The UI is doing a request to the registry, to this particular URL https://tk-registry.terrakubepoc.com/terraform/modules/v1/aws/iam/aws/v5.37.2/download.
That endpoint should include one custom response header called "x-terraform-get" that include the URL to download the module that is require for the UI.
I added some custom logs to the UI using version 2.19.2 but the responder header does not include the header "x-terraform-get", that is the reason for your issue.
Hello @SolomonHD
I got it working.
I fixed just adding "x-terraform-get"
traefik.http.middlewares.terrakube-dex-allow-origin.headers.accessControlExposeHeaders: >
Cache-Control, Content-Language, Content-Length, Content-Type, Expires, Last-Modified,
Pragma, x-amz-server-side-encryption, x-terraform-get, x-amz-request-id, x-amz-id-2, ETag
YES! 🥳🥳 I have it working too. Great effort on this, so happy we got this working 😂😂
ETA: What's annoying is I tried * for headers in the beginning, but that's not allowed when using the Bearer token, you have to specify them,
I'm going clean up the config, remove anything unnecessary. Have a good weekend!
Hey @alfespa17, I re-deployed terrakube inside my company's network and I am still seeing a few issues.
- I can't pull modules from the registry. The
source = "tk.example.com/module/name/provider"is not working.
To confirm, I do not need API keys when running from within TK, correct? Only running Terraform CLI with TK as backend needs API keys.
I checked Minio and the module.zip uploads correctly. A 500 error occurs in the terminal and this error message in the registry container:
2024-04-16 02:16:04.891 INFO 1 --- [nio-8075-exec-9] o.t.r.p.s.aws.AwsStorageServiceImpl : Checking Aws S3 Object exist registry/aws/api-gate/aws/3.0.0/module.zip
2024-04-16 02:16:04.996 INFO 1 --- [nio-8075-exec-9] o.t.registry.service.git.GitServiceImpl : Cloning https://github.service.example.com/LITS/terraform-aws-api-gateway-proxy-cognito.git using 3.0.0
2024-04-16 02:16:05.115 ERROR 1 --- [nio-8075-exec-9] o.t.registry.service.git.GitServiceImpl : Remote branch 'refs/tags/3.0.0' not found in upstream origin
2024-04-16 02:16:05.118 ERROR 1 --- [nio-8075-exec-9] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.zeroturnaround.zip.ZipException: Given directory '/home/cnb/.terraform-spring-boot/git/2d87dbc5-07f6-41a0-8929-c3477f8b0f52' doesn't contain any files!] with root cause
- After creating a workspace and applying a Terraform template, TK creates the resources and uploads the state file to Minio correctly, but the UI displays nothing in the Resources/Outputs tabs in the Overview. Also, the State tab is empty; the run is marked with a red failed box even when it completes successfully.
It appears to not be able to fully parse Minio. Can you try running a few templates in your setup to see if you can replicate? Do you think I'm missing additional headers?
Hello @SolomonHD to download modules from the registry you have two options:
-
Run
terraform login "TERRAKUBE-REGISTRY-HOSTNAME"andterraform login "TERRAKUBE-API-HOSTNAME"that will generate the credentials.tfrc.json file like the following:
-
The second option is to generate a Personal Access Token and you configure credentials in .terraformrc or terraform.rc
credentials "tk-registry.terrakubepoc.com" {
# valid user API token:
token = "xxxxxx.yyyyyy.zzzzzzzzzzzzz"
}
credentials "tk-api.terrakubepoc.com" {
# valid user API token:
token = "xxxxxx.yyyyyy.zzzzzzzzzzzzz"
}
By the way you will need to one credential for the registry and one for the api
For your second question I run a simple example and I can see the state and resources.
Regarding the registry error:
2024-04-16 02:16:04.996 INFO 1 --- [nio-8075-exec-9] o.t.registry.service.git.GitServiceImpl : Cloning https://github.service.example.com/LITS/terraform-aws-api-gateway-proxy-cognito.git using 3.0.0
2024-04-16 02:16:05.115 ERROR 1 --- [nio-8075-exec-9] o.t.registry.service.git.GitServiceImpl : Remote branch 'refs/tags/3.0.0' not found in upstream origin
Make sure that inside your git repository there is a tag or release with name "3.0.0"
https://github.service.example.com/LITS/terraform-aws-api-gateway-proxy-cognito.git
I hope this can help you
Hey @SolomonHD curious question were you able to fix the issue?
I've been busy on other projects last couple of weeks, getting back on this now. IIRC last I tried to replicate the issue on my local and I believe it worked normally like your example. I will test it again on my university infrastructure.
On a side note: Is it possible to get rid of env-config.js? The UI environment variables have the same values. It would clean up the configuration considerably.
I've been busy on other projects last couple of weeks, getting back on this now. IIRC last I tried to replicate the issue on my local and I believe it worked normally like your example. I will test it again on my university infrastructure.
On a side note: Is it possible to get rid of
env-config.js? The UI environment variables have the same values. It would clean up the configuration considerably.
Hello @SolomonHD you can't remove the env-config.js because it is used as an static file for the react UI, I think you could remove the environment variables for the UI those are no longer need it.
Hello @SolomonHD
I created the network and I am using for the domain "terrakubepoc.com" locally editing file /etc/hosts
This is the network information
I used this values in my hosts file:
My env file:
I got the following:
Do you have any idea? Did I miss something?