Like [docker exec][1] or [docker compose exec][0]!
You want to think of it a little differently than the docker version though. With docker, you can exec into any container managed by your own docker engine. With aws, you can't. You have to start an exec capable container whenever you need to debug a docker image.