According to documentation in https://keen.io/docs/streams/amazon-s3/, we were using the default format. For some applications (AWS Athena mostly), we began using the alternate format, and we needed to move the files in the old places to the new folder structure.
This is a very simple script provided as is. Comments are welcome :)