Refactoring from camelCase to snake_case (functions, variables & files)

refactor.md

For a project that sadly will not be made public (See PR) I had to rename some components. This is of course a very specific solution targeted only to our project but maybe it can help/give a starting point for a similar problem. Here is the documentation from the pull request:

Pull request

Trying to rename components to use the snake case naming. (See PEP 8)

I am not happy with the result that the full fledged IDE pycharm brings. (I also just dislike those super heavy programs) Since some of the variable names are used in strings to access fields in the jsons I cannot simply rename every occurrence. Pycharm gives the option to exclude matches in strings etc, but I still would have to do it for every component.

I also wanted to see how far I come with lightweight tools that everybody can have access to.

To document my steps until now:

Renaming from camelCase to snake_case

Rename all files

$ for file in wrapper/*[A-Z]*.py; do
	git mv "$file" "$(sed -E 's/([A-Z])/_\L\1/g' <<< "$file")"
	file_name="${${file#wrapper/}%.py}"
	grep -lr "$file_name" . \
		--exclude-dir=.git --exclude-dir=.idea --exclude-dir=__pycache__ \
	| xargs sed -i "s/$file_name/$(sed -E 's/([A-Z])/_\L\1/g' <<< "$file_name")/g"
done

Rename in- and output formats

$ grep -lEr "(in|out)putFormat" . \
		--exclude-dir=.git --exclude-dir=.idea --exclude-dir=__pycache__ \
		--exclude=test_results.json --exclude=SLR.postman_collection.json \
	| xargs sed -i -E 's/(out|in)putFormat/\1put_format/g'

Rename all functions

Create list of all functions

$ grep -Eroh "def ([a-z]+[A-Z][a-z]+)+" wrapper/ --exclude-dir=__pycache__ \
	| cut -c 5- \
	| sort -u \
	> funcs.txt

results in funcs.txt:

allowedDisplays
allowedResultFormats
allowedSearchFields
...
startAt
translateGetQuery
translateQuery

Rename all previously generated function names

$ for func in `cat funcs.txt`; do
	grep -lr "$func" . \
			--exclude-dir=.git --exclude-dir=.idea --exclude-dir=__pycache__ \
			--exclude=test_results.json --exclude=SLR.postman_collection.json \
			--exclude=funcs.txt \
		| xargs sed -i -E "s/$func/$(sed -E 's/([A-Z])/_\L\1/g' <<< "$func")/g"
done

Rename all variables

Create list of all variables that have to be renamed

Making sure that the fields used for accessing the output_format and the returned dict from the DBs are ignored by ignoring names that are only used after ",',: or @.

$ grep -Eroh "[^\"':@a-zA-Z]_*([a-z]+[A-Z][a-z]+)+" wrapper/ \
		--exclude-dir=__pycache__ \
	| cut -c 2- \
	| sort -u \
	> vars.txt

results in vars.txt:

apiKey
dbQuery
formatDict
...
searchTerms
startRecord
__startRecord

Rename all the previously generated variable names

$ for var in `cat vars.txt`; do
	grep -lr "$var" . \
			--exclude-dir=.git --exclude-dir=.idea --exclude-dir=__pycache__ \
			--exclude=test_results.json --exclude=SLR.postman_collection.json \
			--exclude=vars.txt \
		| xargs sed -i -E "s/([^\"':@a-zA-Z])$var([^a-zA-Z]|$)/\1$(sed -E 's/([A-Z])/_\L\1/g' <<<"$var")\2/g"
done

Here again: Do not change names after ",',: or @

Also: $var has to be the whole word. (By excluding a-zA-Z before and after)