Skip to content

Instantly share code, notes, and snippets.

@santiaago
Last active January 2, 2021 07:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save santiaago/97f21023674c7c9af452 to your computer and use it in GitHub Desktop.
Save santiaago/97f21023674c7c9af452 to your computer and use it in GitHub Desktop.
Python script to insert lines in multiple files following a pattern
a > b
a = foo(c)
b = 2
b = foo(b)
foo
bar
foobar
b = foo(v)
a > b
a1 = foo(c)
b = 2
a3 = foo(b)
foo
bar
foobar
var1 = foo(v)

In the past days I had to build a script to insert some lines of code after a specific statement.

For example, I have the following file:

a > b
a = foo(c)
b = 2
b = foo(b)
foo
bar
foobar
b = foo(v)

And I need to add a line variable = bar(42) after every line of variable = foo(x)

After that treatment my file would be like this:

a > b
a = foo(c)
a = bar(42)
b = 2
b = foo(b)
b = bar(42)
foo
bar
foobar
c = foo(v)
c = bar(42)

Also, I have to perform this task on multiple files at the same time, the files are listed in a visual studio search result that will look like this:

Find all "foo(", Subfolders, Find Results 1, "c:\", "*.txt"
  C:\file1.txt(2):a = foo(c)
  C:\file1.txt(4):b = foo(b)
  C:\file1.txt(8):c = foo(v)
  C:\file2.txt(2):a1 = foo(c)
  C:\file2.txt(4):a3 = foo(b)
  C:\file2.txt(8):variable1 = foo(v)
  Matching lines: 6    Matching files: 2    Total files searched: 214
  Find was stopped in progress.

So we need a way to get the set of files from the visual studio search result and then run the task on each of them.

Here is the main algorithm where:

  • LINE_TO_INSERT is the line to insert, in this case = bar(42)\n
  • PATTERN_OF_LINE in the pattern after which we insert the line In this case '\w+' + '\\ \\=\\ foo\\(' + '\w+' which matches with a = foo(b) for example
  • PATTERN_OF_FILE is the pattern of the files we are looking for in the visual studio search result

The main function is insert_in_files().

def insert_in_files():
	'''Insert LINE_TO_INSERT after all lines with pattern PATTERN_OF_LINE in all files with PATTERN_OF_FILE'''
	files = get_files('vs_search_output.txt')
	for f in files:
		insert_lines(f)
	return None

Build a set of files from a visual studio search result:

Here is the code that will get the set of files in the search result. Where PATTERN_OF_FILE is equal to 'C\\:\\\\\w+.txt'. Which means any files with extension .txt whose name is any word character.

def get_files(path):
	'''Return a set of files from a visual studio search output following a regexp pattern
	
	Format of visual studio search output is the following:
	Find all "pattern to search", Subfolders, Find Results 1, Entire Solution, ""
	  path_to_file.txt(1):		line with text
	  path_to_another_file.txt(22):		line1 some text
	  path_to_another_file.txt(29):		line2 some text
	Matching lines: 2    Matching files: 2    Total files searched: 205
	
	Args:
		path: A path to a file with the search result
		
	Returns:
		A set of all files in search result that match PATTERN_OF_FILE regular expression.
	'''
	from re import search
	from re import escape

	f = open(path, 'r')
	# paths is a set structure with all the different files to look for.
	paths = set()
	count = 0
	for line in f:
		m = search(PATTERN_OF_FILE, line)
		if m:
			paths.add(m.group(0))
	f.close()
	return paths

If you run this function you will get a set of files.

  • insert_in_files is the python file where the get_files is located.
  • vs_search_output.txt is the visual studio search result listed above.

In a python shell:

>>> import insert_in_files
>>> insert_in_files.get_files("vs_search_output.txt")
{'C:\\file1.txt', 'C:\\file2.txt'}
>>>

Search for pattern in file and insert a line:

As a remainder here are the constants we use:

PATTERN_OF_FILE = 'C\\:\\\\\w+.txt'
PATTERN_OF_LINE = '\w+' + '\\ \\=\\ foo\\(' + '\w+' 	# looking for pattern a = foo(b)
FIRST_VAR = '^\w+'
LINE_TO_INSERT = ' = bar(42)\n'

And here is the code to search in a file and insert a line:

def insert_lines(path):
	'''Insert a line with 'FIRST_VAR + LINE_TO_INSERT' after each PATTERN_OF_LINE
	
	Go through a file looking for PATTERN_OF_LINE 
	if found, insert a line just after with 'FIRST_VAR + LINE_TO_INSERT'
	
	Args:
		path: a path to a file to search and insert lines if a match is found
	Returns:
		None
	'''
	from re import escape
	from re import search
	fin = open(path, 'r')
	open('output', 'w').close() 	# clear file before processing
	fout = open('output', 'w')
	
	for line in fin:
		fout.write(line)
		match = search(PATTERN_OF_LINE, line)
		if match:
			res = search(FIRST_VAR, line)
			if res:
				fout.write(res.group(0) + LINE_TO_INSERT)
	fin.close()
	fout.close()
	# save file
	ftemp = open('output', 'r')
	try:
		open(path, 'w').close() 	# clear file before processing
	except:
		print('cannot write in file:', path)
		ftemp.close()
		return None
	f = open(path, 'w')
	for line in ftemp:
		f.write(line)
	f.close()
	ftemp.close()
	return None
PATTERN_OF_FILE = 'C\\:\\\\\w+.txt'
PATTERN_OF_LINE = '\w+' + '\\ \\=\\ foo\\(' + '\w+' # looking for pattern a = foo(b)
FIRST_VAR = '^\w+'
LINE_TO_INSERT = ' = bar(42)\n'
def insert_in_files():
'''Insert LINE_TO_INSERT after all lines with pattern PATTERN_OF_LINE in all files with PATTERN_OF_FILE'''
files = get_files('vs_search_output.txt')
for f in files:
insert_lines(f)
return None
def get_files(path):
'''Return a set of files from a visual studio search output following a regexp pattern
Format of visual studio search output is the following:
Find all "pattern to search", Subfolders, Find Results 1, Entire Solution, ""
path_to_file.txt(1): line with text
path_to_another_file.txt(22): line1 some text
path_to_another_file.txt(29): line2 some text
Matching lines: 2 Matching files: 2 Total files searched: 205
Args:
path: A path to a file with the search result
Returns:
A set of all files in search result that match PATTERN_OF_FILE regular expression.
'''
from re import search
from re import escape
f = open(path, 'r')
# paths is a set structure with all the different files to look for.
paths = set()
count = 0
for line in f:
m = search(PATTERN_OF_FILE, line)
if m:
paths.add(m.group(0))
f.close()
return paths
def insert_lines(path):
'''Insert a line with 'FIRST_VAR + LINE_TO_INSERT' after each PATTERN_OF_LINE
Go through a file looking for PATTERN_OF_LINE
if found, insert a line just after with 'FIRST_VAR + LINE_TO_INSERT'
Args:
path: a path to a file to search and insert lines if a match is found
Returns:
None
'''
from re import escape
from re import search
fin = open(path, 'r')
open('output', 'w').close() # clear file before processing
fout = open('output', 'w')
for line in fin:
fout.write(line)
match = search(PATTERN_OF_LINE, line)
if match:
res = search(FIRST_VAR, line)
if res:
fout.write(res.group(0) + LINE_TO_INSERT)
fin.close()
fout.close()
# save file
ftemp = open('output', 'r')
try:
open(path, 'w').close() # clear file before processing
except:
print('cannot write in file:', path)
ftemp.close()
return None
f = open(path, 'w')
for line in ftemp:
f.write(line)
f.close()
ftemp.close()
return None
Find all "foo(", Subfolders, Find Results 1, "c:\", "*.txt"
C:\file1.txt(2):a = foo(c)
C:\file1.txt(4):b = foo(b)
C:\file1.txt(8):b = foo(v)
C:\file2.txt(2):a1 = foo(c)
C:\file2.txt(4):a3 = foo(b)
C:\file2.txt(8):variable1 = foo(v)
Matching lines: 6 Matching files: 2 Total files searched: 214
Find was stopped in progress.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment