Skip to content

Instantly share code, notes, and snippets.

@jftuga

jftuga/a-description.md

Last active August 6, 2024 13:49
Show Gist options
  • Save jftuga/5e472a3e06b5f5ec62937ed68e5ae80d to your computer and use it in GitHub Desktop.
Save jftuga/5e472a3e06b5f5ec62937ed68e5ae80d to your computer and use it in GitHub Desktop.
Download ZIP
Python insert_lines function
Raw
a-description.md

File Insertion Utility

This Python utility provides two functions for inserting lines into configuration files:

  • insert_lines
  • regex_insert_lines

These functions allow for precise and flexible modification of text files, particularly useful for updating configuration files in networking and system administration tasks.

  • insert_lines: Inserts specified lines into a file before or after a line that starts with a given string.
  • regex_insert_lines: Inserts specified lines into a file before or after a line that matches a given regular expression pattern.

Both functions incorporate safety measures such as creating backups before modification and handling various potential errors, ensuring the integrity of your configuration files during updates.

Config File Examples

These configuration files are designed to work with the examples in the main function (see insert_lines.py below):

In config.txt:

  • The line ip service config matches the search_string "ip service" used in the insert_lines function calls.
  • This will allow the function to insert lines both before and after this line.

In config2.txt:

  • The lines /ip service set www-ssl certificate=dummy-cert.pem_0 and /ip service set api-ssl certificate=dummy-cert.pem_0 both match the regex pattern /ip service set (.*?)-ssl certificate= used in the regex_insert_lines function calls.
  • This will allow the function to insert lines both before the first match and after the last match.

These example configuration files mimic real-world router configurations (a Cisco-style config for config.txt and a MikroTik-style config for config2.txt) while ensuring that the search string and regex pattern used in the main function will find matches.

When you run the main function:

  • It will insert lines before and after ip service config in config.txt.
  • It will insert lines before the first /ip service set ... -ssl certificate= line and after the last such line in config2.txt.

Note: These files muse reside in the same directory as the insert_lines.py script.

Raw
config.txt
# Network Configuration File
hostname Router1
enable secret 5 $1$mERr$hx5rVt7rPNoS4wqbXKX7m0
interface GigabitEthernet0/0
ip address 192.168.1.1 255.255.255.0
no shutdown
interface GigabitEthernet0/1
ip address 10.0.0.1 255.255.255.0
no shutdown
ip route 0.0.0.0 0.0.0.0 203.0.113.1
ip service config
http-server enable
https-server enable
line vty 0 4
login local
transport input ssh
end
Raw
config2.txt
# MikroTik Router Configuration
/interface bridge
add name=bridge1
/interface wireless
set [ find default-name=wlan1 ] ssid=MyWiFi
/ip address
add address=192.168.88.1/24 interface=ether1 network=192.168.88.0
/ip dhcp-server
add address-pool=dhcp_pool1 disabled=no interface=bridge1 name=dhcp1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=no
set ssh disabled=no
set api disabled=yes
set winbox disabled=no
set api-ssl disabled=no
/ip service set www-ssl certificate=dummy-cert.pem_0
/ip service set api-ssl certificate=dummy-cert.pem_0
/system clock
set time-zone-name=America/New_York
/system ntp client
set enabled=yes primary-ntp=0.pool.ntp.org secondary-ntp=1.pool.ntp.org
Raw
insert_lines.py
import os
import re
from datetime import datetime
from typing import List, Literal
def insert_lines(before_or_after: Literal['before', 'after'], file_path: str, search_string: str, lines_to_insert: List[str]) -> None:
r"""
Inserts specified lines into a file either before or after a line that starts with a given string.
This function creates a backup of the original file, then reads the entire file,
finds the first line that starts with the search string, inserts the new lines
either before or after that line based on the 'before_or_after' parameter,
and then writes the modified content to a new file with the original name.
Args:
before_or_after (Literal['before', 'after']): Specifies whether to insert lines
before or after the line starting with the search string.
file_path (str): The path to the file to be modified.
search_string (str): The string to search for at the start of a line.
lines_to_insert (List[str]): The lines to insert.
Raises:
ValueError: If before_or_after is not 'before' or 'after'.
FileNotFoundError: If the specified file does not exist.
StopIteration: If no line starting with the search string is found.
OSError: If there's an issue with file operations (reading, writing, renaming).
Exception: For any other unexpected errors.
Returns:
None
Example:
>>> insert_lines('before', 'config.txt', 'ip service', ['new line 1', 'new line 2'])
>>> insert_lines('after', 'config.txt', 'ip service', ['new line 1', 'new line 2'])
"""
if before_or_after not in ['before', 'after']:
raise ValueError("before_or_after must be either 'before' or 'after'")
backup_path = file_path + '.bak'
try:
# Rename the original file to create a backup
os.rename(file_path, backup_path)
except OSError as e:
raise OSError(f"Failed to create backup file. Error: {e}") from e
try:
# Read the entire file
with open(backup_path, 'r') as file:
content = file.readlines()
# Find the index of the line starting with the search string
insert_index = next(i for i, line in enumerate(content) if line.strip().startswith(search_string))
# Adjust insert_index if inserting after
if before_or_after == 'after':
insert_index += 1
# Insert the new lines at the found index
for i, line in enumerate(lines_to_insert):
content.insert(insert_index + i, line + '\n')
# Write the modified content to the original file path
with open(file_path, 'w') as file:
file.writelines(content)
except Exception as e:
# If any error occurs, attempt to restore the backup
try:
os.rename(backup_path, file_path)
except OSError:
pass # If restoring fails, we can't do much more
raise Exception(f"An error occurred: {e}") from e
else:
# If everything was successful, remove the backup file
try:
os.remove(backup_path)
except OSError:
pass # If we can't remove the backup, it's not critical
def regex_insert_lines(before_or_after: Literal['before', 'after'], file_path: str, search_regex: str, lines_to_insert: List[str]) -> None:
r"""
Inserts specified lines into a file either before or after a line that matches a given regex pattern.
This function creates a backup of the original file, then reads the entire file,
finds the first line that matches the search regex, inserts the new lines
either before or after that line based on the 'before_or_after' parameter,
and then writes the modified content to a new file with the original name.
Args:
before_or_after (Literal['before', 'after']): Specifies whether to insert lines
before or after the line matching the search regex.
file_path (str): The path to the file to be modified.
search_regex (str): The regular expression pattern to search for in each line.
lines_to_insert (List[str]): The lines to insert.
Raises:
ValueError: If before_or_after is not 'before' or 'after'.
FileNotFoundError: If the specified file does not exist.
re.error: If the provided regex pattern is invalid.
RuntimeError: If no line matching the search regex is found.
OSError: If there's an issue with file operations (reading, writing, renaming).
Exception: For any other unexpected errors.
Returns:
None
Example:
>>> regex_insert_lines('before', 'config.txt', r'^ip service', ['new line 1', 'new line 2'])
>>> regex_insert_lines('after', 'config.txt', r'^\s*port\s+\d+', ['new line 1', 'new line 2'])
"""
if before_or_after not in ['before', 'after']:
raise ValueError("before_or_after must be either 'before' or 'after'")
backup_path = file_path + '.bak'
try:
# Compile the regex pattern
pattern = re.compile(search_regex)
except re.error as e:
raise re.error(f"Invalid regex pattern: {e}") from e
try:
# Rename the original file to create a backup
os.rename(file_path, backup_path)
except OSError as e:
raise OSError(f"Failed to create backup file. Error: {e}") from e
try:
# Read the entire file
with open(backup_path, 'r') as file:
content = file.readlines()
# Find the index of the line matching the search regex
insert_index = next((i for i, line in enumerate(content) if pattern.search(line)), None)
if insert_index is None:
raise RuntimeError(f"No line matching the regex '{search_regex}' was found in the file")
# Adjust insert_index if inserting after
if before_or_after == 'after':
insert_index += 1
# Insert the new lines at the found index
for i, line in enumerate(lines_to_insert):
content.insert(insert_index + i, line + '\n')
# Write the modified content to the original file path
with open(file_path, 'w') as file:
file.writelines(content)
except Exception as e:
# If any error occurs, attempt to restore the backup
try:
os.rename(backup_path, file_path)
except OSError:
pass # If restoring fails, we can't do much more
raise Exception(f"An error occurred: {e}") from e
else:
# If everything was successful, remove the backup file
try:
os.remove(backup_path)
except OSError:
pass # If we can't remove the backup, it's not critical
def main():
use_insert_lines = True
use_regex_insert_lines = True
if use_insert_lines:
file_path = 'config.rsc'
search_string = 'ip service'
lines_to_insert = ['new line 1', 'new line 2', 'new line 3']
try:
# Attempt to insert lines before the search string
insert_lines('before', file_path, search_string, lines_to_insert)
print(f"Successfully inserted lines before '{search_string}' in {file_path}")
# Attempt to insert lines after the search string
insert_lines('after', file_path, search_string, lines_to_insert)
print(f"Successfully inserted lines after '{search_string}' in {file_path}")
except ValueError as e:
print(f"Invalid input: {e}")
except FileNotFoundError as e:
print(f"File not found: {e}")
except StopIteration:
print(f"Search string '{search_string}' not found in the file")
except OSError as e:
print(f"File operation error: {e}")
except Exception as e:
print(f"An unexpected error occurred: {e}")
else:
print("All operations completed successfully")
finally:
print("File modification attempt completed")
# Example using regex_insert_lines
if use_regex_insert_lines:
file_path2 = 'config2.txt'
search_regex = r'/ip service set (.*?)-ssl certificate='
lines_to_insert2 = ['# New SSL configuration', '# Added on ' + datetime.now().strftime("%Y-%m-%d")]
try:
# Attempt to insert lines before the regex match
regex_insert_lines('before', file_path2, search_regex, lines_to_insert2)
print(f"Successfully inserted lines before regex match in {file_path2}")
# Attempt to insert lines after the regex match
regex_insert_lines('after', file_path2, search_regex, lines_to_insert2)
print(f"Successfully inserted lines after regex match in {file_path2}")
except ValueError as e:
print(f"Invalid input: {e}")
except FileNotFoundError as e:
print(f"File not found: {e}")
except re.error as e:
print(f"Invalid regex pattern: {e}")
except RuntimeError as e:
print(f"Regex match not found: {e}")
except OSError as e:
print(f"File operation error: {e}")
except Exception as e:
print(f"An unexpected error occurred: {e}")
else:
print("All regex operations completed successfully")
finally:
print("Regex file modification attempt completed")
if __name__ == "__main__":
main()
Raw
notes.md

next()

The next() is a built-in function in Python.

It's quite useful and has some interesting properties.

Basic usage:

  • next() is typically used with iterators. It retrieves the next item from the iterator.
  • In our specific use case:
    • We're using next() with a generator expression:
insert_index = next(i for i, line in enumerate(content) if line.strip().startswith(search_string))

This is equivalent to finding the first occurrence of a line starting with search_string and getting its index.

How it works in this context:

  • The generator expression:
(i for i, line in enumerate(content) if line.strip().startswith(search_string))
  • creates an iterator that yields indices where the condition is true.
  • next() then returns the first value from this iterator.

Error handling

If no line starts with the search string, next() will raise a StopIteration exception. In a more robust version of the script, we might want to handle this case:

try:
    insert_index = next(i for i, line in enumerate(content) if line.strip().startswith(search_string))
except StopIteration:
    raise ValueError(f"No line starting with '{search_string}' found in the file")

Alternatively, you could use the two-argument form of next(), which allows you to specify a default value to return if the iterator is exhausted:

insert_index = next((i for i, line in enumerate(content) if line.strip().startswith(search_string)), None)
if insert_index is None:
    raise ValueError(f"No line starting with '{search_string}' found in the file")

This approach:

  • Uses next() with a default value of None.
  • If no matching line is found, insert_index will be None.
  • We then check if insert_index is None and raise a ValueError if it is.

Geneator Expression Alternative

  • We could have used a for loop to find the index, like this:
for i, line in enumerate(content):
    if line.strip().startswith(search_string):
        insert_index = i
        break

The next() function with a generator expression is more concise but might be slightly less readable for some.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment