Skip to content

Instantly share code, notes, and snippets.

@tlancon
Created June 5, 2020 18:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tlancon/10995e186801c66b3c7912aeb90bb949 to your computer and use it in GitHub Desktop.
Save tlancon/10995e186801c66b3c7912aeb90bb949 to your computer and use it in GitHub Desktop.
Reject outliers from a 1D signal based on a sliding window and a deviation threshold.
def reject_outliers(arr, window_size, threshold):
"""
Given a 1D signal, replace outliers with the average of the surrounding points.
Does the following for every point:
1. Computes the median of the sliding window
2. Computes the percentage difference between the current point and the
sliding window median
3. If that deviation exceeds the given threshold, replaces that point with
the average of the surrounding two points.
4. Otherwise, returns the same point.
Parameters
----------
arr : 1D array
Signal with outliers
window_size : int
Size of sliding window
threshold : float
Minimum acceptable percentage deviation (as decimal)
Returns
-------
result : 1D array
Original signal with outliers removed.
"""
if len(arr.shape) > 1:
print('Only 1D arrays supported.')
return
result = np.empty_like(arr)
for i in range(0, arr.shape[0]):
min_i = int(max(0, np.floor(i - window_size / 2)))
max_i = int(min(arr.shape[0], np.floor(i + window_size / 2 + 1)))
med = np.median(arr[min_i:max_i])
dev = np.abs(arr[i] - med) / med
if dev > threshold:
result[i] = np.average([arr[i-1], arr[i+1]])
else:
result[i] = arr[i]
return result
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment