Skip to content

Instantly share code, notes, and snippets.

@pankkor
Last active July 4, 2024 13:57
Show Gist options
  • Save pankkor/4602959fab83755a41d7332987271a2e to your computer and use it in GitHub Desktop.
Save pankkor/4602959fab83755a41d7332987271a2e to your computer and use it in GitHub Desktop.
macOS input monitoring and data collection

macOS input monitoring aka spyware.

Inspect UI elements under the mouse.

Examples

UIElementInspector sample from 2010

https://developer.apple.com/library/archive/samplecode/UIElementInspector/Introduction/Intro.html Provides information of AXUIElements under the cursor via Accessibility API introduced in Mac OS X version 10.2.

Acessibility Inspector

Example of such an app could be Apple's Accessibility Inspector. (Xcode > Open Developer Tool > Accessibility Inspector) https://developer.apple.com/documentation/accessibility/accessibility-inspector

Uses Apple's private framework AccessibilityAuditDeviceManager internally. AccessibilityAuditDeviceManager can display Objective-C class under cursor. It inspects AXClassName attribute, which is only available for applications with com.apple.private.accessibility.inspection entitlement.

But most of the element under cursos attributes could still be obtained without this entitlement

Available API's

Use Quartz Event Services to tap into system events (from CoreGraphics.framework)

https://developer.apple.com/documentation/coregraphics/quartz_event_services Event taps make it possible to monitor and filter input events from several points within the system, prior to their delivery to a foreground application similar to Win32 SetWinEventHook()

Allows to synchroniously intercept, filter and substitute system wide events.

CGEventTapCreate

https://developer.apple.com/documentation/coregraphics/1454426-cgeventtapcreate Create an even tap (aka event handler). You have around 10ms to hande the event. After that event will receive event types -1 or -2 and won't be called any more.

kCGEventTapDisabledByTimeout = 0xFFFFFFFE,  // -2
kCGEventTapDisabledByUserInput = 0xFFFFFFFF // -1

Example of swapping 'a' and 'z' keystrokes

Inspect elements

  • AX* ApplicationServices.framework, and it's subframework HIServices.framework. ApplicationServices.framework has CoreGraphics.framework as a dependency.

AXUIElementCreateSystemWide

Returns an accessibility object that provides access to system attributes. This is useful for things like finding the focused accessibility object regardless of which application is currently active.

AXUIElement https://developer.apple.com/documentation/applicationservices/axuielement_h

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment