Skip to content

Instantly share code, notes, and snippets.

@ShikiSuen
Last active March 24, 2024 07:29
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ShikiSuen/73b7a55526c9fadd2da2a16d94ec5b49 to your computer and use it in GitHub Desktop.
Save ShikiSuen/73b7a55526c9fadd2da2a16d94ec5b49 to your computer and use it in GitHub Desktop.
Let's talk about what InputMethodKits needs to improve.

Related sample project: https://github.com/vChewing/vChewing-macOS/tree/3.4.9

It seems that individual bug reports doesn't work at all. Besides, the entire InputMethodKit needs a renovation.

This thread will be sent to Apple by certain special approaches after gathering enough usable information.

Let's talk about what InputMethodKits needs to improve. Here's my conclusion. If Apple think there's already an API, then it might be either mulfunctioning or not exposed to Swift.

  1. An official Swift-friendly wrapper with neither "!" nor "?" in the parameters of all provided APIs.

  2. Provide effective official exposed Swift access to:

    1. { get set } WindowLevel of the IMKCandidates, making sure that it won't get covered by NSMenu and Spotlight by default.
    2. { get set } Default candidate fonts as NSFont
    3. { get set } Highlight background color of the IMKCandidate panel
    4. { get set } Window background color tint of the IMKCandidate panel, allowing developers to enable or disable aero-glass transparency.
    5. { get set } Candidate keys definable in both ways: charCodes and keyCodes.
    6. { get set } Get the currently-highlighted TOTAL INDEX of the chosen candidate in the current candidate window.
    7. { get } Add IMKTextInput.isWritingContextVertical.
    8. { get } Add currentMarkedRangeOrigin (NSPoint), currentMarkedRangeTopLeft (NSPoint), currentMarkedRangeTopRight (NSPoint), currentCursorOrigin (NSPoint), currentCursorTopPoint (NSPoint).
  3. IMKInputController.Candidates() can send (Candidate, Reading, Annotation) triplet to the IMKCandidates. Also, IMKInputController.candidateSelected() and IMKInputController.candidateSelectionChanged() retrieves both Candidate and its total index number in the candidate window.

  4. Add Home / End key support in the IMKCandidate window.

  5. Provide a clear and unified Swift API for sending key NSEvents to IMKCandidates, making sure that these sent events are handled there. At least, - (BOOL)handleKeyboardEvent:(NSEvent *)event API_AVAILABLE(macosx(10.14)); needs to be officially exposed to developers.

  6. Add official NSEvent support of detecting whether Shift key has been pressed as a single function key. This is for helping IME developers to provide their own in-method input mode switch. This feature is crucial for users who already got accustomed to how Windows input methods behave. Reasons:

    1. Ergonomics: "Ctrl" / "CapsLock" / "Fn" key are less friendly than "Shift" key to one's pinkle finger.
    2. macOS built-in CapsLock IME toggle can have 1-second latency on certain user's computers with totally unknown reasons.
    3. Most Chinese IME users are from Windows, inheriting their preferences which are uncompromisable at all, period.
    4. IME-Developer-implemented Shift-key alphanumerical mode toggle can provide a fast way of inputting ASCII alphanumerals without moving the pinkle finger away from the Shift key.
  7. Provide a USABLE way of showing a description text at the bottom of the IMKCandidate window. This feature can be used for telling users what this candidate window is prepared for.

  8. Official TouchBar IMKCandidates API for 3rd-party input method developers.

  9. IMKCandidate window instances need to be separated for each InputMethodController. Reason: The IMKController session of an app sometimes deactivates after the one of the another client app gets activated, leading to the mess of handling input states. Imagine that you switched to another app and immediately want to call the symbol menu (shown through IMKCandidates), but the symbol menu window gets disappeared due to deactivateServer() triggered by another client app.

  10. Please state clearly in the documentation of certain APIs (like (de)activateServer() and setValue()) if they are expected to be mission-critical (e.g. expected to finish their processings as fast as possible). Otherwise, this can cause responsiveness issues of an input method if it is using Swift and is not using Grand Central Dispatch.

  11. An official workable solution (with precise sample project) demonstrating how to implement the IME settings pane of the System Preferences / Settings window:

image

  1. The documentation of InputMethodKit needs update regarding how to make sure annotations are shown in IMKCandidates. A Swift-only example is warmly welcomed.

  2. Different IMKInputController session instances might interfere each other: When activating a new instance of such in Safari new tab, the previous instance may deactivate later than the current one. However, this deactivation process may close the IMKCandidate window called by the current instance, confusing the user who are using the current input session because he / she can't tell which input state it currently is: It's actually a candidate state, but the candidate window is closed by the previous deactivated input session.

  3. The menu item "Edit Text Substitutions…" in the input source menu has been mistranslated as "Edit User Dictionary" which is extremely misleading to 3rd-party input method users.

  4. macOS 14 Sonoma needs to provide an official instructive documentation regarding how the icons of 3rd-party input methods should be designed to make sure they are stylisticly unified with system built-in input methods.

  5. The documentation of InputMethodKit needs to be updated to ask developers to change their InputMethodConnectionName to $(PRODUCT_BUNDLE_IDENTIFIER)_Connection in order not to let their input methods occasionally greyed out in the input source menu.

According to WWDC 2006 presentation "Input Method Kit Overview" held by Lee Collins, the InputMethodConnectionName in the info.plist of an input method should "Do not include spaces in the name or periods." However, in recent macOS, it can be occasionally observed that the right answer is $(PRODUCT_BUNDLE_IDENTIFIER)_Connection for sandboxed 3rd-party input methods. Even if you have set your InputMethodConnectionName to something else, in rare cases (mostly after your laptop gets woke up in macOS 13.4 Ventura) the system might dismiss your specified connection name and use $(PRODUCT_BUNDLE_IDENTIFIER)_Connection to make a connection instead, hence refusal by (possibly) Sandbox. At least, in such occasion, you can only see the affected input methods gray out in the input source menu before your mac reboots.

Therefore, it is necessary to update the following page:

https://developer.apple.com/documentation/appkit/nstextinputcontext/1529156-keyboardinputsources

... to add the recommended value for InputMethodConnectionName as $(PRODUCT_BUNDLE_IDENTIFIER)_Connection instead of the one suggested in WWDC2006.

Additional notes: Since macOS 10.14 till now, in order to use IMKCandidates, these following APIs have to be exposed through bridging-header:

@interface IMKCandidates(PROJECT_TARGET_NAME) {}

- (unsigned long long)windowLevel API_AVAILABLE(macosx(10.14));
- (void)setWindowLevel:(unsigned long long)level API_AVAILABLE(macosx(10.14));
- (BOOL)handleKeyboardEvent:(NSEvent *)event API_AVAILABLE(macosx(10.14));
- (void)setFontSize:(double)fontSize API_AVAILABLE(macosx(10.14));

@end

Finally, please allow input methods sellable on Mac App Store. A macOS input method app can be installed in the ~/Library/Input Methods/ and is okay to be Sandboxed (e.g. vChewing since v2.3.0). Therefore, there should be no privacy concerns anymore.

@ShikiSuen
Copy link
Author

ShikiSuen commented Nov 20, 2023

  • FB13259791 // Need instructions on how to customize the IME icon used in the input source indicator.

  • FB13392636 // Please allow Sandboxed input methods to be enabled when SecureEventInput is enabled (except those which have security-concerned entitlements).

  • FB13392574 // Need an official Swift-friendly wrapper with neither "!" nor "?" in the parameters of all provided APIs.

  • FB13392573 // InputMethodKit needs some new Swift APIs exposed to the 3rd-party developers.

  • FB13392580 // Extend what IMKInputController can send / receive to IMKCandidates.

  • FB13392583 // Add Home / End key support in the IMKCandidate window.

  • FB13392588 // Provide a clear and unified Swift API for sending key NSEvents to IMKCandidates.

  • FB13392591 // Add official NSEvent support of detecting whether Shift key has been pressed as a single function key.

  • FB13392595 // Provide a usable way of showing a description text at the bottom of the IMKCandidate window.

  • FB13392597 // Provide official TouchBar IMKCandidates API for 3rd-party input method developers.

  • FB13392598 // IMKCandidate window instances need to be separated for each InputMethodController.

  • FB13392599 // Please state clearly in the documentation of certain APIs (like (de)activateServer() and setValue()) if they are expected to be mission-critical (e.g. expected to finish their processings as fast as possible).

  • FB13392602 // InputMethodKit needs an official workable solution (with precise sample project) demonstrating how to implement the IME settings pane of the System Preferences / Settings window.

  • FB13392604 // The documentation of InputMethodKit needs update regarding how to make sure annotations are shown in IMKCandidates.

  • FB13392607 // Please stop letting IMKInputController sessions interfere each other.

  • FB13392610 // The documentation of InputMethodKit needs to be updated to ask developers to change their InputMethodConnectionName to $(PRODUCT_BUNDLE_IDENTIFIER)_Connection in order not to let their input methods occasionally greyed out in the input source menu.

  • FB13392624 // Please clearly document what a Sandboxed input method shall do in the entitlements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment