Skip to content

Instantly share code, notes, and snippets.

@manuelbl
Created August 3, 2019 09:12
Embed
What would you like to do?
ESP32 as Bluetooth Keyboard

ESP32 as Bluetooth Keyboard

With its built-in Bluetooth capabilities, the ESP32 can act as a Bluetooth keyboard. The below code is a minimal example of how to achieve it. It will generate the key strokes for a message whenever a button attached to the ESP32 is pressed.

For the example setup, a momentary button should be connected to pin 2 and to ground. Pin 2 will be configured as an input with pull-up.

In order to receive the message, add the ESP32 as a Bluetooth keyboard of your computer or mobile phone:

  1. Go to your computers/phones settings
  2. Ensure Bluetooth is turned on
  3. Scan for Bluetooth devices
  4. Connect to the device called "ESP32 Keyboard"
  5. Open an empty document in a text editor
  6. Press the button attached to the ESP32

The code has been written for the Arduino framework. I recommend using PlatformIO for development as it is far superior to the Arduino IDE while still taking full advantage of the Arduino ecosystem (libraries, support etc.)

/*
* Sample program for ESP32 acting as a Bluetooth keyboard
*
* Copyright (c) 2019 Manuel Bl
*
* Licensed under MIT License
* https://opensource.org/licenses/MIT
*/
//
// This program lets an ESP32 act as a keyboard connected via Bluetooth.
// When a button attached to the ESP32 is pressed, it will generate the key strokes for a message.
//
// For the setup, a momentary button should be connected to pin 2 and to ground.
// Pin 2 will be configured as an input with pull-up.
//
// In order to receive the message, add the ESP32 as a Bluetooth keyboard of your computer
// or mobile phone:
//
// 1. Go to your computers/phones settings
// 2. Ensure Bluetooth is turned on
// 3. Scan for Bluetooth devices
// 4. Connect to the device called "ESP32 Keyboard"
// 5. Open an empty document in a text editor
// 6. Press the button attached to the ESP32
#define US_KEYBOARD 1
#include <Arduino.h>
#include "BLEDevice.h"
#include "BLEHIDDevice.h"
#include "HIDTypes.h"
#include "HIDKeyboardTypes.h"
// Change the below values if desired
#define BUTTON_PIN 2
#define MESSAGE "Hello from ESP32\n"
#define DEVICE_NAME "ESP32 Keyboard"
// Forward declarations
void bluetoothTask(void*);
void typeText(const char* text);
bool isBleConnected = false;
void setup() {
Serial.begin(115200);
// configure pin for button
pinMode(BUTTON_PIN, INPUT_PULLUP);
// start Bluetooth task
xTaskCreate(bluetoothTask, "bluetooth", 20000, NULL, 5, NULL);
}
void loop() {
if (isBleConnected && digitalRead(BUTTON_PIN) == LOW) {
// button has been pressed: type message
Serial.println(MESSAGE);
typeText(MESSAGE);
}
delay(100);
}
// Message (report) sent when a key is pressed or released
struct InputReport {
uint8_t modifiers; // bitmask: CTRL = 1, SHIFT = 2, ALT = 4
uint8_t reserved; // must be 0
uint8_t pressedKeys[6]; // up to six concurrenlty pressed keys
};
// Message (report) received when an LED's state changed
struct OutputReport {
uint8_t leds; // bitmask: num lock = 1, caps lock = 2, scroll lock = 4, compose = 8, kana = 16
};
// The report map describes the HID device (a keyboard in this case) and
// the messages (reports in HID terms) sent and received.
static const uint8_t REPORT_MAP[] = {
USAGE_PAGE(1), 0x01, // Generic Desktop Controls
USAGE(1), 0x06, // Keyboard
COLLECTION(1), 0x01, // Application
REPORT_ID(1), 0x01, // Report ID (1)
USAGE_PAGE(1), 0x07, // Keyboard/Keypad
USAGE_MINIMUM(1), 0xE0, // Keyboard Left Control
USAGE_MAXIMUM(1), 0xE7, // Keyboard Right Control
LOGICAL_MINIMUM(1), 0x00, // Each bit is either 0 or 1
LOGICAL_MAXIMUM(1), 0x01,
REPORT_COUNT(1), 0x08, // 8 bits for the modifier keys
REPORT_SIZE(1), 0x01,
HIDINPUT(1), 0x02, // Data, Var, Abs
REPORT_COUNT(1), 0x01, // 1 byte (unused)
REPORT_SIZE(1), 0x08,
HIDINPUT(1), 0x01, // Const, Array, Abs
REPORT_COUNT(1), 0x06, // 6 bytes (for up to 6 concurrently pressed keys)
REPORT_SIZE(1), 0x08,
LOGICAL_MINIMUM(1), 0x00,
LOGICAL_MAXIMUM(1), 0x65, // 101 keys
USAGE_MINIMUM(1), 0x00,
USAGE_MAXIMUM(1), 0x65,
HIDINPUT(1), 0x00, // Data, Array, Abs
REPORT_COUNT(1), 0x05, // 5 bits (Num lock, Caps lock, Scroll lock, Compose, Kana)
REPORT_SIZE(1), 0x01,
USAGE_PAGE(1), 0x08, // LEDs
USAGE_MINIMUM(1), 0x01, // Num Lock
USAGE_MAXIMUM(1), 0x05, // Kana
LOGICAL_MINIMUM(1), 0x00,
LOGICAL_MAXIMUM(1), 0x01,
HIDOUTPUT(1), 0x02, // Data, Var, Abs
REPORT_COUNT(1), 0x01, // 3 bits (Padding)
REPORT_SIZE(1), 0x03,
HIDOUTPUT(1), 0x01, // Const, Array, Abs
END_COLLECTION(0) // End application collection
};
BLEHIDDevice* hid;
BLECharacteristic* input;
BLECharacteristic* output;
const InputReport NO_KEY_PRESSED = { };
/*
* Callbacks related to BLE connection
*/
class BleKeyboardCallbacks : public BLEServerCallbacks {
void onConnect(BLEServer* server) {
isBleConnected = true;
// Allow notifications for characteristics
BLE2902* cccDesc = (BLE2902*)input->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));
cccDesc->setNotifications(true);
Serial.println("Client has connected");
}
void onDisconnect(BLEServer* server) {
isBleConnected = false;
// Disallow notifications for characteristics
BLE2902* cccDesc = (BLE2902*)input->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));
cccDesc->setNotifications(false);
Serial.println("Client has disconnected");
}
};
/*
* Called when the client (computer, smart phone) wants to turn on or off
* the LEDs in the keyboard.
*
* bit 0 - NUM LOCK
* bit 1 - CAPS LOCK
* bit 2 - SCROLL LOCK
*/
class OutputCallbacks : public BLECharacteristicCallbacks {
void onWrite(BLECharacteristic* characteristic) {
OutputReport* report = (OutputReport*) characteristic->getData();
Serial.print("LED state: ");
Serial.print((int) report->leds);
Serial.println();
}
};
void bluetoothTask(void*) {
// initialize the device
BLEDevice::init(DEVICE_NAME);
BLEServer* server = BLEDevice::createServer();
server->setCallbacks(new BleKeyboardCallbacks());
// create an HID device
hid = new BLEHIDDevice(server);
input = hid->inputReport(1); // report ID
output = hid->outputReport(1); // report ID
output->setCallbacks(new OutputCallbacks());
// set manufacturer name
hid->manufacturer()->setValue("Maker Community");
// set USB vendor and product ID
hid->pnp(0x02, 0xe502, 0xa111, 0x0210);
// information about HID device: device is not localized, device can be connected
hid->hidInfo(0x00, 0x02);
// Security: device requires bonding
BLESecurity* security = new BLESecurity();
security->setAuthenticationMode(ESP_LE_AUTH_BOND);
// set report map
hid->reportMap((uint8_t*)REPORT_MAP, sizeof(REPORT_MAP));
hid->startServices();
// set battery level to 100%
hid->setBatteryLevel(100);
// advertise the services
BLEAdvertising* advertising = server->getAdvertising();
advertising->setAppearance(HID_KEYBOARD);
advertising->addServiceUUID(hid->hidService()->getUUID());
advertising->addServiceUUID(hid->deviceInfo()->getUUID());
advertising->addServiceUUID(hid->batteryService()->getUUID());
advertising->start();
Serial.println("BLE ready");
delay(portMAX_DELAY);
};
void typeText(const char* text) {
int len = strlen(text);
for (int i = 0; i < len; i++) {
// translate character to key combination
uint8_t val = (uint8_t)text[i];
if (val > KEYMAP_SIZE)
continue; // character not available on keyboard - skip
KEYMAP map = keymap[val];
// create input report
InputReport report = {
.modifiers = map.modifier,
.reserved = 0,
.pressedKeys = {
map.usage,
0, 0, 0, 0, 0
}
};
// send the input report
input->setValue((uint8_t*)&report, sizeof(report));
input->notify();
delay(5);
// release all keys between two characters; otherwise two identical
// consecutive characters are treated as just one key press
input->setValue((uint8_t*)&NO_KEY_PRESSED, sizeof(NO_KEY_PRESSED));
input->notify();
delay(5);
}
}
@legitlee
Copy link

legitlee commented May 2, 2021 via email

@roblatour
Copy link

roblatour commented May 2, 2021

So, I went way down the rabbit hole on this one, but found the solution.

In my code, I had the following two statements:

#include "HIDKeyboardTypes.h"
#define US_KEYBOARD 1

in that order.

Because of that, the code was being driven as if I had a UK keyboard (where key val 34 is mapped to the @ key as opposed to the double quote).

I changed the order of the statements (as in the example) to:

#define US_KEYBOARD 1
#include "HIDKeyboardTypes.h"

and everything worked as expected.

Thanks for your help, the displays helped me get to this conclusion.

As a side note:
My setup is an ESP32 connected to a Windows computer via Bluetooth, with the ESP32 acting as a keyboard. More specifically I am using the ESP32 as a keyboard macro device. When I press a button connected to the ESP32 it sends a string of keys to the Windows computer via the Bluetooth connection.

As often I find myself typing (or copying and pasting) the MQTT command to subscribe to publications, it is that command that I wanted to automate in this way, and which I was having trouble sending.

It's all now working perfectly.

Again, with thanks,

@manuelbl
Copy link
Author

manuelbl commented May 2, 2021

@mihailnik It should be possible to connect a keyboard via Bluetooth to an ESP32. However, this code achieves the opposite: emulating a keyboard with an ESP32.

You might want to have a look at https://github.com/chegewara/esp32-hid-keyboard-client.

@LevitateGamer
Copy link

LevitateGamer commented Jun 22, 2021

@

@mihailnik It should be possible to connect a keyboard via Bluetooth to an ESP32. However, this code achieves the opposite: emulating a keyboard with an ESP32.

You might want to have a look at https://github.com/chegewara/esp32-hid-keyboard-client.

Can you help me with this I am using Arduino IDE and I am trying to make something similar
Code:

// ---------------------------------
// Key definitions
#define BUTTON_KEY1 KEY_F13
#define BUTTON_KEY2 KEY_F14
#define BUTTON_KEY3 KEY_F15
#define BUTTON_KEY4 KEY_F16
#define BUTTON_KEY5 KEY_F17
#define BUTTON_KEY6 KEY_F18
#define BUTTON_KEY7 KEY_F19

// Pin definitions
#define BUTTON_PIN1 12
#define BUTTON_PIN2 27
#define BUTTON_PIN3 26
#define BUTTON_PIN4 23
#define BUTTON_PIN5 32
#define BUTTON_PIN6 33
#define BUTTON_PIN7 25
// ---------------------------------

#include <BleKeyboard.h>

BleKeyboard bleKeyboard;

// Button helper class for handling press/release and debouncing
class button {
  public:
    const char key;
    const uint8_t pin;

    button(uint8_t k, uint8_t p) : key(k), pin(p) {}

    void press(boolean state) {
      if (state == pressed || (millis() - lastPressed  <= debounceTime)) {
        return; // Nothing to see here, folks
      }

      lastPressed = millis();

      state ? bleKeyboard.press(key) : bleKeyboard.release(key);
      pressed = state;
    }

    void update() {
      press(!digitalRead(pin));
    }

  private:
    const unsigned long debounceTime = 30;
    unsigned long lastPressed = 0;
    boolean pressed = 0;
} ;

// Button objects, organized in array
button buttons[] = {
  {BUTTON_KEY1, BUTTON_PIN1},
  {BUTTON_KEY2, BUTTON_PIN2},
  {BUTTON_KEY3, BUTTON_PIN3},
  {BUTTON_KEY4, BUTTON_PIN4},
  {BUTTON_KEY5, BUTTON_PIN5},
  {BUTTON_KEY6, BUTTON_PIN6},
  {BUTTON_KEY7, BUTTON_PIN7},
};

const uint8_t NumButtons = sizeof(buttons) / sizeof(button);
const uint8_t ledPin = 2;

void setup() {
  Serial.begin(115200);

  bleKeyboard.begin();
  // Safety check. Ground pin #1 (RX) to cancel bleKeyboard inputs.
  pinMode(1, INPUT_PULLUP);
  if (!digitalRead(1)) {
    failsafe();
  }

  // Set LEDs Off. Active low.
  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH);

  for (int i = 0; i < NumButtons; i++) {
    pinMode(buttons[i].pin, INPUT_PULLUP);
  }
}

void loop() {
  for (int i = 0; i < NumButtons; i++) {
    buttons[i].update();
  }
}

void failsafe() {
  for (;;) {} // Just going to hang out here for awhile :D
}

@manuelbl
Copy link
Author

You quote a comment about connecting a Bluetooth keyboard to an ESP32 and show code of an ESP32 doing the opposite (acting as a keyboard). Your code includes a header file called BleKeyboard.h that I don't know. You don't describe what you are trying to achieve and where you are possibly stuck. I'm afraid I can't help you.

@Zibri
Copy link

Zibri commented Aug 26, 2021

how about emulating a USB keyboard just by plugging the esp32 into a pc USB port? is that possible?

@manuelbl
Copy link
Author

Most ESP32s do not contain a USB peripheral. Instead, the development board contains a USB-to-UART converter. They are restricted to a serial connection and cannot emulate a keyboard. The only exception is the ESP32-S2. It contains a USB peripheral. It would be able to emulate a USB keyboard but it is not yet supported by the stable Arduino version. You might be able to use it with the ESP-IDF framework, see https://github.com/iot-components/examples_tinyusb_esp32sx/blob/master/device/hid_device_touchpad/main/touchpad_hid_device_main.c

@krisherliu
Copy link

My PC (Windows 10+CSR4.0 BT) can't find "ESP32 Keyboard" on the list after the scan. My android phone can find and connect to "ESP32 Keyboard".

I can connect ESP32 to my Windows PC with this code:

`
#include "BluetoothSerial.h"
BluetoothSerial SerialBT;
void setup()
{
SerialBT.begin("ESP32test");
delay(1000);
}

void loop()
{
String inputFromOtherSide;
if (SerialBT.available()) {
inputFromOtherSide = SerialBT.readString();
SerialBT.println("You had entered: ");
SerialBT.println(inputFromOtherSide);
}
}
`
Can you help me with this? Thank you.

@manuelbl
Copy link
Author

manuelbl commented Sep 6, 2021

Given the limited information, I can't really help you. Since I have published this code, a far more capable library has been published: ESP32-BLE-Keyboard. Why don't you give it a try?

@thiagosanches
Copy link

Hey @manuelbl , I'm trying to use your snippet but got the following error:

In file included from c:\users\thiago.desktop-tt0eta0\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-97-gc752ad5-5.2.0\xtensa-esp32-elf\include\c++\5.2.0\locale:41:0,
                 from c:\users\thiago.desktop-tt0eta0\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-97-gc752ad5-5.2.0\xtensa-esp32-elf\include\c++\5.2.0\iomanip:43,
                 from C:\Users\thiago.DESKTOP-TT0ETA0\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.5\libraries\BLE\src\BLEAddress.cpp:13:
c:\users\thiago.desktop-tt0eta0\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-97-gc752ad5-5.2.0\xtensa-esp32-elf\include\c++\5.2.0\bits\locale_facets_nonio.h:2008:35: fatal error: bits/messages_members.h: No such file or directory
compilation terminated

error: bits/messages_members.h: No such file or directory
compilation terminated

Am I missing something?

@thiagosanches
Copy link

Never mind, I was using the wrong board type, it should be ESP32 Dev Module.

@discapacidad5
Copy link

Hello, I am developing a device for people with disabilities. I need to use the HID braille protocol but I am not getting anything with HID for esp32, could you help me with a sample code of how I can implement the HID braille protocol?

here are references to braille HID

https://usb.org/sites/default/files/hutrr78_-_creation_of_a_braille_display_usage_page_0.pdf

https://github.com/nvaccess/nvda/blob/00cd67b737bc5b23a6f5e31cf28110b64ebf2fee/devDocs/hidBrailleTechnicalNotes.md

my repository is the following
https://github.com/brailletouch/Brailletouch

@manuelbl
Copy link
Author

manuelbl commented Dec 6, 2021

In order to make it work for a braille display, the InputReport, OutputReport and REPORT_MAP structures need to be considerably adapted. The linked PDF seems to contain the report map. So that's a good starting point for adapting the REPORT_MAP.

The report map describes the data structures of the input and output report. So you will need to understand the details of this complex HID construct in order to figure out how the InputReport and OutputReport look like.

I'm afraid it's not trivial and difficult without having access to working braille devices.

@discapacidad5
Copy link

I think that it is easier to understand and modify a ready-made structure than to do unfortunately there are example of a structure made of keyboards for gaming devices but there is no structure made from which one can guide to modify and adapt an HD braille device would be It is very helpful that someone will provide a h & d braile structure that serves as an example for one to adapt it to the particular project or adapt the project to that structure

I basically need a structure for a 40-cell braille display
With a braille keyboard which contains eight brsille keys a space bar day chif keys
Basically its keyboard structure is similar to that of Freedom Scientific Focus 40 braille displays with an additional navigation key up arrow down arrow left arrow right arrow and center button and on either side a tab key and on the other side tab chip these keys are for move within a document or within the menus from a cell phone in a simple way here I leave more or less a manual of how the screen would be arranged shine

A general HID tuning document with this data would be very helpful.

Introduction

.a line Braille brailletouch is open source in both hardware and software.

The Braille brailletouch line consists of:

40, touch cells 1, Braille cells A sensor button (cursor scroll button) above the Braille cell. 8-key Braille keyboard, Perkins style Two scroll buttons two rocker keys two selectors two Shift keys one multiple scroll keyboard right left up down select and from the sides tab and chif tab One navigation rocker One mode button at either end of the line, for quick navigation through files, lists and menus. USB connection to computer Compatibility with select phones and other mobile devices via screen readers Physical description

On the left side of the line there are, starting from the position closest to you, a power button and a micro USB port. standard. The USB port allows you to connect the line to a computer using the USB cable or the power adapter.

Braille cells are located on the front of the right side of the unit. There is a sensor above the Braille cell. At both ends of the Braille display surface there is a navigation rocker whose purpose is to facilitate navigation. Above each of the navigation rockers is a mode button to toggle between the different navigation modes.

the braille display has 40 touch cells that are located on the center front of the unit

On the left is the multi-scroll keyboard, it has Jockey-style diction and selection keys and stubborn left and right corresponding to tab and chif + tab

Above the touch cells is an eight-key Braille (Perkins-style) keyboard. These keys are used to type text and execute commands. On the front edge, located in the center and below the Braille cells, is the SPACEBAR. This is used in combination with the Braille keys to enter commands. The commands that are executed with the SPACEBAR are called Cor. For example, COR L or COR 1-2-3. on the side and side of the SPACEBAR are the left SHIFT key, right SHIFT key

The following controls are located on the front of the equipment, from left to right:

brailletouch 40: left selector, left scroll button, left rocker key, right rocker key, right scroll button and right selector.

The following sections describe the use of these controls when working with Braille Touch and NVDA. If you are working with the brailletouch line and another screen reviewer, both on a computer and with a mobile device, consult the documentation for that software, as the functions of these controls may be different from those of NVDA.

Navigation Rockers and Mode Buttons

The Navigation Rockers on the brailletouch allow you to quickly scroll through files, dialogs, lists and menus. In files, the wheels allow you to scroll through lines, sentences, paragraphs or scroll left and right. To switch between the four available navigation modes, press the Mode Button located above the Navigation Rocker. In the dialogs, you can scroll through the available controls. In menus, by menu items. It is also possible to completely deactivate the Navigation Rockers by pressing the Left or Right Mode Button and the SPACEBAR at the same time. To activate them, run the same command again.

Scroll buttons

The scroll buttons allow you to scroll a display to the left or right. Press the Left Scroll button, which displays a raised double left arrow, to scroll left; press the Right Scroll button, which displays a raised right double arrow, to scroll right.

Rocker keys

The rocker keys allow you to move to the next or previous line. Press the upper part of the rocker key to go to the previous line, and the lower part to go to the next line. In combination with the scroll buttons, the rocker keys allow you to move the cursor to the beginning or end of the current line. To move the cursor to the beginning of the line, press a scroll button plus the top of a rocker key; To move the cursor to the end of the line, press a scroll button plus the bottom of a rocker key;

Selectors

If used individually, the selectors (concave in shape) allow you to configure Auto Advance. Pressing both Selector buttons at the same time will turn auto advance on or off. Press the Left selector to decrease the speed of the automatic advance, and press the Right selector to increase it.

When pressed in combination with other keys, they perform various functions. For example, if you press a Selector in conjunction with the top or bottom of a rocker key, you will page up or down. You can also press a Selector together with the Scroll left or right button to go to the beginning or end of a file.

Multi-scroll keyboard

it behaves like a Jockey with the keys arrow up arrow down arrow right arrow left and in the center select, from the left side a key corresponding to tabulator and from the right side a key corresponding to Shift + tabulator

Shift buttons

The Shift buttons are used in combination with the SPACEBAR, Braille keys, and other controls on the line to execute commands.

the Sensor button

There is a sensor button above the braille cell. The sensor button allows you to move the cursor to the corresponding character currently displayed in the cell, or to open links on web pages and email messages. In Line mode, the sensor allows you to open menus or select a menu item.

If you hold down the LEFT or RIGHT SCROLL BUTTON and press the sensor, you will execute a Right Mouse Click on the position corresponding to the character displayed on the braille display below the sensor.

The sensor button also allows you to select text in a document. To select using the Braille display sensor button, press and hold the LEFT SHIFT button and press the sensor on the character you want to initiate selection. Release both keys. Move to the position where you want to end the selection and do the same operation again. You can use any navigation command, including the Navigation Rockers, to move between the beginning and the end of the text you want to select.

touch sensors

The braille braille touch screen has 40 tactile sensors and a horizontal mine that represents virtually 40 braille bristles, when a finger moves through the touch sensors the character that is represented by each position is shown in the brille brille and the router above the braille bristle it becomes the router for that character at that time.

The power button

The Power button turns the brailletouch screen on and off when it is not connected. I think it is easier to understand and modify a ready-made structure than to do unfortunately there are examples of a structure made of keyboards for gaming devices but there is no structure made of Which one can be guided to modify and adapt a HD braille device would be very helpful if someone will contribute a h & d braille structure that serves as an example for one to adapt it to the particular project or adapt the project to that structure

I basically need a structure for a 40-cell braille display
With a braille keyboard which contains eight brsille keys a space bar day chif keys
Basically its keyboard structure is similar to that of Freedom Scientific Focus 40 braille displays with an additional navigation key up arrow down arrow left arrow right arrow and center button and on either side a tab key and on the other side tab chip these keys are for move within a document or within the menus from a cell phone in a simple way here I leave more or less a manual of how the screen would be arranged shine

A general HID tuning document with this data would be very helpful.

Introduction

.a line Braille brailletouch is open source in both hardware and software.

The Braille brailletouch line consists of:

40, touch cells 1, Braille cells A sensor button (cursor scroll button) above the Braille cell. 8-key Braille keyboard, Perkins style Two scroll buttons two rocker keys two selectors two Shift keys one multiple scroll keyboard right left up down select and from the sides tab and chif tab One navigation rocker One mode button at either end of the line, for quick navigation through files, lists and menus. USB connection to computer Compatibility with select phones and other mobile devices via screen readers Physical description

On the left side of the line there are, starting from the position closest to you, a power button and a micro USB port. standard. The USB port allows you to connect the line to a computer using the USB cable or the power adapter.

Braille cells are located on the front of the right side of the unit. There is a sensor above the Braille cell. At both ends of the Braille display surface there is a navigation rocker whose purpose is to facilitate navigation. Above each of the navigation rockers is a mode button to toggle between the different navigation modes.

the braille display has 40 touch cells that are located on the center front of the unit

On the left is the multi-scroll keyboard, it has Jockey-style diction and selection keys and stubborn left and right corresponding to tab and chif + tab

Above the touch cells is an eight-key Braille (Perkins-style) keyboard. These keys are used to type text and execute commands. On the front edge, located in the center and below the Braille cells, is the SPACEBAR. This is used in combination with the Braille keys to enter commands. The commands that are executed with the SPACEBAR are called Cor. For example, COR L or COR 1-2-3. on the side and side of the SPACEBAR are the left SHIFT key, right SHIFT key

The following controls are located on the front of the equipment, from left to right:

brailletouch 40: left selector, left scroll button, left rocker key, right rocker key, right scroll button and right selector.

The following sections describe the use of these controls when working with Braille Touch and NVDA. If you are working with the brailletouch line and another screen reviewer, both on a computer and with a mobile device, consult the documentation for that software, as the functions of these controls may be different from those of NVDA.

Navigation Rockers and Mode Buttons

The Navigation Rockers on the brailletouch allow you to quickly scroll through files, dialogs, lists and menus. In files, the wheels allow you to scroll through lines, sentences, paragraphs or scroll left and right. To switch between the four available navigation modes, press the Mode Button located above the Navigation Rocker. In the dialogs, you can scroll through the available controls. In menus, by menu items. It is also possible to completely deactivate the Navigation Rockers by pressing the Left or Right Mode Button and the SPACEBAR at the same time. To activate them, run the same command again.

Scroll buttons

The scroll buttons allow you to scroll a display to the left or right. Press the Left Scroll button, which displays a raised double left arrow, to scroll left; press the Right Scroll button, which displays a raised right double arrow, to scroll right.

Rocker keys

The rocker keys allow you to move to the next or previous line. Press the upper part of the rocker key to go to the previous line, and the lower part to go to the next line. In combination with the scroll buttons, the rocker keys allow you to move the cursor to the beginning or end of the current line. To move the cursor to the beginning of the line, press a scroll button plus the top of a rocker key; To move the cursor to the end of the line, press a scroll button plus the bottom of a rocker key;

Selectors

If used individually, the selectors (concave in shape) allow you to configure Auto Advance. Pressing both Selector buttons at the same time will turn auto advance on or off. Press the Left selector to decrease the speed of the automatic advance, and press the Right selector to increase it.

When pressed in combination with other keys, they perform various functions. For example, if you press a Selector in conjunction with the top or bottom of a rocker key, you will page up or down. You can also press a Selector together with the Scroll left or right button to go to the beginning or end of a file.

Multi-scroll keyboard

it behaves like a Jockey with the keys arrow up arrow down arrow right arrow left and in the center select, from the left side a key corresponding to tabulator and from the right side a key corresponding to Shift + tabulator

Shift buttons

The Shift buttons are used in combination with the SPACEBAR, Braille keys, and other controls on the line to execute commands.

the Sensor button

There is a sensor button above the braille cell. The sensor button allows you to move the cursor to the corresponding character currently displayed in the cell, or to open links on web pages and email messages. In Line mode, the sensor allows you to open menus or select a menu item.

If you hold down the LEFT or RIGHT SCROLL BUTTON and press the sensor, you will execute a Right Mouse Click on the position corresponding to the character displayed on the braille display below the sensor.

The sensor button also allows you to select text in a document. To select using the Braille display sensor button, press and hold the LEFT SHIFT button and press the sensor on the character you want to initiate selection. Release both keys. Move to the position where you want to end the selection and do the same operation again. You can use any navigation command, including the Navigation Rockers, to move between the beginning and the end of the text you want to select.

touch sensors

The braille braille touch screen has 40 tactile sensors and a horizontal mine that represents virtually 40 braille bristles, when a finger moves through the touch sensors the character that is represented by each position is shown in the brille brille and the router above the braille bristle it becomes the router for that character at that time.

The power button

The Power button turns the brailletouch display on and off

@discapacidad5
Copy link

I have created a blank repository within my project for anyone who wants to help me. https://github.com/brailletouch/HID-braille-ESP32

my device has the same key structure as focus 40
I am a baby in programming I would like to have a model code that I can modify and adapt

@discapacidad5
Copy link

Research news

I found this library for esp32 on arduino https://github.com/T-vK/ESP32-BLE-Keyboard/blob/master/BleKeyboard.cpp

And this other https://github.com/T-vK/ESP32-BLE-Mouse/blob/master/BleMouse.cpp

I think that modifying it could adapt to the braille HID takes care of automatically making the description table for the controller you just have to load the list of the hid braille and then one in the code adds the keys or functions that you need to use

the libraries that are called to do all the work are "BLEHIDDevice" and "HIDTypes.h" en https://github.com/espressif/arduino-esp32/blob/master/libraries/BLE/src/

// BleKeyboard.cpp

#include "BleKeyboard.h"     // modify by Blebraille.h

static const uint8_t _hidReportDescriptor[] = {
  USAGE_PAGE(1),      0x01,   // USAGE_PAGE (Generic Desktop Ctrls)  // modify by HID braille   USAGE_PAGE(1),      0x41, 
  USAGE(1),           0x06,          // USAGE (Keyboard)  // modify by   USAGE(1),           0x01,  Braille Display
  COLLECTION(1),      0x01,          // COLLECTION (Application)
  
};
 ..............

and


// BleMouse.cpp

static const uint8_t _hidReportDescriptor[] = {
  USAGE_PAGE(1),       0x01, // USAGE_PAGE (Generic Desktop)   //  modify by HID braille     USAGE_PAGE(1),      0x41,
  USAGE(1),            0x02, // USAGE (Mouse)  // modify by  el HID  USAGE(1),           0x01,  Braille Display
  COLLECTION(1),       0x01, // COLLECTION (Application)
  USAGE(1),            0x01, //   USAGE (Pointer)
  COLLECTION(1),       0x00, //   COLLECTION (Physical)
  // ------------------------------------------------- Buttons (Left, Right, Middle, Back, Forward)
  // USAGE_PAGE(1),       0x09, //     USAGE_PAGE (Button)
  // USAGE_MINIMUM(1),    0x01, //     USAGE_MINIMUM (Button 1)
  // USAGE_MAXIMUM(1),    0x05, //     USAGE_MAXIMUM (Button 5)
  // LOGICAL_MINIMUM(1),  0x00, //     LOGICAL_MINIMUM (0)
  // LOGICAL_MAXIMUM(1),  0x01, //     LOGICAL_MAXIMUM (1)
  // REPORT_SIZE(1),      0x01, //     REPORT_SIZE (1)
  // REPORT_COUNT(1),     0x05, //     REPORT_COUNT (5)
  // HIDINPUT(1),         0x02, //     INPUT (Data, Variable, Absolute) ;5 button bits
  // ------------------------------------------------- Padding
  // REPORT_SIZE(1),      0x03, //     REPORT_SIZE (3)
  // REPORT_COUNT(1),     0x01, //     REPORT_COUNT (1)
  // HIDINPUT(1),         0x03, //     INPUT (Constant, Variable, Absolute) ;3 bit padding
  // ------------------------------------------------- X/Y position, Wheel
  // USAGE_PAGE(1),       0x01, //     USAGE_PAGE (Generic Desktop)
  // USAGE(1),            0x30, //     USAGE (X)
  // USAGE(1),            0x31, //     USAGE (Y)
 // USAGE(1),            0x38, //     USAGE (Wheel)
 // LOGICAL_MINIMUM(1),  0x81, //     LOGICAL_MINIMUM (-127)
  // LOGICAL_MAXIMUM(1),  0x7f, //     LOGICAL_MAXIMUM (127)
 // REPORT_SIZE(1),      0x08, //     REPORT_SIZE (8)
 // REPORT_COUNT(1),     0x03, //     REPORT_COUNT (3)
 // HIDINPUT(1),         0x06, //     INPUT (Data, Variable, Relative) ;3 bytes (X,Y,Wheel)
  // ------------------------------------------------- Horizontal wheel
  // USAGE_PAGE(1),       0x0c, //     USAGE PAGE (Consumer Devices)
 //  USAGE(2),      0x38, 0x02, //     USAGE (AC Pan)
  // LOGICAL_MINIMUM(1),  0x81, //     LOGICAL_MINIMUM (-127)
  // LOGICAL_MAXIMUM(1),  0x7f, //     LOGICAL_MAXIMUM (127)
  // REPORT_SIZE(1),      0x08, //     REPORT_SIZE (8)
  // REPORT_COUNT(1),     0x01, //     REPORT_COUNT (1)
  // HIDINPUT(1),         0x06, //     INPUT (Data, Var, Rel)
  // END_COLLECTION(0),         //   END_COLLECTION
  // END_COLLECTION(0)          // END_COLLECTION
// };
..................

Those examples could be replaced by the HID braille example

//	Braille Display Page (0x41)

// Sample Report Descriptor ‐ Braille Display
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
    0x05, 0x41,        // Usage Page (Braille)
    0x09, 0x01,        // USAGE (Braille Display)
    0xA1, 0x01,        // Collection (Application)
    0x1A, 0x01, 0x02,  //   Usage Minimum (Braille Keyboard Dot 1)
    0x2A, 0x08, 0x02,  //   Usage Maximum (Braille Keyboard Dot 8)
    0x75, 0x01,        //   Report Size (1)
    0x95, 0x08,        //   Report Count (8)
    0x15, 0x00,        //   Logical Minimum (0)
    0x25, 0x01,        //   Logical Maximum (1)
    0x81, 0x02,        //   Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x05, 0x41,        //   Usage Page (Braille)
    0x0A, 0x0A, 0x02,  //   Usage (Braille Keyboard Left Space)
    0x0A, 0x0B, 0x02,  //   Usage (Braille Keyboard Right Space)
    0x0A, 0x10, 0x02,  //   Usage (Braille Joystick Center)
    0x0A, 0x11, 0x02,  //   Usage (Braille Joystick Up)
    0x0A, 0x12, 0x02,  //   Usage (Braille Joystick Down)
    0x0A, 0x13, 0x02,  //   Usage (Braille Joystick Left)
    0x0A, 0x14, 0x02,  //   Usage (Braille Joystick Right)
    0x75, 0x01,        //   Report Size (1)
    0x95, 0x07,        //   Report Count (7)
    0x15, 0x00,        //   Logical Minimum (0)
    0x25, 0x01,        //   Logical Maximum (1)
    0x81, 0x02,        //   Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x75, 0x01,        //   Report Size (1)
    0x95, 0x01,        //   Report Count (1)
    0x81, 0x03,        //   Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x0A, 0x0D, 0x02,  //   Usage (Braille Left Controls)
    0xA1, 0x02,        //   Collection (Logical)
    0x05, 0x09,        //     Usage Page (Button)
    0x19, 0x01,        //     Usage Minimum (Button 1)
    0x29, 0x03,        //     Usage Maximum (Button 3)
    0x75, 0x01,        //     Report Size (1)
    0x95, 0x03,        //     Report Count (3)
    0x15, 0x00,        //     Logical Minimum (0)
    0x25, 0x01,        //     Logical Maximum (1)
    0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0xC0,              //   End Collection
    0x05, 0x41,        //   Usage Page (Braille)
    0x0A, 0x0E, 0x02,  //   Usage (Braille Right Controls)
    0xA1, 0x02,        //   Collection (Logical)
    0x05, 0x09,        //     Usage Page (Button)
    0x19, 0x01,        //     Usage Minimum (Button 1)
    0x29, 0x03,        //     Usage Maximum (Button 3)
    0x75, 0x01,        //     Report Size (1)
    0x95, 0x03,        //     Report Count (3)
    0x15, 0x00,        //     Logical Minimum (0)
    0x25, 0x01,        //     Logical Maximum (1)
    0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0xC0,              //   End Collection
    0x75, 0x02,        //     Report Size (2)
    0x95, 0x01,        //     Report Count (1)
    0x81, 0x03,        //     Input (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position) //2 bit pad
    0x05, 0x41,        //   Usage Page (Braille)
    0x0A, 0x0C, 0x02,  //   Usage (Braille Face Controls)
    0xA1, 0x02,        //   Collection (Logical)
    0x05, 0x09,        //     Usage Page (Button)
    0x19, 0x01,        //     Usage Minimum (Button 1)
    0x29, 0x03,        //     Usage Maximum (Button 4)
    0x75, 0x01,        //     Report Size (1)
    0x95, 0x04,        //     Report Count (4)
    0x15, 0x00,        //     Logical Minimum (0)
    0x25, 0x01,        //     Logical Maximum (1)
    0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x75, 0x04,        //     Report Size (4)
    0x95, 0x01,        //     Report Count (1)
    0x81, 0x03,        //     Input (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position)
    0xC0,              //   End Collection
    
    0x05, 0x41,        //   Usage Page (Braille)
    0x09, 0x02,        //   USAGE (Braille Row)
    0xA1, 0x02,        //   Collection (Logical)
    0x09, 0x02,        //     Usage (8 Dot Braille Cell)
    0x15, 0x00,        //     Logical Minimum (0)
    0x26, 0xFF, 0x00,  //     Logical Maximum (255)
    0x75, 0x08,        //     Report Size (8)
    0x95, 0x14,        //     Report Count (20)
    0x91, 0x03,        //     Output (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position,Non‐volatile)
    
    0x09, 0xFA,        //     USAGE (Router Set 1)
    0xA1, 0x02,        //     Collection (Logical)
    0x0A, 0x00, 0x01,  //       Usage (Router Key)
    0x15, 0x00,        //       Logical Minimum (0)
    0x25, 0x01,        //       Logical Maximum (1)
    0x75, 0x01,        //       Report Size (1)
    0x95, 0x14,        //       Report Count (20)
    0x81, 0x02,        //       Input (Data,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position)
    0x75, 0x04,        //       Report Size (4)
    0x95, 0x01,        //       Report Count (1)
    0x81, 0x03,        //       Input (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position) //4‐bit pad
    0xC0,              //     End Collection
    0xC0,              //   End Collection
    0xC0,              // End Collection


It is only to order it as a library so that later in the code you can call the library and pass the parameters that are needed such as the number of braille cells, buttons, etc.

something like that


#include < Blebraille..h>

 Blebraille  Blebraille(40); // para definir una pantalla de 40 caracteres

void setup() {
  Serial.begin(115200);
  Serial.println("Starting BLE work!");
  Blebraille.begin();
}

void loop() {
  if(Blebraille.isConnected()) {
    Serial.println("Sending 'Hello world'...");
Blebraille.print("Hello world");
if (Blebraille.available()) { // Si hay dato en el enviado por el lector de pantalla
             dato = Blebraille.read(); // Lo lee HID ble
              }


    delay(1000);

    Serial.println("Sending Enter key...");
    Blebraille.write(BRAILLE_KEYBOARD_DOT_1);

    delay(1000);

   
  }

  Serial.println("Waiting 5 seconds...");
  delay(5000);
}

	

This is what I have been able to understand so far

I understand that with 0x05 I must pass how many braille cells my screen has but I don't know how to do that

according to the hid code nvda source brailleDisplayDrivers hid.py is defined like this

class BraillePageUsageID(enum.IntEnum):
	UNDEFINED = 0
	BRAILLE_DISPLAY = 0x1
	BRAILLE_ROW = 0x2
	EIGHT_DOT_BRAILLE_CELL = 0x3
	SIX_DOT_BRAILLE_CELL = 0x4
	NUMBER_OF_BRAILLE_CELLS = 0x5
	SCREEN_READER_CONTROL = 0x6
	SCREEN_READER_IDENTIFIER = 0x7
	ROUTER_SET_1 = 0xFA
	ROUTER_SET_2 = 0xFB
	ROUTER_SET_3 = 0xFC
	ROUTER_KEY = 0x100
	ROW_ROUTER_KEY = 0x101
	BRAILLE_BUTTONS = 0x200
	BRAILLE_KEYBOARD_DOT_1 = 0x201
	BRAILLE_KEYBOARD_DOT_2 = 0x202
	BRAILLE_KEYBOARD_DOT_3 = 0x203
	BRAILLE_KEYBOARD_DOT_4 = 0x204
	BRAILLE_KEYBOARD_DOT_5 = 0x205
	BRAILLE_KEYBOARD_DOT_6 = 0x206
	BRAILLE_KEYBOARD_DOT_7 = 0x207
	BRAILLE_KEYBOARD_DOT_8 = 0x208
	BRAILLE_KEYBOARD_SPACE = 0x209
	BRAILLE_KEYBOARD_LEFT_SPACE = 0x20A
	BRAILLE_KEYBOARD_RIGHT_SPACE = 0x20B
	BRAILLE_FACE_CONTROLS = 0x20C
	BRAILLE_LEFT_CONTROLS = 0x20D
	BRAILLE_RIGHT_CONTROLS = 0x20E
	BRAILLE_TOP_CONTROLS = 0x20F
	BRAILLE_JOYSTICK_CENTER = 0x210
	BRAILLE_JOYSTICK_UP = 0x211
	BRAILLE_JOYSTICK_DOWN = 0x212
	BRAILLE_JOYSTICK_LEFT = 0x213
	BRAILLE_JOYSTICK_RIGHT = 0x214
	BRAILLE_DPAD_CENTER = 0x215
	BRAILLE_DPAD_UP = 0x216
	BRAILLE_DPAD_DOWN = 0x217
	BRAILLE_DPAD_LEFT = 0x218
	BRAILLE_DPAD_RIGHT = 0x219
	BRAILLE_PAN_LEFT = 0x21A
	BRAILLE_PAN_RIGHT = 0x21B
	BRAILLE_ROCKER_UP = 0x21C
	BRAILLE_ROCKER_DOWN = 0x21D
	BRAILLE_ROCKER_PRESS = 0x21E

I don't know if I'm on the right track or I'm totally lost, but if someone understands and knows how I can continue this structure, it would be good to help.

@datajohan-karlberg
Copy link

Hi
I wonder how to send media keys?
I want to send key combination for "volume up" (that is for to take picture in smartphone app)

some programs use:
const MediaKeyReport KEY_MEDIA_VOLUME_UP = {32, 0};
But i have no idea how to send it with this program.

@manuelbl
Copy link
Author

manuelbl commented Mar 2, 2022

The code on this page is not capable of sending media keys. The HID descriptor and input report do not cover it. The library at https://github.com/T-vK/ESP32-BLE-Keyboard can do it. That's probably where you have KEY_MEDIA_VOLUME_UP from.

@datajohan-karlberg
Copy link

Thank you for fast reply. So sad. I have tested "https://github.com/T-vK/ESP32-BLE-Keyboard2" but it have a little problem...
my phone can not do the Bluetooth pairing. But with your code Bluetooth pairing works without any problems.

@Ariful369
Copy link

Yes I have the keypad setup. I loaded the sketch below and the buttons work #include "Keypad.h" const byte ROWS = 4; //four rows const byte COLS = 4; //four columns

char keys[ROWS][COLS] = { {'1','2','3','A'}, {'4','5','6','B'}, {'7','8','9','C'}, {'*','0','#','D'} };

// For Arduino Microcontroller //byte rowPins[ROWS] = {9, 8, 7, 6}; //byte colPins[COLS] = {5, 4, 3, 2};

// For ESP8266 Microcontroller //byte rowPins[ROWS] = {D1, D2, D3, D4}; //byte colPins[COLS] = {D5, D6, D7, D8};

// For ESP32 Microcontroller byte rowPins[ROWS] = {23, 22, 3, 21}; byte colPins[COLS] = {19, 18, 5, 17};

Keypad keypad = Keypad(makeKeymap(keys), rowPins, colPins, ROWS, COLS);

void setup() { Serial.begin(115200); }

void loop() { char key = keypad.getKey(); Serial.println("hello\n");

if (key){ Serial.println("letter\n"); Serial.println(key); } delay(10); }

****From the monitor when pressing two

entry 0x400806b8 2 2 2

740580

@manuelbl
Copy link
Author

What's the context of your post? Do you have a question?

@gomcodoctor
Copy link

Thank you for fast reply. So sad. I have tested "https://github.com/T-vK/ESP32-BLE-Keyboard2" but it have a little problem... my phone can not do the Bluetooth pairing. But with your code Bluetooth pairing works without any problems.

Same here..this code worked like charm, that library did not work at all..

Reconnection issue + unable to pair second device even if first is disconnected with keyboard library

Thanks for sharing this code

@HeadHodge
Copy link

HeadHodge commented Dec 23, 2022

Hi,

i'm an arduino newbee, and your example worked perfect on my feathers3 esp32-s3 stick the very 1st try! It saved me so many hours of time, that i wanted to take some of the extra time i now have to thank you for sharing this great ble hid example (aka HOG, hid over gatt). I verified it working well on my Win10, Chromebox, and Firestick.

for anyone interested.... i was able to successfully modify your example to use the nimBLEHID library instead of the older BLEHID library.

I also modified your ReportMap to add two additional reports to support Consumer Media Control and Basic Mouse Control codes.

The key to using the extra hid reports is to:

  • add additional reports to reportmap at line 98
  • change reportid to use desired report on line 230
  • use the proper report format on line 274

Thanks again... here is copy of my working code changes to your example:

#include <NimBLEDevice.h>
#include <NimBLEUtils.h>
#include <NimBLEScan.h>
#include <NimBLEAdvertisedDevice.h>

/* 
 * Sample program for ESP32 acting as a Bluetooth keyboard
 * 
 * Copyright (c) 2019 Manuel Bl
 * 
 * Licensed under MIT License
 * https://opensource.org/licenses/MIT
 */

//
// This program lets an ESP32 act as a keyboard connected via Bluetooth.
// When a button attached to the ESP32 is pressed, it will generate the key strokes for a message.
//
// For the setup, a momentary button should be connected to pin 2 and to ground.
// Pin 2 will be configured as an input with pull-up.
//
// In order to receive the message, add the ESP32 as a Bluetooth keyboard of your computer
// or mobile phone:
//
// 1. Go to your computers/phones settings
// 2. Ensure Bluetooth is turned on
// 3. Scan for Bluetooth devices
// 4. Connect to the device called "ESP32 Keyboard"
// 5. Open an empty document in a text editor
// 6. Press the button attached to the ESP32

#define US_KEYBOARD 1
#include <Arduino.h>
#include "NimBLEDevice.h"
#include "NimBLEHIDDevice.h"
#include "HIDTypes.h"
#include "HIDKeyboardTypes.h"

// Change the below values if desired
#define BUTTON_PIN 2
#define MESSAGE "h"
#define DEVICE_NAME "ESP32 Keyboard"

// Forward declarations
void bluetoothTask(void*);
void typeText(const char* text);
bool isBleConnected = false;

void setup() {
    Serial.begin(115200);

    // configure pin for button
    pinMode(BUTTON_PIN, INPUT_PULLUP);

    // start Bluetooth task
    xTaskCreate(bluetoothTask, "bluetooth", 20000, NULL, 5, NULL);

    Serial.print("BleStart: ");
    Serial.println();

}

void loop() {  
    //if (isBleConnected && digitalRead(BUTTON_PIN) == LOW) {
    if (isBleConnected) {
        // button has been pressed: type message
        Serial.println(MESSAGE);
        typeText(MESSAGE);
    }

    delay(3000);
}

// Message (report) sent when a key is pressed or released
struct InputReport {
    uint8_t modifiers;       // bitmask: CTRL = 1, SHIFT = 2, ALT = 4
    uint8_t reserved;        // must be 0
    uint8_t pressedKeys[6];  // up to six concurrenlty pressed keys
};

struct ConsumerReport {
    uint16_t pressedKey;     // two byte keycode
};

struct MouseReport {
    uint8_t pressedButton;     // one byte buttoncode
    uint8_t offsetX;           // move mouse on X-sxis
    uint8_t offsetY;           // move mouse on Y-sxis
};

// Message (report) received when an LED's state changed
struct OutputReport {
    uint8_t leds;            // bitmask: num lock = 1, caps lock = 2, scroll lock = 4, compose = 8, kana = 16
};

// The report map describes the HID device (a keyboard in this case) and
// the messages (reports in HID terms) sent and received.
static const uint8_t REPORT_MAP[] = {
//
//ReportId = 1 (Keyboard Conrol Input)
    USAGE_PAGE(1),      0x01,       // Generic Desktop Controls
    USAGE(1),           0x06,       // Keyboard
    COLLECTION(1),      0x01,       // Application
    REPORT_ID(1),       0x01,       //   Report ID (1)
    USAGE_PAGE(1),      0x07,       //   Keyboard/Keypad
    USAGE_MINIMUM(1),   0xE0,       //   Keyboard Left Control
    USAGE_MAXIMUM(1),   0xE7,       //   Keyboard Right Control
    LOGICAL_MINIMUM(1), 0x00,       //   Each bit is either 0 or 1
    LOGICAL_MAXIMUM(1), 0x01,
    REPORT_COUNT(1),    0x08,       //   8 bits for the modifier keys
    REPORT_SIZE(1),     0x01,       
    HIDINPUT(1),        0x02,       //   Data, Var, Abs
    REPORT_COUNT(1),    0x01,       //   1 bytes (for up to 1 concurrently pressed keys)
    REPORT_SIZE(1),     0x08,
    LOGICAL_MINIMUM(1), 0x00,
    LOGICAL_MAXIMUM(1), 0xE7,       //   101 keys
    USAGE_MINIMUM(1),   0x00,
    USAGE_MAXIMUM(1),   0xE7,
    HIDINPUT(1),        0x00,       //   Data, Array, Abs
    END_COLLECTION(0),              // End application collection
//
//Report Reference Id = 2 (Media Control Input)
    USAGE_PAGE(1), 0x0C,            // Usage Page (Consumer)
    USAGE(1), 0x01,                 // Usage (Consumer Control)
    COLLECTION(1), 0x01,            // Collection (Application)
    REPORT_ID(1), 0x02,             //   Report ID (2)
    REPORT_SIZE(1), 0x10,           //   Report Size (16)
    REPORT_COUNT(1), 0x01,          //   Report Count (1)
    LOGICAL_MINIMUM(1), 0x01,       //   Logical Minimum (1)
    LOGICAL_MAXIMUM(2), 0xff, 0x07, //   Logical Maximum (2047)
    USAGE_MINIMUM(1), 0x01,         //   Usage Minimum (Consumer Control)
    USAGE_MAXIMUM(2), 0xff, 0x07,   //   Usage Maximum (0x07FF)
    HIDINPUT(1), 0x00,              //   Input (Data,Array,Abs,No Wrap,Linear,Preferred State,No Null Position)
    END_COLLECTION(0),              // End Collection
//
//Report Reference Id = 3 (Mouse Control Input)
    USAGE_PAGE(1), 0x01,            // Usage Page (Generic Desktop Ctrls)
    USAGE(1), 0x02,                 // Usage (Mouse)
    COLLECTION(1), 0x01,            // Collection (Application)
    REPORT_ID(1), 0x03,             //   Report ID (3)
    USAGE(1), 0x01,                 //   Usage (Pointer)
    COLLECTION(1), 0x00,            //   Collection (Physical)
    USAGE_PAGE(1), 0x09,            //     Usage Page (Button)
    USAGE_MINIMUM(1), 0x01,         //     Usage Minimum (0x01)
    USAGE_MAXIMUM(1), 0x03,         //     Usage Maximum (0x03)
    LOGICAL_MINIMUM(1), 0x00,       //     Logical Minimum (0)
    LOGICAL_MAXIMUM(1), 0x01,       //     Logical Maximum (1)
    REPORT_COUNT(1), 0x03,          //     Report Count (3)
    REPORT_SIZE(1), 0x01,           //     Report Size (1)
    HIDINPUT(1), 0x02,              //     Input (Data,Var,Abs,No Wrap,Linear,...)
    REPORT_COUNT(1), 0x01,          //     Report Count (1)
    REPORT_SIZE(1), 0x05,           //     Report Size (5)
    HIDINPUT(1), 0x03,              //     Input (Const,Var,Abs,No Wrap,Linear,...)
    USAGE_PAGE(1), 0x01,            //     Usage Page (Generic Desktop Ctrls)
    USAGE(1), 0x30,                 //     Usage (X)
    USAGE(1), 0x31,                 //     Usage (Y)
    LOGICAL_MINIMUM(1), 0x81,       //     Logical Minimum (-127)
    LOGICAL_MAXIMUM(1), 0x7F,       //     Logical Maximum (127)
    REPORT_SIZE(1), 0x08,           //     Report Size (8)
    REPORT_COUNT(1), 0x02,          //     Report Count (2)
    HIDINPUT(1), 0x06,              //     Input (Data,Var,Rel,No Wrap,Linear,...)
    END_COLLECTION(0),              //   End Collection
    END_COLLECTION(0)               // End Collection
};

NimBLEHIDDevice* hid;
NimBLECharacteristic* input;
NimBLECharacteristic* output;

const InputReport NO_KEY_PRESSED = { };

/*
 * Callbacks related to BLE connection
 */
class BleKeyboardCallbacks : public BLEServerCallbacks {

    void onConnect(BLEServer* server) {
        isBleConnected = true;

        // Allow notifications for characteristics
        //BLE2902* cccDesc = (BLE2902*)input->getDescriptorByUUID(NimBLEUUID((uint16_t)0x2902));
        //cccDesc->setNotifications(true);

        Serial.println("Client has connected");
    }

    void onDisconnect(BLEServer* server) {
        isBleConnected = false;

        // Disallow notifications for characteristics
        //BLE2902* cccDesc = (BLE2902*)input->getDescriptorByUUID(NimBLEUUID((uint16_t)0x2902));
        //cccDesc->setNotifications(false);

        Serial.println("Client has disconnected");
    }
};

/*
 * Called when the client (computer, smart phone) wants to turn on or off
 * the LEDs in the keyboard.
 * 
 * bit 0 - NUM LOCK
 * bit 1 - CAPS LOCK
 * bit 2 - SCROLL LOCK
 */
 class OutputCallbacks : public BLECharacteristicCallbacks {
    void onWrite(BLECharacteristic* characteristic) {
        //OutputReport* report = (OutputReport*) characteristic->getValue();
        std::string value = characteristic->getValue();
        OutputReport* report = (OutputReport*)value.data();
//Alternatively use the getValue template:
//my_struct_t myStruct = pChr->getValue<my_struct_t>();

        
        Serial.print("LED state: ");
        Serial.print((int) report->leds);
        Serial.println();
    }
};

void bluetoothTask(void*) {

    // initialize the device
    NimBLEDevice::init(DEVICE_NAME);
    NimBLEServer* server = NimBLEDevice::createServer();
    server->setCallbacks(new BleKeyboardCallbacks());

    // create an HID device
    hid = new NimBLEHIDDevice(server);
    input = hid->inputReport(2); // report ID
    output = hid->outputReport(1); // report ID
    output->setCallbacks(new OutputCallbacks());

    // set manufacturer name
    hid->manufacturer()->setValue("Maker Community");
    // set USB vendor and product ID
    hid->pnp(0x02, 0xe502, 0xa111, 0x0210);
    // information about HID device: device is not localized, device can be connected
    hid->hidInfo(0x00, 0x02);

    // Security: device requires bonding
    BLESecurity* security = new BLESecurity();
    security->setAuthenticationMode(ESP_LE_AUTH_BOND);

    // set report map
    hid->reportMap((uint8_t*)REPORT_MAP, sizeof(REPORT_MAP));
    hid->startServices();

    // set battery level to 100%
    hid->setBatteryLevel(100);

    // advertise the services
    NimBLEAdvertising* advertising = server->getAdvertising();
    advertising->setAppearance(HID_KEYBOARD);
    advertising->addServiceUUID(hid->hidService()->getUUID());
    advertising->addServiceUUID(hid->deviceInfo()->getUUID());
    advertising->addServiceUUID(hid->batteryService()->getUUID());
    advertising->start();

    Serial.println("BLE ready");
    delay(portMAX_DELAY);
};

void typeText(const char* text) {
    int len = strlen(text);
    for (int i = 0; i < len; i++) {

        // translate character to key combination
        uint8_t val = (uint8_t)text[i];
        if (val > KEYMAP_SIZE)
            continue; // character not available on keyboard - skip
        KEYMAP map = keymap[val];

        // create input report

       /*
       //REPORT 1
        InputReport report = {
            //.modifiers = map.modifier,
            .modifiers = map.modifier,
            .reserved = map.usage
            //.pressedKeys = {
            //   map.usage,
            //    0, 0, 0, 0, 0
            //}
        };
        */
       
        //REPORT 2
        ConsumerReport report = {
            //.pressedKey = 0xE9 //Volume Up
            .pressedKey = 0x46 //FireTV Back
        };

        /*        
        //REPORT 3
        MouseReport report = {
            .pressedButton = 0,
            .offsetX = 100,
            .offsetY = 100
        };
        */

        // send the input report
        input->setValue((uint8_t*)&report, sizeof(report));
        input->notify();

        delay(5);

        // release all keys between two characters; otherwise two identical
        // consecutive characters are treated as just one key press
        input->setValue((uint8_t*)&NO_KEY_PRESSED, sizeof(NO_KEY_PRESSED));
        input->notify();

        delay(5);
    }
}

@manuelbl
Copy link
Author

@HeadHodge Thanks a lot for the feedback and the code.

@djavorek
Copy link

djavorek commented Jan 28, 2023

Just a small feedback, if you want your device to connect again after it was disconnected, you have to start advertising again at onDisconnect hook.
See: espressif/arduino-esp32#6016

void onDisconnect(BLEServer *server)
  {
    isBleConnected = false;

    // Disallow notifications for characteristics
    BLE2902 *cccDesc = (BLE2902 *)input->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));
    cccDesc->setNotifications(false);

    server->getAdvertising()->start();
    Serial.println("BLE client has disconnected.");
  }

Also @manuelbl I have used this code in my project (added your license), reply if you have any concerns.
https://gist.github.com/djavorek/bbab52e010818f8e5717b881217f1903

@manuelbl
Copy link
Author

@djavorek Thanks for the feedback. You are very welcome to use the code.

@alex-k8
Copy link

alex-k8 commented Feb 19, 2023

Hello @manuelbl, first of all thanks for this great project!
I can't seem to understand how I can control the caps lock/scroll lock/num lock leds on my keyboard. I successfully receive the OutputReport, but how do I use this to set the LED on my keyboard?
Also, is it somehow possible to control the RGB of the keyboard?

@manuelbl
Copy link
Author

@alex-k8 In this example, the ESP32 acts as a keyboard. It is connected to a computer. The computer will send output reports indicating what LEDs need to be turned on. As the ESP32 usually does not implement LEDs representing Caps Lock etc., the output reports can be safely ignored.

In your setup, the ESP32 and the keyboard seem to be two different things. I don't understand your setup and can't tell you what to do with the output reports.

@manuelbl
Copy link
Author

@alex-k8 I'm assuming your keyboard is connected by USB. USB and Bluetooth work very similar when it comes to keyboards. The both exchange input and output reports in exactly the same format. So to control the LEDs, you would need to forward the output report ot the keyboard. But I have no idea what USB/keyboard API you are using. So this might be possible, or you might need to go a different approach.

@alex-k8
Copy link

alex-k8 commented Feb 19, 2023

@manuelbl Thank you for your insight, I think I'll figure it out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment