t0ny-peng/README.md

## README.md

      
    Raw
  

              README.md
            
          
    The original repository of toyota-na was removed following a DMCA takedown notice, as detailed here. Consequently, the integration featured in this tutorial will not function for legal reasons. I regret to say I can no longer offer updates related to this tutorial. It has been a rewarding experience working on this project, and I am confident that you will discover alternative solutions to make it operational.

How to use the Home Assistant Toyota NA integration


How to use the Home Assistant Toyota NA integration

Use IMAP sensor

Troubleshooting


Use your own domain

Use AWS

Register AWS account and setup billing
Create AWS S3 bucket
Create AWS lambda function
Configure AWS S3 to allow SES action to write and Lambda to read/write
Create the AWS SES email domain and verify
Setup AWS SES rule set
Verify the setup
Auto-forward your Toyota App OTP to AWS SES email
Setup the HA Toyota NA integrations
Troubleshooting AWS


The Toyota AP now requires 2FA to be enabled for all accounts. This makes the
current Toyota NA integration in Home Assistant unusable. I've been playing
with the OPT for a while and found two ways to retrieve the OTP in an automated
way.
The IMAP solution is easy to start with, but not very stable as the verification
code often get lost. The personal domain + AWS solution is more reliable, but
requires a bit more setup.
If you have any suggestions or questions, please feel free to leave a comment.
Use IMAP sensor

This method applies if you do not have your own domain.


Setup the IMAP integration in HA using your Toyota app account email
and password. The IMAP server is different for different email providers.
https://www.home-assistant.io/integrations/imap/

For Gmail you need to create a app password if you have 2FA enabled.

Make sure the value of "sensor.imap_<your_email>" is the correct number of unread message in your
email inbox, though we won't use this sensor directly.


Setup a Home assistant file notification service in your configurations.yaml. This
will be used later when an email is received in the IMAP event.
notify:
  - name: toyota_na_verification_code
    platform: file
    filename: "/config/www/verification_code/code.txt" # Or any other path under www. 


Create this file if it doesn't exist. Make sure you can access this file in your browser
via https://<your HA URL>/local/verification_code/code.txt


Add an automation that triggers when an IMAP email is received. Then parse the content
of the email to retrieve 6 digit verification and call the file notification service
to store it last line of the file.

Change the email to your Toyota app account email.

##========== Toyota NA verification code ==========##
- id: toyota_na_verification_code_received
  alias: "Toyota NA verification code received"
  trigger:
    platform: event
    event_type: "imap_content"
  condition:
    condition: and
    conditions:
      - condition: template
        value_template: "{{ trigger.event.data['username'] == '<your toyota email>' }}"
      - condition: template
        value_template: "{{ 'verification code:' in trigger.event.data['text'] }}"
  action:
    - service: notify.toyota_na_verification_code
      data_template:
        message: >-
          {{ trigger.event.data["text"] | regex_findall_index("verification code:\s+([0-9]+)") }}


The ha-toyota-na integration has been modified to work with toyota-na-custom, a
a customized version of toyota-na that supports OTP since I can't find
where the original toyota-na repo is.
Clone the following repo to your local. I'll make a PR to the original repo later.
https://github.com/t0ny-peng/ha-toyota-na
Copy the custom_components/toyota_na folder to /config/custom_components on your Home
Assistant instance.


Restart Home assistant. Add the integration. You will be prompted to enter
these information:

Toyota app account email
Toyota app account password
OTP URL. https://<your HA URL>/local/verification_code/code.txt
OTP Timeout. Typically 30 seconds is enough, but can be increased
if the email is not received in time.


You should be able to see the integration is setup successfully. The integration
can refresh the token when necessary.


Troubleshooting

If you cannot get the integration to work, try to trouble shoot with the
following steps:

Check if you received the email in your inbox. If not, make sure the email
account is correct.
Use any of your other email to send a test email to your Toyota app account
email. Check if the IMAP sensor is updated.
Use the debugging tool in HA to listen to the imap_content event. Make sure
the event fires when you receive the email.
Open the OTP URL and monitor if the new verification code is written to the
file.
Check the log of the integration. It should print the error message if there's
any.

Use your own domain

Sometimes the IMAP sensor cannot retrieve the verification code email in time,
resulting in failure of setting up the integration. Thus it's better to use
AWS Email service for a more reliable setup.
Prerequisite:


You have a domain that you can manage the DNS records. E.g., mydomain.com
You can register a domain from any domain registrar. I use Cloudflare for
my domain.


Use AWS

AWS is the most reliable way to receive an email and trigger a webhook via lambda.
The workflow is:
Email send to your (sub) domain -> Received by AWS SES -> Email content saved to
S3, AND Trigger a lambda function -> The verification is saved to a file on
S3, accessible by HA
The setup is a bit complicated but once setup, it's very reliable.
Register AWS account and setup billing

You'll get $300 for the first year, but in reality the workflow will cost
almost nothing, if any. A credit card is required for registration.
Create AWS S3 bucket

I'm close to us-west-2 so I choose this region. You can choose any region.
This tutorial use the name my-s3-bucket as an example.
Go to AWS S3 us-west-2 and click
Create bucket. Enter a unique name. ⚠️In the Block Public Access settings for this bucket
section, unselect Block all public access and check the 4 checkboxes below as:

ON
ON
OFF
OFF

This is to make sure the verification can be accessed from HA Toyota integration.
Create AWS lambda function

Go to AWS Lambda us-west-2
and click Create function.
Make sure you select/input the following

The type is Author from scratch
ToyotaAppVerificationCode as the function name
Python 3.11 as the Runtime
Keep the rest unchanged

Then click Create function.

IIRC the lambda function by default has access to the Internet, unless it's
connected to a VPC. But let me know if you run into timeout error later.

Go back to the lambda page and click the newly created and only one lambda
function. In the coding section, paste the following code:
⚠️Don't forget to change the S3 bucket name⚠️
import json
import boto3
import http.client

def find_verification_code(content):
    # Define the prefix that appears before the verification code
    prefix = "verification code:"

    # Find the start of the verification code
    start_index = content.find(prefix)
    if start_index != -1:
        # Adjust the start index to the beginning of the actual code
        start_index += len(prefix)

        # Extract the verification code
        verification_code = content[start_index:start_index+7].strip()
        return verification_code
    else:
        return ""
        
def lambda_handler(event, context):
    message_id = event["Records"][0]["ses"]["mail"]["messageId"]
    print(f"messageId: {message_id}")
    
    aws_bucket = "⚠️⚠️YOUR AWS S3 BUCKET NAME⚠️⚠️"
    object_key = "code.txt" ## Using code.txt as example, but could be any name,
                            ## also change every occurrence of code.txt below

    # Create an S3 client
    s3 = boto3.client('s3')

    # Retrieve the email content from S3
    email_object = s3.get_object(Bucket=aws_bucket, Key=message_id)
    content = email_object["Body"].read().decode('utf-8')
    verification_code = find_verification_code(content)
    # Log the email content
    # print("content: ", content)
    print(f"verification code is: {verification_code}")

    ## If `verification_code` is empty, return error
    if verification_code == "":
        return {"statusCode": 400, "body": json.dumps("No verification code found")}


    # Create an S3 client
    s3 = boto3.client('s3')

    # Putting the verification code into the S3 bucket
    try:
        s3.put_object(Bucket=aws_bucket, Key=object_key, Body=verification_code)
        return {
            'statusCode': 200,
            'body': json.dumps('Verification code saved successfully')
        }
    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps('Error saving verification code: ' + str(e))
        }
Then press CMD + S(or CTRL + S on Windows) to save the code. Click Deploy
to deploy the code.
Before leaving this page, go to the Configuration -> Permission tab,
click the Role name link and save its ARN somewhere. Its format
is like arn:aws:iam::<your AWS ID>:role/service-role/<Lambda name>-role-<xxxxxx>
Configure AWS S3 to allow SES action to write and Lambda to read/write

Go back to the AWS S3 page and click the my-s3-bucket bucket. Currently, it's
not accessible by Home Assistant, SES or Lambda. Permission needs to be given
so that,

SES can save the email to the S3 bucket, under a unique key filename
Lambda can read the email content, and save the verification to a file code.txt
Home Assistant(or the public Internet) can read the verification code from the file
code.txt

Go to the permission tab and scroll down to Bucket policy. By default it should be empty.
Paste the following policy. ⚠️⚠️Remember to change Lambda Role ARN and Bucket Name⚠️⚠️
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowLambdaGetObject",
            "Effect": "Allow",
            "Principal": {
                "AWS": "<Lambda Role ARN>"
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetObjectVersion"
            ],
            "Resource": "arn:aws:s3:::<⚠️⚠️Bucket name⚠️⚠️>/*"
        },
        {
            "Sid": "AllowSESPuts",
            "Effect": "Allow",
            "Principal": {
                "Service": "ses.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::<⚠️⚠️Bucket name⚠️⚠️>/*",
            "Condition": {
                "StringEquals": {
                    "AWS:SourceAccount": "<⚠️⚠️Account ID⚠️⚠️>"
                }
            }
        },
        {
            "Sid": "PublicReadAccessForCodeTxt",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<⚠️⚠️Bucket name⚠️⚠️>/code.txt"
        }
    ]
}
Create the AWS SES email domain and verify

Go to the AWS SES
page. Click Verified Identities on the left. It's empty by default. Now add
your subdomain to the verified list, e.g., mail.aws.mydomain.com by clicking
the Create Identity button.
Select Domain as Identity type and enter mail.aws.mydomain.com in the blank.
Keep the rest unchanged and click Create Identity. The identity is created, but
needs to be verified.
In the next page, you'll see some DNS records to set in the Publish DNS records
tab. Go to your domain DNS management page and add these records.
In addition, very important, add a MX record, with mail.aws as the Name,
inbound-smtp.us-west-2.amazonaws.com as the value and 10 as the priority. This
ensures that all the emails sent to *@mail.aws.mydomain.com will be handled by
AWS SES.
Wait for several minutes for the DNS records to propagate, and this identity
will be verified by AWS.
Setup AWS SES rule set

With the domain verified and MX record added, AWS SES can now receive emails.
However, rule set is needed to inform AWS SES what to do with the received
emails.
Go to the Email Receiving
page and create a rule set. Enter ToyotaOTPRuleSet as the name, then the
empty rule set is created. Click Create Rule, give it a name,
uncheck Spam and virus scanning, then click Next.
In the recipient conditions page, add an email address toyota@mail.aws.domain.com.
This tells AWS SES to only handle emails sent to this address, and this email
will become your Toyota app account email(see below). Click Next.
In the Actions page, two actions are needed.

Deliver to S3 bucket, then choose the S3 bucket created above.
Invoke AWS Lambda function, choose the lambda function created above.

Upon finalizing adding the rule, you'll be prompted with a question whether to
add permission to allow this rule to invoke Lambda function. Click Add Permission.
Go back to the Email receiving page and click the rule set ToyotaOTPRuleSet.
Click Set As Active button to activate the rule set.
Verify the setup

Use any email account to send a test email to toyota@mail.aws.domain.com with
the content verification code: 123456. Then download the code.txt file using
the following link:
curl https://s3.<your-aws-region>.amazonaws.com/<my-s3-bucket>/code.txt
The file should contain the 6 digit verification code.
If not, refer to the following troubleshooting section.
Auto-forward your Toyota App OTP to AWS SES email

E.g., if your Toyota app account email is john.doe@gmail.com, we need it
to auto forward the Toyota App OTP email to the AWS SES trigger email
address toyota@mail.aws.domain.com.

Ideally, you could use toyota@mail.aws.domain.com as the Toyota app
account email, but toyota seems having trouble sending verification email
to that account. Therefore I forward the email from Gmail to AWS SES,
but you are welcome to try it.

Go to Gmail settings, Forwarding and POP/IMAP tab, click Add a forwarding
button and enter toyota@mail.aws.domain.com. After a while, you should see
a new file in the AWS S3 bucket. Download it and open it with any text editor.
You should see a link in the email content to approve this forward. Open that link
in browser and approve the forward.
Then go back to Gmail, add a search filter with:

From being donotreply@toyotaconnectedservices.com
Has the words being verification code:

Click continue or create filter, select Forward it to and use
the toyota@mail.aws.domain.com as target.
Setup the HA Toyota NA integrations

Follow steps 5 and 6 in the Use IMAP sensor section to
install the customized integration. ⚠️Remember to use the following link
in the OTP URL field:
https://s3.<your-aws-region>.amazonaws.com/<my-s3-bucket>/code.txt

If everything is setup correctly, you
should be able to see the integration initialization finishes successfully and some new
entities are created. The integration will automatically refresh the token
when necessary, and resort to OTP when the token is expired, e.g., if Home
Assistant is restarted.
Troubleshooting AWS

If you cannot get the integration to work, try to trouble shoot with the
following steps:


Verify that the CNAME, TXT(if any) and MX records of your domain are set correctly.


Verify that the identity status is "Verified" in the AWS SES page.


Make sure the AWS S3 bucket permission is set correctly without typo, or
wrong name of lambda role.


Use any email to send a test email to toyota@mail.aws.domain.com, and
use the Cloudwatch log to check if the lambda function is triggered.
You can find the Cloudwatch log in Lambda function -> Monitoring -> View CloudWatch logs button.


Download the file in AWS S3, and open it with any text editor to make sure
the content is an email content.


If you see the code.txt file created in S3 bucket page, but the link
is not downloadable from, e.g., shell or browser, check the bucket
permission and the policy.


If the Cloudwatch log of Lambda function reports 403 as return code, this is likely that your Home Assistant is proxied
by Cloudflare and the Bot Fight Mode is turned on. Try disable it.