mapillary_upload.md

Mapillary Upload Photos

This document show how to upload images on Mapillary. Mapillary currently don't have any good documentation on how to upload the images, only some confuse Python code with a lot of things that nobody wants if was implementing on own code.

So, looking at the Python code, I have concluded that to upload a photo, or a photo sequence, you need the following information:

---

Photo File

The photo file must contains some EXIF information, specially:

• Exif.GPSInfo.GPSLatitudeRef;
• Exif.GPSInfo.GPSLatitude;
• Exif.GPSInfo.GPSLongitudeRef;
• Exif.GPSInfo.GPSLongitude;
• Date time (must be one of the following):
  • Exif.Image.DateTime;
  • Exif.Image.DateTimeOriginal;
  • Exif.Photo.DateTimeOriginal;
  • Exif.Photo.DateTimeDigitized;
  • Exif.GPSInfo.GPSDateStamp;

Note: I don't have checked all the EXIF Date field. I always use Exif.Image.DateTime and Exif.GPSInfo.GPSDateStamp.

More information about the EXIF fields, can be checked at: http://www.exiv2.org/tags.html

---

User info

To upload, you need the following information about the user account that will be used to upload the photo file:

• username;
• permission hash;
• signature hash.

username: the username (login username);

permission hash: Access the user profile on Mapillary website (little gear icon on the right), then Developers tab. Copy the Permission Hash on Upload from script section.

signature hash: Access the user profile on Mapillary website (little gear icon on the right), then Developers tab. Copy the Signature Hash on Upload from script section.

Additional Info

Some additional info is necessary to upload the photos to Mapillary server.

Upload URL: Fixed value. For some reason, the URL is a directly URL to Amazon. No futher documentation about what URL must be used, so, use the same URL from mapillary_tools code:

https://s3-eu-west-1.amazonaws.com/mapillary.uploads.manual.images

Uploading Photos

To upload a Photo, or a Sequence of photos, you will need generate a key. I believe this key must be unique for each sequence. So, If you want upload 100 photos, all the 100 photos use the same key (*).

Update: Apparently, a unique sequence can't contains more than 1000 photos, or this can cause a error on Mapillary Server (legacy dashboard). See this issue for more info: https://github.com/mapillary/mapillary_issues/issues/2799

Generating a key: Again, no documentation about this, how the format of the key, what must be included, if accept all chars, etc. In mapillary_tools, the key was generated with uuid library, using uuid4(). The generated key follow this format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx where x is a char between a and f or a number between 0 and 9 (all lowercase).

After the key was generated, the username and image name must be included on key, using the following format: {username}/{key}/{filename}

*Note: Only the username and key part appears to be the same on all photos of the same sequence.

Sending a POST to Mapillary Server with a Photo

To send a Photo to Mapillary Server, you must POST to the Amazon URL (Upload URL), and this upload need to be a multipart/form-data with the following parameters:

• AWSAccessKeyId;
• acl;
• key;
• signature;
• policy;
• Content-Type;
• file;

AWSAccessKeyId: No idea, again, because of the lack of documentation about this is, use the same value used in mapillary_tools. The value must be:

AKIAI2X3BJAT2W75HILA

acl: No documentation again, use the same value used in mapillary_tools. The value must be:

private

key: This is the key that you must generate. See the Generating a key info on this document.

signature: the signature hash of the user. See the signature hash info on section User info section of this document.

policy: the permission hash of the user. See the permission hash info on section User info section of this document.

Content-Type: Fixed value: image/jpeg. Really don't have tried to send images on another format, but I believe that Mapillary Server don't accept other types of images than jpeg.

File: The content of the file, with filename.

Note: For some reason, the parameters must follow this order. If some parameter was on different order, maybe a 400 forbidden response will be returned.

After send, and if everything was correctly, the Mapillary Server will return a 204 no content response.

---

POST body example:

POST /mapillary.uploads.manual.images HTTP/1.1
Host: s3-eu-west-1.amazonaws.com
Content-Type: multipart/form-data; boundary=0f16fabcff574d5abd81cfff9808f63b
Cache-Control: no-cache

--0f16fabcff574d5abd81cfff9808f63b
Content-Disposition: form-data; name="AWSAccessKeyId"

AKIAI2X3BJAT2W75HILA
--0f16fabcff574d5abd81cfff9808f63b
Content-Disposition: form-data; name="acl"

private
--0f16fabcff574d5abd81cfff9808f63b
Content-Disposition: form-data; name="key"

{generated_key}
--0f16fabcff574d5abd81cfff9808f63b
Content-Disposition: form-data; name="signature"

{signature_hash}
--0f16fabcff574d5abd81cfff9808f63b
Content-Disposition: form-data; name="policy"

{permission_hash}
--0f16fabcff574d5abd81cfff9808f63b
Content-Disposition: form-data; name="Content-Type"

image/jpeg
--0f16fabcff574d5abd81cfff9808f63b
Content-Disposition: form-data; name="file"; filename="{filename}"

{file_content}
--0f16fabcff574d5abd81cfff9808f63b--

---

CURL example:

curl -X POST \
  https://s3-eu-west-1.amazonaws.com/mapillary.uploads.manual.images \
  -H 'cache-control: no-cache' \
  -H 'content-type: multipart/form-data;' \
  -F AWSAccessKeyId=AKIAI2X3BJAT2W75HILA \
  -F acl=private \
  -F key={generated_key} \
  -F signature={signature_hash} \
  -F policy={permission_hash} \
  -F Content-Type=image/jpeg \
  -F 'file=@{path_to_file}'

---

Python example:

Below a sample code in Python to do a upload. Why another Python example if mapillary_tools exists? Because mapillary_tools has too much code for a "simple" thing, and the encode_multipart function on uploader.py it's not necessary.

import uuid
import os
import requests


def upload_file(filepath, permission, signature, key=None):
    """
    Send a photo to the Mapillary Server
    :param filepath: Full path to the jpg file
    :param permission: permission hash
    :param signature: signature hash
    :param key: sequence key
    """

    upload_url = "https://s3-eu-west-1.amazonaws.com/mapillary.uploads.manual.images"
    filename = os.path.basename(filepath)
    s3_filename = filename
    s3_key = key + s3_filename if key is not None else s3_filename

    # because the request parameters must follow a order, the parameters must be a list, not a dictionary
    # check the codes here:
    # https://stackoverflow.com/a/21091873/4794469
    # https://stackoverflow.com/a/23131823/4794469
    parameters = [
        ('AWSAccessKeyId', (None, 'AKIAI2X3BJAT2W75HILA')),
        ('acl', (None, 'private')),
        ('key', (None, s3_key)),
        ('signature', (None, signature)),
        ('policy', (None, permission)),
        ('Content-Type', (None, 'image/jpeg')),
        ('file', (s3_filename, open(filepath, 'rb'))),
    ]

    response = requests.post(upload_url, files=parameters)
    if response.status_code == 204:
        print "uploaded"


if __name__ == '__main__':
    # login username
    login = 'projevias_rco'

    # script permission hash
    permission_hash = 'YOUR_PERMISSION_HASH_HERE'

    # script signature hash
    signature_hash = 'YOUR_SIGNATURE_HASH_HERE'

    sequence_id = uuid.uuid4()
    sequence_key = login + "/" + str(sequence_id) + "/"

    file_list = ['/tmp/image01.jpg', '/tmp/image02.jpg']
    for filepath in file_list:
        upload_file(
            filepath=filepath,
            permission=permission_hash,
            signature=signature_hash,
            key=sequence_key)

Accept the sequence

After upload the sequence, you must approve the upload (really don't know why!)! So, after upload everything, access the Legacy Dashboard at http://legacy.mapillary.com/map/upload/im and approve the upload sequence

2017-11-20: Include the section about "Accept the sequence" and a link to the issue when a sequence has more than 1k photos.