Dropbox to Google Photos

README.md

App release: Dropbox to Google Photos

One of the exciting features we've added since our alpha release is the ability to manage files in your Dexter Apps. In order to demonstrate this useful new system, we're launching a simple new App that will let you copy all the photos inside a Dropbox folder into a Google Photos album.

Anatomy of the App

[screenshot]

The basic app has a very simple structure. We start with an "App Configured" trigger, which simply means that the app runs as soon as the user "Uses" the app and finishes configuring it.

We then go to the first of the two special modules we've made for this app: Dropbox: Get Photos. The fingerprint for this module is pretty straightforward:

requires: dropbox oauth
inputs:   dropbox folder path
outputs:  all photo files under path

We configure the module to ask the user for a folder path that we want to scrape for images ("/Camera Uploads", "/Screenshots", "/Cats", etc.). It then searches under that folder for all jpg, png, and gif files, stitches the results together, and formats them using the Dexter file microformat (see "How Files Work" below for details). Finally, all the results are exposed as the output "photos".

That brings us to the next special module, Google Photos: Add to Album:

requires: google oauth
inputs:   files, album name
outputs:  -

The dropbox module's output gets wired up to this module's files input, and we ask the user to give us the name of an existing album where the new photos should go ("Misc", "Great Cat Pics 2015", "Suff from my phone", etc.).

We then do a dance involving XML, HTTP requests, and Dexter's Google Oauth integration to find the requested albumn and send it the photos you found before.

When everything is done, you should be able to refresh your albumn and see all the shiny new Dropbox pictures inside!

Inside Dexter: How files work

[screenshot]

There are 3 stages to file handing in Dexter: sourcing, transmission, and consumption.

Sourcing

In order to get files into a Dexter App, you first need to source them. Right now, that means using a provider-specific function to generate a list of one or more files (Dropbox: get folder contents, AWS S3: get logs, etc.).

These source modules have 2 jobs:

1. They establish a specific provider in the App. For example, a Dropbox source module will force the App user to authenticate with Dropbox before they can use the app. Because this source module has force the authentication, other modules in the system don't need to ask for Dropbox authentication - it's already baked into the App! This lets consumers be provider agnostic.
2. They normalize file provider output into a universally understood Dexter file pointer.

In this example app, we're using a Dropbox-specific module to get a list of files. By doing so, we've guaranteed that the App has the proper credentials required to access a user's Dropbox account, and we've created a list of files that the system can recognize as having been sourced from Dropbox. How? Well, that's a matter of...

Transmission

Files are moved around a Dexter app by means of a Dexter file pointer:

{
    source: 'dropbox',
    path: '/foo/bar',
    id: '/foo/bar',,
    size: 3920235
}

Dexter's built-in tools recognize this structure as a reference to an external file. The source tells it which provider owns the file, the id gives it the reference it needs to interact with the file, and the other values can be used to filter, log, or otherwise interact with the file without hitting any additional API endpoints.

The files outputted from our Dropbox Photos module are then easily recognizable as having come from Dropbox, and any files-aware module will be able to manage the file in Dropbox using the information given. In the future, you'll have access to other file-type providers like Amazon's S3, Google Drive, Box, and more - each of which will share this common file pointer structure. That means apps can deal with files generically, unless they have a need for...

Consumption

Unless a consumer wants to do something special with a module, there's no need for it to know exactly where a file came from. So long as the file was correctly sourced from a supported provider, a consumer simply needs to ask Dexter to generate a Buffer, which it can then use as needed:

dexter.files.get({
    source: 'dropbox',
    path: '/foo/bar',
    id: '/foo/bar',,
    size: 3920235
}).then(function(buffer) {
    //...
});

Alternatively, if there's something special you want to do with files from a particualr filesystem (like attaching a file to an Evernote note, or exporting a Google Drive document as a PDF), all the information you need to internact with that particular system is in the pointer.

Note that unless you need special access to a particular provider's SDK to perform a task, you don't need to explicitly ask for file providers' OAuth credentials in order to work with their files. Basic operations are wrapped up and handled transparently by the Dexter module wrapper, so getting (and soon moving, creating, or deleting) files can be done without explicit authorization - Dexter takes care of it for you. However, if you want to use the Google SDK to convert a Doc to a PDF, you'll need to explicitly ask for Google OAuth credentials so you can configure the SDK.

Further modificiations

As with all Dexter Apps, if you ALMOST love "Dropbox to Google Photos", you're free to remix and customize it as necessary! To get started, just click the Remix button on the App's page. This will give you your own copy of the App which you can modify however you want! Here's a few examples of what you can do:

Only upload JPGs

If your folders have a mix of JPG and GIF, you might not want the GIFs to be a part of your Google Photos album. After all, Nyan Cat and your pictures of Mr. Scruffles' birthday shouldn't be allowed to mix! What you need is a filter for your files.

This is a simple change - all we need to do is stick a special filter module in the middle of our App. Here's how:

1. In the left-hand menu, switch to Modules, then click the big + in the lower-right corner to add a new module. Search for "file filter"...one of the first results should be "Files: Extension Filter". Add that to your App and click Done.
2. Drag the new module onto the canvas, dropping it somwhere between the Dropbox and Google Photos modules (you don't have to be exact)
3. Drag the connector from Dropbox's output to your new module's input, then drag the new module's output to Google Photos' input
4. Configure your new module: drag Dropbox's photos output to the files input, and set extensions to be manually entered. Type in "jpg, jpeg".
5. Configure your Google Photos module: drag the Filter's files output to the photos input.

That should be it! No more Nyan Cat spoiling the precious memories of kitty's party.

Make sure the album exists

Right now, the App will fail if the album you assign to the Google Photos module doesn't exist. If you want to create the album before the migration begins, you'll need to add another module to the app. We can fix that by adding in a module that creates an album if it doesn't already exist.

To make this change, we'll do 2 things. First, we'll add and wire up the new module, just like we did for the previous two changes. Second, we'll turn the album name into an environment variable so it can be reused. Why do this? Both the album creation and the photo upload modules want an album name as an input parameter. We could just set both inputs to be entered by the user, but then the user would have to enter the same album name twice. That's both confusing and annoying. By setting up a user-generated environment varaible, then using that environment variable for both inputs, we make our app easier to understand and easier to use.

1. In the left-hand menu, switch to Modules, then click the big + in the lower-right corner to add a new module. Search for "create album"...one of the first results should be "Google Photos: Create Album". Add that to your App and click Done.
2. Drag the new module onto the canvas, dropping it above your Dropbox module.
3. Drag the trigger's output to the new module's input, and the new module's output to Dropbox's input
4. Switch to "Settings" and add a new variable called "Album name". Set it to prompt the user for a value.
5. Configure your new module: set album to be come from your "Album name" environment variable
6. Configure Add to Album: change album to come from the visual editor, then use the visual editor to wire it up to your new environment variable

Now you don't have to try to pre-create an album before using your app!

Email yourself a log

If you're dealing with a lot of pictures, you might want to keep track of which pictures are new to your album. To do this, we can set up the App to report the list of uploaded files when it's done.

To do this, we'll add 2 new modules: one to turn our file pointers into useable data, and another to send out the email log.

1. In the left-hand menu, switch to Modules, then click the big + in the lower-right corner to add a new module. Search for "file metadata"...the first results should be "Files: Metadata". Add that to your App and click Done.
2. Drag the new module onto the canvas, dropping it after your Google Photos module.
3. Drag the Google Photos output to the new module's input.
4. Configure your new module: use the visual editor to hook the output of your Dropbox module up to the module's lone input
5. Drag a Dexter Emailer module onto the canvas and wire it up after your metadata module
6. Configure the emailer by mapping the "path" output of the metadata module to "HTML body", explicitly setting the subject to "Google Photo upload results", and prompting the user to enter a value for "To"

That's it! Now, when the app is done, the user will get an email detailing the files that were uploaded.

ALL THE THINGS!!!

Rather than bore you with a mile-long list combining all the above features, we'll just leave this here instead. Have fun!