Skip to content

Instantly share code, notes, and snippets.

@robertmarsal
Forked from mtowers/README.md
Last active March 20, 2022 02:26
Show Gist options
  • Star 28 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
  • Save robertmarsal/9546339 to your computer and use it in GitHub Desktop.
Save robertmarsal/9546339 to your computer and use it in GitHub Desktop.
Google Analytics Real Time Visitors Widget for Dashing with OAuth2 Authentication

Description

A Dashing widget for displaying the number of current visitors (in real time) to your website, as reported by Google Analytics.

This widget is a fork of https://gist.github.com/mtowers/5986576

screen shot

Dependencies

google-api-ruby-client

Add it to dashing's gemfile:

gem 'google-api-client'

and run bundle install.

Usage

To use this widget, you'll first need to set up a Google API project and attach it to the Google Analytics profile you wish to monitor, as well as request access to the Google Analytics Real Time API.

1. Request acces to the Google Analytics Real Time API

  1. Go to http://goo.gl/7ZPlMI and fill the form

2. Create and download a new private key for Google API access.

  1. Go to https://code.google.com/apis/console
  2. Click 'Create Project'
  3. Enable 'Analytics API' service and accept both TOS's
  4. Click 'API Access' in the left-hand nav menu
  5. Click 'Create an OAuth 2.0 Client ID'
  6. Enter a product name (e.g. Dashing Widget) - logo and url are optional
  7. Click 'Next'
  8. Under Application Type, select 'Service Account'
  9. Click 'Create Client ID'
  10. Click 'Download private key' NOTE: This will be your only opportunity to download this key.
  11. Note the password for your new private key ('notasecret')
  12. Close the download key dialog
  13. Find the details for the service account you just created and copy it's email address which will look something like this: 210987654321-3rmagherd99kitt3h5@developer.gserviceaccount.com - you'll need it in your ruby code later

3. Attach your Google API service account to your Google Analytics profile

Note: you will need to be an administrator of the Google Analytics profile

  1. Log in to your Google Analytics account: http://www.google.com/analytics/
  2. Click 'Admin' in the upper-right corner
  3. Select the account containing the profile you wish to use
  4. Select the property containing the profile you wish to use
  5. Select the profile you wish to use
  6. Click the 'Users' tab
  7. Click '+ New User'
  8. Enter the email address you copied from step 13 above
  9. Click 'Add User'

4. Locate the ID for your Google Analytics profile

  1. On your Google Analytics profile page, click the 'Profile Settings' tab
  2. Under 'General Information' copy your Profile ID (e.g. 654321) - you'll need it in your ruby code later

5. Start coding (finally)

  1. Copy the visitor_count.rb file in to your dashing jobs\ folder.
  2. Update the service_account_email, key_file, key_secret and profileID variables
    service_account_email = '[YOUR SERVICE ACCOUTN EMAIL]' # Email of service account
    key_file = 'path/to/your/keyfile.p12' # File containing your private key
    key_secret = 'notasecret' # Password to unlock private key
    profileID = '[YOUR PROFILE ID]' # Analytics profile ID.
  1. Add the widget HTML to your dashboard
    <li data-row="1" data-col="1" data-sizex="1" data-sizey="1"> 
        <div data-id="visitor_count_real_time" data-view="Graph" data-title="Current Visitors"></div> 
    </li>

Notes

If you want to modify this plugin to pull other data from Google Analytics, be sure to check out the Google Analytics Query Explorer.

require 'google/api_client'
require 'date'
# Update these to match your own apps credentials
service_account_email = '[YOUR SERVICE ACCOUNT EMAIL]' # Email of service account
key_file = 'path/to/your/keyfile.p12' # File containing your private key
key_secret = 'notasecret' # Password to unlock private key
profileID = '[YOUR PROFILE ID]' # Analytics profile ID.
# Get the Google API client
client = Google::APIClient.new(
:application_name => '[YOUR APPLICATION NAME]',
:application_version => '0.01'
)
visitors = []
# Load your credentials for the service account
key = Google::APIClient::KeyUtils.load_from_pkcs12(key_file, key_secret)
client.authorization = Signet::OAuth2::Client.new(
:token_credential_uri => 'https://accounts.google.com/o/oauth2/token',
:audience => 'https://accounts.google.com/o/oauth2/token',
:scope => 'https://www.googleapis.com/auth/analytics.readonly',
:issuer => service_account_email,
:signing_key => key)
# Start the scheduler
SCHEDULER.every '5s', :first_in => 0 do
# Request a token for our service account
client.authorization.fetch_access_token!
# Get the analytics API
analytics = client.discovered_api('analytics','v3')
# Execute the query
visitCount = client.execute(:api_method => analytics.data.realtime.get, :parameters => {
'ids' => "ga:" + profile_id,
'metrics' => "ga:activeVisitors",
})
visitors << { x: Time.now.to_i, y: visits.to_i }
# Update the dashboard
send_event('visitor_count_real_time', points: visitors)
end
@manishval
Copy link

Updated version:

require 'google/api_client'
require 'date'

# Update these to match your own apps credentials
service_account_email = ENV['GOOGLE_SERVICE_ACCOUNT_EMAIL'] # Email of service account
key_file = 'path/to/key.p12' # File containing your private key
key_secret = 'notasecret' # Password to unlock private key
profile_id = ENV['GOOGLE_ANALYTICS_PROFILE_ID'] # Analytics profile ID.

# Get the Google API client
client = Google::APIClient.new(
  :application_name => ENV['GOOGLE_APPLICATION_NAME'],
  :application_version => '0.01'
)

visitors = []

# Load your credentials for the service account
key = Google::APIClient::KeyUtils.load_from_pkcs12(key_file, key_secret)
client.authorization = Signet::OAuth2::Client.new(
  :token_credential_uri => 'https://accounts.google.com/o/oauth2/token',
  :audience => 'https://accounts.google.com/o/oauth2/token',
  :scope => 'https://www.googleapis.com/auth/analytics.readonly',
  :issuer => service_account_email,
  :signing_key => key)

# Start the scheduler
SCHEDULER.every '5s', :first_in => 0 do

  # Request a token for our service account
  client.authorization.fetch_access_token!

  # Get the analytics API
  analytics = client.discovered_api('analytics','v3')

  # Execute the query
  response = client.execute(:api_method => analytics.data.realtime.get, :parameters => {
    'ids' => "ga:" + profile_id,
    'metrics' => "ga:activeVisitors",
  })

  visitors << { x: Time.now.to_i, y: response.data.rows }

  # Update the dashboard
  send_event('visitor_count_real_time', points: visitors)
end

@acheloos-m
Copy link

Just tried it, but i'm somehow struggelin with the o_auth.. it gives the following error:

scheduler caught exception: Authorization failed. Server message: { "error" : "invalid_request", "error_description" : "Required parameter is missing: grant_type" } /usr/local/lib/ruby/gems/2.0.0/gems/signet-0.5.0/lib/signet/oauth_2/client.rb:883:in fetch_access_token'
/usr/local/lib/ruby/gems/2.0.0/gems/signet-0.5.0/lib/signet/oauth_2/client.rb:898:in fetch_access_token!' /home/mheyne/bidash/jobs/visitor_count_real_time.rb:31:in block in <top (required)>'
/usr/local/lib/ruby/gems/2.0.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in call' /usr/local/lib/ruby/gems/2.0.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in trigger_block'
/usr/local/lib/ruby/gems/2.0.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:204:in block in trigger' /usr/local/lib/ruby/gems/2.0.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in call'
/usr/local/lib/ruby/gems/2.0.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in block in trigger_job'

Any experiences with that one? I followed the steps as written..

Cheers

@claylevering
Copy link

This looks like this might an issue with the fact that Google is currently considering this realtime analytics to be invitation only Beta. Womp womp. I'm getting the same thing:

scheduler caught exception:
Authorization failed.  Server message:
{
  "error" : "invalid_grant"
}
/home/ubuntu/.rvm/gems/ruby-2.1.1/gems/signet-0.5.0/lib/signet/oauth_2/client.rb:883:in `fetch_access_token'
/home/ubuntu/.rvm/gems/ruby-2.1.1/gems/signet-0.5.0/lib/signet/oauth_2/client.rb:898:in `fetch_access_token!'
/home/ubuntu/sweet_dashboard_project/jobs/visitor_count_real_time.rb:29:in `block in <top (required)>'
/home/ubuntu/.rvm/gems/ruby-2.1.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in `call'
/home/ubuntu/.rvm/gems/ruby-2.1.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in `trigger_block'
/home/ubuntu/.rvm/gems/ruby-2.1.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:204:in `block in trigger'
/home/ubuntu/.rvm/gems/ruby-2.1.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in `call'
/home/ubuntu/.rvm/gems/ruby-2.1.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in `block in trigger_job'

@bhaku
Copy link

bhaku commented Apr 11, 2014

The same problem as acheloos-m ;/

@lynnaloo
Copy link

lynnaloo commented Aug 1, 2014

I am getting this message after it has been running a while: {"error":"server_error","error_description":"JWT has expired."}. Shouldn't the fetch_access_token method be getting a new token every time this runs?

@thorsten
Copy link

thorsten commented Sep 2, 2014

Works greate except there's no graph visible for us. Any hints? The embedded SVG shows only the coordinate numbers...

@thorsten
Copy link

Okay, Update after 3 weeks of usage:

  • Graph renders after a day
  • Dashboard is getting slower and slower every day as all realtime informations are stored and calculated in the widget
  • Everything else is awesome :)

@EZPK
Copy link

EZPK commented Sep 30, 2014

Do you have an explanation of why it takes a day to render ? I'm having some issues with the axe Y data and I think it may be because of that.

@bobthemagicman
Copy link

Was there any update on the "invalid_grant" situation? I just finished getting it all setup and am seeing the same error mentioned above.

@EwanValentine
Copy link

Also getting "invalid_grant" any response on this? Also:
"copy the visitor_count.rb file in to your dashing jobs\ folder." - There is no file with that name included in the gist? Should that not be visitor_count_real_time.rb? Also, the first comment claims to be the updated version, is that official??

@liam1027
Copy link

liam1027 commented Nov 8, 2014

I'm having trouble getting dashing to start after adding this widget. I get the following error:

/var/lib/gems/1.9.1/gems/backports-3.6.3/lib/backports/std_lib.rb:9:in `require': cannot load such file -- google/api_client (LoadError)

I ran gem install google-api-client then bundle install. If I do gem list, I can see google-api-client listed, but if I do bundle update, I don't see it there.

I'm a ruby noob, so I'm not really sure how to fix this.

@thopit
Copy link

thopit commented Dec 3, 2014

The dashboard is getting slower after continued usage, because old values are never deleted.
I would suggest adding a variable at the top of the file:

max_amount = 100

as well as adding after this line

visitors << { x: Time.now.to_i, y: response.data.rows }

the following code:

if visitors.size > max_amount
    visitors.shift
end

So the first added element is removed.

@tcope25
Copy link

tcope25 commented Dec 14, 2014

I'm still getting 'invalid_grant' response errors. Anyone know how to fix this?

@franciscoruiz
Copy link

For those not being granted access:

The Real Time Reporting API is currently available as a developer preview in limited beta. If you're interested in signing up, request access to the beta.

Extracted from the Real Time Reporting API overview

@codeArtist2015
Copy link

This plugin works for me, but the problem I have is that it stops reporting the number of active users after 5 minutes. Any ideas how to fix this ?

@sxslex
Copy link

sxslex commented Apr 12, 2017

What is wrong? How to solve?

MINGW64 /c/Teste
$ ruby -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [i386-mingw32]

MINGW64 /c/Teste
$ gem environment
RubyGems Environment:
  - RUBYGEMS VERSION: 2.6.11
  - RUBY VERSION: 2.3.3 (2016-11-21 patchlevel 222) [i386-mingw32]
  - INSTALLATION DIRECTORY: C:/Ruby23/lib/ruby/gems/2.3.0
  - USER INSTALLATION DIRECTORY: C:/Users/marcelovilela.mg/.gem/ruby/2.3.0
  - RUBY EXECUTABLE: C:/Ruby23/bin/ruby.exe
  - EXECUTABLE DIRECTORY: C:/Ruby23/bin
  - SPEC CACHE DIRECTORY: C:/Users/marcelovilela.mg/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: C:/ProgramData
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-mingw32
  - GEM PATHS:
     - C:/Ruby23/lib/ruby/gems/2.3.0
     - C:/Users/marcelovilela.mg/.gem/ruby/2.3.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - C:\Users\marcelovilela.mg\bin
     - C:\Program Files\Git\mingw64\bin
     - C:\Program Files\Git\usr\local\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\mingw64\bin
     - C:\Program Files\Git\usr\bin
     - C:\Users\marcelovilela.mg\bin
     - C:\Ruby23\bin
     - C:\ProgramData\Oracle\Java\javapath
     - C:\Program Files\Common Files\Microsoft Shared\Windows Live
     - C:\Windows\system32
     - C:\Windows
     - C:\Windows\System32\Wbem
     - C:\Windows\System32\WindowsPowerShell\v1.0
     - C:\Program Files\Intel\DMIX
     - C:\Program Files\TortoiseSVN\bin
     - C:\python26
     - C:\python26\scripts
     - C:\cheetah-2.4.4\bin
     - C:\Program Files\nodejs
     - C:\Program Files (x86)\Skype\Phone
     - C:\Program Files\Git\cmd
     - C:\Users\marcelovilela.mg\AppData\Roaming\npm
     - C:\Python27\Scripts
     - C:\Python27
     - C:\Program Files\Git\usr\bin\vendor_perl
     - C:\Program Files\Git\usr\bin\core_perl

MINGW64 /c/Teste
$ gem list google-api-client -d
google-api-client (0.11.1)
    Authors: Steven Bazyl, Tim Emiola, Sergio Gomes, Bob Aman
    Homepage: https://github.com/google/google-api-ruby-client
    License: Apache 2.0
    Installed at: C:/Ruby23/lib/ruby/gems/2.3.0

    Client for accessing Google APIs

MINGW64 /c/Teste
$ ruby visitor_count.rb
C:/Ruby23/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- google/api_client (LoadError)
        from C:/Ruby23/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from visitor_count.rb:1:in `<main>'

@davidkendrickmbll
Copy link

davidkendrickmbll commented May 2, 2017

We recently implemented this widget on our dashboard and applied some modifications.

We added the ability to set the timezone (activesupport) as our server is set to UTC.

We are storing 12 hours of results at 1 per minute instead of 1 per second (max 720) in memcached (dalli) so if the service is restarted it retains the graph data.

Add to smashings gem file:

gem 'activesupport'
gem 'dalli'

and run bundle install

require 'google/api_client'
require 'date'
require 'active_support/all'
require 'dalli'

Time.zone = '[YOUR TIMEZONE]'

# Update these to match your own apps credentials
service_account_email = '[YOUR SERVICE ACCOUNT EMAIL]' # Email of service account
key_file = 'path/to/your/keyfile.p12' # File containing your private key
key_secret = 'notasecret' # Password to unlock private key
profile_id = '[YOUR PROFILE ID]' # Analytics profile ID.

# Get the Google API client
client = Google::APIClient.new(:application_name => '[YOUR APPLICATION NAME]', 
  :application_version => '0.01')

# Load your credentials for the service account
key = Google::APIClient::KeyUtils.load_from_pkcs12(key_file, key_secret)
client.authorization = Signet::OAuth2::Client.new(
  :token_credential_uri => 'https://accounts.google.com/o/oauth2/token',
  :audience => 'https://accounts.google.com/o/oauth2/token',
  :scope => 'https://www.googleapis.com/auth/analytics.readonly',
  :issuer => service_account_email,
  :signing_key => key)

visitors = []

# Memcached initialization
options = { namespace: "dashboard", compress: true }
dc = Dalli::Client.new('localhost:11211', options)

# Query visitors data from memcached on start
visitors = dc.get('visitor_count') if visitors.empty?

# Initialize the graph data
send_event('visitor_count_real_time', points: visitors)

# Total number of values to store
max_amount = 720
  
# Start the scheduler
SCHEDULER.every '1m', :first_in => 0 do
  # Request a token for our service account
  client.authorization.fetch_access_token!

  # Get the analytics API
  analytics = client.discovered_api('analytics','v3')

  # Execute the query
  response = client.execute(:api_method => analytics.data.realtime.get, :parameters => {
    'ids' => "ga:" + profile_id,
    'metrics' => "rt:activeUsers",
  })

  # Get the current visit count from the query
  visit_count = response.data.rows[0][0].to_i
  
  visitors << { x: Time.now.to_i, y: visit_count }
  
  # Remove the oldest value if total number is more than allowed
  if visitors.size > max_amount
    visitors.shift
  end
  
  # Store visitors data in memcached
  dc.set('visitor_count', visitors)

  # Update the dashboard
  send_event('visitor_count_real_time', points: visitors)
end

@edbo
Copy link

edbo commented Jul 3, 2018

I've done an update which does a few things:

  • Instead of oauth I created a service account in google auth (Note the private key is base64 encoded which I did to avoid some encoding issues when passing it through environment variables we use Docker).
  • I've set the timer to run every five seconds and store 24 hours of data
  • I've set the query to group by dimensions as we're planning to add a view that shows the most popular pages
require 'base64'
require 'date'
require 'google/apis/analytics_v3'

# Decode the private key
ENV['GOOGLE_PRIVATE_KEY'] = Base64.strict_decode64(ENV['GOOGLE_PRIVATE_KEY']).gsub('\n', "\n")

scopes = %w(https://www.googleapis.com/auth/analytics.readonly)
authorization = Google::Auth::ServiceAccountCredentials.make_creds(scope: scopes)
Google::Apis::RequestOptions.default.authorization = authorization

# Schedule the token refresh
SCHEDULER.every '30m', first_in: 1800 do
  # Refresh the token for our service account
  authorization.fetch_access_token!
end

# Setup services and variables
Analytics = Google::Apis::AnalyticsV3
analytics = Analytics::AnalyticsService.new

dimensions = %w(rt:pagePath)
metrics = %w(rt:activeUsers)
sort = %w(rt:pagePath)

visitors = []
max_amount = 8640

# Schedule the data updating
SCHEDULER.every '10s', first_in: 0 do
  # Get the analytics API
  response = analytics.get_realtime_data("ga:#{ENV['GOOGLE_ANALYTICS_PROFILE_ID']}", 
    metrics.join(','), 
    dimensions: dimensions.join(','),
    sort: sort.join(','))

  visitors << { 
    x: Time.now.to_i,
    y: response.totals_for_all_results['rt:activeUsers'].to_i
  }
  visitors.shift if visitors.size > max_amount

  # Update the dashboard
  send_event('visitor_count_real_time', points: visitors)
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment