Skip to content

Instantly share code, notes, and snippets.

@dergachev

dergachev/blackfire-outline-drupalcamp-nyc.md Secret

Last active May 7, 2016 19:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dergachev/a90e8b8b668a686f91b55bd37a0332de to your computer and use it in GitHub Desktop.
Save dergachev/a90e8b8b668a686f91b55bd37a0332de to your computer and use it in GitHub Desktop.
Download ZIP
Raw
blackfire-outline-drupalcamp-nyc.md

High level overview

  • A - Intro + motivation
  • A - Profiling methodology + philosophy
  • D - Demo 1: Coursecal
  • D - Blackfire tour
  • A - Blackfire basics: Terminology, advantages, installation
  • A - Blackfire features (basic + intermediate)
  • D - Demo 2: TQ, copy as curl
  • A - Blackfire + Drupal tricks
  • V - Demo 3: Block visibility
  • A - If time allows: General Drupal performance tips
  • Q & A

Intro

  • Title Slide
  • About Evolving Web
  • Outline
  • Why Profiling is important
  • UX, $, concurrency + scalability
  • What we aren't measuring
    • loading, front-end, browser rendering
  • What we are measuring
    • profiling is like xray glasses {IMG} for your code
    • what profiling shows
    • page generation time, CPU + memory info
    • blocking operations: SQL + external requests
  • Example scenario of Drupal slowness
  • Blackfire.io is a great PHP profiling tool
  • Profiling gets DRAMATIC results, pretty fast:
    • Demo 1: McGill Course Calendar
      • didn't use node_load_multiple
      • 260ms -> 225ms, 13%
      • took 1h to locate and fix a problem
    • Demo 2: Client X
      • slow redirect (390ms -> 95ms), took an hour to diagnose and fix
      • references_dialog old buggy version
        • 1s to 770ms (23%)
        • took 30m to diagnose, instant fix
      • uncached menu
        • 770ms to 480ms
        • took 2 hours to diagnose, several hours to fix
      • uncached takes a few sec (?) to load
      • Took a day to diagnose and fix these PARTIAL problems, on an unfamiliar codebase
    • Demo 3: D8 evolvingweb.ca site
      • block visibility (80ms out of 450ms, 18%)
      • metatag module patch (saves 30ms)
      • took 2 hours to identify problems, + 2 days to fix
    • Linux Foundation / AllSeen Alliance CAWT
      • Views handling of revisions inefficient; runs entity_load on each one
      • used xdebug + reading code to figure out why
      • Took 3 hours to diagnose + add revision cache
      • 980ms -> 420ms

Methodology

  • Identify performance goals: what does it mean to be fast?

    • VS other sites
    • VS user expectations
    • Isolate front-end from back-end
    • Understand cached vs uncached behavior
    • why varnish isn't enough

  • Define behavior externally (path, logged in, environment, isolation, caching...)

  • Study call graph to get a diagnosis

  • Log your runs, later it will be hard to remember all you've changed

  • Look for any low hanging fruit, bottleneck

    • (easily cachable requests, bad SQL, blocking requests, unecessary entity loads, watchdog,...)

  • Look for signs of overall sluggishness (eg swapping, hard-drive contention, network issues, slow/shared server, lack of APC)

  • Build a hypothesis on the bottleneck

  • Log the scenario, mark it as a reference (baseline)

  • Make a change, do a comparison

    • In drupal, static caching means removing "slow" code just pushes it to later in request

  • Iteration

  • Know when to stop profiling

    • compare variations: pages, site, server env, enable/disable modules, comment out code

  • Tools {INLINE THIS?}

  • Chrome dev tools (YSlow, GTMetrix, WebPageTest.org, Google PageSpeed Insights)

  • ab (Apache bench)

  • devel sql query log

  • APM: newrelic/zen

  • xhprof / blackfire

  • Segue to blackfire

    • Why backend is important
      • Drupal core is not exactly lightweight, contrib varies, custom + legacy code
      • We deal with many projects, working on slow ones makes me sad
    • Why blackfire
      • free*, easy to install
      • intuitive GUI and process (comparisions, collaboration)
      • does the job

Blackfire Basics: Demo 1 node_load_multiple

  • Intro (Why blackfire)
  • blackfire UI tour
  • How to read a call-graph
  • exclusive vs inclusive
  • WALL = CPU + IO (disk, network);
  • Memory
  • SQL + HTTP requests (premium only; otherwise use devel query log)
  • callers and callees
  • problem description
  • solution patch

Understanding Blackfire

  • Blackfire.io

    • what it does, who it's by
    • advantages over xhprof
      • distribution (docker / chef / ansible), embedded (magento cloud, heroku)
      • SAAS nature
      • interactive callgraph, better user experience
      • don't display each node, just >1%
      • actively maintained... support for PHP 7
      • safer for production instrumentation
      • documentation, installation

  • Installing blackfire

    • Blackfire PHP C extension "probe", agent
    • Companion (Chrome extension), command-line client
    • Install steps on ubuntu 
      wget -O - https://packagecloud.io/gpg.key | sudo apt-key add -
echo "deb http://packages.blackfire.io/debian any main" | sudo tee /etc/apt/sources.list.d/blackfire.list
sudo apt-get update
sudo apt-get install blackfire-agent blackfire-php
# fill in server-id and server-token
sudo blackfire-agent --register
sudo /etc/init.d/blackfire-agent start

# for command-line use, fill in client-id and client-token
blackfire config

# disable xhprof and xdebug php extensions
# restart apache or php-fpm
    • More info in Blackfire Install Docs
      • includes instructions for OSX, Red hat, Windows, docker, chef, ...

  • Advanced features

    • Comparison
    • Copy as curl (ajax, cookies, POST requests)
    • profiling command-line / drush commands
      • (drush.launcher)

CASE STUDY 2: Client X / Redirect

  • TQ redirection
  • copy-as-curl (good for redirects, POST, AJAX)
  • solution: moving from _preprocess_page to hook_init
  • comparison profile
  • learning about a foreign codebase
  • when you find one issue, you might find more

Advanced

  • aggregation (10 requests, averaged)

    • Turn aggregation to control for caching and side effects

  • Blackfire doesn't keep arguments (or 1 at most)

  • Sampling, not tracing!

  • sharing profiles with your team, persistence

  • use blackfire to learn new codebase (contrib)

  • xdebug conflict + necessity

  • Blackfire PHP SDK

  • Tradeoff: memory vs time

  • Caching and dirty runs

    • D7 + D8 cache killing

  • diagnostic technques

    • references / comparison
    • xdebug
    • enableProbe / disableProbe
    • argument capturing

CASE STUDY 3: evolvingweb.ca on D8

  • Problem: why is core D8 slower than D7?
    • Few contrib or custom modules, just core + config + theme
  • Diagnosis: block visiblity
  • Solution: block_acces_records module

Pro Blackfire features

  • environments
    • groups of profiles and team members
    • one for dev / stage / prod, per project
  • data retention
  • CI + scenarios + notifications
    • trigger via web service
    • slack integration, etc.
  • assertions
  • custom metrics
  • recommendations

Generic Drupal Backend Tips

  • know these: varnish/memcache/APCu/Opcache
    • memcache only helps speed up cache_set/ cache_get and overall load on DB
  • D8 render cache with tags + context
  • D7 vs D8 (complexity Vs caching)
  • Number of contrib modules
  • Mysql tuning http://www.jeffgeerling.com/articles/web-design/2010/drupal-performance-white-paper
  • Cron job, search, watchdog, SSD, multiple app heads, CDN, php7, fpm, nginx for files
  • cookies + page cache
  • devel web profiler
  • entity_load_multiple()

Following Up

  • Please provide feedback on our session
  • Join us for Code Sprints
    • Friday, May 13 at the Convention Center
    • First-Time Sprinter Workshop - 9am-12pm in Room 271-273
    • Mentored Core Sprint - 9am-6pm in Room 275-277
    • General Sprints - 9am-6pm in Room 278-282
  • Follow @dergachev and @djvasi on twitter
    • write us for help
  • Go to blackfire booth to pickup book
    • Blackfire.io coupon - DRUPALNOLA
  • Questions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment