public
Created

TLC Transients — On the fly WordPress transients with soft expiration and one-liner chained syntax.

  • Download Gist
README.md
Markdown

TLC Transients

A WordPress transients interface with support for soft-expiration (use old content until new content is available), background updating of the transients (without having to wait for a cron job), and a chainable syntax that allows for one liners.

Examples

In this simple example, we're defining a feed-fetching callback, and then using tlc_transient with a chain to point to that callback and use it, all in one line. Note that since we haven't used background_only(), the initial load of this will cause the page to pause.

<?php>
// Define your callback (other examples use this)
function my_callback() {
    return wp_remote_retrieve_body(
        wp_remote_get( 'http://example.com/feed.xml', array( 'timeout' => 30 ) )
    );
}

// Grab that feed
echo tlc_transient( 'example-feed' )
    ->updates_with( 'my_callback' )
    ->expires_in( 300 )
    ->get();
?>

This time, we'll set background_only() in the chain. This means that if there has been a hard cache flush, or this is the first-ever request, it will return false. So your code will have to be written to gracefully degrade if the feed isn't yet available. This, of course, triggers a background update. And once it is available, it will start returning the content.

<?php
echo tlc_transient( 'example-feed' )
    ->updates_with( 'my_callback' )
    ->expires_in( 300 )
    ->background_only()
    ->get();
?>

We don't have to chain, of course.

<?php
$t = tlc_transient( 'example-feed' );
if ( true ) {
    $t->updates_with( 'my_callback' );
} else {
    $t->updates_with( 'some_other_callback' );
}

$t->expires_in( 300 );
echo $t->get();
?>

We can even pass parameters to our callback.

<?php
// Define your callback
function my_callback_with_param( $param ) {
    return str_replace(
        'foo',
        $param,
        wp_remote_retrieve_body( wp_remote_get( 'http://example.com/feed.xml', array( 'timeout' => 30 ) ) ),
    );
}

// Grab that feed
echo tlc_transient( 'example-feed' )
    ->updates_with( 'my_callback_with_param', array( 'bar' ) )
    ->expires_in( 300 )
    ->background_only()
    ->get();
?>
tlc-transients.php
PHP
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162
<?php
 
class TLC_Transient_Update_Server {
public function __construct() {
add_action( 'init', array( $this, 'init' ) );
}
 
public function init() {
if ( isset( $_POST['_tlc_update'] ) ) {
$update = get_transient( 'tlc_update__' . $_POST['key'] );
if ( $update && $update[0] == $_POST['_tlc_update'] ) {
tlc_transient( $update[1] )->expires_in( $update[2] )->updates_with( $update[3], (array) $update[4] )->set_lock( $update[0] )->fetch_and_cache();
}
exit();
}
}
}
 
new TLC_Transient_Update_Server;
 
if ( !class_exists( 'TLC_Transient' ) ) {
class TLC_Transient {
public $key;
private $lock;
private $callback;
private $params;
private $expiration = 0;
private $force_background_updates = false;
 
public function __construct( $key ) {
$this->key = $key;
}
 
public function get() {
$data = get_transient( $this->key );
if ( false === $data ) {
// Hard expiration
if ( $this->force_background_updates ) {
// In this mode, we never do a just-in-time update
// We return false, and schedule a fetch on shutdown
$this->schedule_background_fetch();
return false;
} else {
// Bill O'Reilly mode: "We'll do it live!"
return $this->fetch_and_cache();
}
} else {
// Soft expiration
if ( $data[0] !== 0 && $data[0] < time() )
$this->schedule_background_fetch();
return $data[1];
}
}
 
private function schedule_background_fetch() {
if ( !$this->has_update_lock() ) {
set_transient( 'tlc_update__' . $this->key, array( $this->new_update_lock(), $this->key, $this->expiration, $this->callback, $this->params ) );
add_action( 'shutdown', array( $this, 'spawn_server' ) );
}
return $this;
}
 
public function spawn_server() {
$server_url = home_url( '/?tlc_transients_request' );
wp_remote_post( $server_url, array( 'body' => array( '_tlc_update' => $this->lock, 'key' => $this->key ), 'timeout' => 0.01, 'blocking' => false, 'sslverify' => apply_filters( 'https_local_ssl_verify', true ) ) );
}
 
public function fetch_and_cache() {
// If you don't supply a callback, we can't update it for you!
if ( empty( $this->callback ) )
return false;
if ( $this->has_update_lock() && !$this->owns_update_lock() )
return; // Race... let the other process handle it
$data = call_user_func_array( $this->callback, $this->params );
$this->set( $data );
$this->release_update_lock();
return $data;
}
 
public function set( $data ) {
// We set the timeout as part of the transient data.
// The actual transient has no TTL. This allows for soft expiration.
$expiration = ( $this->expiration > 0 ) ? time() + $this->expiration : 0;
set_transient( $this->key, array( $expiration, $data ) );
return $this;
}
 
public function updates_with( $callback, $params = array() ) {
$this->callback = $callback;
if ( is_array( $params ) )
$this->params = $params;
return $this;
}
 
private function new_update_lock() {
$this->lock = md5( uniqid( microtime() . mt_rand(), true ) );
return $this->lock;
}
 
private function release_update_lock() {
delete_transient( 'tlc_update__' . $this->key );
}
 
private function get_update_lock() {
$lock = get_transient( 'tlc_update__' . $this->key );
if ( $lock )
return $lock[0];
else
return false;
}
 
private function has_update_lock() {
return (bool) $this->get_update_lock();
}
 
private function owns_update_lock() {
return $this->lock == $this->get_update_lock();
}
 
public function expires_in( $seconds ) {
$this->expiration = (int) $seconds;
return $this;
}
 
public function set_lock( $lock ) {
$this->lock = $lock;
return $this;
}
 
public function background_only() {
$this->force_background_updates = true;
return $this;
}
}
}
 
// API so you don't have to use "new"
function tlc_transient( $key ) {
$transient = new TLC_Transient( $key );
return $transient;
}
 
// Example:
 
function sample_fetch_and_append( $url, $append ) {
$f = wp_remote_retrieve_body( wp_remote_get( $url, array( 'timeout' => 30 ) ) );
$f .= $append;
return $f;
}
 
function test_tlc_transient() {
$t = tlc_transient( 'foo' )
->expires_in( 30 )
->background_only()
->updates_with( 'sample_fetch_and_append', array( 'http://coveredwebservices.com/tools/long-running-request.php', ' appendfooparam ' ) )
->get();
var_dump( $t );
if ( !$t )
echo "The request is false, because it isn't yet in the cache. It'll be there in about 10 seconds. Keep refreshing!";
}
 
add_action( 'wp_footer', 'test_tlc_transient' );

Don't you think it's time to transform this into a proper git repo?

That way, you can use the fancy new ACE editor to edit files in the browser. ;)

I only just put it up here! :-) Hadn't even told anyone about it yet. Yeah, it'll go into a proper GH repo soon! In the meantime, what do you think?

It looks pretty neat. It could also be done via a one-time wp_cron job, no?

Would this be compatible with plugins that replace WordPress' cache with persistent cache systems like Xcache?

Yep, since it only uses the transients API.

Love it. It would be awesome to have the ability to sleep and try get the cached data again if the lock is already owned by another request for items that are required.

I'd honestly love to see this functionality in core. Until then though I'll test this out some in a couple of my plugins that could really use it.

@scribu — yeah, it could have used a cron job. But cron jobs are annoying and flaky. I had an early version using cron jobs, but it wasn't as dependable. Also, the initial sync takes longer, because cron only runs once a minute.

What happens if the updates_with function can't get the data it needs? Like if it relies on a web service that's down? It seems that the data is still set, just set to null or false or whatever the updates_with function returns on failure. What do you think about allowing a way for that function to return something that will keep the data from being set? Maybe null or false, or even allow for an exception to be thrown?

Something like this:

try {
    $data = call_user_func_array( $this->callback, $this->params );
    $this->set( $data );
} catch( Exception $e ) {}

Mark,

I'm working with a large site containing 4,000 videos (cpt). The client is asking to track hits on single videos within a two week period for use with displaying a "What's Popular Now" widget. What are your thoughts performance wise using a transient approach to track hits stored as such day->video_id->hits. Day transients would expire after two weeks. Thanks for the great writeup.

@aaroncampbell — see the update. Like that?

@inspectorfegter — This should probably not be used for stuff that changes frequently. The way it works is that it essentially "forks" and runs each update in its own process. This is great for long-running updates that happen less frequently than once a minute. Transients should never be used as primary storage for any data, as they can go away at any time. For your hit counter, if it's not super-critical that the information be 100% accurate, you could store it in a transient and then write the transient value to a permanent storage location if the number of hits is divisible by 10, or 20, or 50, etc. That way you're not constantly writing to a permanent storage location. And if you unexpectedly lose the transient, you repopulate from the master, and at worst you lose the contents of the hit buffer.

Wouldn't returning a WP_Error be more WordPress like for avoiding the cache update? Especially cases where you may normally receive a WP_Error result anyway.

Not only that, but if the update failed for whatever reason, not updating the timestamp on the current transient makes the site keep trying constantly. There needs to be a way to say don't try again for x-time when failing.

Thanks Mark. That's exactly what I'm looking for.

@markjaquith - Yep, that's exactly what I did.

@prettyboymp - I suppose WP_Error would the "WordPress way". I feel like WP_Error is sort of the poor mans Exceptions because we weren't able to use Exceptions. I can see your point though, especially in the cases where you can do something like use wp_remote_request() directly with updates_with(). It means some errors are handled automatically by WordPress. However, not having your own function between the two means you can't handle things like empty (but successful) responses, invalid JSON or XML, etc.

As for a timeout for failed attempts, I suppose that would make sense but should probably be different from the expires_in, so you could set expires_in(30)->retry_in(5) to make it retry 5 minutes after a failed update and 30 minutes after a successful one.

@aaroncampbell — expires_in() takes a seconds input, FYI. :-) I like the retry_in() idea.

@markjaquith - Yeah, my examples were bad...in my plugin I'm using it right though :-)

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.