Skip to content

Instantly share code, notes, and snippets.

@dasilvaluis
Last active September 27, 2021 09:34
Show Gist options
  • Save dasilvaluis/ebca42b8b8d70e81f8917f675a784060 to your computer and use it in GitHub Desktop.
Save dasilvaluis/ebca42b8b8d70e81f8917f675a784060 to your computer and use it in GitHub Desktop.
Gettext Scanner Gulp Script for Twig Projects
/**
* Gettext Scanner Script for Twig Projects
* v1.3
*
* Developed by Luís Silva
* https://github.com/luism-s
*/
/**
* Purpose:
* Scan Twig and PHP files in the given directories for gettext function calls and output a POT file for translation.
*
* Description:
* While working with Wordpress using the Twig template engine, one might find easier to use gettext
* functions in .twig files for string translation. To simplify the scanning of .twig files for those same functions, this
* script was built to parse .twig files, wrap occurrences of gettext function calls in php tags and
* output the result as a .php file and from that generate a POT file.
* It also scans PHP files as well.
*
* Context: https://github.com/timber/timber/issues/1465
*
* Usage: `gulp pot`
*
* Logic:
* - Iterates over all given .twig files
* - Search and replace for gettext functions in Twig files and wraps them around PHP tags
* - Outputs each file as .php into a cache folder
* - Scan all .php files for gettext functions using 'gulp-wp-pot' (cache included)
* - Generate .pot file
*
* Dependencies:
* `npm install gulp gulp-if del gulp-wp-pot gulp-replace gulp-rename run-sequence`
*
* Warning:
* This script has only ben tested in the context of Wordpress theme development using Timber.
*
* TODO:
* Cover `translate_nooped_plural` function.
*/
const gulp = require('gulp');
const gulpif = require('gulp-if');
const del = require('del');
const wpPot = require('gulp-wp-pot');
const replace = require('gulp-replace');
const rename = require('gulp-rename');
const runSequence = require('run-sequence');
/**
* Configuration Options
*
* All paths are as if this script is
* located in the root of the theme and all the Twig
* files are located under /views
*/
const config = {
"text_domain" : "theme-test", // Replace with your domain
"twig_files" : "views/**/*.twig", // Twig Files
"php_files" : "**/*.php", // PHP Files
"cacheFolder" : "views/cache", // Cache Folder
"destFolder" : "languages", // Folder where .pot file will be saved
"keepCache" : true // Delete cache files after script finishes
};
/**
* __
* _e
* _x
* _xn
* _ex
* _n_noop
* _nx_noop
* translate -> Match __, _e, _x and so on
* \( -> Match (
* \s*? -> Match empty space 0 or infinite times, as few times as possible (ungreedy)
* ['"] -> Match ' or "
* .+? -> Match any character, 1 to infinite times, as few times as possible (ungreedy)
* , -> Match ,
* .+? -> Match any character, 1 to infinite times, as few times as possible (ungreedy)
* \) -> Match )
*/
const gettext_regex = {
// _e( "text", "domain" )
// __( "text", "domain" )
// translate( "text", "domain" )
// esc_attr__( "text", "domain" )
// esc_attr_e( "text", "domain" )
// esc_html__( "text", "domain" )
// esc_html_e( "text", "domain" )
simple: /(__|_e|translate|esc_attr__|esc_attr_e|esc_html__|esc_html_e)\(\s*?['"].+?['"]\s*?,\s*?['"].+?['"]\s*?\)/g,
// _n( "single", "plural", number, "domain" )
plural: /_n\(\s*?['"].*?['"]\s*?,\s*?['"].*?['"]\s*?,\s*?.+?\s*?,\s*?['"].+?['"]\s*?\)/g,
// _x( "text", "context", "domain" )
// _ex( "text", "context", "domain" )
// esc_attr_x( "text", "context", "domain" )
// esc_html_x( "text", "context", "domain" )
// _nx( "single", "plural", "number", "context", "domain" )
disambiguation: /(_x|_ex|_nx|esc_attr_x|esc_html_x)\(\s*?['"].+?['"]\s*?,\s*?['"].+?['"]\s*?,\s*?['"].+?['"]\s*?\)/g,
// _n_noop( "singular", "plural", "domain" )
// _nx_noop( "singular", "plural", "context", "domain" )
noop: /(_n_noop|_nx_noop)\((\s*?['"].+?['"]\s*?),(\s*?['"]\w+?['"]\s*?,){0,1}\s*?['"].+?['"]\s*?\)/g,
};
/**
* Main Task
*/
gulp.task('pot', function(callback) {
runSequence('compile-twigs', 'generate-pot', callback);
});
/**
* Generate POT file from all .php files in the theme,
* including the cache folder.
*/
gulp.task('generate-pot', () => {
const output = gulp.src(config.php_files)
.pipe(wpPot({
domain: config.text_domain
}))
.pipe(gulp.dest(`${config.destFolder}/${config.text_domain}.pot`))
.pipe(gulpif(!config.keepCache, del.bind(null, [config.cacheFolder], { force: true })));
return output;
});
/**
* Fake Twig Gettext Compiler
*
* Searches and replaces all occurences of __('string', 'domain'), _e('string', 'domain') and so on,
* with <?php __('string', 'domain'); ?> or <?php _e('string', 'domain'); ?> and saves the content
* in a .php file with the same name in the cache folder.
*
* Functions supported:
*
* Simple: __(), _e(), translate()
* Plural: _n()
* Disambiguation: _x(), _ex(), _nx()
* Noop: _n_loop(), _nx_noop()
*/
gulp.task('compile-twigs', () => {
del.bind(null, [config.cacheFolder], {force: true})
// Iterate over .twig files
gulp.src(config.twig_files)
// Search for Gettext function calls and wrap them around PHP tags.
.pipe(replace(gettext_regex.simple, match => `<?php ${match}; ?>`))
.pipe(replace(gettext_regex.plural, match => `<?php ${match}; ?>`))
.pipe(replace(gettext_regex.disambiguation, match => `<?php ${match}; ?>`))
.pipe(replace(gettext_regex.noop, match => `<?php ${match}; ?>`))
// Rename file with .php extension
.pipe(rename({
extname: '.php',
}))
// Output the result to the cache folder as a .php file.
.pipe(gulp.dest(config.cacheFolder));
});
{
"name": "twig-gettext-pot-parser",
"version": "1.2.0",
"dependencies": {
"gulp": "3.9.1",
"del": "^3.0.0",
"gulp-if": "^2.0.2",
"gulp-rename": "^1.2.2",
"gulp-replace": "^0.6.1",
"gulp-wp-pot": "^2.2.0",
"run-sequence": "^2.2.1"
},
"scripts": {
"twig-pot": "gulp pot"
}
}
<?php
/**
* Include Wordpress gettext functions in Twig if necessary
*/
// Init twig engine
global $twig;
$loader = new Twig_Loader_Filesystem( '/views' );
$twig = new Twig_Environment( $loader, [] );
// Add wordpress gettext functions
$twig->addFunction(new Twig_SimpleFunction('__', function( $text, $domain = 'default' ) {
return __($text, $domain);
} ));
$twig->addFunction(new Twig_SimpleFunction('_n', function( $single, $plural, $number, $domain = 'default' ) {
return _n($single, $plural, $number, $domain);
} ));
$twig->addFunction(new Twig_SimpleFunction('_x', function( $text, $context, $domain = 'default' ) {
return _x($text, $context, $domain);
} ));
$twig->addFunction(new Twig_SimpleFunction('_ex', function( $text, $context, $domain = 'default' ) {
return _ex($text, $context, $domain);
} ));
$twig->addFunction(new Twig_SimpleFunction('_nx', function( $single, $plural, $number, $context, $domain = 'default' ) {
return _nx($single, $plural, $number, $context, $domain);
} ));
@vyskoczilova
Copy link

vyskoczilova commented Aug 29, 2019

Hi, thanks for your script! If you're using Gulp 4 you have better to remove run-sequence as a dependency and rather use gulp.series and don't forget to signalize that the script has ended with done(); function

/**
 * Main Task
 */
gulp.task('pot', gulp.series('compile-twigs', 'generate-pot'));

@dasilvaluis
Copy link
Author

dasilvaluis commented Sep 1, 2019

Hi, thanks for your script! If you're using Gulp 4 you have better to remove run-sequence as a dependency and rather use gulp.series and don't forget to signalize that the script has ended with done(); function

/**
 * Main Task
 */
gulp.task('pot', gulp.series('compile-twigs', 'generate-pot'));

Hi @vyskoczilova This script is using gulp 3. At the time gulp 4 wasn't mature enough so I left it as it is. Maybe I'll update it in the future. Thank you!

@asterion
Copy link

👍 Gracias :)

@csalmeida
Copy link

Had to make a change in order to get it to run on Gulp 4 but this was so useful to create .pot files out of Twig templates, obrigado! 🙏

@dasilvaluis
Copy link
Author

dasilvaluis commented Sep 30, 2020

Had to make a change in order to get it to run on Gulp 4 but this was so useful to create .pot files out of Twig templates, obrigado! 🙏

@csalmeida could you please share your solution? Thinking of adding a file for gulp 4 too.

@csalmeida
Copy link

Sure, I have adapted quite a lot of it for my specific use case but I think it boils down to this:

I've changed the main task to use series() instead of runSequence():

gulp.task('pot', gulp.series(['compile-twigs', 'generate-pot']));

Returned the stream of the compile-twigs task:

gulp.task('compile-twigs', () => {
  del.bind(null, [config.cacheFolder], {force: true})
  // Iterate over .twig files
  const output = gulp.src(config.twig_files)
    // Search for Gettext function calls and wrap them around PHP tags. 
    .pipe(replace(gettext_regex.simple, match => `<?php ${match}; ?>`))
    .pipe(replace(gettext_regex.plural, match => `<?php ${match}; ?>`))
    .pipe(replace(gettext_regex.disambiguation, match => `<?php ${match}; ?>`))
    .pipe(replace(gettext_regex.noop, match => `<?php ${match}; ?>`))
    // Rename file with .php extension
    .pipe(rename({
      extname: '.php',
    }))
    // Output the result to the cache folder as a .php file.
    .pipe(gulp.dest(config.cacheFolder));

    return output;
});

Hope this helps! Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment