micapam/preprocess.md

## preprocess.md

      
    Raw
  

              preprocess.md
            
          
    Nanoc preprocessor analysis

Apparent uses

The following apparent uses of the preprocessor show up in sites’ sources (see below):

Read from external world into config

From environment (e.g. NANOC_ENV)
From side-effect-free command output (e.g. nanoc --version)


Set attributes based on other attributes, identifier, or raw content

Convert from one type to another (e.g. string to Time, or split tags string on comma)
Not currently possible: make changes depending on item’s dependencies
Assign next/previous item (identifier)


Create item (not based on anything)
Create items based on config details
Create items based on other items (sometimes paginated)

Items by category
Items by language
Items by year
Items by tag


Delete items that match certain criteria (e.g. is_draft, is_published)
Generate a variant copy of an item (e.g. .merge(offline_mode: true))
Create a config with some preprocessing (e.g. mangle(e-mail address))
Fetch data from external sources (e.g. GitHub)

… based on existing items (e.g. fetch repo info for all item[:github_repo])


Delete files generated as part of another process (e.g. output/search-index.json)
rsync static to output
Set item attributes to be compatible with the Blogging helper
Set item attributes to be compatible with the Sitemap helper

Lessons learnt

Modify to work with helpers

The preprocessor is used to modify items so they work with the helpers.
Having more flexible helpers (configurable data-obtaining methods) will help.
Precompute properties

The preprocessor is used to precompute properties of an item.
Having memoizable helpers will help.
Dependent data source

The preprocessor is used as a data source that depends on items being available. For example, it occasionally happens that GitHub API calls are made wth information obtained from items that are already loaded.
Env data

The preprocessor is used quite often to pull in data from the environment. For example, access credentials for deployment, but also output of commands to use as example input (e.g. nanoc --version).
IDEA: @env['FOO'] with dependency tracking? @system['command'] for commands? (Not quite the right syntax…)
The Nanoc philosophy

…
Sources


from: https://github.com/moll/nutikaitse/blob/master/Rules
preprocess do
  @config[:env] = ENV["NANOC_ENV"] ? ENV["NANOC_ENV"].downcase : "production"

	@items.each do |item|
    KINDS.find do |type, path|
      item[:kind] = type if item.identifier =~ path
    end

    LANGUAGES.find do |lang|
      item[:lang] = lang if item.identifier.end_with?(".#{lang}/")
    end

    # is_hidden is for sidemap
    item[:is_hidden] = true if item[:hidden]
  end
end

from: https://github.com/avdgaag/nanoc-template
preprocess do
  create_robots_txt
  create_webmaster_tools_authentications
  create_sitemap
end

# Preprocessor helpers
#
# This file has a collection of methods that are meant to be used in the
# preprocess-block in the Nanoc Rules file.
#
# @author Arjan van der Gaag


# Generate a sitemap.xml file using Nanoc's own xml_sitemap helper method by
# dynamically adding a new item.
#
# Make items that should not appear in the sitemap hidden. This by default
# works on all image files and typical assets, as well as error pages and
# htaccess. The is_hidden attribute is only explicitly set if it is absent,
# allowing per-file overriding.
#
# @todo extract hidden file types into configuration file?
def create_sitemap
  return unless @site.config[:output_generated_assets]

  @items.each do |item|
    if %w{png gif jpg jpeg coffee scss sass less css xml js txt}.include?(item[:extension]) ||
        item.identifier =~ /404|500|htaccess/
      item[:is_hidden] = true unless item.attributes.has_key?(:is_hidden)
    end
  end
  @items << Nanoc3::Item.new(
    "<%= xml_sitemap %>",
    { :extension => 'xml', :is_hidden => true },
    '/sitemap/'
  )
end

# Use special settings from the site configuration to generate the files
# necessary for various webmaster tools authentications, such as the services
# from Google, Yahoo and Bing.
#
# This loops through all the items in the `webmaster_tools` setting, using
# its properties to generate a new item.
#
# See config.yaml for more documentation on the input format.
def create_webmaster_tools_authentications
  return unless @site.config[:output_generated_assets]

  @site.config[:webmaster_tools].each do |file|
    next if file[:identifier].nil?
    content    = file.delete(:content)
    identifier = file.delete(:identifier)
    file.merge({ :is_hidden => true })
    @items << Nanoc3::Item.new(
      content,
      file,
      identifier
    )
  end
end

# Generate a robots.txt file in the root of the site by dynamically creating
# a new item.
#
# This will either output a default robots.txt file, that disallows all
# assets except images, and points to the sitemap file.
#
# You can override the contents of the output of this method using the site
# configuration, specifying Allow and Disallow directives. See the config.yaml
# file for more information on the expected input format.
def create_robots_txt
  return unless @site.config[:output_generated_assets]

  if @site.config[:robots]
    content = if @site.config[:robots][:default]
      <<-EOS
User-agent: *
Disallow: /assets
Allow: /assets/images
Sitemap: #{@site.config[:base_url]}/sitemap.xml
      EOS
    else
      [
        'User-Agent: *',
        @site.config[:robots][:disallow].map { |l| "Disallow: #{l}" },
        (@site.config[:robots][:allow] || []).map { |l| "Allow: #{l}" },
        "Sitemap: #{@site.config[:robots][:sitemap]}"
      ].flatten.compact.join("\n")
    end
    @items << Nanoc3::Item.new(
      content,
      { :extension => 'txt', :is_hidden => true },
      '/robots/'
    )
  end
end

from: https://github.com/dnsimple/dnsimple-support
preprocess do
  create_category_pages
end

  def create_category_pages
    articles_by_category.each do |category, items|
      @items << Nanoc::Item.new(
        "<%= render('category_index', :category => '#{category}') %>",
        {
            :title => "Articles in #{category}",
            :h1 => "#{category} articles",
            :items => items,
            :categories => category
        },
        url_for_category(category),
        :binary => false
      )
    end
  end

from https://github.com/scheibler/nanoc-multilingual-theme/blob/master/Rules.example
preprocess do
    # redirect index page
    items << Nanoc::Item.new(
        "<%= render 'redirect' %>", { :kind => 'redirect' }, "/")
    # create various blog sites in all supported languages
    # if you don't use the blog pattern at your site, you can delete the rest of the preprocess block
    for language in LANGUAGE_CODE_TO_NAME_MAPPING.keys do
        article_list = sorted_blog_article_list_for(language)
        # blog main page
        items << Nanoc::Item.new( "",
            { :kind => 'article_list', :title => 'Blog', :canonical_identifier => 'blog_home', :start_index => 0 },
            "/" + language + "/blog/")
        # blog index pages for older articles
        article_index = @site.config[:number_of_articles_at_blog_index_page]
        site_index = 2
        while article_index < article_list.length
            items << Nanoc::Item.new( "",
                { :kind => 'article_list', :title => translate_string(language, 'blog_title_page_number') % site_index,
                    :canonical_identifier => "blog_home%d" % site_index, :start_index => article_index },
                "/" + language + "/blog/index-%.2d/" % site_index)
            article_index += @site.config[:number_of_articles_at_blog_index_page]
            site_index += 1
        end
        # blog rss feed
        items << Nanoc::Item.new(
            "<%= atom_feed :title => @site.config[:site_title], :author_name => @site.config[:site_author],
                :author_uri => @site.config[:author_uri], :limit => @site.config[:number_of_rss_feed_entries],
                :articles => sorted_blog_article_list_for(language_code_of(item)) %>",
            { :kind => 'feed' }, "/" + language + "/blog/feed/")
        # tags main page
        items << Nanoc::Item.new( "",
            { :kind => 'tag_list', :title => 'Tags', :canonical_identifier => 'tag_list', :start_index => 0 },
            "/" + language + "/blog/tags/")
        all_tags(language, true).each do |tag|
            items << Nanoc::Item.new( "",
                { :kind => 'article_list_for_tag', :title => 'Tag %s' % tag[0], :tag => tag[0],
                    :count => tag[1], :canonical_identifier => 'articles_for_tag_%s' % tag[0].gsub(' ','_') },
                "/" + language + "/blog/tags/" + tag[0].gsub(' ','-') + "/")
        end
    end
end

from: https://github.com/democratech/LaPrimaire/blob/master/Rules
preprocess do
	create_robots_txt
end

def create_robots_txt
	if @site.config[:robots]
		content = if @site.config[:robots][:default]
				  <<-EOS
User-agent: *
Disallow: /admin/
Disallow: /citoyen/
EOS
			  else
				  [
					  'User-Agent: *',
					  @site.config[:robots][:disallow].map { |l| "Disallow: #{l}" },
					  (@site.config[:robots][:allow] || []).map { |l| "Allow: #{l}" },
					  "Sitemap: #{@site.config[:robots][:sitemap]}"
				  ].flatten.compact.join("\n")
				  end
		@items << Nanoc3::Item.new(
			content,
			{ :extension => 'txt', :is_hidden => true },
			'/robots/'
		)
	end
end

from: https://github.com/blinry/morr.cc/blob/master/Rules
preprocess do
    @items.each do |item|
        if item[:published]
            item[:published] = DateTime.parse(item[:published].to_s)
        end
        if item[:updated]
            item[:updated] = DateTime.parse(item[:updated].to_s)
        end
        if item[:tags]
            item[:tags] = item[:tags].split(",").map{|t| t.strip}
        end
    end

    tags.each do |tag|
        content = '<%= box(with_tag("'+tag+'")) %>'
        title = "Content with tag '#{tag}'"
        identifier = "/tag/#{tag}/index.md"
        @items.create(content, {:title => title, :noindex => true}, identifier)
    end

    categories.each do |name, items|
        content = "<%= box(categories[\"#{name}\"]) %>"
        title = "#{name}"
        identifier = "/#{name.downcase}/index.md"
        @items.create(content, {:title => title, :headless => true, :noindex => true}, identifier)
    end

    # rebuild these variables for each compilation
    $categories = nil
    $things = nil
end

from: https://github.com/bobthecow/genghisapp.com/blob/master/Rules
preprocess do
  config[:genghis_version] = get_genghis_version
end

from: https://github.com/nanoc/nanoc.ws/blob/master/Rules
preprocess do
  config[:nanoc_version_info] = Nanoc.version_information.strip
  config[:gem_version_info]   = Gem::VERSION
  config[:ruby_version_info]  = `ruby --version`.strip
  config[:generate_pdf] = !ENV['PDF'].nil?
end

from: https://github.com/ddfreyne/stoneship-site/blob/master/Rules
preprocess do
  def hide_assets
    items.each do |item|
      if item.identifier =~ /^\/assets/
        item[:is_hidden] = true
      end
    end
  end

  def delete_drafts
    items.delete_if { |i| i[:is_draft] }
  end

  def convert_dates
    items.each do |i|
      if i[:published_on]
        i[:published_on] = Date.parse(i[:published_on])
      end
    end
  end

  def assign_cachebuster_id
    stylesheet = @items["/assets/style/style.*"]
    digest = Digest::SHA1.base64digest(stylesheet.raw_content)
    stylesheet[:cbid] = digest.gsub(/[^A-Za-z0-9]+/, '')
  end

  hide_assets
  delete_drafts
  convert_dates
  assign_cachebuster_id

from: https://github.com/davidcox/coxlab-website/blob/master/Rules
preprocess do
  news_item_years.each do |y|
    @items << Nanoc3::Item.new(
        "",
        { :title => "News Items from #{y}",
          :year  => y },
        "/news/archive/#{y}/"
      )
  end
end

from: https://github.com/kana/hatokurandom/blob/master/Rules
preprocess do
  def items.find_by_identifier(identifier)
    find {|i| i.identifier == identifier} or
      throw ArgumentError.new("No such item with identifier '#{identifier}'")
  end

  root_item = items.find_by_identifier('/')
  items << Nanoc::Item.new(
    root_item.raw_content,
    root_item.attributes.merge({offline_mode: true}),
    '/offline/'
  )
end

from: https://github.com/pelletier/blog/blob/master/Rules
# Do some preprocessing on articles: inject some attributes into them.
preprocess do
  items.each do |item|
    # Make them articles if they are in this directory (I seriously don't want
    # to type this each time a write a new one).
    if item.identifier =~ %r{^/articles/(.*)$}
      item[:kind] = 'article'

      edited_stamp = item[:edited] or item[:edited_at]
      if not edited_stamp.nil?
        item[:edited_at] = Time.parse(edited_stamp)
      end
    end
    # Add the :created_at attribute, based on the filename (thanks to Time.parse
    # magic).
    if item.identifier =~ %r{^/articles/\d{4}/\d{2}/(\d{4}-\d{2}-\d{2}).+$}
      item[:created_at] = Time.parse($1)
    end
  end
end


[TODO]
from: https://github.com/oblac/jodd-site/blob/master/Rules
preprocess do
  # assign a 'site_path' to each item
  @items.each do |item|
    collect_path(item)
  end
end

	def collect_path(item)
		#puts item.raw_filename

		path = item.identifier.to_s

		# remove raw prefix
		if (path.start_with?('/static/'))
			path = path[7..-1]
		end

		# puts "---->" + path

		# collect documents (format: /path/number+name.md)
		ndx = path.index('+')
		if (ndx != nil)
			last = path.rindex('/')
			key = path[0..last]
			number = path[(last+1)..(ndx-1)]
			path = key + path[(ndx+1)..-1]

			add_doc(key, number, item)
		end

		# set paths
		if (path.end_with?('.md'))
			path = path[0..-3] + 'html'
		else
			index = path.rindex('.')
			if (index != nil)
				ext = item[:extension]
				index2 = ext.rindex('.')
				if (index2 != nil)
					index2 += 1
					ext = ext[index2..-1]
				end
				path = path[0..index] + ext
			end
		end

		item[:site_path] = path

		if (item[:title] == nil)
			title = extract_md_title(item)
			if (title != nil)
				item[:title] = title
			end
		end
	end

from: https://github.com/kana/nanoc-test/blob/master/Rules
preprocess do
  def mangle(email)
    sp = '<span class="nospam">&#9251;</span>'
    email.gsub(/[@.]/, "#{sp}\\0#{sp}")
  end
  config[:site] = begin
    h = {}
    h[:author] = 'Kana Natsuno'
    h[:email] = mangle("kana\100whileimautomaton.net")
    h[:domain] = 'whileimautomaton.net'
    h[:name] = 'while ("im automaton");'
    h[:prefix] = "http://#{h[:domain]}"
    h[:signature] = "#{h[:author]} &lt;#{h[:email]}&gt;"
    h[:uri] = "#{h[:prefix]}/"
    h
  end
  config[:recent_topic_count] = 5

  topics_per_month =
    items
    .select(&:topic?)
    .group_by {|i| i.identifier.match(%r{^(/\d+/\d+/)\d+/$})[1]}
  topics_per_month.each do |id, topics|
    items << Nanoc::Item.new(
      "",
      {
        :title => id.sub(%r{/(\d+)/(\d+)/}, '\1-\2'),
        :topics => topics,
      },
      id
    )
  end
end

from: https://github.com/mklabs/web-learn-jquery-com/blob/master/Rules
preprocess do
  @chapterOrder = [
    "getting-started",
    "javascript-101",
    "jquery-basics",
    "using-jquery-core",
    "events",
    "effects",
    "ajax",
    "plugins",
    "performance",
    "code-organization",
    "custom-events",
    "how-to"
  ]

  @chapters = {}

  @github_users = {
    "jquery" =>  nil
  }

  @items.each do |item|
    item[:chapter] = item[:filename].split('/')[1]
    item[:chapter_title] = item[:chapter].gsub(/-/, " ").upcase
    if item[:github]
      @github_users[ item[:github] ] = nil
    else
      item[:github] = "jquery"
    end
  end

  @github_users.each do |username, wat|
    request = Curl::Easy.http_get("https://api.github.com/users/"+username)
    request.perform
    @github_users[ username ] = JSON.parse request.body_str
  end


  @groupedItems = @items.group_by {|item| item[:chapter]}

  @orderedItems = []

  @chapterOrder.each do |folder|
    myitems = @groupedItems[ folder ]
    @chapters [ folder] = {}
    @chapters[ folder ][ :items ] = @groupedItems[folder].sort_by {|i| i[:section] || 0 }
    @orderedItems = @orderedItems + @chapters[ folder ][ :items ]
    @chapters[ folder ][ :title ] = folder.gsub(/-/, " ").upcase
    @chapters[ folder ][ :folder ] = folder
  end

  @items.each do |item|
    i = item[:ordinal_index] = @orderedItems.index(item)
    if i
      item[:next_item] = @orderedItems[ i+1 ]
      item[:previous_item] = @orderedItems[ i-1 ]
    end
    item[:github_user] = @github_users[ item[:github] ]
  end

  @site.config[:chapters] = @chapters
  @site.config[:orderedItems] = @orderedItems
end

from: https://github.com/gjtorikian/testing/blob/master/Rules
preprocess do
  File.delete("output/search-index.json") if File.exists?("output/search-index.json")
end

from: https://github.com/lifepillar/nanoc4-template/blob/master/Rules
preprocess do
  @config[:production] = !ENV['NANOC_ENV'].nil? && ENV['NANOC_ENV'] == 'production' # See https://github.com/nanoc/nanoc/issues/487

  # See: http://nanoc.ws/docs/api/Nanoc/Helpers/Blogging.html
  # Assume all items inside /blog are blog articles unless otherwise specified.
  @items.select { |item| item.identifier.to_s =~ %r{^/blog/posts/.+} }.each do |item|
    item[:kind] ||= 'article' # Required by Nanoc::Helpers::Blogging
  end

  # Assign a date to all items (not only blog posts) unless they have it already defined.
  @items.each do |item|
    if item.key?(:created_at)
      item[:created_at] = attribute_to_time(item[:created_at])
    else
      item[:created_at] = Time.now
    end
    if item.key?(:updated_at)
      item[:updated_at] = attribute_to_time(item[:updated_at])
    end
  end

  # Build tag pages for blog posts
  build_tag_pages(articles())

  # Build yearly and monthly archives of blog posts
  build_archives(articles())

from: https://github.com/spf13/blog.zacharyvoase.com/blob/master/Rules
preprocess do
  system('rsync -a static/ output')  # Copy static files.
end

from: https://github.com/bjornd/jvectormap-site
preprocess do
  items.each { |i| map_preprocessing(i) if i[:map_params] || i[:map_params_variants] }
  build_jvectormap
  generate_doc
end

def map_preprocessing(item)
  item[:map_params_variants] ||= [Hash.new]
  proc_config = File.read(item.raw_filename.sub('.html', '_config.json'))
  params = JSON.parse(proc_config, :symbolize_names => true)

  item[:js_assets] = []

  item[:map_params_variants].each_index do |index|
    variant_params = params.clone
    variant_params[0] = variant_params[0].merge( item[:map_params_variants][index] )
    variant_params[-1][:params] = variant_params[-1][:params].merge( item[:map_params_variants][index] )
    item[:map_params_variants][index][:projection] = variant_params[0][:projection]
    item[:map_params_variants][index][:proc_config] = proc_config

    variant_params[0][:file_name] = @config[:maps_path]+'/'+variant_params[0][:file_name]
    #if @config[:maps_default_encoding] && !variant_params[-1][:params][:input_file_encoding]
    #  variant_params[-1][:params][:input_file_encoding] = @config[:maps_default_encoding]
    #end

    map_id = Digest::MD5.hexdigest(variant_params.to_json)
    map_name = 'jquery-jvectormap-'+variant_params[-1][:params][:name]+'-'+variant_params[0][:projection]
    output_file_path = 'tmp/'+map_id+'.js'
    variant_params[-1]['file_name'] = output_file_path

    if !File.exists? output_file_path
      converter_command =
          'echo \''+variant_params.to_json+'\' | '+
          'python '+
          'external/jvectormap/converter/processor.py '
      system(converter_command)
    end

    @items << Nanoc3::Item.new(
      File.open(output_file_path, "r").read,
      {},
      "/js/"+map_name+'/'
    )

    item[:map_params_variants][index][:download_link] = '/js/'+map_name+'.js'
    item[:map_params_variants][index][:file_size] = File.size output_file_path
    item[:map_params_variants][index][:name] = variant_params[-1][:params][:name]+'_'+variant_params[0][:projection]

    item[:js_assets] << '/js/'+map_name+'.js'

    if index == 0
      map_content = File.read(output_file_path)
      item[:regions] = JSON.parse(map_content[map_content.index('{') .. map_content.rindex('}')])['paths'].to_a.map do |region|
        {code: region[0], name: region[1]['name']}
      end
    end
  end
end

def generate_doc
  hash = get_jvectormap_commit_hash
  FileUtils.mkpath('tmp/doc') if !File.exists?('tmp/doc')
  tmpDir = 'tmp/doc/'+hash+'/'
  if !File.exists?(tmpDir)
    `external/jsdoc/jsdoc -t ../jsdoc_template/ -d #{tmpDir} external/jvectormap/src/`
  end

  Dir.foreach(tmpDir) do |fname|
    next if !['jvm-dataseries.html', 'jvm-map.html', 'jvm-multimap.html', 'jvm-proj.html', 'jvm-legend.html'].index(fname)
    itemTile, itemText = File.open(tmpDir + fname, "rb").read.split("\n", 2)

    @items << Nanoc3::Item.new(
      itemText,
      {title: itemTile, submenu: true},
      "/documentation/javascript-api/"+File.basename(tmpDir + fname, '.html')+"/"
    )
  end
end

def build_jvectormap
  hash = get_jvectormap_commit_hash
  FileUtils.mkpath('tmp/jvectormap') if !File.exists?('tmp/jvectormap')
  tmpFile = 'tmp/jvectormap/'+hash
  if !File.exists?(tmpFile)
    `external/jvectormap/build.sh #{tmpFile}`
  end
  js_file_name = "jquery-jvectormap-#{@config[:jvectormap_version]}.min"
  @items << Nanoc3::Item.new(
    File.open(tmpFile, "rb").read,
    {},
    "/js/#{js_file_name}/"
  )
  css_file_name = "jquery-jvectormap-#{@config[:jvectormap_version]}"
  @items << Nanoc3::Item.new(
    File.open("external/jvectormap/jquery-jvectormap.css", "rb").read,
    {},
    "/css/#{css_file_name}/"
  )

  FileUtils.mkpath('tmp/jvectormap-zip') if !File.exists?('tmp/jvectormap-zip')
  FileUtils.remove(Dir.glob('tmp/jvectormap-zip/*'))
  FileUtils.copy_file(tmpFile, "tmp/jvectormap-zip/#{js_file_name}.js")
  FileUtils.copy_file('external/jvectormap/jquery-jvectormap.css', "tmp/jvectormap-zip/#{css_file_name}.css")
  `cd tmp/jvectormap-zip; zip jquery-jvectormap-#{@config[:jvectormap_version]}.zip *.css *.js`
  @items << Nanoc3::Item.new(
    File.open("tmp/jvectormap-zip/jquery-jvectormap-#{@config[:jvectormap_version]}.zip", "rb").read,
    {},
    "/binary/jquery-jvectormap-#{@config[:jvectormap_version]}/"
  )
end

from: https://github.com/coderanger/coderanger.net/blob/master/Rules
preprocess do
  # Anything with an explicit published: false should vanish
  items.reject! {|item| item[:published] == false}

  items.each do |item|
    # Extract date from filename if present
    item.identifier.match(%r{(/.*)?/(\d\d\d\d-\d\d-\d\d)-(.*)/}) do |md|
      item.identifier = "#{md[1]}/#{md[3]}/"
      item[:date] ||= md[2]
    end

    # Set the kind for certin subfolders
    parts = item.identifier.split('/')
    if parts.length > 2
      case parts[1]
      when 'posts'
        item[:kind] ||= 'post'
      when 'talks'
        item[:kind] ||= 'talk'
      end
    end

    # Parse the date if present
    item[:date] = Date.parse(item[:date]) if item[:date].is_a?(String)

    # Set the extra timestamps for the Blogging helper
    item[:created_at] = item[:date].to_time if item[:date]
    item[:updated_at] = item[:mtime] if item[:mtime]

    # Convert the identifier into the title if not present
    if item[:kind]
      item[:title] ||= begin
        words = item.identifier.split('/').last.split('-')
        words.first.capitalize! # Always cap the first word
        words.each {|word| word.capitalize! if word.length > 3}
        words.join(' ')
      end
    end
  end
end

from: https://github.com/DivineDominion/nanoc-boilerplate/blob/master/Rules
preprocess do
  # authors may unpublish items by setting meta attribute publish: false
  items.delete_if { |item| item[:publish] == false }
end

from: https://github.com/Caster/Ploggy/blob/master/Rules
preprocess do
    # setting a URL
    items.select{ |item| item[:title] != nil }.each{ |item|
        item[:title_full] = 'Ploggy - ' + item[:title]
        splitId = item.identifier.chop.split('/')
        joinId = splitId[2..-1].join('/')
        item[:url] = (joinId.length > 0 ? '/' : '') + joinId + '/'
    }

    # Ploggy-specific handling
    logs_found = 0
    items.select{ |item| item.identifier.chop.split('/')[1] == 'logs' }
    .each{ |item|
        PloggyHelper.log_items.push(item)
         logs_found += 1
    }
    Nanoc::CLI::Logger.instance.log(:low,
        sprintf('Found %d log entries.', logs_found))
end

from: https://github.com/Caster/denvelop/blob/master/site/Rules
preprocess do
    items.select{ |item| item[:title] != nil }.
        each{ |item|
            # set title of the item
            item[:title_full] = 'Denvelop - ' + item[:title]
            # build the URL of the item, which depends on the language of the
            # item and the default language of the site
            splitId = item.identifier.without_ext.split('/')
            # special handling for index files
            if splitId[-1] == 'index' then
                splitId.pop
            end
            joinId = (splitId[2] == default_language ?
                      splitId[3..-1] : splitId[2..-1]).join('/')
            # set the URL
            item[:url] = (joinId.length > 0 ? '/' : '') + joinId + '/'
        }
end

from: https://github.com/driftyco/ionic-learn/blob/master/Rules
preprocess do
  video_items = items.select do |item|
    split_item = item.identifier.split('/').delete_if(&:empty?)
    split_item.length > 1 and split_item.first == "videos"
  end
  video_names = video_items.map do |item|
    item.identifier.split('/').delete_if(&:empty?)[1]
  end.uniq

  all_formulas.each do |formula|
    formula[:titles] = [formula[:name], "Formulas"]
  end

  video_names.each do |item|
    detail_page = items[detail_identifier.call(item)]
    if !detail_page.nil? && detail_attributes = detail_page.attributes
      last_modified = Date.parse(detail_attributes[:date]).to_time
      if detail_page[:live] == true
        items << Nanoc::Item.new(
          "<%= render 'video_detail' %>",
          detail_attributes.merge({
            titles: [detail_attributes[:name], 'Videos'],
            kind: 'video',
            details: detail_identifier.call(item),
            transcript: transcript_identifier.call(item)
          }),
          video_identifier.call(item),
          last_modified
        )
      end
    end
  end

  video_difficulties.each do |difficulty|
    items << Nanoc::Item.new(
      "<%= render 'difficulty_list' %>",
      {
        difficulty: difficulty,
        kind: 'difficulty_list',
        titles: [difficulty.capitalize, 'Videos'],
        body_css: 'videos'
      },
      "/videos/#{difficulty}/"
    )
  end

  formula_categories.each do |category|
    items << Nanoc::Item.new(
      "<%= render 'category_list' %>",
      {category: category,
       kind: 'category_list',
       titles: [category, 'Formulas'],
       body_css: 'formulas'},
      "/formulas/#{category.downcase.split(' ').join('-')}/"
    )
  end
end

from: https://github.com/EasyRPG/easyrpg.org/blob/master/Rules
preprocess do
  hide_items_from_sitemap
end

def hide_items_from_sitemap
  @items.each do |item|
    if %w{css xml js txt}.include?(item[:extension]) || item.identifier =~ /404/
      item[:is_hidden] = true if item[:is_hidden].nil?
    end
  end
end

from: https://github.com/seth/userprimary.net/blob/master/Rules
preprocess do
  add_missing_info
  create_tags_pages
  create_month_archives
end

def add_missing_info
  items.each do |item|
    if item[:file]
      # nanoc3 >= 3.1 will have this feature, add for older versions
      item[:extension] ||= item[:file].path.match(/\..*$/)[0]
    end
  end
end

def create_tags_pages
  tags = Hash.new { |h, k| h[k] = 0 }
  items.each do |item|
    if item[:kind] == "article"
      if item[:tags]
        item[:tags].each { |t| tags[t] += 1 }
      end
    end
  end

  tags.each do |tag, count|
    content = %[= render('tag', :tag_name => "#{tag}", :tag_count => "#{count}")]
    items << Nanoc3::Item.new(content,
                              { :title => "#{tag}",
                                :tag_name => tag,
                                :tag_count => count.to_s
                              },
                              "/tags/#{tag}/")
  end
end

def create_month_archives
  bymonth = Hash.new { |h, k| h[k] = [] }
  items.each do |item|
    if item[:kind] == "article"
      t = Time.parse(item[:created_at])
      bymonth[t.strftime("%Y-%B")] << item
    end
  end

  bymonth.each do |year_month, posts|
    posts = posts.sort_by { |p| Time.parse(p[:created_at]) }.reverse
    most_recent = Time.parse(posts.first[:created_at])
    items << Nanoc3::Item.new(render('bymonth', :posts => posts,
                                     :year_month => year_month),
                              { :title => "#{year_month}",
                                :post_count => posts.count,
                                :most_recent => most_recent,
                                :month => most_recent.strftime("%B"),
                                :year => most_recent.strftime("%Y")
                              },
                              "/archives/#{year_month}/")
  end
end

from: https://github.com/rvm/rvm-site/blob/master/lib/auto.rb
preprocess do
  AutoHelper.new(File.dirname(__FILE__), self).auto

  # TODO see https://github.com/nanoc/nanoc/pull/481
  # force reload of items, does the damn warning about defining
  # preprocess second time - not a problem, no loop here
  #self.site.instance_variable_set(:@loaded, false)
  #self.site.instance_variable_set(:@items_loaded, false)
  #self.site.load
end

class AutoHelper
  attr_reader :root, :site, :tags, :authors

  def initialize(root, site)
    @root = Pathname(root)
    @site = site
    @tags = []
    @authors = {}
  end

  def parse_tags(item)
    item.attributes[:tags].each { |tag| @tags << tag unless @tags.include?(tag) }
  end

  def parse_author(item)
    authors[item.attributes[:author]] = item.attributes[:author_full]
  end

  def parse_site
    site.items.each do |item|
      parse_tags  (item) if item.attributes[:tags]
      parse_author(item) if item.attributes[:author]
    end
  end

  def write_file(file, content)
    file.open('w') { |file| file.write(content) }
  end

  def render_layout(layout_name, mapping = {})
    layout = @site.layouts.find { |l| l.identifier == layout_name.cleaned_identifier }
    filter = Nanoc::Filter.named('erb').new(mapping)
    filter.run(layout.raw_content)
  end

  def create_tag_page(tag_page, tag)
    write_file(tag_page, render_layout("templates/tag_page", tag: tag))
  end

  def create_tag_feed(feed_page, tag)
    write_file(feed_page, render_layout("templates/feed", type: 'tag', filter: tag))
  end

  def ensure_tag_page(tags_dir, tag)
    tag_page  = tags_dir + "#{tag}.haml"
    feed_page = tags_dir + "#{tag}_feed.haml"

    create_tag_page(tag_page, tag)  unless tag_page.exist?
    create_tag_feed(feed_page, tag) unless feed_page.exist?
  end

  def ensure_tag_pages
    tags_dir  = root + 'content' + 'tags'
    tags_dir.mkpath

    tags.each do |tag|
      ensure_tag_page(tags_dir, tag)
    end
  end

  def create_author_page(author_page, author, author_full)
    write_file(author_page, render_layout("templates/author_page", author: author, author_full: author_full))
  end

  def create_author_feed(feed_page,   author, author_full)
    write_file(feed_page, render_layout("templates/feed", type: 'author', filter: author))
  end

  def ensure_author_page(authors_dir, author, author_full)
    author_page = authors_dir + "#{author}.haml"
    feed_page   = authors_dir + "#{author}_feed.haml"

    create_author_page(author_page, author, author_full) unless author_page.exist?
    create_author_feed(feed_page,   author, author_full) unless feed_page.exist?
  end

  def ensure_author_pages
    authors_dir  = root + 'content' + 'authors'
    authors_dir.mkpath

    authors.each_pair do |author, author_full|
      ensure_author_page(authors_dir, author, author_full)
    end
  end

  def auto
    parse_site
    ensure_tag_pages
    ensure_author_pages
  end
end

from: https://github.com/yazgoo/blag/blob/master/Rules
preprocess do
  # for tag in all_tags
  #   @items << Nanoc::Item.new("",
  #       {:title => tag.capitalize, :tag => tag, :layout => "tag", :extension => 'html'},
  #        "/blog/tags/#{tag.to_url}/")
  # end

  for date in get_timeline
    # @items << Nanoc::Item.new("", {
    #           :title => "Blog posts from #{date.year}",
    #           :menu_title => date.year, :year => date.year,
    #           :layout => "timeline",
    #           :extension => "html"}, "/blog/#{date.year}/")
    @items << Nanoc::Item.new("", {
              :title => "Blog posts from #{Date::MONTHNAMES[date.month.to_i]} #{date.year}",
              :menu_title => Date::MONTHNAMES[date.month.to_i],
              :year => date.year,
              :month => date.month,
              :layout => "timeline",
              :extension => "html"}, "/blog/#{date.year}/#{'%02d' % date.month}/")
  end

end

from: https://github.com/martinrehfeld/inside.glnetworks.de/blob/master/Rules
preprocess do
  # authors may unpublish items by setting meta attribute publish: false
  items.delete_if { |item| item[:publish] == false }

  # set sensible defaults for attributes
  items.each do |item|
    item[:is_hidden] = true if item.binary?
    if item.identifier =~ %r(^/articles/) && !item.binary?
      item[:kind] ||= 'article'
      item[:updated_at] ||= item.mtime
    end
  end

  create_category_pages

  copy_static
end

# Create category/tag pages (uses layouts/category.haml
def create_category_pages
  tags.keys.each do |tag|
    items << Nanoc3::Item.new(
      "= render('category', :tag => '#{tag}')",
      {
        :title => "Category: #{tag}",
        :changefreq => 'daily',
        :priority => 0.4
      },
      "/categories/#{tag.downcase.parameterize}/",
      :binary => false
    )
  end
end

# Copy static assets outside of content instead of having nanoc3 process them.
def copy_static
  FileUtils.cp_r 'static/.', 'output/', :preserve => true
end

from: https://github.com/github/developer.github.com/blob/master/Rules
preprocess do
  add_created_at_attribute
  add_kind_attribute
  create_individual_blog_pages
  generate_redirects(config[:redirects])
  @items.each do |item|
    ConrefFS.apply_attributes(@config, item, :default)
  end
end

def add_created_at_attribute
  @items.each do |item|
    date = date_from_filename(item[:filename])
    item[:created_at] = date unless date.nil?
  end
end

def add_kind_attribute
  @items.each do |item|
    next unless item[:filename].to_s.starts_with?('content/changes/2')
    item[:kind] = 'change'
  end
end

module Paginate
  BLOG_TYPE = 'changes'

  def paginated_items(items)
    items.select { |i| i.identifier =~ %r(/#{BLOG_TYPE}/\d{4}) }.sort_by { |b| Time.parse(b[:created_at].to_s) }
  end

  def create_individual_blog_pages
    paginated_blog_items = paginated_items(items)

    # create individual blog pages
    blog_pages = []
    blog_pages << paginated_blog_items.slice!(0...PER_PAGE) until paginated_blog_items.empty?

    blog_pages.each_index do |i|
      next_i = i + 1 # accounts for 0-index array
      first = i * PER_PAGE
      last  = (next_i * PER_PAGE) - 1

      @items.create(
        "<%= renderp '/pagination_page.html',
          :current_page => #{next_i},
          :per_page => PER_PAGE,
          :first => #{first}, :last => #{last} %>",
        { :title => 'GitHub API Changes', :layout => 'blog' },
        "/changes/#{next_i}"
      )
    end
  end
end

module RedirectGenerator
  def generate_redirects(redirects)
    redirects.each do |pairs|
      pairs.each_pair do |old_url, new_url|
        filename = old_url.to_s.sub(%r{/$}, '')
        @items.create(
          "<%= renderp '/redirect.*',
            { :new_url => '#{new_url}'
            }
          %>",
          { :filename => filename },
          filename
        )
      end
    end
  end
end

from: https://github.com/DataDog/documentation/blob/master/Rules
preprocess do
  create_redirect_pages
  # def rchomp(sep = $/)
  #   self.start_with?(sep) ? self[sep.size..-1] : self
  # end

  # def snippetizer(item)
  #   if item["has_snippets"] == "True"
  #     doc = Nokogiri::HTML.fragment(item.content)
  #     doc.css(".snippetizer").each do |snippet|
  #       print snippet
  # #     # id = @item.identifier.gsub("(/(.+)/", "$&")
  #       id = item.identifier.chomp('/').rchomp('/') + '-' + snippet['id']
  # #     # print id
  # #     # print snippet.content
  #       new_snippet = {id => snippet.content}
  #       print new_snippet
  #       $global_snippets.merge(new_snippet)
  #     end
  #   end
  # end

  if ENV.has_key?('github_personal_token')
    $client = $client ||= Octokit::Client.new(:access_token => ENV['github_personal_token'])
    $client.user.login
  end

  $cbfingerprints = get_cache_bust_fingerprints()
  $en_local_hash = get_local_hash('en')
  $ja_local_hash = get_local_hash('ja')
  $example_items = collect_example_items()
  $video_items = collect_video_items()
  $ja_example_items = collect_ja_example_items()
  $ja_video_items = collect_ja_video_items()
  $integration_items = collect_integration_items()
  $ja_integration_items = collect_ja_integration_items()
  $guide_items = collect_guide_items()
  $ja_guide_items = collect_ja_guide_items()
  create_tag_pages($example_items,{:identifier => "/examples/%%tag%%/"})
  create_tag_pages($video_items,{:identifier => "/videos/%%tag%%/"})
  create_tag_pages($ja_example_items,{:identifier => "/ja/examples/%%tag%%/", :template => "example-subindex-ja"})
  create_tag_pages($ja_video_items,{:identifier => "/ja/videos/%%tag%%/", :template => "video-subindex-ja"})

  @items.each do |item|
    language = "en"
    otherlang = "ja"
    langpath = ""
    phrases = $en_local_hash
    if item.identifier.match('/ja/')
      language = "ja"
      otherlang = ""
      langpath = "/ja"
      phrases = $ja_local_hash
    end
    item["language"] = language
    item["otherlang"] = otherlang
    item["langpath"] = langpath
    phrases.each do | key, val |
      item[key] = val
    end
    # snippetizer(item)
  end
end

from: https://github.com/ddfreyne/neps-site/blob/master/Rules
preprocess do
  # Remove non-NEPs from /neps/*/
  items.delete_if do |i|
    parts = i.identifier.scan(/[^\/]+/)
    parts.size == 2 && parts[0] == 'neps' && parts[1] !~ %r{^NEP-}
  end

  # Remove : from neps
  items.each do |i|
    i.identifier.sub!(':', '')
  end

  # Remove /neps/ prefix
  items.each do |i|
    i.identifier.sub!(/^\/neps/, '')
  end

  # Assign number and title
  neps.each do |nep|
    nep[:number] = nep.identifier[/\d+/].to_i
    nep[:title]  = nep.identifier.match(/\d+-(.*)\/$/)[1].gsub('-', ' ')
  end
end

from: https://github.com/andreareginato/conference_site/blob/master/Rules
preprocess do
  copy_static # add assets to the output directory
end

from: https://github.com/cdlm/website-nanoc/blob/master/rules.rb
preprocess do

  # setup blog items
  all_feeds.each do |feed|
    feed.chain_entries
    feed.set_info
    feed.generate
  end

  # sitemap
  hide_items do |item|
    case item.identifier
    when %r{/publications/\d\d\d\d/.*}
      false
    when /404|500|htaccess/, %r{/(scripts|stylesheets)/.*}
      true
    else
      item.binary? || @site.config[:hidden_extensions].include?(item[:extension])
    end
  end
  create_sitemap
end

  def all_feeds
    Enumerator.new do |feeds|
      @items.each do |i|
        feeds << Feed.new(@site, i) if i.feed?
      end
    end
  end

    def chain_entries
      prev = nil
      entries.each do |current|
        unless prev.nil?
          prev[:next] = current.identifier
          current[:prev] = prev.identifier
        end
        prev = current
      end
    end

    def set_info
      entries.each do |e|
        e.attributes.update(@root[:entries_info])
      end
    end

    def generate
      @site.items << archive_item unless @root[:archives].nil?
      @site.items.concat yearly_archive_items unless @root[:archives_yearly].nil?
      @site.items << tag_item unless @root[:tags].nil?
    end

from: https://github.com/spree/spree/blob/master/guides/Rules
preprocess do
  config[:deploy][:default][:region] = ENV['S3_REGION']
  config[:deploy][:default][:bucket] = ENV['S3_BUCKET_NAME']
  config[:deploy][:default][:aws_access_key_id] = ENV['AWS_ACCESS_KEY_ID']
  config[:deploy][:default][:aws_secret_access_key] = ENV['AWS_SECRET_ACCESS_KEY']
end

from: https://github.com/matsimitsu/roytomeij.com/blob/master/Rules
preprocess do
  # authors may unpublish items by setting meta attribute publish: false
  items.delete_if { |item| item[:publish] == false }

  copy_static
 # create_tag_pages
  add_update_item_attributes
end

from: https://github.com/chickenboot/vintageinvite/blob/master/Rules
preprocess do
  build_tag_pages(items)
end

def all_tags(items = nil, sort = false)
  items ||= @items # default to all items if no items passed
  tags = {}
  items.each do |i|
    (i[:tags] || []).each{|t| tags[t] ||= 0; tags[t] += 1 }
  end
  # if sort is true, sort by item count descending
  sort ? tags.sort {|tl, tr| tr[1] <=> tl[1]} : tags
end

def build_tag_pages(items)
  all_tags(items).each do |tag,count|
    items << Nanoc3::Item.new(
      "= render('_blog_page', :tag => '#{tag}', :page_title => 'Tag: #{tag}')",
      { :title => "Tag: #{tag}" },  # , :is_hidden => true
      "/blog/tags/#{tag}/", :binary => false
    )
  end
end

from: https://github.com/CootCraig/nanoc_blog/blob/master/blog/Rules
preprocess do
  items << Nanoc::Item.new(
    "",
    {},
    "/post_index/"
  )
  tag_list = Set.new
  items.each do |item|
    item[:tags].each do |tag|
      if tag.length > 0
        tag_list.add(tag.downcase)
      end
    end if item[:tags]
  end
  items << Nanoc::Item.new(
    "",
    { :tag_list => tag_list },
    "/tag_index/"
  )
  tag_list.each do |tag|
    items << Nanoc::Item.new(
      "",
      { :tag => tag },
      "/tags/#{tag}/"
    )
  end
end

from: URL
___CODE___

from: URL
___CODE___

from: URL
___CODE___

from: URL
___CODE___

from: URL
___CODE___

from: URL
___CODE___

from: URL
___CODE___