ansible - example of using filters to change each item in a list

00_description

The problem:

I wanted to use the jinja 'map' filter to modify each item in a string, in this simple 
example, adding '.conf' to each item.

The 'format' filter in jinja takes arguments (value, *args, **kwargs).  Unfortunately, 
it uses 'value' as the pattern. When called inside map 'value' is the current item in 
the list, or in other words *args as far as format is concerned. So it's the wrong way 
around.

The following example creates a simple filter that has the format args in an order we 
can use when calling 'map'.  Note this is extremely simplestic, and only allows for 
adding a prefix or suffix to an item. i.e. prefix%s or %ssuffix.  You can't do more 
advanced ones like '%s - %s' with this example as that requires passing the current 
item as an array and (at least in python 2x), variable length args have to come after 
named args.

_Note: A better approach might be to use the ansible regex_replace filter within map, 
but when I tried that, it returned \u0001.conf, so I'm guessing that there's some 
unicode foo (or lack thereof) happening. i.e. I tried
  mylist2: "{{ mylist | map('regex_replace', '^(.*)$', '\\1.conf' ) | list }}" 

01_map_format.py

# plugins/filter/map_format.py (check path in ansible.cfg)

# This code is essentially a direct copy of the jinja 'format' filter with
# some minor mods to the args and their use, see do_format in
# https://github.com/mitsuhiko/jinja2/blob/master/jinja2/filters.py
# for the original

from jinja2.utils import soft_unicode

def map_format(value, pattern):
    """
    Apply python string formatting on an object:
    .. sourcecode:: jinja
        {{ "%s - %s"|format("Hello?", "Foo!") }}
            -> Hello? - Foo!
    """
    return soft_unicode(pattern) % (value)

class FilterModule(object):
    ''' jinja2 filters '''

    def filters(self):
        return {
            'map_format': map_format,
        }

02_play.yml

---
- hosts: all 
  remote_user: vagrant
  sudo: true
  vars:
    mylist: 
      - Alice
      - Bob 
      - Carol
    mylist2: "{{ mylist | map('map_format', '%s.conf' ) | list }}" 
  tasks:
    - debug: var=mylist2

03_output

...
TASK: [debug var=mylist2] ***************************************************** 
ok: [vagrant] => {
    "var": {
        "mylist2": [
            "Alice.conf", 
            "Bob.conf", 
            "Carol.conf"
        ]
    }
}
...

I needed this to work and discovered that running the following using 2 backward slashes works for me when rendering within a jinja template "however" when running the following code it outputs correctly to stdout when using 4 backward slashes -

- hosts: 127.0.0.1
  vars:
    php_config:
      - php
      - php55
  tasks:
    - name: "Step 1"
      debug:
        msg: "{{ php_config | map('regex_replace', '(.*)', '\\\\1_aa') | list}}"

I needed this to work and discovered that running the following using 2 backward slashes works for me when rendering within a jinja template "however" when running the following code it outputs correctly to stdout when using 4 backward slashes -

- hosts: 127.0.0.1
  vars:
    php_config:
      - php
      - php55
  tasks:
    - name: "Step 1"
      debug:
        msg: "{{ php_config | map('regex_replace', '(.*)', '\\\\1_aa') | list}}"

def modify_list(values=[], pattern='', replacement='', ignorecase=False):
    ''' Perform a `re.sub` on every item in the list'''
    if ignorecase:
        flags = re.I
    else:
        flags = 0
    _re = re.compile(pattern, flags=flags)
    return [_re.sub(replacement, value) for value in values]

class FilterModule(object):
    def filters(self):
        return {'modify_list': modify_list}

example

- set_fact: m_dirs={{ dirs | modify_list('(.*)','\\1/aba') }}

def modify_list(values=[], pattern='', replacement='', ignorecase=False):
    ''' Perform a `re.sub` on every item in the list'''
    if ignorecase:
        flags = re.I
    else:
        flags = 0
    _re = re.compile(pattern, flags=flags)
    return [_re.sub(replacement, value) for value in values]

class FilterModule(object):
    def filters(self):
        return {'modify_list': modify_list}

example

- set_fact: m_dirs={{ dirs | modify_list('(.*)','\\1/aba') }}

I think I managed to achieve a similar thing doing this:

{{ ansible_play_hosts | zip_longest([], fillvalue=':2181') | map('join') | join(',') }}

Looks ugly, but given ansible_play_hosts=['a', 'b'] it produces a:2181,b:2181

I think I managed to achieve a similar thing doing this:

{{ ansible_play_hosts | zip_longest([], fillvalue=':2181') | map('join') | join(',') }}

Looks ugly, but given ansible_play_hosts=['a', 'b'] it produces a:2181,b:2181

Doubling double back-slash worked for me :)
Item : ["intl","mysqlnd","curl","ldap","gd","dom"]

Action : "{{ item|map('regex_replace','^(.*)$','php5-\\\\1')|list }}"

Doubling double back-slash worked for me :)
Item : ["intl","mysqlnd","curl","ldap","gd","dom"]

Action : "{{ item|map('regex_replace','^(.*)$','php5-\\\\1')|list }}"

@EddyP23 excellent solution!
I updated it for readability, moving the colon into the map statement:
{{ ansible_play_hosts | zip_longest([], fillvalue='2181') | map('join', ':') | join(',') }}

This way it becomes a bit more obvious what the mapped join does.

@EddyP23 excellent solution!
I updated it for readability, moving the colon into the map statement:
{{ ansible_play_hosts | zip_longest([], fillvalue='2181') | map('join', ':') | join(',') }}

This way it becomes a bit more obvious what the mapped join does.

When there is a need to add both prefix and suffix (and making everything a list), look at:

  set_fact:
    extended_etcd_endpoints_list: "{{ groups['etcd'] | map('extract', hostvars, ['ansible_default_ipv4','address']) | map('regex_replace', '^(.*)$','https://\\1:2379') | list  }}"

What is does: takes the list of all machines in the group etcd, extracts the ipv4 address, adds a prefix of 'https://' and a suffix of ':2379'.
At the end, everything is transformed to a list.

When there is a need to add both prefix and suffix (and making everything a list), look at:

  set_fact:
    extended_etcd_endpoints_list: "{{ groups['etcd'] | map('extract', hostvars, ['ansible_default_ipv4','address']) | map('regex_replace', '^(.*)$','https://\\1:2379') | list  }}"

What is does: takes the list of all machines in the group etcd, extracts the ipv4 address, adds a prefix of 'https://' and a suffix of ':2379'.
At the end, everything is transformed to a list.