It seems we can't toggle a filter per buffer (we have @ but it applies to all the filters of the buffer). Here is a hack that enable or disable the irc_smart filter globally depending on a buffer localvar
/script install buffer_autoset.py
/key bind meta-! /mute -core /eval /toggle buffer_autoset.buffer.${buffer.full_name}.localvar_set_irc_smart 1 0;/mute -core /eval /filter ${if:${buffer_autoset.buffer.${buffer.full_name}.localvar_set_irc_smart}!=?enable:disable} irc_smart
/trigger addreplace irc_smart_toggle signal buffer_switch
/trigger set irc_smart_toggle command "/mute -core /filter ${if:${buffer_autoset.buffer.${buffer[${tg_signal_data}].full_name}.localvar_set_irc_smart}?enable:disable} irc_smart"
Then, on buffer switch, if the buffer have a irc_smart localvar, it will enable the filter. To add the localvar irc_smart to a buffer, press meta-!