slack2html

slack2html.php

<?
/////////////////////
// slack2html
// by @levelsio
/////////////////////
//
/////////////////////
// WHAT DOES THIS DO?
/////////////////////
//
// Slack lets you export the chat logs (back to the first messages!), even if
// you are a free user (and have a 10,000 user limit)
//
// This is pretty useful for big chat groups (like mine, #nomads), where you
// do wanna see the logs, but can't see them within Slack
//
// Problem is that Slack exports it as JSON files, which is a bit unusable,
// so this script makes it into actual HTML files that look like Slack chats
//
///////////////////////
// CHANGES IN THIS FORK
///////////////////////
//
// This fork was created to allow the script to also work for Slack history exported using
// https://github.com/hisabimbola/slack-history-export - a command line module to allow you to download your Slack history.
//
// When you export all your history from Slack, it does not include history from private groups/channels and DMs. The
// slack-history-export tool provides a way to export history from these private groups/channels and DMs, but the
// original version of this script was not compatible with the exported JSON files from this tool. This fork updates
// the script to make it compatible so that you can combine both your exported public channel history from Slack, and
// your private group/channel and DM history from the slack-history-export tool and convert all of this to HTML.
//
// The slack-history-export tool exports individual JSON files, but this script expects all history to be contained in
// folders, so you need to put each JSON file inside its own folder and place these folders alongside the folders for
// the public channels exported from Slack. The name you give to the folder is the name that will be used for the
// private group/channel or DM when it appears in the index.html file, and will also be the name of the HTML file that
// is generated for the private group/channel.
//
// For example, if the file exported from the slack-history-export tool is called
// 1461648973171-my-private-group-history.json then you should place this file inside a new folder called
// my-private-group and then put this my-private-group folder alongside the folders for the public channels exported
// from Slack.
//
///////////////////
// INSTRUCTIONS
///////////////////
//
// Run this script inside the directory of an extracted (!) Slack export zip
// e.g. "/tmp/#nomads Slack export Aug 25 2015" like this:
// MacBook-Pro:#nomads Slack export Aug 25 2015 mbp$ php slack2html.php
//
// It will then make two dirs:
// 	/slack2html/json
// 	/slack2html/html
//
// In the JSON dir it will put each channels chat log combined from all the
// daily logs that Slack outputs (e.g. /channel/2014-11-26.json)
//
// In the HTML dir it will generate HTML files with Slack's typical styling.
// It will also create an index.html that shows all channels
// 
///////////////////
// FEEDBACK
///////////////////
//
// Let me know any bugs by tweeting me @levelsio
//
// Hope this helps!
// 
// Pieter @levelsio
//
/////////////////////

    ini_set('memory_limit', '1024M');
	date_default_timezone_set('UTC');
	mb_internal_encoding("UTF-8");
	error_reporting(E_ERROR);

	// <config>
		$stylesheet="
			* {
				font-family:sans-serif;
			}
			body {
				text-align:center;
				padding:1em;
			}
			.messages {
				width:100%;
				max-width:700px;
				text-align:left;
				display:inline-block;
				word-wrap: break-word;
			}
			.messages img {
				background-color:rgb(248,244,240);
				width:36px;
				height:36px;
				border-radius:0.2em;
				display:inline-block;
				vertical-align:top;
				margin-right:0.65em;
			}
			.messages .time {
				display:inline-block;
				color:rgb(200,200,200);
				margin-left:0.5em;
			}
			.messages .username {
				display:inline-block;
				font-weight:600;
				line-height:1;
			}
			.messages .message {
				display:inline-block;
				vertical-align:top;
				line-height:1;
				width:calc(100% - 3em);
			}
			.messages .message .msg {
				line-height:1.5;
			}
		";
	// </config>

    // <compile daily logs into single channel logs>
		$files=scandir(__DIR__);
        $baseDir=__DIR__.'/../slack2html';
		$jsonDir=$baseDir.'/'.'json';
		if(!is_dir($baseDir)) mkdir($baseDir);
		if(!is_dir($jsonDir)) mkdir($jsonDir);

		foreach($files as $channel) {
			if($channel=='.' || $channel=='..') continue;
			if(is_dir($channel)) {
				$channelJsonFile=$jsonDir.'/'.$channel.'.json';
				if(file_exists($channelJsonFile)) {
					echo "JSON already exists ".$channelJsonFile."\n";
					continue;
				}

				unset($chats);
				$chats=array();

				echo '====='."\n";
				echo 'Combining JSON files for #'.$channel."\n";
				echo '====='."\n";

				$dates=scandir(__DIR__.'/'.$channel);
				foreach($dates as $date) {
					if(!is_dir($date)) {
						echo '.';
						$messages=json_decode(file_get_contents(__DIR__.'/'.$channel.'/'.$date),true);
						if(empty($messages)) continue;
						foreach($messages as $message) {
							array_push($chats,$message);
						}
					}
				}
				echo "\n";

				file_put_contents($channelJsonFile,json_encode($chats));
				echo number_format(count($chats)).' messages exported to '.$channelJsonFile."\n";
			}
		}
    // </compile daily logs into single channel logs>

	// <load users file>
		$users=json_decode(file_get_contents(__DIR__.'/'.'users.json'),true);
		$usersById=array();
        $usersByName=array();
		foreach($users as $user) {
			$usersById[$user['id']]=$user;
            $usersByName[$user['name']]=$user;
		}
	// </load users file>

	// <load channels file>
		$channels=json_decode(file_get_contents(__DIR__.'/'.'channels.json'),true);
		$channelsById=array();
		foreach($channels as $channel) {
			$channelsById[$channel['id']]=$channel;
		}
	// </load channels file>

	// <generate html from channel logs>
		$htmlDir=$baseDir.'/'.'html';
		if(!is_dir($htmlDir)) mkdir($htmlDir);
		$channels=scandir($jsonDir);
		$channelNames=array();
		$mostRecentChannelTimestamps=array();
		foreach($channels as $channel) {
			if($channel=='.' || $channel=='..') continue;
			if(is_dir($channel)) continue;

			$mostRecentChannelTimestamp=0;

            if($message['ts']) {
                $message_timestamp = $message['ts'];
            }
            else {
                $message_timestamp = strtotime($message['date']);
            }
            
			if($message_timestamp>$mostRecentChannelTimestamp) {
				$mostRecentChannelTimestamp=$message_timestamp;
			}
			$array=explode('.json',$channel);
			$channelName=$array[0];
			
			$channelHtmlFile=$htmlDir.'/'.$channelName.'.html';
			if(file_exists($channelHtmlFile)) {
				echo "HTML already exists ".$channelJsonFile."\n";
				continue;
			}

			array_push($channelNames,$channelName);
			echo '====='."\n";
			echo 'Generating HTML for #'.$channelName."\n";
			echo '====='."\n";
			$messages=json_decode(file_get_contents($jsonDir.'/'.$channel),true);
			if(empty($messages)) continue;
			$htmlMessages='<html><body><style>'.$stylesheet.'</style><div class="messages">';
			foreach($messages as $message) {
				if(empty($message)) continue;
				if(empty($message['text'])) continue;
				echo '.';
				
				// change <@U38A3DE9> into levelsio
				if(stripos($message['text'],'<@')!==false) {
					$usersInMessage=explode('<@',$message['text']);
					foreach($usersInMessage as $userInMessage) {
						$array=explode('>',$userInMessage);
						$userHandleInBrackets=$array[0];
						$array=explode('|',$array[0]);
						$userInMessage=$array[0];
						$username=$array[1];

						if(empty($username)) {
							$username=$usersById[$userInMessage]['name'];
						}

                        if(empty($username)) {
                            $username=$usersByName[$userInMessage]['name'];
                        }

						$message['text']=str_replace('<@'.$userHandleInBrackets.'>','@'.$username,$message['text']);
					}
				}

				// change <#U38A3DE9> into #_chiang-mai
				if(stripos($message['text'],'<#')!==false) {
					$channelsInMessage=explode('<#',$message['text']);
					foreach($channelsInMessage as $channelInMessage) {
						$array=explode('>',$channelInMessage);
						$channelHandleInBrackets=$array[0];
						$array=explode('|',$array[0]);
						$channelInMessage=$array[0];
						$channelNameInMessage=$array[1];
						if(empty($username)) {
							$channelNameInMessage=$channelsById[$channelInMessage]['name'];
						}
						if(!empty($username)) {
							$message['text']=str_replace('<#'.$channelHandleInBrackets.'>','#'.$channelNameInMessage,$message['text']);
						}
					}
				}
				// change <http://url> into link
				if(stripos($message['text'],'<http')!==false) {
					$linksInMessage=explode('<http',$message['text']);
					foreach($linksInMessage as $linkInMessage) {
						$array=explode('>',$linkInMessage);
						$linkTotalInBrackets=$array[0];
						$array=explode('|',$array[0]);
						$linkInMessage=$array[0];
						$message['text']=str_replace('<http'.$linkTotalInBrackets.'>','<a href="http'.$linkInMessage.'">http'.$linkInMessage.'</a>',$message['text']);
					}
				}

				// change @levelsio has joined the channel into
				// @levelsio\n has joined #channel
				if(stripos($message['text'],'has joined the channel')!==false) {
					$message['text']=str_replace('the channel','#'.$channelName,$message['text']);
					$message['text']=str_replace('@'.$usersById[$message['user']]['name'].' ','',$message['text']);
				}

				$array=explode('.',$message['ts']);
				$time=$array[0];
                
                if($message['ts']) {
                    $time_formatted = date('Y-m-d H:i',$message['ts']);
                }
                else {
                    $time_formatted = date('Y-m-d H:i',strtotime($message['date']));
                }

				//$message['text']=utf8_decode($message['text']);
                $message_user_identifier = $message['user'];
                
                if(!$message_user_identifier) {
                    $message_user_identifier = $message['comment']['user'];
                }
                
                $message_user = $usersById[$message_user_identifier];

                if(!$message_user) {
                    $message_user = $usersByName[$message_user_identifier];
                }
                
                $message_text = $message['text'];

                // Convert quotes and apostrophes
                $message_text = str_replace('“', '"', $message_text);
                $message_text = str_replace('\u201c', '"', $message_text);
                $message_text = str_replace('”', '"', $message_text);
                $message_text = str_replace('\u201d', '"', $message_text);
                $message_text = str_replace('’', "'", $message_text);
                $message_text = str_replace('\u2019', "'", $message_text);

                // Convert new lines
                $message_text = nl2br($message_text);

                // Convert remaining unicode characters
                $message_text_working = json_encode($message_text);
                $message_text_working = preg_replace('/\\\u([0-9a-z]{4})/', '&#x$1;', $message_text_working);
                $message_text = json_decode($message_text_working);
				
				$htmlMessage='';
				$htmlMessage.='<div><img src="'.$message_user['profile']['image_72'].'" /><div class="message"><div class="username">'.$message_user['name'].'</div><div class="time">'.$time_formatted.'</div><div class="msg">'.$message_text."</div></div></div><br/>\n";
				$htmlMessages.=$htmlMessage;
			}

			$htmlMessages.='</div></body></html>';
			file_put_contents($channelHtmlFile,$htmlMessages);
			$mostRecentChannelTimestamps[$channelName]=$mostRecentChannelTimestamp;
			echo "\n";
		}
		asort($mostRecentChannelTimestamps);
		$mostRecentChannelTimestamps=array_reverse($mostRecentChannelTimestamps);
	// </generate html from channel logs>

	// <make index html>
		$html='<html><body><style>'.$stylesheet.'</style><div class="messages">';
		foreach($mostRecentChannelTimestamps as $channel => $timestamp) {
			$html.='<a href="./'.$channel.'.html">#'.$channel.'</a> '.date('Y-m-d H:i',$timestamp).'<br/>'."\n";
		}
		$html.='</div></body></html>';
		file_put_contents($htmlDir.'/index.html',$html);
	// </make index html>

?>

Thanks for the great idea, however it would be nice, if a little bit more introduction is provided to use this script for people without php knowledge. There is one other problem that this tool assumes that the input file is standard slack zip export. Normal users do not have the possibility to download slack archives and hence this script is only restricted to admins.

Actually, I found several other scripts but each with it's own limitation:

1. https://gist.github.com/Chandler/fb7a070f52883849de35 - This is the one I used and gives a dump of channel including private channels and messages. However unable to convert to suitable html format
2. https://gist.github.com/levelsio/122907e95956602e5c09- Requires php knowledge also assumes a standard slack zip export
3. https://github.com/hfaran/slack-export-viewer - Probably provides the desired functionality, but also assumes standard slack zip export
4. https://github.com/humor4fun/slack-backup/blob/master/slack-backup.sh - L
   functionality, but also assumes standard slack zip export
5. https://github.com/humor4fun/slack-backup/blob/master/slack-backup.sh - Linux based
6. https://github.com/joefitzgerald/slack-dump - Unsure about the programming language used and how the tool would work and how to install
7. https://www.npmjs.com/package/slack-history-export - Assumes that one have a working knowledge of npm. It took a while to actually install npm first and then install this tool. But after installation I'm not sure how to actually run this tool?
8. https://gist.github.com/jordanmkoncz/0ce0ce11a3359209f48949eefee945ce - This tool requires import from slack-history-export tool, which I find unable to run (See #6)

Any help guys.. some newbie instructions would be great

I'm afraid this doesn't grab any user info because slack-history-export doesn't generate a users file, which this seems to depend on. Is it possible to get this fixed?

How did you get it working with https://github.com/hisabimbola/slack-history-export without having the users file?

Getting:

$ ../php slack2html.php --help

Fatal error: Uncaught Error: Call to undefined function mb_internal_encoding() in C:\Users\PC118\Desktop\php-7.1.9-Win32-VC14-x64\slack\slack2html.php:76
Stack trace:
#0 {main}
thrown in C:\Users\PC118\Desktop\php-7.1.9-Win32-VC14-x64\slack\slack2html.php on line 76

I had to enable a module: https://stackoverflow.com/a/6658975/750216

Running the tool just gives empty json folder and an html file containing only the styles and no messages.

Example content of log.json that is not parsed correctly:

[{
  "type": "message",
  "user": "U2Q5YL49K",
  "text": "yes",
  "ts": "1504256348.000341",
  "timestamp": 1504256348000,
  "isoDate": "2017-09-01T08:59:08.000Z"
},{
  "type": "message",
  "user": "U0D0M9U1L",
  "text": "are you working on it?",
  "ts": "1504256342.000156",
  "timestamp": 1504256342000,
  "isoDate": "2017-09-01T08:59:02.000Z"
}]