GSoC'25 at Zulip - Work Product

gsoc-2025-at-zulip-work-product.md

Google Summear of Code 2025 at Zulip - Work Product

Hi, I’m Mohammad Reza Kianifar, and this is a summary of my work contributing to Zulip during Google Summer of Code 2025.

My project, Migrate Zulip’s Direct Message Recipient Data Structures to a New Model, focused on migrating 1:1 conversations from the legacy Personal recipient model to the DirectMessageGroup model.

This migration enables:

• Per-conversation settings for all direct messages
• Enhanced features such as pinning/hiding DM conversations
• A unified recipient model across all direct messages

📋 Project Information

Title	Migrate Zulip’s Direct Message Recipient Data Structures to a New Model
Size	Medium (175+ hrs)
Duration	May 2025 - September 2025
Organization	Zulip
Issue	#25713
Mentor	Sumanth Rao
Technologies	Python, PostgreSQL, Django, Database Migration
Contributor	Mohammad Reza Kianifar

🏗️ Architecture Background

Each Zulip message has a sender and a recipient. Before this project, Zulip used three recipient types:

1. Personal → Single user (1:1 and self DMs)
2. Stream → Channel/topic-based messages
3. DirectMessageGroup → Group conversations (with more than one user))

The migration transitions 1:1 and self messages from type #1 to type #3 while maintaining full backward compatibility during the process.

Once complete, the legacl Personal‍ type (including its code paths and related database columns) can be removed.

🚀 My Contributions

Phase 1: Foundation & Compatibility Layer

In this phase, I added the support for using DirectMessageGroup as a safe alternative to Personal for 1:1 and self DMs. These changes did not affect production yet, since such DirectMessageGroup models were not created at that time. The PRs including:

1. direct_messages: Use DM groups for 1:1 or self DMs if present. #34307 - ✅ Merged
   • A preparatory refactor for migrating the internal structure of Recipient objects for group DMs to use the DirectMessageGroup type, not the legacy Personal type. This step has the message-sending code path check if a DirectMessageGroupe exists and prefer it if available.events about muted users.
2. messages: Add unread 1:1/self DMs to pm_dict if DM group exists. #34787 - ✅ Merged
   • Overrides get_raw_unread_data behavior to populate pm_dict for 1:1 and self DMs when a DirectMessageGroup exists. This maintains the API compability.
3. export_search: Support --dm option for 1:1 DMs using DM group. #34789 - ✅ Merged
   • Updates the export_search command to first checks for a DirectMessageGroup for the two users and uses it if present. If not, we fall back to the legacy logic.
4. notification: Use existing email format for missed 1:1 DM via DM group. #34822 - ✅ Merged
   • Renders the email notification for missed 1:1 direct messages using DirectMessageGroup with the same format as messages sent to a Personal recipient.
5. push_notification: Use existing payload for 1:1 DM using DM group. #34864 - ✅ Merged
   • Generates notification payloads for 1:1 messages via DirectMessageGroup in the same format as those sent to a Personal recipient.
6. scheduled_messages: Set read_by_sender for self-DMs using DM group. #34842 - ✅ Merged
   • Ensures read_by_sender is set correctly for when scheduling a 1:1 message using DirectMessageGroup.
7. welcome_bot: Support auto-reply for 1:1 DMs using DM groups. #34852 - ✅ Merged
   • Ensures Welcome Bot auto-replies to users in 1:1 DMs, including when the DirectMessageGroup recipient model is used.
8. report_message: Use existing template for reporting 1:1 DM via DM group. #34884 - ✅ Merged
   • Builds the existing report message format for 1:1 DMs using DirectMessageGroup.
9. analytics: Support DM groups in count message type query. #35348 - ✅ Merged
   • Updates the analytics count query to treat DM groups of size 2 or less as private messages.
10. Support DirectMessageGroup when displaying scheduled messages to self. #35785 - ✅ Merged
    1. get_recipient_ids: Include user ID for self-messages using DM group.
       • Adds user ID for self-messages in get_recipient_ids when DirectMessageGroup is used.
    2. scheduled_messages: Add support for self-DM identification.
       • Shows "You" as the header for self-scheduled to improve the user experience. This is not only limited to when DirectMessageGroup is used.
11. settings: Add variable to control recipient type for 1:1 DMs. #35469 - 📋 Open
    • Adds PREFER_DIRECT_MESSAGE_GROUP setting to control the recipient type for 1:1 and self DMs.
    • Sets to False by default.
    • Uses @override_settings(PREFER_DIRECT_MESSAGE_GROUP=True) to test the new behaviour.
12. personal_recipient: Handle cases with none values. #34900 - 📋 Open
    • Handle cases where the Personal recipient is None to ensure DirectMessageGroup is a safe alternative option when Personal recipient does not exist.
13. Import 1:1 DMs as DirectMessageGroup if enabled. #35903 - 📋 Open
    1. import: Create DM group for 1:1 messages if enabled.
       • Uses DirectMessageGroup as the recipient type for 1:1 messages when importing from Mattermost and RocketChat.
    2. slack: Refactor functions to use dataclasses.
       • Uses DirectMessageGroup as the recipient type for 1:1 messages when importing from Slack.
       • Creates dataclsses to be used as return types of import Slack functions.

Phase 2: Migration

Here, I enabled DirectMessageGroup as the default recipient for 1:1 and self DMs, and created a backfill migration to update existing records. The PRs are:

1. Transition 1:1 DMs to use direct message groups when enabled #35769 - 📋 Open
   1. settings: Enable preference for DM group over personal recipient.
      • Sets the PREFER_DIRECT_MESSAGE_GROUP setting variable as True.
   2. direct_messages: Create DM groups for 1:1 DMs if enabled.
      • Updates the code path of selecting the recipient type for messages to create a DirectMessageGroup for 1:1 and self DMs if the setting variable is enabled. Otherwise, falls back to the legacy Personal recipient.
2. migrations: Add backfill to update personal DMs to rely on DM group. #35795 - 📋 Open
   • Updates recipients of 1:1 or self messages to rely on DirectMessageGroup. The affected models include Draft, ScheduledMessage, Message, and ArchivedMessage. I fetch all records and update the recipient type in batches for all direct messages between two (or one) users.

Phase 3: Clean-Up

Once migration is complete, legacy Personal code paths can be removed. The related PRs are:

1. personal_recipients: Stop creating when DM groups preferred for 1:1. #35347 - 📋 Open
   • Stops creating Personal recipients and their subscriptions when PREFER_DIRECT_MESSAGE_GROUP is True.
   • Affects user creation in single/bulk operations and data imports from other platforms.
2. personal_recipients: Delete legacy code paths. #36107 - 📋 Open
   • Deletes all code paths where the legacy Personal recipient was used.
   • Removes the PREFER_DIRECT_MESSAGE_GROUP setting variable.
3. migrations: Delete personal recipients. (TODO)
   • Deletes all recipients of type Personal
   • Drops UserProfile.recipient field.

Phase 4: Adding New Columns to DirectMessageGroup

Based on the community discussions in the Zulip channel, we decided to add three fields to the DirectMessageGroup model for efficiency and future use. This work is done in the following PR, including four commits:

1. Add first_message, last_message, and total_messages to DirectMessageGroup #36045 - 📋 Open
   1. dm_groups: Add fields for message statistics tracking.
      • Adds the new fields to the DirectMessageGroup model + the migration.
   2. message_send: Update direct message group statistics.
      • Sets/Updates the new fields when new direct messages are sent.
   3. retention: Update DM group statistics on archive and restore.
      • Updates the fields when archiving/restoring direct messages.
   4. migrations: Add backfill to populate DM group statistics.
      • Adds backfill migration to populate the new fields for existing DirectMessageGroups.

🔜 Future Work

I will continue working on the pending PRs to get them merged, finalize the migration, and complete clean-up tasks.

💡 Personal experience

Working on this project has been one of the most impactful experiences of my career. I learned a great deal about large-scale migrations, the rebase Git strategy for managing PRs and commits, and Zulip’s development practices and architecture.

I’ve grown as a software engineer and look forward to continuing my contributions to Zulip, exploring other areas of the codebase, and learning from the amazing community here.

🌟 Acknowledgements

I would like to thank Google and Zulip for giving me this opportunity to work on such an exciting project.

A special thanks to Tim Abbott for his invaluable feedback and collaboration, which greatly influenced the quality and progress of this work.

I truly appreciate the constant support, guidance, and accurate reviews of to my mentor Sumanth Rao.

I also sincerely thank Rohan Gudimetla and the entire Zulip community for their helpful reviews, feedback, and encouragement throughout the project.