Skip to content

Instantly share code, notes, and snippets.

@reetp
Last active November 23, 2023 00:19
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save reetp/a66149d5f060f260643a353ca7067a98 to your computer and use it in GitHub Desktop.
Save reetp/a66149d5f060f260643a353ca7067a98 to your computer and use it in GitHub Desktop.
Debugging

Rocket.Chat - Asking for help

For users with issues with Rocket.Chat please do yourselves a few favours before asking for help.

Before you start.

First, you must ALWAYS test on the latest version of Rocket.Chat - you can also try https://open.rocket.chat which always runs development versions. A large amount of issues are fixed with an update.

You are going to need to provide some basic information if you want a chance of resolving your issue.

If you haven't got this information to hand please go and get it now.

At a minimum you will need this:

  • Server Setup Information:

    • Server hardware: VPS/hypervisor/bare metal
    • Version of Rocket.Chat Server:
    • Operating System: Ubuntu/Redhat etc
    • Deployment Method: snap/docker/tar/etc
    • Number of Running Instances:
    • DB Replicaset Oplog:
    • NodeJS Version:
    • MongoDB Version:
  • Client information:

    • Client type: Electron app/React-Native app/Browser and version
  • A list of the steps requires to replicate the issue.

    • If we can't replicate it we can't debug it.

Contents

How to ask for help

To pay or not to pay

If you think your issue is urgent, then you have two choices.

a) Pay for support - Pricing

Email: support@rocket.chat

b) Patiently wait for a volunteer to consider helping you.

You are entitled to exactly what you paid for

No more, and no less.

Please remember:

Abusing any community member will get you banned completely.

  • YOU are the one with the problem. You would not be here otherwise
  • Most people in the chat are volunteers who will try and help for free
  • Volunteers do this in their spare time and are not getting paid for this. Respect them
  • Just because it is urgent for you does mean that it is urgent for them
  • Your time might be valuable. Theirs is invaluable
  • They have jobs to do and mouths to feed. Don't waste their time
  • Answer their questions. They are not asking for fun but to try and understand
  • Being rude to them will not help you at all
  • Repeating your question 10 times in one channel will not help you.
  • Posting the same question in several channels will not help you
  • Do not DM anyone unless you know them, or they DM you first
  • Do not randomly @ people or @all hoping to get attention

Abuse of these things you will just get you muted, blocked or banned and you will not get any help

Don't open a bug yet

Do NOT open a new issue in github until you have read the rest of this document. Most 'issues' are configuration issues, not bugs.

Do NOT ask a question, wait for 30 seconds, and then decide you are way too important to wait around and your client is nagging and you'll open a bug to get it fixed faster. It will probably just get closed unless it is well documented and really is a bug.

If you ask in the channels or forums then devs or other community helpers will tell you if you really need to open an issue.

Frequently if there really IS a problem you won't be the first to experience it, so always check github carefully for duplicates. Use lots of different search terms and make sure you check closed Issues as well (see below)

Do not ask the same question in 5 different channels. Just one channel please.

Common channels are:

Support

React-Native

Ubuntu Snap

Raspberry Pi

Omnichannel

Desktop

Federation

Is it a feature request?

If Rocket works but doesn't do something the way you want it to then this is a feature request NOT a bug. Open a request in Feature Requests

RTFM

Have you read ALL the documentation? At least twice?

A lot of the answers can be found there. Read through slowly and carefully.

It is easy to miss the obvious because you think you know the answer you are just looking for confirmation of this , rather than reading to actually understand (and discovering you might just have been wrong)

Search

Have you looked through the bug reports? At least 2 different searches? Opened AND Closed bugs?

A lot of answers can be found there.

Check the various repos - Rocket itself, the mobile apps, electron etc.

Rocket.Chat server

Rocket.Chat server feature requests

React Native Mobile Client

Electron desktop application

Rocket.Chat documentation

When searching change the filter box to both open and closed issues like this:

is:issue is open

to

is:issue my search word

e.g.

is: issue theme

Try some variations - you won't hit the sweet spot first time usually. And the more often you do this the better you will get at learning what to search for and how. It is an art.

Still stuck?

Don't open an Issue yet.

The next step is to ask your well documented question in the forums or Rocket.Chat

How to Report Bugs Effectively

It is impossible to diagnose something when you just say 'it's broken'. The helpers are not mind readers. They cannot see your screen or your your set-up. They have not watched you set this up, and the mistakes you made along the way. They are effectively blind. So you have got to describe things clearly and concisely.

Your problem may appear complex but it is often the basics that catch you out. It is hard to tell at first if a problem is simple or complex so please be patient if you get asked 'the stupid questions'. We are just trying to get a baseline and making sure you have done the basics, and trying to understand how you got to this position. It is extremely common for people to miss steps in their set-up.

Don't hide things away because you are embarrassed. They are most likely going to find out, so you might as well be honest. We have ALL made mistakes.

This is a really good guide on how to report issues clearly and (more importantly) how to get the attention of developers and helpers - the people who have the solutions to your issue.

How to ask a smart question that will attract attention

This one is very relevant and it can waste a lot of time:

The XY Info problem

Tell us about the problem you were originally trying to solve, not the problem that you are experiencing right now. Frequently the one you are experiencing now is the result of not understanding or being able to fix the original issue.

Remember, if you phoned the garage about your car and said 'it runs funny sometimes' what would they ask you? Make, model, age... etc.

This is no different.

Use this as a template to ask your question. Get all the right information. It will help if you then have to submit a real Issue.

Complete a bug template

Server Set-up Information:

  • Version of Rocket.Chat Server:
  • Operating System:
  • Deployment Method:
  • Number of Running Instances:
  • DB Replicaset Oplog:
  • NodeJS Version:
  • MongoDB Version:

Some background will help, and will avoid XY issues. Is it a fresh install? Is it an upgraded install? Has it always been broken? When it did the issue start? Did you make any changes that could have caused it?

Make sure you post some BRIEF log snippets or use https://pastebin.com/

Don't block up the channel with ridiculously long messages.

If your problem relates to a connection to say a database, or LDAP, that might be on another server then check the logs for those as well.

Here is a great example of a good bug report:

Good bug reporting example

Last, be patient. Answer the questions you are asked. They may appear odd, but they are probably for a good reason. Debugging is a logical process and you need to help establish the facts first.

Remember, you are the one with the problem. They can just go back to work or their wife and kids and ignore you.

A classic is:

I want to know how to do X

Why do you want to do X?

Because I do.

But X is not a very good thing to be doing.

But I want to.

Tell us why because there may be a different solution, even though it might not be the one you want to hear.

Shut up bozo. You know nothing (So why are you asking us then?? Confirmation bias...). Just tell me what I want to know

I can tell you that is not the correct solution and will cause you untold grief and in a few days you'll be back complaining your system is broken/hacked/otherwise shredded and you have thousands of users shouting at you.

And you'll probably starting asking with "HELP URGENT HELP REQUIRED NOW"


Remember it is you asking for help. It is you with the problem. So please don't bite the hand that feeds you.


Open an Issue

Got this far with no solution? If you haven't already been told to do so then open an Issue with all the right information in the right repo.

Please use the templates provided. Add all the relevant information.

Stick to facts, not opinions, assumptions or guesses.

Rocket.Chat server

Rocket.Chat server feature requests

React Native Mobile Client

Electron desktop application

Rocket.Chat documentation

FAQ

Slow connections. iOS connection error

95% or more of these are due to improperly configure reverse proxies where the websocket is not working properly

If a websocket cannot be correctly established, the client will try and fall-back to an XHR connection. It is extremely slow and flaky. The iOS app will not connect at all without a websocket.

Please use Chrome/ium and dev tools to check for the presence of a websocket.

Websockets

  • Right Click, Inspect

  • Network, F5 to refresh

  • Look for the WS filter

  • Should see websocket

    websocket - wss://open.rocket.chat/sockjs/123/pbiz45rm/websocket

Click it and look on the right for 'Headers'

Has this got a green light?

 Status Code: 101 Switching Protocols

What do the headers say?

Response headers
 Connection: Upgrade
 Sec-WebSocket-Accept: nXd/PfaHuA1Kx/ggJWAAy+96Ee4=
 Upgrade: websocket

Notifications

A lot of issues with notifications are down to set-up/configuration

Please read this for the current notification logic:

Push Notification Logic

General debugging

It isn't a bug unless you can repeat it.

So, make a note of everything you do. I always use a note editor and paste in histories of commands, logs, files. changes, hacks or whatever.

I tidy up afterwards and then test that I can repeat it.

If I can I know I have a case.

The more you document, the easier it is for a dev to find, and fix.

Make sure you have all the correct versions of software used.

Start your little history. Be clear. Be concise. Don't waffle. Stick to the facts, not your guesses.

Add your documentation.

Sit back and feel smug :-)

Upgrading Rocket.Chat

Upgrading is a fraught business at any time. First you have Rocket itself, and then the Mongo database.

Snaps should auto-update so this is aimed at those who use other methods.

  • Rule One. Backup. Lots of them and lots of ways, and practice restores as well.
  • Rule Two. Use virtual machines for testing. They are so cheap.

Upgrading Mongo

  • Did I mention backups? Read how to do this. Practice it.
  • Use the Mongo documentation. There are leaps between 2.x and 3.x
  • Then there are more leaps as you proceed trough 3.x and then again to 4.x
  • As of writing 3.4 is going EoL so you need to be on 3.6
  • For now I would get to 3.6 and then start preparing for 4.x
  • To convert from mmapv1 -> wiredtiger you will need to backup, convert, restore

Upgrading Rocket

Did I mention backups? Yup - boring. Until you need them.

The problem with Rocket is not the actual code itself, but the fact that most updates also update the database schema - the core layout.

Now, theoretically apart from adding oplogs to Mongo for Rocket 1.x + you should be able to merrily jump from say 0.62 through to todays 2.1.x

Except..... those schema changes. If you are going to hit a problem, it will be one of those.

The devs test changes from one version to another, and maybe even over a few versions. But not every single version up to latest or say 0.74.3 -> 1.3.2

That's where it may go wrong.

So a recommend path is something like this:

-> 0.73.x -> 1.0.x -> 1.32 -> 2.0.x -> 2.1.x

Or possibly even more cautious:

 -> 073.x - 1.0.x -> 1.1.x -> 1.2.x -> 1.3.x -> 2.0.x -> 2.1.x

( 'x' means the last patched version e.g. 1.3.2 Check tags or releases )

I have seen people make some big jumps. And some disasters too. If in doubt, take you time.

And did I mention backups?

@reetp
Copy link
Author

reetp commented Aug 13, 2019

Thanks to https://stackedit.io/app for helping with the Markdown :-)

@reetp
Copy link
Author

reetp commented Oct 16, 2019

ToC added using Atom editor with https://atom.io/packages/markdown-toc

@reetp
Copy link
Author

reetp commented May 10, 2021

Another editor I found:

https://jbt.github.io/markdown-editor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment