Get a shell on Exchange Online by following Microsoft's guide.
Publish the calendar for the room:
Set-MailboxCalendarFolder RoomNameHere:\Calendar -DetailLevel fulldetails -PublishEnabled $true -reseturl
Show the .html link, as well as the .ics link: