The primary reason that results seem like they might be returned out of order is because there is a random element to the rank that we give each search document. This allows us to expose courses that might otherwise have been buried too far down the list to be seen, which both helps authors get exposure and improves the feedback that we get because we see users interact with more courses.
The randomness added to the query is seeded by the user id, so each time an individual user searches for something the results will be in the same order. However, if two different users attempted the same search, they might see results in different orders. This sort of behavior is fairly common among searches on marketplaces, like ecommerce websites, because it helps prevent winner-take-all effects.
The current behavior is that all results are ranked without randomness, and then the top ten highest ranking results are reordered in a random way. This means that you see no randomness after the tenth result. The amount of randomness that we add to the search may decrease as we learn more from search behavior, but it is unlikely to ever go away altogether.