Don't you need to add new rewrite rule via "rewrite_rules_array" filter? Like this:
$rules = array( 'my_regexp_url' => '?raw_url' );
return $rules + $incoming;
I haven't tried that yet, but I thought add_rewrite_endpoint would handle creating the rewrite rules. It works on standard posts, pages, and the main blog without adding any rewrite rules.
You're probably correct there, I never used endpoint before and looking at the codex it seems should be handing rewriting as well.