I promised /u/thirty-five- that I would give him another writeup for the wiki, and I've got some time today so I figured why the hell not tackle drivetrain and derailleur compatibility?
As usual, the late great Sheldon Brown is a fantastic resource, but following his passing in 2008 it is rather hit or miss whether you can find compatibility information on newer stuff.
To start off with a disclaimer, this information is compiled from a variety of sources and while I believe it to be correct, there could be mistakes. Feel free to point them out and I will correct them.
To start off, we must explain how a derailleur works. At the simplest level, the point of the derailleur is to move the chain sideways at the point where it meshes with the chainrings or sprockets, forcing the chain onto the adjacent chainring or sprocket. In ye olden days, that meant thou wouldst taketh thine friction paddle shifter and pusheth or pulleth thine shifter until the chain doth droppeth upon thy desired sprocket, then thou must procedeth to attack thine enemies. Eventually, people got sick of this, and modern indexed shifters have specific detents that line up with each cog, so you shift it one click to the next cog. This introduces quite a bit more challenge in terms of compatibility and maintenance. In a friction shifted drivetrain, the rider figures out where the cogs are by feel, and there is no need for cable tension adjustments or having a certain number of cogs or whatever – you kept pulling until it lined up. So for the purposes of this discussion, I’m going to focus on 8,9,10, and 11 speed Shimano drivetrains. 8 or less speeds, Sheldon Brown has much better info than I could put together.
Let’s take a step back in time to Shimano 8 speed indexed shifting. You have a cassette with 8 sprockets. Each sprocket is 1.8mm thick, with a 3mm spacer between each one, making a 4.8mm center to center from the adjacent sprockets, and a total cassette width of 35.4mm. To shift gears, the derailleur must slide the cage 4.8mm to the side. The key thing to understand here is that the derailleur provides a mechanical gain ratio of 1.7 on an 8-speed group. 1mm of cable pull from the shifter moves the derailleur 1.7mm. When each shift occurs, the shifter pulls the cable 2.82mm and then the derailleur, with its 1.7 mechanical gain ratio, moves 4.8mm.
Now let’s advance to Shimano 9 speed indexed shifting. We’ll keep the same width freehub, so to make things nicely the sprockets must get thinner and closer together. Now we’re down to 1.78mm sprockets with 2.56mm spacers, giving a center-to-center spacing of 4.34mm and a total width of 36.5mm (which still fits on our old 8 speed freehub). The cool thing is that our derailleur doesn’t give two shits how many sprockets we have. The cable moves X mm, it moves 1.7X mm. It doesn’t know where the sprockets are. What needs to change is that now, instead of pulling 2.82mm of cable per shift, we need our shifter to pull 2.55mm of cable, which will get multiplied by 1.7 and give us the desired 4.34mm.
Well, that was easy. Let’s take our same old 1.7-ratio rear derailleur and make a 10 speed cassette. Now we are at 1.6mm sprockets with 2.35mm spacers, giving us 3.95mm center to center and a total width of 37.2mm. Shimano gives us a 1mm spacer to install our new cassette on a 8/9 speed freehub. Divide 3.95mm by 1.7, you know the drill, our shifter now needs to pull 2.3mm of cable. Here’s the problem, though. 2.3mm is not a whole lot of cable. Our system is now getting quite sensitive to cable adjustment. It’s like trying to steer a bike with a really short stem. It doesn’t take a whole lot of error to throw us off. And our cable is under a lot of tension. This works okay with the 10 speed groups with side-exit shift cables (Dura Ace 7800, Ultegra 6600, 105 5600, Tiagra 4600). But once Shimano finally caught up to the cool kids and is doing under-the-tape routing of the shift cable, this cable has to go through a whole bunch of sharp bends, and our shifting isn’t as crisp as it used to be on D-A 7900, Ultegra 6700, and 105 5700.
Now, on the mountain bike side of things, Shimano knew this was going to be a problem once you have dirty cables on your mountain bike. So they thought to themselves “Hey, what if we made the derailleur give us less of a gain?”. A longer cable pull per shift would allow more tolerance in our system. The downside would be a bit more force to shift, but now it’s the 2000s and we have all sorts of great low-friction cables and housings, so they figured it would be okay. And in typical Shimano style, they launched it with a cool albeit confusing name, Dyna-Sys. Our new 10-speed Dyna-Sys derailleur now only moves 1.2 times as far as the cable pulls it, not 1.7 like our old derailleur we used for 8 and 9 speed road and mountain, and for 10 speed road. Our 10 speed sprocket is still spaced at 3.95mm, but now with our 1.2x rear derailleur, our 10 speed Dyna-Sys shifter pulls 3.29mm of cable instead of 2.3mm, which means we don’t need as much force on the cable and we’re more tolerant of it being a little out of whack.
Sticking with MTB for a moment, 11 speed added another sprocket. But because we’ve got a gigantic 11th cog (all the cool kids are running 11-40 cassettes or bigger), we don’t need to make the freehub any wider – we can use a spider to hang that bigass 40T cog over the hub flange, and keep our 8/9/10 speed freehub. Now, I personally don’t have any 11s Shimano mountain stuff, so I can’t verify what the pull ratio of the 11s MTB derailleur is, or the cog center-to-center spacing. I’ve read one source that claims the derailleur is now 1.1x (even less than DynaSys 10 speed’s 1.2x), and that the cog spacing is 3.9mm center-to-center.
Back to road, we aren’t as lucky. No way is a 28T cog going to clear a hub flange. We need to make the freehub 1.85mm wider. Don’t worry, we give everyone a 1.85mm spacer if they want a 8/9/10 speed cassette. And if you’re doing 10 speed, you need to use that and the 1mm spacer we gave you with the 10s cassette. Why did we make 10s narrower than 9s again? Don’t ask questions. So now we’ve gone down to only 3.72mm center-to-center spacing on our cogs. We get the brainiacs in the room to crunch the numbers. 3.72mm divided by 1.7 is only 2.19mm. Pocket protectors get thrown on the floor. This is bullshit, this ain’t going to shift. Our 7900/6700/5700 10-speed under-the-tape 1.7x groupsets break their shift cables every 2,000 miles already and we can’t keep making the cable pulls shorter. Luckily, we can solve this problem, we literally just solved it for MTB with a 1.2x Dyna-Sys rear derailleur. Don’t get your hopes up though, we can’t make stuff interchangeable. What’s a good number between 1.7 and 1.2? Why not a 1.4x ‘Goldilocks’ rear derailleur? A 1.4x ratio gets us up to 2.66mm of cable pull per shift. That’s even more than our old 9 speed! Woohoo! The boys in the lab are cheering. We launch Dura Ace 9000 (take that, Vegeta!) with a 1.4x rear derailleur, and everyone promptly throws their 10 speed under-the-tape groups in the trash in the rush to turn this bitch up to 11 (speeds). This trickles down to Ultegra 6800, 105 5800, and is retained for Dura Ace R9100 and Ultegra R8000.
All is well in Shimano land. Remember, we never made a Tiagra derived from the 7900/6700/5700 groups. Tiagra 4600 has those side exit shift cables, and without all those bends, pulling 2.3mm per shift ain’t pretty, but it gets the job done. But the market research is in. People think under-the-tape shift cables are so hot right now. Plus, the all powerful bar tape lobby is pushing for a way to trick people into replacing their bar tape whenever they replace their shift cables. Basically, the stars align and Tiagra 4700 is going to need to have under the tape cables. We don’t want to try to pull 2.3mm of cable any more. What if we take our Goldilocks 1.4x rear derailleur and use that? Now we’re pulling a whopping 2.82mm of cable, the same as our ancient prehistoric 8-speed stuff, even more than our fancy pants 11 speed groups. And we don’t need to redesign a rear derailleur, let’s just take a 105 11-speed derailleur, find a way to make it a couple bucks cheaper, paint it a different color, and call it a Tiagra 4700 10-speed rear derailleur.
Much to the chagrin of people who previously enjoyed making sweeping generalizations such as “any N speed Shimano stuff works with any other N speed Shimano stuff”, who already had to eat crow and modify it to “N speed stuff except Dyna-Sys”, we’ve now got what’s basically a cheapified 11 speed group that shifts on 10 cogs. Which is actually quite nice if you dropped a boatload of money on some blingy carbon wheels with a 10 speed freehub and now you can’t go 11 speed without offering three month’s of salary to your new fiancé Miss Zipp. Although if you had just waited another two years, you would have seen Shimano release the HG800 11-34T 11-speed cassette that fits on a 10 speed freehub (remember that trick the MTB guys did with their big ol’ 40T cog? Turns out we can pull the same trick with a 34T cog and make a great climbing cassette for people with non-11S wheels)
communist under the bar tape cable routing resolution to the proletariat, Shimano gave Sora R3000 and Claris R2000 the same styling (so hot right now!) but because 9 and 8 speed stuff already had a decently long pull, they just left that stuff with the ‘Oldschool’ 1.7x rear derailleur pull ratio, meaning any old 1.7x derailleur will work with them.
As an aside, let me touch on front shifting. In ye olden days, you had two kinds of front shifters – doubles and triples. They used a relatively short cable pull, with “short arm” front derailleurs. With R9000, Shimano decided that we all need to harden the fuck up and jettison our granny rings, so no triples were offered for 11 speed. The double FD got replaced with a “long arm” that does essentially the same thing as they did on the back – you’re pulling the cable further, with less force to change gears. Long arm FDs work great most of the time, but they are sensitive to the angle at which the cable comes up towards the FD. With the 9000, Shimano included some sort of mystical divining ride for you to figure out where on the arm to attach the cable to get that slick and beautiful long arm shifting. This long arm style FD trickled down to the budiget-concious Freds’ 11 speed groups, Ultegra 6800 and 105 5800. But it didn’t stop there – Tiagra 4700, Sora R3000, and Claris R2000 all got long-armed. The only exception to this is that the triple crankset versions of these get the old short arm triple FD and pull ratio, since there’s no such thing as a long arm triple. Triples aren’t dead! Triples will rise again!
Anyway, the magic cable routing guide tool confused and infuriated many people. In an effort to appease the masses, Shimano said “how can we make a really complicated front derailleur that acts like a long arm, is as compact as a short arm, allows housing to be run all the way to it, includes tension adjustment, and is more complicated than a Rube Goldberg machine?”. They achieved this lofty goal with the new R9100 “toggle” style front derailleur, which has cable routing out of a MC Escher drawing and requires a Master’s degree in Euclidean geometry to set up, but did it’s job. The pros got this with the Dura-Ace R9100 generation, and it trickled down to the dentists with Ultegra R8000. Some peasant with 105 begged Shimano for this fantastic new design, and the CEO’s wife said “Let them eat cake!”. Thus was born the 105 5801 front derailleur, which gives you bourgeois front shifting and proletariat rear shifting, ironically relegating Long Arm to the 8/9/10 speed groups.
I should also add as an addendum that at some point along the line, Shimano introduced a Shadow (or “Direct Mount” rear derailleur scheme that moves the pivot point back. This changes how the derailleur looks and how it moves, but it not related at all to pull ratio. Various Shadow derailleurs exist for 1.7x Oldschool, 1.4x Goldilocks, and 1.2x Dyna-Sys. So don’t get confused there.