Libertarianism when applied to the social and political structure of technology and open source (Meritocracy) perpetuates inequality by failing to acknowledge the role of power and current state of inequality.
American libertarianism is uniquely American, unfortunately Silicon Valley happens to be in America. It is a political and social philosophy that believes individual ownership and power should supplant collective ownership of any kind. Most immediately the ownership and interventionist powers of government should be dismantled and the free market should be left with the responsibility of creating equality and individuals expected to protect their own interests.
Filtered through the lens of technology this becomes a more unified theory of Meritocracy. Being that we don't trade goods for currency we don't have a traditional market and libertarians must invent one. A strange mix of experience, social capital, and skill are a sort of currency in technology communities. Meritocracy is the social theory that allows that currency and its exchange to become power the way actually currency becomes power in a capitalist free market.
This is where techno-libertarians and people attempting to build a system of equality collide. To build a more equitable system you have to re-distribute power. If there is an imbalance of race and gender (there obviously is) then it means the under-represented are not engaged and have no power under the existing standards of "merit." To alter representation we must alter the process and culture of technology creation to subvert the value of "merit" so that people without it can engage.
Attempting to alter existing gender or race imbalance is always in conflict with libertarian meritocracy because being black or female has no currency in the system they've created. Such a system must be called out for what it is: a process of subjugation that perpetuates existing power structures. Existing power happens to be white male dominated. Libertarians would then argue that the fact that existing power is white male dominated does not mean the system itself is racist or sexist because the system doesn't value race or sex, only "merit."
To which advocates of equality must respond that any system which perpetuates historical racism and sexism inherits those traits by willful ignorance.
Libertarianism is the philosophy of non-intervention. That no system should exist which attempts to mitigate the private exchange of currency, goods and services in a free market. Any intervention, even intervention that would create gender or racial equality, cannot be tolerated without admitting that the free market is a poor arbiter of freedom.
In the birth of a new culture where a market and currency do not exist as they do elsewhere libertarianism invents the tyranny of meritocracy so that we might extend even further the domination of those who have inherited power over those who currently lack it. If we are to create a more equitable culture we must reject it.
Can you clarify what core values are driving the opinions expressed here? I'm having difficulty understanding exactly what kind of utopia you're striving for here and what the most important things are that you feel so strongly in need of defending or rectifying. I would suggest that the disagreement you're having with what you're characterising (in a straw-man kind of way) as "American libertarianism" have mostly to do with core values and the outcomes that you think are important to achieve.
Can you give some specifics about where you believe that "merit" is currently set up in such a way as to exclude the minorities that you're concerned about? For instance, are you concerned about the representation of women as speakers at tech conferences; if so, what is the "merit" system that is holding them back now, why is it a bad system and and what is the ideal system that would supplant the existing one and how is that overall more beneficial and for whom exactly does it need to be beneficial for to be a "better" system? Just looking for some examples so I can understand better the core values driving these kinds of arguments.