CHAOSScon - D&I

Panel:
Diversity & Inclusion: Innovating to Create a Consistent Definition, Data, and Metrics.

Dawn Foster
Two aspects to research. Interested in metrics in general. Background in open source, community building.
Lots of time justifying my own existence. :) This data is helpful.

Not as much experience w/ diversity & inclusion specifically, but lots of interest.

As part of PhD lots of research on Linux Kernel.

=====

Emma Irwin: Developer for 15 years primarily in open source; incredible opportunity to build skills, find mentorship, etc. Moving towards helping other people find that. Moved towards Mozilla, where I can focus on enabling other peoples’ success in open source.

Designing strategy for diversity & inclusion in a global community. Different bandwidth, access to education…

Lots of qualitative research.

Take all shitty data we have, people asking different questions in different ways…. Pull it into some kind of cohesive narrative. Very hard.

Came out with initial metrics for working group, and determination to get standards + best practices for diversity & inclusion.

===

Sean.

Spent 15 years looking at peoples’  behavior online (not-evil NSA :D)
Lots of mini of GitHub and pulling data into different repositories.
Metrics to help people tell stories.
Dashboard to show metrics not useful without story you’re trying to tell.

Tool: Auger (?) we’re trying to build, lots of different pieces pulling in lots of different pieces of data.

Two things we’re doing:
1) Make all data available via an API to build your own front-end. Allow people to look at project you’re familiar wit and project you’re not familiar with
2) 

====

Jesus:
Been involved in D&I since may years ago.
Diverst n free software.
10 years ago.

My concern started to happen when there were large parts of the world where they use lots of free software, but not represented in FLOSS community (e.g. South America). Not just gender, but also geographical, language POV.

Working in metrics. Want to be able to hone in on marginalized groups.

====

Implications/problems/complexities?

Demographics.
Non-male contributors
Socioeconomic status — how do we think about that?
Age groups
Caregivers/parents…

How do we ensue we’re asking those questions for the right reasons? Not just for dashboard; being intentional.
Intentionality is super important; not quite there yet.

Ethics around Consent.
If someone gives personally identifiable information (email address) is it ok to scrape publicly available info from elsewhere?
Be empathetic about why we’re doing this…

We know diversity adds value
But remember that at the end of the day, it’s about people.


—

Work I do is mostly numbers. How it affects D&I is metrics don’t have objectivity. They’re precise, high degree of accuracy…. But just because it’s quantitative doesn’t make it objective. We make all kinds of decisions on how those numbers are interpreted, what story we’re trying to tell.

Need to step back and say “ok, these have value, but how are we going to use them?”

—

One challenge with D&I in general is that there’s a lot of things we can’t really measure, but have huge influence. Evening events based entirely around alcohol. People who don’t drink do they choose to attend anyway? Stay at home? 

Language we use, “technical vs. non-technical” How does this influence diversity?

Very difficult to measure. If the person stayed home, is it because they were sick? Is it because they didn’t like it? etc.

===

We have tools to measure basics like how many males/females there are, but not tools to measure why.

From the POV of policy, if you’re not attractive enough, you need to put more effort in. But if you are attractive, you’re going to bring in people who you don’t want.

Need metrics for “this community is biased because of this and this and need to improve in this way”

===

Burden isn’t just on women to be caregivers. How do we shift away from thinking of it as a gendered issue?

Focus on “less amount of time to contribute,” regardless of the reason.

Open source does offer flexibility for you to work on any time schedule. So offers opportunities. But has to be made accessible/easy to get started and stay engaged.

===

Lots of tech jobs say “We want you to move to San Francisco” This is stupid for a lot of reasons (laughter) but from a metrics perspective, interesting to see if companies who insist on you moving to SF are more likely to attract white men who want to live in a bunk bed situation, vs. companies who allow remote employees with flexible hours. Do they attract/retain women / caregivers for longer?


Looking at metrics of mailing list, where people live who collaborate with one another, location doesn’t have any bearing. You don’t tend to collaborate more with people in your timezone necessarily, etc.

===

Maintaining diversity in your projects/companies.. lots of people disengage silently. Lots of sexist behavior, women will tend to quit silently.

If you track the gender of the person across the lifecycle of a project, and there’s a difference between collaboration patterns (e.g. 2 pull requests and they’re out) this is worth drilling into.

If you look at these “drop-off” collaboration patterns more broadly, you can find patterns that cross factors such as gender.

Mozilla looking at qualitative research right now, specifically around caregiving roles, looking at forks (who’s forked and left), who might be a technical non-male contributors (have to go by name :\), talking us through what that means. Still in learning phase, but hoping to get valuable information from this.

===

Is this field too risky? For example, StackOverflow, grouping by gender by name / tag … look at how tag is different and changed over time, more/less diversity… But then when I went to publish, I was encouraged not to. Look at time zones instead. (Safer.)

The first time D&I was talked about in the CHAOSS project was over a year ago. There was no presence of these larger issues of gender diversity at all. Lots of people contributing to make this visible. There is reticence to actually tackle this issue. It’s a hard issue to talk about, eps. In the open source community (for whatever reason). But finding ways to talk about this phenomena that we can all observe and face-validate is necessary. It needs to be easier to bring it up.

D&I is sensitive topic, if I mention it, I’ll have to look at how terribly I’m doing about it. :) Talk selection, etc. always uncomfortable to see the data staring back to you.

Working across projects is so important. How can we help you answer this question? Is there concern about “outing” individuals? (Transgender, times that they’re working…)

If you’re starting to measure data about D&I in a community that hasn’t traditionally been very D&I, there’s a concern “is this data going to be used against me? Weaponized?” For example, tying this to performance. Any metric that is produced can be “weaponized” (e.g. commits, most weaponized metric). No new threats from D&I and if we start measuring it now, it’ll only get better.

We need real breakthrough research in this, vs. a lot of opinions/emotions. Getting the facts is fundamental, because right now we don’t know what the problem is exactly or how to track it.

The cool thing is working with academics is they can help us refine the data to be less opinionated.

===

When you have those interactions where you call out discrimination and diversity problems, you get “where’s your proof?”

Harder for a metrics/data person to put gender in as an API.

Qualitative data is valuable. Something valuable about sitting in front of someone and them telling you their story. A benefit in developing the qyantiative metrics layered on top of that qualitative base.

Need both. Numbers tell you something, but not necessarily why.

===

When people quietly leave, there’s usually been explicit complaints before hand. Do we track open source project’s organizational response to those complaints and their effectiveness? For example, if you have an issue, you’ve been harassed… is there any metrics for how open source community responds? The gauge effectiveness of resolution.

At Mozilla, dug into codes of conduct and enforcement… found that 45% of people in open source communities did not feel safe or empowered by their code of conduct. So trying to build processes and measure sentiment about it. Also enforcement : tracking responses, decision-making… trying to bring these measurements to CHAOSS as well. Another anecdote: people getting together in a women-only group. At first thought that was great, but found out it was in response because men were creating a toxic environment.

===

For well-being of user, how is consent compliance done? How is data being used? If you look someone up on the metrics, would you be able to find e.g. their GitHub profile, their gender, etc?

You tend to analyze “chunks” of data. Gender first, commits second, cross-reference between. Then you can prove if there are correlation or not. Not a privacy issue. However, looking at specific patterns (e.g. a person leaving) you need to get consent from the person for that, because in that case you are essentially targeting a person.

Also have to be careful about how you anonymize things and how much data you give. They can say enough stuff that even though you don’t have the name you know exactly who they’re talking about. For example, pulling together a table with gender might identify who are the 3 women in the group.

Also, someone can match exported data with source data, and find individuals this way.

Tools like genderize.io and take a name and derive gender by it. But definitely not bullet proof. “Jesus Maria” — Male or female?

===

From a name POV, non-causausion names hard to determine male vs. female. Biggest problem I have is contributors from China, they almost always change their timezone so as not to be seen UTC+8. How do you measure demographics in this situation?

Well, you also can’t tell Europe from Africa in that case. You can look at other indicators; for example people are in “UK” but if they don’t change times in the summer, they’re not UK. Or look at when activity is happening,;’