Skip to content

Instantly share code, notes, and snippets.

@pombredanne

pombredanne/shared-mime-info DMCA.md

Last active March 30, 2021 12:58
Show Gist options
  • Save pombredanne/7d6b3689a1b796c9a509c83b6b87f274 to your computer and use it in GitHub Desktop.
Save pombredanne/7d6b3689a1b796c9a509c83b6b87f274 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
shared-mime-info DMCA.md

This is a notice received and originally from https://github.zendesk.com/attachments/token/eTJTaIjPp5pqbcAldaowe2N4E/?name=2021-03-22-freedesktop.rtf

Are you the copyright holder or authorized to act on the copyright owner's behalf?

Yes, I am the copyright holder.

Please describe the nature of your copyright ownership or authorization to act on the owner's behalf.

I'm [private] of the software that some code was taken from, and [private] of it for more than 15 years.

Please provide a detailed description of the original copyrighted work that has allegedly been infringed. If possible, include a URL to where it is posted online.

The shared-mime-info package contains:

  • The core database of common MIME types, their file extensions and icon names.
  • The update-mime-database command, used to extend the DB and install a new MIME data.
  • The freedesktop.org shared MIME database spec.

The core database was copied wholesale:
[private]
with translations merged:
[private]

What files should be taken down? Please provide URLs for each file, or if the entire repository, the repository’s URL.

https://github.com/zRedShift/mimemagic/blob/master/cmd/parser/freedesktop.org.xml
https://github.com/13521900025/mimemagic/blob/master/cmd/parser/freedesktop.org.xml
https://github.com/backwardn/mimemagic/blob/master/cmd/parser/freedesktop.org.xml
https://github.com/brandfolder/mimemagic/blob/master/cmd/parser/freedesktop.org.xml
https://github.com/developgo/mimemagic/blob/master/cmd/parser/freedesktop.org.xml
https://github.com/Kycklingar/mimemagic/blob/master/cmd/parser/freedesktop.org.xml
https://github.com/pombredanne/mimemagic/blob/master/cmd/parser/freedesktop.org.xml
https://github.com/simplesurance/mimemagic/blob/master/cmd/parser/freedesktop.org.xml

Have you searched for any forks of the allegedly infringing files or repositories? Each fork is a distinct repository and must be identified separately if you believe it is infringing and wish to have it taken down.

Yes.

Is the work licensed under an open source license? If so, which open source license? Are the allegedly infringing files being used under the open source license, or are they in violation of the license?

Is the work licensed under an open source license?

Yes.

If so, which open source license?

The GNU General Public License v2 or later:
https://gitlab.freedesktop.org/xdg/shared-mime-info/-/blob/master/COPYING

Are the allegedly infringing files being used under the open source license, or are they in violation of the license?

They're using the file under an MIT license which is not compatible with the GNU GPL v2 or later.

What would be the best solution for the alleged infringement? Are there specific changes the other person can make other than removal? Can the repository be made private?

Relicense the project under a license compatible with the GNU GPL v2 or later, or remove it.

Do you have the alleged infringer’s contact information? If so, please provide it.

No.

I have a good faith belief that use of the copyrighted materials described above on the infringing web pages is not authorized by the copyright owner, or its agent, or the law.

I have taken fair use into consideration.

I swear, under penalty of perjury, that the information in this notification is accurate and that I am the copyright owner, or am authorized to act on behalf of the owner, of an exclusive right that is allegedly infringed.

I have read and understand GitHub's Guide to Submitting a DMCA Takedown Notice.

So that we can get back to you, please provide either your telephone number or physical address.

[private]
[private]
[private]
[private]

Please type your full legal name below to sign this request.

[private]

@pombredanne
Copy link
Author

@Pizzacus Thank you for your comprehensive post and opinion!

@pombredanne
Copy link
Author

The readme of the package starts with this note:

This package contains:

  • The freedesktop.org shared MIME database spec.
  • The merged GNOME and KDE databases, in the new format.
  • The update-mime-database command, used to install new MIME data.

I could not find anything about its license. I assume this would be GPL?

  • The merged GNOME and KDE databases.
    I assume both would be GPL alright but there is again no notice in the provided XML database file freedesktop.org.xml which is the topic at issue here. There is a notice in the related file freedesktop.org.xml.in as a comment though:
<!--
The freedesktop.org shared MIME database (this file) was created by merging
several existing MIME databases (all released under the GPL).

It comes with ABSOLUTELY NO WARRANTY, to the extent permitted by law. You may
redistribute copies of update-mime-database under the terms of the GNU General
Public License. For more information about these matters, see the file named
COPYING.

The latest version is available from:

	http://www.freedesktop.org/wiki/Software/shared-mime-info/

To extend this database, users and applications should create additional
XML files in the 'packages' directory and run the update-mime-database
command to generate the output files.
-->
  • The update-mime-database command, used to install new MIME data.
    This comes alright with a Copyright (C) 2003 Thomas Leonard e.g. @talex5 and this notice in update-mime-database.c
#define COPYING								\
	     N_("Copyright (C) 2003 Thomas Leonard.\n"			\
		"update-mime-database comes with ABSOLUTELY NO WARRANTY,\n" \
		"to the extent permitted by law.\n"			\
		"You may redistribute copies of update-mime-database\n"	\
		"under the terms of the GNU General Public License.\n"	\
		"For more information about these matters, "		\
		"see the file named COPYING.\n")

The man page at update-mime-database.1 has this notice: This manpage is in the public domain.

@pombredanne
Copy link
Author

I also created this ticket https://gitlab.freedesktop.org/xdg/shared-mime-info/-/issues/154 so that this issue can be discussed publicly in the upstream project

@cfergeau
Copy link

I am not redistributing any binary, but how would code and data that side-by-side in a repo make a whole codebase a derived work?
I am not legally-trained so I have no idea what this would mean in general, and in particular here. I think this is at best a case of side-by-side redistribution aka "mere aggregation" per the GPL 2.0 Section2 : https://www.gnu.org/licenses/old-licenses/gpl-2.0.html

What about https://github.com/zRedShift/mimemagic/blob/master/magicsigs.go , https://github.com/zRedShift/mimemagic/blob/master/mediatypes.go , ... ? I'm quite sure these are generated from shared-mime-info.xml using https://github.com/zRedShift/mimemagic/blob/master/cmd/parser/main.go , which make this code base more than "shipping go files side by side with a GPL xml file", these go files are a derivative work from shared-mime-info.xml.

@hadess
Copy link

hadess commented Mar 23, 2021

I'm not going to read the whole thing, because I only have one life.

A quick note though that I do have the right to enforce my own copyright (and I wrote the majority of the contents of freedesktop.org.xml.in above the person that imported the original database, and own that copyright), and I don't need to ask other rights holders.

I've pointed out the changes I'd like to see made in zRedShift/mimemagic#4 (comment)

@pombredanne
Copy link
Author

@cfergeau good points!

FWIW, https://gitlab.freedesktop.org/xdg/shared-mime-info/-/issues/154#note_850132 was closed by @hadess who does not seem to be willing to discuss the matter further.

@hadess
Copy link

hadess commented Mar 23, 2021

FWIW, https://gitlab.freedesktop.org/xdg/shared-mime-info/-/issues/154#note_850132 was closed by @hadess who does not seem to be willing to discuss the matter further.

?

@pombredanne
Copy link
Author

@cfergeau I further updated the README at https://github.com/pombredanne/mimemagic/blob/22e9e89765540fcd8062db2471432a7a12b7e9b6/README.md

@Pizzacus
Copy link

It's not GPLv2 or later though .-.

@pombredanne
Copy link
Author

@hadess you wrote:

I'm not going to read the whole thing, because I only have one life.
A quick note though that I do have the right to enforce my own copyright (and I wrote the majority of the contents of freedesktop.org.xml.in above the person that imported the original database, and own that copyright), and I don't need to ask other rights holders.

I am not disputing your rights there as I have no idea whether you have proper standing or not in this matter and what the GPL means for data files: you brought a serious legal action in the first place so I think it is fair to discuss this. You closed and locked the ticket in https://gitlab.freedesktop.org/xdg/shared-mime-info/-/issues/154#note_850132 for further comment... but this still needs to be discussed. As mentioned before your demands and guess lack the clarity that is required to be actionable in zRedShift/mimemagic#4

@cfergeau
Copy link

@cfergeau I further updated the README at https://github.com/pombredanne/mimemagic/blob/22e9e89765540fcd8062db2471432a7a12b7e9b6/README.md

This is fairly light and misleading imo. The generated code is gplv2 if it's a straight rip from the xml database. This also means any go program using this code as a go module effectively has to be shipped under the gplv2. This is very unusual in the go ecosystem, so it might e safer to just remove that generated code, and reimplement it properly (parsing the xml file at runtime maybe?)

@pombredanne
Copy link
Author

pombredanne commented Mar 23, 2021 •
edited
Loading

@Pizzacus you wrote:

It's not GPLv2 or later though .-.

Let me fix that further, though the plot thickens and I need some extra clarity there first before being able to resolve the issues without compounding the problem:

The notice in freedesktop.org.xml.in is:

The freedesktop.org shared MIME database (this file) was created by merging
several existing MIME databases (all released under the GPL).

It comes with ABSOLUTELY NO WARRANTY, to the extent permitted by law. You may
redistribute copies of update-mime-database under the terms of the GNU General
Public License. For more information about these matters, see the file named
COPYING.

The latest version is available from:

        http://www.freedesktop.org/wiki/Software/shared-mime-info/

To extend this database, users and applications should create additional
XML files in the 'packages' directory and run the update-mime-database
command to generate the output files.

Per the GPL-2.0, Section 9 https://www.gnu.org/licenses/old-licenses/gpl-2.0.html#section9 this means any version of the GPL:

If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation.

And the GPL COPYING text is for the GPL-2.0. I am not really able to resolve whether that's a GPL-2.0-or-later or a GPL-2.0 or any of the GPL 1,2 or 3 or any other version that applies.

@hadess do you know what is the exact license of this project? I need to know so I can carry this forward accurately.

@Pizzacus
Copy link

@cfergeau I hadn't thought of that... You're absolutely right, that's true, it does mean that if the work is a module, then GPL will apply to any other program that depends on it...

Well, if you're in the US, you might be able to just regenerate the file, because databases are not protected by copyright in the US.

See https://meta.wikimedia.org/wiki/Wikilegal/Database_Rights

Essentially, in the US, the structure and organisation of a database is protected by copyright. But the content, if it's just factual, non-creative, is not.

So you could create your own database of mime types, with the content of the original one, but structured in your own way. I think it would no longer be covered by GPL, but PLEASE MAKE SURE OF THAT BEFORE YOU DO IT 😨

@pombredanne
Copy link
Author

@cfergeau in reply to https://gist.github.com/pombredanne/7d6b3689a1b796c9a509c83b6b87f274#gistcomment-3676356

That's a good point. I made further updates at https://github.com/pombredanne/mimemagic/blob/b433f99f4a226778cdeb1a2f2887b36929e5ca8e/README.md to clarify what the GPL impact may be on the generated code. Note that this (e.g. the license of data fact under a the GPL and what happens in this case of generation) is a grey area as one may be able to make a point that file types and their magic are non-copyrightable facts? Alwayer's take on this would be much welcomed.

@hadess
Copy link

hadess commented Mar 23, 2021

It's supposed to be GPLv2+, after verification. This MR attempts to clarify this:
https://gitlab.freedesktop.org/xdg/shared-mime-info/-/merge_requests/119

@cfergeau
Copy link

Essentially, in the US, the structure and organisation of a database is protected by copyright. But the content, if it's just factual, non-creative, is not.

In the mimemagic, I suspect the structure of the data is heavily based on shared-mime-info structure, so it's not just reusing individual 'facts' in its own way. And well, if the code can be MIT if ??? (developer? user? distributor?) is in the US, but GPL in other parts of the world, this still feels very complicated :)

Note that this (e.g. the license of data fact under a the GPL and what happens in this case of generation) is a grey area as one may be able to make a point that file types and their magic are non-copyrightable facts?

My recommendation would be to err on the safe side, and consider this code to be GPL, with all the implications that go with that. You are of course free to have your own interpretation on the matter. If you go with the 'safe side', maybe you are going too far, but you know for sure you won't have more problems because of this. Going with your own interpretation is full of 'maybe's (the ones you used yourself in this discussion, and in the README file).

@pombredanne
Copy link
Author

@hadess you wrote:

This MR attempts to clarify this:
https://gitlab.freedesktop.org/xdg/shared-mime-info/-/merge_requests/119

MO is that this MR is missing some proper historical references to the license of the original data from Gnome and KDE that were merged to create that database. This would need to be researched and documented before a change in licensing documentation to avoid adding more layers of confusion to the topic.

@hadess
Copy link

hadess commented Mar 23, 2021

MO is that this MR is missing some proper historical references to the license of the original data from Gnome and KDE that were merged to create that database. This would need to be researched and documented before a change in licensing documentation to avoid adding more layers of confusion to the topic.

Except that, as Zander mentioned, it doesn't matter one bit in this discussion.

You can choose to pin your copy of the database to a particular version of the GPL, say, GPLv2 and you would still not be following that license's terms properly.

@R030t1
Copy link

R030t1 commented Mar 24, 2021

Essentially, in the US, the structure and organisation of a database is protected by copyright. But the content, if it's just factual, non-creative, is not.

In the mimemagic, I suspect the structure of the data is heavily based on shared-mime-info structure, so it's not just reusing individual 'facts' in its own way.

@cfergeau: Reverse engineering for interoperability is explicitly protected. Discussions elsewhere talk about replacing the disputed file with a drop in public domain one, so the suggestion this project lacks of structure imposed from this specific database is, contingent on the existence of those other databases, very defensible.

Then, the GPL-ness extends just to the MIME type database; @hadess needs to be going after people who are distributing binaries but not the source to the MIME database or its changes without the copyright notice. I am very pro-GPL but even if you considered XML a programming language (it's not) it is different than the implementation language in every case. Its inclusion in a forest of other files does not mean all of those files must be similarly licensed; see current handling of e.g. router firmware.

@pombredanne
Copy link
Author

For reference, see also these related pages:

@pombredanne
Copy link
Author

For reference here is the status of this DMCA takedown so far:

The head original fork of @zRedShift https://github.com/zRedShift/mimemagic was relicensed under the GPL and all past releases "retracted" Go package-wise and git tags deleted.

My fork https://github.com/pombredanne/mimemagic/ has been filtered and purged from all and any Freedesktop shared-mime-info content

Eventually GitHub published the DMCA takedown here https://github.com/github/dmca/blob/master/2021/03/2021-03-22-freedesktop.md with redacted names.

@pombredanne
Copy link
Author

On the Ruby side, following mimemagicrb/mimemagic#97 which was worded the same way as this DMCA takedown but was not yet a DMCA action @jellybob and @minad 's https://github.com/mimemagicrb/mimemagic/ was briefly relicensed under the GPL and all past versions and tags yanked then eventually was rewritten to remove all generated parts and relicensed under the MIT to read a system-installed mime database after having created quite a stir for downstream users, including major ones such as Rails and all Rails users.

@pombredanne
Copy link
Author

More related issues created by @hadess which I am not sure I always understand:

They have this typical content I guess asking for repository removal.

Remove repo #1
Hey,

I know you're just trying to get your Ruby on Rails stuff working, but if old
versions of a repo were removed because their license is incorrect, the right
way to fix this isn't to reupload stuff that was using the wrong license.

The upstream discussion:
rails/rails#41750

And more takedown requests impacting other places:

Remove mimemagic 0.3.6 The license listed is invalid in: https://github.com/KON-ch/ActorConnection/tree/master/vendor/cache/ruby/3.0.0/gems/mimemagic-0.3.6
Please refer to: rails/rails#41750

I've historically been the maintainer of shared-mime-info for around 15 years, and script/freedesktop.org.xml looks like it's a copy of the database shipped with shared-mime-info, which is released under the GPL, with shared-mime-info's translators work merged in, and the GPL header removed.
The license that you're shipping mimemagic under (MIT) isn't compatible with shared-mime-info's.
There are a number of possibilities to fix this problem:

change the mimemagic license to be GPL compatible
parse the XML file that shared-mime-info ships at runtime, and don't ship it in a codebase with an incompatible license

Using a GPL file as a source makes your whole codebase a derived work, making it all GPL, so I think it's pretty important that this problem gets corrected before somebody uses it in a pure MIT codebase, or a closed-source application.

You will also need to re-add the GPL header to the shared-mime-info XML file as a matter of urgency. It was stripped in release tarballs by the tool used to merge translations, but is visible in the .in version of the same file.

And some also contain DMCA takedown threats:
gedhean/mimemagic#2 (comment)

hadess commented 15 minutes ago
Thanks for the advice, @hadess. I'll remove the repo soon.

Please fix it now, otherwise I'll have to file a DMCA takedown request, and it's more work for me, and more hassle for you.

@pombredanne
Copy link
Author

There is quite a bit of twitter chatter about the impact of the actions listed here in the Rails world https://twitter.com/search?q=mimemagic&src=typed_query

@pombredanne
Copy link
Author

And some interesting article (translated from German by Google translate): https://www.heise.de/news/Ruby-on-Rails-Durch-Lizenzproblem-entfallene-Library-erzeugt-Dominoeffekt-5999197.html

Ruby on Rails: Library lost due to license problem creates domino effect
Half a million open source projects are likely to be affected by the chaos caused by a library that was initially incorrectly licensed and then withdrawn.

In the middle of this week, Bastien Nocera, the maintainer of an open source software library called shared-mime-infothe maintainer of the Ruby library, mimemagichad notified that mimemagicthe wrong license was being used. Noceras Library is registered under the GPLv2 license, and projects based on it would have to use the same license. The Ruby library, however, was registered with the MIT license. The discovery might have been a side note, but the licensing problem has sparked a chain reaction that now affects around 500,000 open source projects.

@pombredanne
Copy link
Author

And an interesting article by @cseeman https://dev.to/cseeman/what-s-up-with-mimemagic-breaking-everything-he1

And some weird twists: https://news.ycombinator.com/item?id=26571086

In a twist of irony, the software for which the copyright claim breaking rails was made is hosted on the free edition of gitlab, which is based on rails.

and https://news.ycombinator.com/item?id=26573161

And according to the twitter-bio of the individual, who brought this up, he's related to Red Hat, which are also affected [^1].
[^1]RedHatInsights/compliance-backend#79...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment