Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
cURL examples to query Wikidata

SPARQL Queries (with cURL command) on Wikidata

This gist resulted to be just the spark for a proper article, and won't be maintained here anymore.

The SPARQL endpoint is http://wdqs-beta.wmflabs.org/bigdata/namespace/wdq/sparql and it has a Web form to fire queries. However http://www.wikidata.org/prop/direct/P31 ("instance of") tells you what the entity is.

The repository doesn't have named graphs, or at least the SPARQL endpoint rejects graph queries. The classes of entities (rdf:type) are not described in the repository.

To find the HTML page of an entity (such as https://www.wikidata.org/entity/Q866405), simply replace /entity/ with /wiki/.

By default the SPARQL endpoint returns the results in SPARQL Results XML format but

  • adding -H "Accept: application/json" to the cURL command gets them in JSON Query Results
  • adding -H "Accept: text/csv" to the cURL command gets them in CSV format (the most readable).

Don't forget to URL encode the query ;-)

The list of entity types

#Selects the first 20 types of entities encountered:

select distinct ?type where {
?thing a ?type
}
limit 20

curl http://wdqs-beta.wmflabs.org/bigdata/namespace/wdq/sparql?query=select%20distinct%20%3Ftype%20where%20%7B%0A%3Fthing%20a%20%3Ftype%0A%7D%0Alimit%2020 -H "Accept: text/csv"

Description of an entity (East Antarctica)

describe queries return all the triples in which the selected entity is subject (position 1/3). Certain SPARQL endpoints (such as this one) also return the triples in which the entity is object (position 3/3). The query result being a graph and not a table like for SPARQL Results, the default format is RDF/XML. JSON and Turtle (text/turtle) can also be requested.

describe <http://www.wikidata.org/entity/Q866405>

curl http://wdqs-beta.wmflabs.org/bigdata/namespace/wdq/sparql?query=describe%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2FQ866405%3E -H "Accept: text/turtle"

@ColinMaudry

This comment has been minimized.

Copy link
Owner Author

@ColinMaudry ColinMaudry commented May 28, 2015

If you need a specific SPARQL request, you can request (uh uh) it here and I'll see what I can do.

@lucaswerkmeister

This comment has been minimized.

Copy link

@lucaswerkmeister lucaswerkmeister commented May 29, 2015

curl can also do the URL encoding for you:

curl -G https://wdqs-beta.wmflabs.org/bigdata/namespace/wdq/sparql --data-urlencode query='
select distinct ?type where {
?thing a ?type
}
limit 20
'

(Note the -G option to use a GET request)

@ColinMaudry

This comment has been minimized.

Copy link
Owner Author

@ColinMaudry ColinMaudry commented May 29, 2015

Nice thanks, I didn't know about it

However, if I paste that it my console, it treats the line breaks as Enter keypresses, and thus as many commands as the number of lines. But that may work in a shell script.

My objective here was to give one-liners to paste in console.

@lucaswerkmeister

This comment has been minimized.

Copy link

@lucaswerkmeister lucaswerkmeister commented May 29, 2015

Weird, the shell shouldn’t start executing the command until the quote is closed… which shell are you using?

@ColinMaudry

This comment has been minimized.

Copy link
Owner Author

@ColinMaudry ColinMaudry commented May 29, 2015

Apparently it's a Windows limitation 😬

Via SSH on my Ubuntu server, it worked.

I tried with both ConEmu and the standard command prompt from my Windows laptop: fails. In order to leave it cross-platform, I'll leave the URL encoding steps. Thanks for the input!

@vorachet

This comment has been minimized.

Copy link

@vorachet vorachet commented Jun 8, 2017

I try to use https://query.wikidata.org/sparql with HTTP POST. It does not work. This manual (https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual) said

"POST requests also accepts query in the body of the request, instead of URL, allowing to run larger queries without hitting URL length limit."

Any one uses https://query.wikidata.org/sparql with POST?

@dr0i

This comment has been minimized.

Copy link

@dr0i dr0i commented Jun 23, 2017

@vorachet At https://en.wikibooks.org/wiki/SPARQL/Wikidata_Query_Service it is said that PUT is forbidden. Normally, you don't need POST to run large queries - use GET. I do it like @lucaswerkmeister commented on 29 May 2015. Put this into a file.sh:

curl --header "Accept: application/sparql-results+json"  -G 'https://query.wikidata.org/sparql' --data-urlencode query='
SELECT ?s WHERE {
your complex query here
}'

and run it e.g. in bash with "bash file.sh".

@ppKrauss

This comment has been minimized.

Copy link

@ppKrauss ppKrauss commented Aug 2, 2018

More one example. Generating a CSV file (countries.csv) with only item labels (cuts the Wikidata URL) and some data.

curl -o countries.csv -G 'https://query.wikidata.org/sparql' \
     --header "Accept: text/csv"  \
     --data-urlencode query='
 SELECT DISTINCT ?iso2 ?qid ?osm_relid ?itemLabel
 WHERE {
  ?item wdt:P297 _:b0.
  BIND(strafter(STR(?item),"http://www.wikidata.org/entity/") as ?qid).
  OPTIONAL { ?item wdt:P1448 ?name .}
  OPTIONAL { ?item wdt:P297 ?iso2 .}
  OPTIONAL { ?item wdt:P402 ?osm_relid .}
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]" . }
 }
 ORDER BY ?iso2
'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.