Skip to content

Instantly share code, notes, and snippets.

@stuppie
Created May 3, 2017 00:33
Show Gist options
  • Save stuppie/4e9db11a10aeef6f6f9f9fa972eb282b to your computer and use it in GitHub Desktop.
Save stuppie/4e9db11a10aeef6f6f9f9fa972eb282b to your computer and use it in GitHub Desktop.
Comparing resources for json.loads vs requests json
#!/usr/bin/env bash
/usr/bin/time -v python3 json_loads.py
/usr/bin/time -v python3 requests_json.py
import json
import requests
endpoint = 'https://query.wikidata.org/sparql'
query = """
PREFIX wd: <http://www.wikidata.org/entity/>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX p: <http://www.wikidata.org/prop/>
PREFIX v: <http://www.wikidata.org/prop/statement/>
PREFIX q: <http://www.wikidata.org/prop/qualifier/>
PREFIX ps: <http://www.wikidata.org/prop/statement/>
#Tool: wdi_core fastrun
SELECT ?p ?label WHERE {
?p p:P686/ps:P686 ?zz .
OPTIONAL {
?p skos:altLabel ?label FILTER (lang(?label) = "en") .
}
}
"""
params = {
'query': query,
'format': 'json'
}
headers = {
'Accept': 'application/sparql-results+json',
'User-Agent': 'wikidataintegrator: github.com/SuLab/WikidataIntegrator'
}
response = requests.get(endpoint, params=params, headers=headers)
d = json.loads(response.text)
import requests
endpoint = 'https://query.wikidata.org/sparql'
query = """
PREFIX wd: <http://www.wikidata.org/entity/>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX p: <http://www.wikidata.org/prop/>
PREFIX v: <http://www.wikidata.org/prop/statement/>
PREFIX q: <http://www.wikidata.org/prop/qualifier/>
PREFIX ps: <http://www.wikidata.org/prop/statement/>
#Tool: wdi_core fastrun
SELECT ?p ?label WHERE {
?p p:P686/ps:P686 ?zz .
OPTIONAL {
?p skos:altLabel ?label FILTER (lang(?label) = "en") .
}
}
"""
params = {
'query': query,
'format': 'json'
}
headers = {
'Accept': 'application/sparql-results+json',
'User-Agent': 'wikidataintegrator: github.com/SuLab/WikidataIntegrator'
}
response = requests.get(endpoint, params=params, headers=headers)
d = response.json()
Command being timed: "python3 json_loads.py"
User time (seconds): 177.47
System time (seconds): 0.86
Percent of CPU this job got: 97%
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:02.17
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 2558900
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 267015
Voluntary context switches: 110
Involuntary context switches: 425
Swaps: 0
File system inputs: 1440
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Command being timed: "python3 requests_json.py"
User time (seconds): 0.54
System time (seconds): 0.08
Percent of CPU this job got: 15%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.95
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 231952
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 46770
Voluntary context switches: 68
Involuntary context switches: 6
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment