Skip to content

Instantly share code, notes, and snippets.

@pmrt
Created May 7, 2019 11:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pmrt/fbdbf611d0588b2a7d607816bf8c89f8 to your computer and use it in GitHub Desktop.
Save pmrt/fbdbf611d0588b2a7d607816bf8c89f8 to your computer and use it in GitHub Desktop.
crawler
const api = 'https://www.autodoc.es/ajax/selector/car_selector.json'
/*
error throws an error with a provided `msg`
*/
function error(msg) {
console.error(`Ha ocurrido un error: ${msg}`)
}
/*
GetManufacturers using a hardcoded selector.
*/
function getManufacturers() {
return document.querySelectorAll("optgroup[label='Los fabricantes están en orden alfabético'] > option")
}
async function fetcher(body) {
try {
const res = await fetch(api, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
body,
})
const data = await res.json()
return JSON.parse(data)
} catch (err) {
error(err)
}
}
/*
fetch the models for a given manufacturer ID
*/
async function getModelsByManufacturer(id) {
return await fetcher(`maker_id=${id}&changed=maker`)
}
/*
fetch type by a given manufacturer ID and model ID
*/
async function getType(makerId, modelId) {
return await fetcher(`maker_id=${makerId}&model_id=${modelId}&changed=model`)
}
async function crawl() {
let res = []
for (let maker of getManufacturers()) {
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment