XavierVal

[QA_TEST] Tech  

With the intention of analysing the data of the internet portals, we want to process the content of a list of webs. We provide to you a list of URLs. You should code a solution that access to this URLs from a navigator and retrieve the title for each of the accessed URLs and store it.  

Once each URL in the list has been visited you should store the titles and group those titles per each distinct domain (domian1 -> title1, title2). 

Also it is required to publish this info against a web service. The info that should be sent per each domain is the median (not the AVG) of the lengths of the titles associated to each domain group. 

You are required to deliver a solution to fulfil this requirements: 

XavierVal

# Valido

´´´
curl -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "Fiware-Service: serv12" -H "Fiware-ServicePath: /sub12" -d '{
  "entities": [
    {
      "id": "entity_.*",
      "isPattern": "true",
      "type": "entity_type"
    }

XavierVal

#####
vi listen.js 
######
//script version
var version="0.0.2"

//verbosity flag
var vm=false;

XavierVal

import ckanapi

class CkanUtils(object):


    def ckanDataset(comp, service, secure=False):
        # collect all the input data and mixed it with the template to send
        url = world.c[comp]["endpoint"]
        ckan = ckanapi.RemoteCKAN(world.c[comp]["endpoint"], apikey=world.c[comp]["apikey"],
            user_agent='ckanapiexample/1.0 (+http://example.com/my/website)')

XavierVal

[cloud-user@prueba ~]$ cat /usr/cygnus/conf/cygnus.conf 
########################################
# FILE MANAGED BY PUPPET
# for config latest
########################################

# APACHE_FLUME_HOME/conf/cygnus.conf

# The next tree fields set the sources, sinks and channels used by Cygnus. You could use different names than the
# ones suggested below, but in that case make sure you keep coherence in properties names along the configuration file.