Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Proof-of-concept ABAP Program to extract sales order data to an Elasticsearch server as seen on http://ceronio.net
*--------------------------------------------------------------------*
* Example program to extract data to Elasticsearch
*--------------------------------------------------------------------*
* Be sure to read the accompanying article:
* http://ceronio.net/2015/02/sap-business-intelligence-with-elasticseach-and-kibana/
*&---------------------------------------------------------------------*
*& WARNING!: This program is just a proof of concept. For a proper
*& production-quality extraction program, you would:
*& 1. Use a database cursor for selection
*& 2. Do bulk updates to Elasticsearch for better performance
*& 3. Cater for delta/full selection (and additional criteria)
*& 3. Use language-dependent texts everywhere
*& 4. Investigate the quickest way to get values into the (stab) table
*& 5. Possibly normalize the data in Elasticsearch?
*& 6. etc.
*& IN ADDITION, YOU WOULD HAVE TO SET MOST STRING FIELDS TO "NOT ANALYZED"
*& IF YOU WANT TO REPORT ON THEM!!!
*&---------------------------------------------------------------------*
report zslsord_extr_to_es.
* Set this to point to your Elasticsearch installation
data: gv_es_url type string value 'http://10.0.0.2:9200'.
* For JSON transformation:
data gr_writer type ref to cl_sxml_string_writer.
data gv_json type xstring.
* Data needed for HTTP communication
data: gv_client type ref to if_http_client.
data: gv_request type ref to if_http_request.
data: gv_response type ref to if_http_response.
data: gv_code type i.
data: gv_reason type string.
* For RTTI of structure with fields and dynamic transfer of data
data: gr_sdesc type ref to cl_abap_structdescr.
data: gt_comp type abap_component_tab.
data: gs_comp type abap_componentdescr.
data: gv_fname type string.
field-symbols: <field> type any.
* Because we don't want an unnecessary root element in our JSON and we want all
* the keys lower case, we use (stab)
data: gt_stab type abap_trans_srcbind_tab.
data: gs_stab type abap_trans_srcbind.
* Definition of the data we want to retrieve from the database
data: begin of gs_order,
vbeln type vbak-vbeln,
erdat type vbak-erdat,
erzet type vbak-erzet,
audat type vbak-audat,
vbtyp type vbak-vbtyp,
auart type vbak-auart,
vkorg type vbak-vkorg,
vtweg type vbak-vtweg,
spart type vbak-spart,
bstnk type vbak-bstnk,
kunnr type vbak-kunnr,
name1 type kna1-name1,
posnr type vbap-posnr,
matnr type vbap-matnr,
matkl type vbap-matkl,
pstyv type vbap-pstyv,
netwr type vbap-netwr,
waerk type vbap-waerk,
kwmeng type vbap-kwmeng,
vrkme type vbap-vrkme,
end of gs_order.
data: gt_order like table of gs_order.
* Get a description of the above data structure for its fields
gr_sdesc ?= cl_abap_structdescr=>describe_by_data( gs_order ).
gt_comp = gr_sdesc->get_components( ).
* Data selection which must be maintained to correspond to the above data definition
select k~vbeln k~erdat k~erzet k~audat k~vbtyp k~auart k~vkorg k~vtweg k~spart k~bstnk k~kunnr k1~name1
p~posnr p~matnr p~matkl p~pstyv p~netwr p~waerk p~kwmeng p~vrkme
into table gt_order
from vbak as k
join vbap as p
on k~vbeln = p~vbeln
join kna1 as k1
on k~kunnr = k1~kunnr.
* up to 2000 rows.
* Create HTTP client
call method cl_http_client=>create_by_url
exporting
url = gv_es_url
importing
client = gv_client
exceptions
argument_not_found = 1
plugin_not_active = 2
internal_error = 3
others = 4.
if sy-subrc <> 0.
* Here you can report the problem somewhere, like a list and exit the program
write: / 'Failure instantiating HTTP client'.
exit.
endif.
* Get handles on the request and response obejcts
gv_request = gv_client->request.
gv_response = gv_client->response.
loop at gt_order into gs_order.
* Fill the stab table with values of the current record
clear gt_stab.
loop at gt_comp into gs_comp.
assign component gs_comp-name of structure gs_order to <field>.
if sy-subrc = 0.
gs_stab-name = to_lower( gs_comp-name ).
get reference of <field> into gs_stab-value.
append gs_stab to gt_stab.
endif.
endloop.
* Transform data into JSON string
gr_writer = cl_sxml_string_writer=>create( type = if_sxml=>co_xt_json ).
call transformation id
source (gt_stab)
result xml gr_writer.
gv_json = gr_writer->get_output( ).
* Set up HTTP request to post the data to ElasticSearch
gv_request->set_method( 'POST' ).
call method cl_http_utility=>set_request_uri
exporting
request = gv_request
uri = |/orders/order_item/{ gs_order-vbeln }{ gs_order-posnr }|.
gv_request->set_data( gv_json ).
call method gv_client->send
exceptions
http_communication_failure = 1
http_invalid_state = 2
http_processing_failed = 3
http_invalid_timeout = 4
others = 5.
if sy-subrc <> 0.
* Here you can report the problem somewhere, like a list
write: / 'Failed exporting (HTTP send)', gs_order-vbeln, gs_order-posnr.
continue.
endif.
call method gv_client->receive
exceptions
http_communication_failure = 1
http_invalid_state = 2
http_processing_failed = 3
others = 5.
if sy-subrc <> 0.
* Here you can report the problem somewhere, like a list
write: / 'Failed exporting (HTTP receive)', gs_order-vbeln, gs_order-posnr.
continue.
endif.
call method gv_response->get_status
importing
code = gv_code
reason = gv_reason.
if gv_code >= 400.
write: / 'Failed exporting (HTTP response)', gv_reason, gs_order-vbeln, gs_order-posnr.
* Here you can report the problem somewhere, like a list
endif.
endloop.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment