Source Code Search with HANA Fulltext Index

ZBC_SOURCE_CODE_INDEX.abap

*&---------------------------------------------------------------------*
*& Report ZBC_SOURCE_CODE_INDEX
*& Martin Ceronio, May 2017, martin@ceronio.net
*&---------------------------------------------------------------------*
*& Full text index of all sources in SAP system
*&---------------------------------------------------------------------*
report zbc_source_code_index.

data: lt_src type string_table.
data: ls_sci type zbc_sci.
data: lt_sci type table of zbc_sci.
types: tt_sci type standard table of zbc_sci with default key.
data: lt_sci_upd type table of zbc_sci.
data: lv_count type i.

data: pkg_size type i value 100.
* Update progress every:
data: prog_upd type i.
data: empty_sources type i.

parameters: p_idxdel radiobutton group op1 default 'X'.
SELECT-OPTIONS: s_progn for ls_sci-progname.
parameters: p_idxful radiobutton group op1.
parameters: p_idxcre radiobutton group op1.

*--------------------------------------------------------------------*
* FORM INDEX_DELTAS
*--------------------------------------------------------------------*
form index_deltas.
* Get a list of all sources that are not yet indexed or that have been changed
* since they were last indexed
  select r~progname, r~udat, r~utime from reposrc as r
    left outer join zbc_sci as s
      on r~progname = s~progname
    where r~r3state = 'A'
      and ( r~udat > s~udat or ( r~udat = s~udat and r~utime > s~utime ) or s~udat is null )
      and r~datalg > 0 "Only those units with actual source
      and r~progname in @s_progn
    into table @lt_sci.

  loop at lt_sci into ls_sci.
* Read the active source and concatenate it into a single string
    read report ls_sci-progname into lt_src.
    concatenate lines of lt_src into ls_sci-source separated by cl_abap_char_utilities=>cr_lf.

* Keep track of how many sources are actually empty (despite having some length in REPOSRC)
    if strlen( ls_sci-source ) = 0.
      add 1 to empty_sources.
*      continue.
    endif.

* Add to updates
    append ls_sci to lt_sci_upd.

* Provide an update every so often:
    add 1 to prog_upd.
    add 1 to lv_count.
    if prog_upd >= 5000.
      clear prog_upd.
      data(lv_message) = |{ lv_count } of { lines( lt_sci ) } completed|.
      if sy-batch = abap_true.
        message lv_message type 'I'.
      else.
        call function 'SAPGUI_PROGRESS_INDICATOR'
          exporting
            text       = lv_message.
      endif.
    endif.

* Update when package size has been reached
    if lines( lt_sci_upd ) >= pkg_size.
* Normally, I always advocate commits in updates, but here is an exception:
      modify zbc_sci from table lt_sci_upd.
      commit work.
      clear lt_sci_upd.
    endif.
  endloop.

* Last remaining entries
  if lines( lt_sci_upd ) > 0.
    modify zbc_sci from table lt_sci_upd.
    commit work.
  endif.
  message |Extraction complete, stored { lines( lt_sci ) } sources, { empty_sources } empty sources not saved| type 'I'.
endform.

* Prior to doing delta, deletes index table, causing everything to be recreated
form index_full.
  data: lv_answer type char1.
  call function 'POPUP_TO_CONFIRM'
    exporting
      titlebar        = 'Really delete and recreate source index?'
      diagnose_object = ''
      text_question   = 'You are about to delete and extract the sources for all selected programs. Continue?'
    importing
      answer          = lv_answer.
  check lv_answer = '1'.

  delete from zbc_sci where progname in s_progn.
  commit work and wait.
  perform index_deltas.

endform.

*--------------------------------------------------------------------*
* FORM CREATE_INDEX
*--------------------------------------------------------------------*
form create_index raising cx_sql_exception.
* Create (or delete and create) the fulltext index on the source field
  data(lr_ddl) = new cl_sql_statement( ).

* First, drop the indexes if they exist
  data(lv_ddel) = |DROP FULLTEXT INDEX ZBCSL_SCI_SRC|.

* (Re)Create the fulltext indexes
  data(lv_dcre) = |CREATE FULLTEXT INDEX ZBCSL_SCI_SRC ON ZBC_SCI (SOURCE)|.

* Ignore error on drop as it may not exist (and DROP FULLTEXT has no IF EXIST clause)
  try.
      lr_ddl->execute_ddl( lv_ddel ).
    catch cx_sql_exception.
  endtry.

  lr_ddl->execute_ddl( lv_dcre ).
  message |Index created| type 'I'.
endform.

start-of-selection.

  try.
      case 'X'.
        when p_idxdel.
          perform index_deltas.
        when p_idxful.
          perform index_full.
        when p_idxcre.
          perform create_index.
      endcase.
    catch cx_root into data(lx_root).
      message lx_root->get_text( ) type 'I'.
  endtry.

ZBC_SOURCE_CODE_SEARCH.abap

*&---------------------------------------------------------------------*
*& Report ZBC_SOURCE_CODE_SEARCH
*& Martin Ceronio, May 2017, martin@ceronio.net
*&---------------------------------------------------------------------*
*& Program that searches index on program sources in ZBC_SCI
*& Use program ZBC_SOURCE_CODE_INDEX to index/re-index sources
*&---------------------------------------------------------------------*
report zbc_source_code_search.

* Search term to search for
parameters: p_query type string lower case.
parameters: p_maxh type i default 100.

* Search result table
data: gr_alv type ref to cl_salv_table.
data: gr_cont type ref to cl_gui_simple_container.
data: gv_height type i.

types: begin of ts_result,
         progname type progname,
         snippets type string,
         score    type p decimals 7,
       end of ts_result.
data: gt_result type table of ts_result.

class lcl_handler definition.
  public section.
    class-methods: on_double_click
                  for event double_click of cl_salv_events_table
      importing row.
endclass.
class lcl_handler implementation.
  method on_double_click.
    read table gt_result into data(ls_result) index row.
    check sy-subrc = 0.
    CALL FUNCTION 'EDITOR_PROGRAM'
      exporting
        display           = abap_true
*        line              = '000001'    " Cursor line
        program           = ls_result-progname
      exceptions
        application       = 1
        others            = 2.
    if sy-subrc <> 0.
     message id sy-msgid type sy-msgty number sy-msgno
                with sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
    endif.
*    editor-call for report ls_result-progname display-mode.
  endmethod.
endclass.

*--------------------------------------------------------------------*
* FORM SEARCH
*--------------------------------------------------------------------*
form search raising cx_sql_exception.
  refresh gt_result.
*  data(lv_query) = |SELECT PROGNAME FROM ZBC_SCI WHERE CONTAINS ((SOURCE), '{ p_query } LIMIT { p_maxh }') |.
  data(lv_query) =
  |SELECT PROGNAME, snippets(SOURCE), score() as score FROM ZBC_SCI WHERE CONTAINS (SOURCE, '{ p_query }') order by score desc limit { p_maxh }|.
  data(lr_sql) = new cl_sql_statement( ).
  data(lr_res) = lr_sql->execute_query( lv_query ).
  lr_res->set_param_table( ref #( gt_result ) ).
  data(lv_recs) = lr_res->next_package( ).
  gr_alv->get_columns( )->set_optimize( abap_true ).
  gr_alv->refresh( ).
  message |{ lv_recs } records retrieved| type 'S'.
endform.

at selection-screen.
  if sy-ucomm = space.
    try.
        perform search.
      catch cx_root into data(lx_root).
        message lx_root->get_text( ) type 'I'.
    endtry.
  endif.

at selection-screen output.
  if gr_alv is not bound.
    try.
        gr_cont = new cl_gui_simple_container( parent = cl_gui_container=>default_screen autoalign = abap_true lifetime = 1 ).
        gr_cont->get_height( importing height = gv_height ).
        cl_gui_cfw=>flush( ).
        gr_cont->set_height( gv_height - 50 ).
        gr_cont->set_top( 20 ).
        call method cl_salv_table=>factory
          exporting
            r_container  = gr_cont
          importing
            r_salv_table = gr_alv
          changing
            t_table      = gt_result.
        data(lr_event) = gr_alv->get_event( ).
        set handler lcl_handler=>on_double_click for lr_event.
        gr_alv->display( ).
      catch cx_salv_msg into data(lx_salv).
        message lx_salv->get_text( ) type 'I'.
    endtry.
  endif.