Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save varocarbas/5ba3b9af59328dd02806fc9305bac430 to your computer and use it in GitHub Desktop.
Save varocarbas/5ba3b9af59328dd02806fc9305bac430 to your computer and use it in GitHub Desktop.
Friendliness, safety, efficiency and adaptability -- accessory_java

Friendliness, safety, efficiency and adaptability -- accessory_java

Introduction

The current article discusses the main concerns defining accessory_java (main repository): friendliness, safety, efficiency and adaptability. This library is meant to help programmers deal with different aspects of programming in Java. There are also multiple references to ib (main repository), another Java library easing the communication with Interactive Brokers.

accessory_java

I started developing accessory_java while working on a personal project. It was meant to be a container of generic resources which I could eventually use in other applications. Basically, a library including not just useful methods and variables, but also a solid starting point to face virtually any project in Java. Or, in other words, a way to adapt Java to my programming approach, to help me easily overcome the language peculiarities. That initial code has kept growing and evolving until reaching the current stage, which I consider mature, comprehensive and reliable enough.

Main features of accessory_java:

  • Built from scratch, by relying on stable functionalities and with the minimum amount of external dependencies possible. At the moment, there is only one external dependency: the MySQL connector. And it can even be ignored in case of not using the database (DB) resources.
  • Including virtually no comments or documentation, but with an overall clear structure and quite rigid rules and conventions which are applied in a very systematic and consistent way. Actually, this article and potential future ones might be gradually compensating the aforementioned lacks.
  • Main priorities (ordered): friendliness, safety (e.g., default validity checks everywhere or no uncaught exceptions), efficiency/speed, scalability/adaptability.
  • Theoretically technology agnostic, although currently mostly focused on MySQL and Linux.

Before using any of the resources of this library, one of the start overloads in the accessory._ini class has to be called, as shown in the code below.

String app_name = samples.APP;
boolean includes_legacy = false; //When true, it looks for a package called "legacy" which might 
                                 //be missing anyway.

//------
//dbs_setup includes the setup to be used by default in all the DBs. 
//It can be populated by using one of the parent_ini_db.get_setup_vals methods or via adding valid
//types.CONFIG_DB_SETUP_[*]-value pairs. Any of those values can be overwritten by the specific setups 
//in the _ini_db class, which are included when adding a new DB through the populate_db method called 
//from populate_all_dbs.

String db_name = samples.PERFECT_NAME;
String username = samples.PERFECT_USERNAME;
String password = samples.PERFECT_PASSWORD;
        
//--- These two variables work together: if encrypt is true, user has to be non-null and the DB 
//credentials will be retrieved from the corresponding encrypted file.
boolean encrypt = true;
String user = samples.USER; //This is the crypto ID associated with the given encryption information, 
                            //which can be generated by calling the method db.encrypt_credentials.
//---
        
//If encrypt is true, a set of encrypted credentials located in a specific file is expected.
//These credentials can be generated by calling db.encrypt_credentials/db.update_credentials. 
//But that call or other related actions (e.g., changing the directory for credentials) can't 
//be performed before _ini.start is called. That is, if delay_encryption is set to false, 
//dbs_setup would be updated with previously-generated credentials. Alternatively, delay_encryption 
//can be true, dbs_setup would include a meaningless placeholder and the credentials would be updated 
//after _ini.start is called. 
boolean delay_encryption = true;
        
HashMap<String, Object> dbs_setup = null;
        
if (samples.USE_DB)
{
    String setup = null; //This variable can be null (i.e., default setup) or equal to the given DB
                         //(i.e., corresponding types.CONFIG_[*] constant). 
    String host = null; //A null value indicates that the default host will be used 
                        //(e.g., "localhost" in MySQL).
            
    if (encrypt && !delay_encryption) 
    {
        dbs_setup = parent_ini_db.get_setup_vals(db_name, setup, user, host, encrypt);
    }
    else dbs_setup = parent_ini_db.get_setup_vals(db_name, setup, username, password, host);
}
//------

_ini.start(app_name, includes_legacy, dbs_setup);
        
if (encrypt && delay_encryption)
{
    paths.update_dir(paths.DIR_CRYPTO, samples.PERFECT_LOCATION);
    paths.update_dir(paths.DIR_CREDENTIALS, samples.PERFECT_LOCATION);

    db.update_credentials(db_crypto.SOURCE, user, username, password);
}

permalink

public abstract class samples
{
    public static final String APP = "whatevs";
    public static final String ID = "whichevs";
    public static final String USER = "whoevs";
    public static final String PLACEHOLDER = "wherevs";
    
    //Set this variable to true only if there is a valid DB setup in place 
    //(i.e., MySQL connector, valid DB name and valid credentials).
    public static final boolean USE_DB = false;
    
    //--- I would input my own values where these constants are used if I were you.
    public static final String PERFECT_NAME = PLACEHOLDER;
    public static final String PERFECT_LOCATION = PLACEHOLDER;
    public static final String PERFECT_USERNAME = PLACEHOLDER;
    public static final String PERFECT_PASSWORD = PLACEHOLDER;
    //---

    public static void print_error(String sample_id_) { print_message(sample_id_, null, false); }
    
    public static void print_message(String sample_id_, String message_, boolean is_ok_)
    {
        String message = sample_id_ + misc.SEPARATOR_CONTENT;
        
        if (is_ok_) message += message_; 
        else message += "ERROR";
        
        generic.to_screen(message);
    }
    
    public static void print_messages(String sample_id_, String[] messages_)    
    {
        String message = sample_id_;

        for (String message2: messages_) { message += misc.SEPARATOR_CONTENT + message2; }
        
        generic.to_screen(message);
    }
    
    public static void print_end() { generic.to_screen("sample code completed successfully"); }
}

permalink

Previous articles about accessory_java:

ib

A Java library easing the automated communication with Interactive Brokers (IB). It relies on and emulates the defining ideas of accessory_java. An analysis of this library is beyond the scope of the present article and its code will be merely used to support the explanations about accessory_java.

Preliminary Ideas

Getting into the specifics of different parts of the code of these libraries is beyond the current scope and will probably be discussed in future articles. In any case, some clarifications are already required to fully understand the next sections.

The main code dealing with DB management in the accessory package is included in the DB classes: db and all the classes whose names start with "db_". At the moment, MySQL is the only supported DB type. The peculiar evolution of this part of the code has provoked the appearance of three modes: the initial, default one (accessory.db querying methods with is_quick_ being false), quick (accessory.db querying methods with is_quick_ being true or accessory.db_quick ones when accessory.db_quick.is_quicker returns false) and quicker (accessory.db_quicker querying methods or accessory.db_quick ones when accessory.db_quick.is_quicker returns true). More precisely, the two last modes are considered part of the same quick reality, where the quicker option is just an evolved alternative which is used whenever possible. And this happens every time because the only requirement of the given DB type having its own quicker implementation is always met.

Other important aspect of the way in which these libraries deal with DBs is the distinction between sources and tables, fields and columns (or "cols" as referred in the code); between the code-relevant, actual-DB-independent constants and the variables storing the current DB information. For example, SOURCE1, including FIELD1 and FIELD2, is used everywhere in the code and associated with the MySQL table "table1", inside the DB "db1", which has the columns "col1" (FIELD1) and "col2" (FIELD2). The way in which the fields/columns bipartition is being handled by the querying methods is one of the main factors justifying the aforementioned modes and their performance differences.

public static String get_string
(
    String source_, String field_col_, String where_, String wrong_
) 
{ return get_string(source_, field_col_, where_, wrong_, DEFAULT_IS_FIELD, is_quick(source_)); }

public static String get_string
(
    String source_, String field_col_, String where_, String wrong_, 
    boolean is_field_, boolean is_quick_
) 
{ 
    String output = 
    (
        is_quick_ ? db_quick.select_one_string
        (
            source_, get_field_quick_col
            (
                source_, field_col_, is_field_, is_quick_
            ), 
            where_, db.DEFAULT_ORDER
        ) 
        : db.select_one_string(source_, field_col_, where_, db.DEFAULT_ORDER)
    ); 

    return (!db.WRONG_STRING.equals(output) ? output : wrong_);
}

public static int get_int(String source_, String field_col_, String where_, int wrong_) 
{ 
    return get_int(source_, field_col_, where_, wrong_, DEFAULT_IS_FIELD, is_quick(source_)); 
}	

public static int get_int
(
    String source_, String field_col_, String where_, 
    int wrong_, boolean is_field_, boolean is_quick_
) 
{ 
    int output = 
    (
        is_quick_ ? db_quick.select_one_int
        (
            source_, get_field_quick_col(source_, field_col_, is_field_, true), 
            where_, db.DEFAULT_ORDER
        ) 
        : db.select_one_int(source_, field_col_, where_, db.DEFAULT_ORDER)
    ); 

    return (output != db.WRONG_INT ? output : wrong_);
}

permalink

In parallel to the management of native Java types/classes (e.g., int or ArrayList), which is mostly performed by methods from the accessory.generic class, accessory._types introduces the concept of "types": simple string constants which define an efficient and adaptable way to store categorised information and which represent a cornerstone of these libraries. For practical reasons, many parts of the code are expected to exclusively deal with a shortlisted number of native classes. And, for the purpose of this article, only a subset of them are relevant, the following ones:

  • Small/big: double/Double, long/Long, int/Integer, boolean/Boolean, byte/Byte, char/Character.

  • Collections (referred as "arrays" in the code): ArrayList (only big), HashMap (only big) and native arrays (both small and big).

static Class<?>[] populate_all_classes()
{
    ArrayList<Class<?>> classes = new ArrayList<Class<?>>();

    classes.add(Class.class);
    classes.add(Method.class);
    classes.add(Exception.class);
    classes.add(LocalDateTime.class);
    classes.add(LocalDate.class);
    classes.add(LocalTime.class);

    classes.add(size.class);
    classes.add(data.class);
    classes.add(db_field.class);
    classes.add(db_where.class);
    classes.add(db_order.class);

    classes.add(String.class);
    classes.add(Boolean.class);
    classes.add(boolean.class);
    classes.add(Byte.class);
    classes.add(byte.class);
    classes.add(Character.class);
    classes.add(char.class);
    classes.add(Object.class);

    for (Class<?> type: numbers.get_all_classes()) { classes.add(type); }
    for (Class<?> type: arrays.get_all_classes()) { classes.add(type); }

    return classes.toArray(new Class<?>[classes.size()]);
}

permalink

static final Class<?>[] populate_all_classes()
{
    ArrayList<Class<?>> output = new ArrayList<Class<?>>();

    output.add(HashMap.class);
    output.add(ArrayList.class);

    output.addAll(new ArrayList<Class<?>>(Arrays.asList(get_all_classes_array())));
    output.addAll(new ArrayList<Class<?>>(Arrays.asList(get_all_classes_small())));

    return output.toArray(new Class<?>[output.size()]);	
}

static Class<?>[] populate_all_classes_small() 
{ 
    return new Class<?>[] 
    { 
        double[].class, long[].class, int[].class, 
        boolean[].class, byte[].class, char[].class 
    };
}

static Class<?>[] populate_all_classes_array()
{
    return new Class<?>[]
    {
        Array.class, String[].class, Boolean[].class, Integer[].class, 
        Long[].class, Double[].class, Byte[].class, Character[].class, 
        Class[].class, Method[].class, Exception[].class, LocalTime[].class, 
        LocalDateTime[].class, size[].class, data[].class, 
        db_field[].class, db_where[].class, db_order[].class		
    };
}

permalink

The classes in the ib package whose names start with "async_data" deal with the market data feeds provided by IB. The development of this part has also had a curious evolution and it can be considered the ib equivalent of the accessory_java DB modes. In fact, I wrote most of the code for both implementations more or less at the same time. Similarly to what happens with the DB modes, I currently care almost exclusively about the quicker alternatives.

public static void start(String symbol_, int id_)
{
    boolean is_restart = async_data_quicker.id_is_ok(async_data_quicker._get_id(symbol_, false));
    boolean start_vals = !_is_only_db(false);

    int i = async_data_quicker.get_i(id_, true);

    if (APP.equals(async_data_watchlist_quicker._APP)) 
    {
        async_data_watchlist_quicker._last_id = id_;

        async_data_watchlist_quicker._symbols[i] = symbol_;
        if (start_vals) async_data_watchlist_quicker._vals[i] = new double[_vals_size];

        async_data_watchlist_quicker.start(symbol_, is_restart);
    }
    else if (APP.equals(async_data_trades_quicker._APP)) 
    {
        async_data_trades_quicker._last_id = id_;

        async_data_trades_quicker._symbols[i] = symbol_;
        if (start_vals) async_data_trades_quicker._vals[i] = new double[_vals_size];

        async_data_trades_quicker.start(symbol_, is_restart);
    }
    else if (APP.equals(async_data_market_quicker._APP)) 
    {
        async_data_market_quicker._last_id = id_;

        async_data_market_quicker._symbols[i] = symbol_;
        if (start_vals) async_data_market_quicker._vals[i] = new double[_vals_size];

        async_data_market_quicker.start(symbol_, is_restart);
    }
}

permalink

In the ib package, orders is the main public class dealing with all what relates to market orders. ib.execs is the main public class handling IB's executions, which are actions automatically triggered after an order is filled.

Of course, the corresponding articles are the best places to find reliable information about already discussed issues like encryption or conventions.

Friendliness

Ideas like programmer friendliness, usability, intuitiveness, feeling like relying on the corresponding resources right away or not having second thoughts about how a specific scenario could be (mis)managed describe the highest priority of these libraries, even the main reason why they exist at all. In fact, all the other concerns described in the next sections are, in some way, defining that friendliness, which wouldn't be such if those other priorities weren't adequately addressed (how could I intuitively trust an unsafe or inefficient code?).

The original scope of the libraries was much more limited, almost just "advancing work". Meaning generalising a code likely to be used in different parts of this specific development or in general, when working on any Java project. Those initially modest expectations gradually evolved into much wider goals, already with a major focus on programmer-friendliness, but firstly mostly meant for facilitating future maintenance. Such an expectation was quickly deemed somewhat irrelevant via the evident positive impact that the overall friendliness was having on my current programming work.

I am a fairly experienced programmer who is quite used to write code, to create from scratch rather than to use third-party resources. At the beginning of this project, I wasn't too experienced with Java, at least not for a development of this sort, and not as much as with other (pretty similar) languages like C#. In any case, I wasn't expecting to almost redefine different aspects of my programming approach, to learn to appreciate the positive impact of certain (small) details on my work. This isn't more than me and my expertise doing what I have been doing for many years, although the application of certain ideas in such a systematic way had surely never happened before. It is also true that I really never had the time, freedom and motivation to truly do everything my way, to fully focus on what is objectively best for the development without having to comply with more or less arbitrary constraints.

Some of the examples listed here and in the next sections might seem trivial or, at least, not worth the effort, but this is surely not the case, not as per my recent experiences. My work has become appreciably faster, more reliable and more stress-free. And I am not precisely a stress-prone person, much less while doing something I enjoy. On top of that, the final outcomes are clearly superior, for now and for the future. To not mention the evident work-done benefits, because I do tend to use the same kind of approaches and resources, the ones already included in these libraries, whose friendliness is evidently conditioned by their suitability to offer me what I want.

In summary, the friendliness of these libraries can be roughly defined as being able to throw anything at them (and feeling like doing so!), to not have to spend much time thinking about how to interact with/to eventually modify certain part, no matter how long ago I created it or how wrong the given inputs might be.

Conventions

  • Starting/ending variable names with underscores to indicate whether they are global or local (method parameters or not) is surprisingly helpful to speed up writing clean code, mainly when you tend to use similar variable names.
public static String update(String input_, String format_, String unit_, long increase_)
{
    String output = strings.DEFAULT;

    LocalDateTime input = from_string(input_, format_);
    if (input == null || !strings.is_ok(unit_)) return output;

    return (increase_ == 0 ? input_ : to_string(update(input, unit_, increase_), format_));
}

permalink

  • The systematic use of descriptive names and certain expressions facilitates a lot the location of any resource and appreciably reduces the number of required checks. If I want to use certain method to deal with strings, for example, I type "strings." in my IDE (Eclipse by default) and, afterwards, a meaningful word like "split" or "escape" or "contain". Then, thanks to the descriptive parameter names, I can choose the best overload right away. Note that I have created a big number of different methods, in some cases to address very specific needs. I surely don't (and shouldn't) know what is in each of them, much less months or years later.

  • The fact that the public methods are error-proof is also speeding up certain actions. For example, if I want to split a string in a simplistic way (i.e., supported by the Java inbuilt methods) and the inputs are known and valid, I would use the inbuilt resources because of being faster. In any other case, I would use one of the accessory.strings split overloads, because I know that all of them can gracefully deal with invalid inputs/outputs. I can reliably make split-second decisions which, without accessory_java, would require some thought or writing more code.

DB

The original version of this part was completely focused on friendliness. Actually, it was too focused on friendliness, which was precisely the reason why the quick modes eventually appeared. In any case, the whole evolution of this part is very descriptive of the main concerns here: firstly, friendliness; and, only afterwards and when proven required, anything else.

The original accessory.db querying methods are systematically checking pretty much everything. Inputs have to meet all the validity requirements: valid source/fields, valid data types and valid values.

public static ArrayList<HashMap<String, String>> select_quick
(
    String source_, String[] cols_, String where_cols_, int max_rows_, String order_cols_
) 
{ return db_queries.select_quick(source_, cols_, where_cols_, max_rows_, order_cols_); }

public static int select_count(String source_) { return select_count(source_, DEFAULT_WHERE); }

public static int select_count(String source_, String where_cols_) 
{ 
    return db_queries.select_count(source_, where_cols_); 
}

public static <x> void insert_update
(
    String source_, HashMap<String, x> vals_raw_, db_where[] where_
) 
{ insert_update(source_, vals_raw_, db_where.to_string(where_)); }

public static <x> void insert_update
(
    String source_, HashMap<String, x> vals_raw_, String where_cols_
) 
{ 
    if (exists(source_, where_cols_)) update(source_, vals_raw_, where_cols_);
    else insert(source_, vals_raw_);
}

permalink

The validity of the input values is firstly crosschecked against the accessory.data class requirements, which involve more than just compatibility with the given Java native types (e.g., Integer vs. String), and later against the ones of the corresponding DB type.

public static final String STRING_SMALL = _types.DATA_STRING_SMALL;
public static final String STRING_BIG = _types.DATA_STRING_BIG;
public static final String TINYINT = _types.DATA_TINYINT;
public static final String INT = _types.DATA_INT;
public static final String LONG = _types.DATA_LONG;
public static final String DECIMAL = _types.DATA_DECIMAL;
public static final String TIMESTAMP = _types.DATA_TIMESTAMP;
public static final String BOOLEAN = _types.DATA_BOOLEAN;

public static final String STRING = STRING_SMALL;

public static final String TRUE = _types.DATA_BOOLEAN_TRUE;
public static final String FALSE = _types.DATA_BOOLEAN_FALSE;

public static final double MIN_DECIMAL = numbers.MIN_DECIMAL;
public static final double MIN_LONG = numbers.MIN_LONG;
public static final double MIN_TINYINT = numbers.MIN_INT;
public static final double MIN_INT = numbers.MIN_INT;
public static final double MIN_STRING = 0.0;
public static final double MIN_STRING_SMALL = MIN_STRING;
public static final double MIN_STRING_BIG = MIN_STRING;
public static final double MIN_BOOLEAN = 2.0;
public static final double MIN_TIMESTAMP = 0.0;

public static final double MAX_DECIMAL = numbers.MAX_DECIMAL;
public static final double MAX_LONG = numbers.MAX_LONG;
public static final double MAX_TINYINT = db.get_max_value(TINYINT);
public static final double MAX_INT = numbers.MAX_INT;
public static final double MAX_STRING_SMALL = db.get_max_size(STRING_SMALL);
public static final double MAX_STRING_BIG = db.get_max_size(STRING_BIG);
public static final double MAX_BOOLEAN = 2.0;
public static final double MAX_TIMESTAMP = dates.get_length(dates.FORMAT_TIMESTAMP);

permalink

private static long get_max_size_static(String data_type_)
{
    long max = 0;

    String data_type = data.check_type(data_type_);
    if (!strings.is_ok(data_type)) return max;

    if (data.is_boolean(data_type)) max = 1;
    else if (data_type.equals(data.TIMESTAMP)) max = DEFAULT_SIZE_TIMESTAMP;		
    else if (data_type.equals(data.DECIMAL)) max = 64;
    else if (data_type.equals(data.TINYINT)) max = 3;
    else if (data_type.equals(data.INT)) max = numbers.MAX_DIGITS_INT;
    else if (data_type.equals(data.LONG)) max = numbers.MAX_DIGITS_LONG;
    else if (data_type.equals(data.STRING)) max = (long)get_max_value_static(data_type);
    else if (data_type.equals(data.STRING_BIG)) max = (long)get_max_value_static(data_type);

    return max;
}

permalink

When using the corresponding overloads, even the validity of the information in the where/order statements is checked via the db_where/db_order classes in the accessory package.

private boolean is_ok
(
    String source_, String field_col_, String operand_, 
    Object value_, String link_, boolean is_quick_
)
{
    _temp_source = db.check_source(source_);
    _temp_field_col = (is_quick_ ? field_col_ : db.check_field(_temp_source, field_col_));
    _temp_operand = check_operand(operand_);
    _temp_link = check_link(link_);

    return 
    (
        strings.are_ok
        (
            new String[] { _temp_source, _temp_field_col, _temp_operand }
        ) 
        && (value_ != null)
    );
}

permalink

private boolean is_ok
(
    String source_, String field_col_else_, 
    String order_, boolean is_field_col_, boolean is_quick_
)
{
    _temp_source = db.check_source(source_);
    _temp_field_col_else = 
    (
        (is_field_col_ && !is_quick_) ? db.check_field
        (
            _temp_source, field_col_else_
        ) 
        : field_col_else_
    );
    _temp_order = _types.check_type(order_, _types.DB_ORDER);

    return (strings.are_ok(new String[] { _temp_source, _temp_field_col_else, _temp_order }));
}

permalink

Just the fact of having permanent access to all the sources/tables, their structures and their constituent fields/columns is tremendously friendly and allows to perform actions which wouldn't be possible otherwise. For example, I find accessory.db.create_table particularly useful: it reliably generates a non-straightforward outcome from the simplest inputs (i.e., source and the option to drop an existing table). The systematic reliance on sources in most of the accessory.db methods is, in itself, a quite friendly feature. It is possible, for example, to use a source to get information from the DB including it.

public static void backup_db_to_file(String any_source_) 
{ 
    get_valid_instance(any_source_).backup_db_to_file(any_source_); 
}

public static void recover_db_from_file(String any_source_) 
{ 
    get_valid_instance(any_source_).restore_db_from_file(any_source_); 
}

public static String get_db_backup_path(String any_source_) 
{ 
    return get_valid_instance(any_source_).get_db_backup_path(any_source_); 
}

public static String get_db_restore_path(String any_source_) 
{ 
    return get_valid_instance(any_source_).get_db_restore_path(any_source_); 
}

permalink

Despite being formally considered an "ini" class, part of the startup, not one of the DB classes, accessory._ini_db does indeed represent an important part of the code handling DB management. I find the addition of sources via accessory.parent_ini_db.add_source, together with all the complementary resources meant to ease these actions, very friendly. For example, to add a new source to the default DB in accessory, being fully recognised by both libraries and already having a good starting structure for any actions affecting it, you would only need to emulate the setup of any other source. This can be easily accomplished by following these steps:

  • In accessory._types, add the new source/fields via CONFIG_NAME_DB constants by emulating the CONFIG_CRYPTO_DB ones.

  • Create a new db_name class with the most basic db_crypto structure. Just adding the public constants for source and fields would be enough.

  • In accessory._ini_db, write sources = add_source_name(db, sources); inside populate_all_dbs and create add_source_name by emulating add_sources_crypto and by relying on the friendly accessory.db_common methods (or on the db_ib.common ones) to add the fields. Although having to use the accessory.db_field constructors directly wouldn't be too unfriendly either.

public static db_field get_field_decimal_tiny() { return get_field_decimal(3); }

public static db_field get_field_decimal() { return get_field_decimal(DEFAULT_SIZE_DECIMAL); }

public static db_field get_field_decimal(int size_) 
{ 
    return new db_field(data.DECIMAL, size_, numbers.DEFAULT_DECIMALS); 
}

public static db_field get_field_tiny() { return get_field_tiny(false); }

public static db_field get_field_tiny(boolean is_unique_) 
{ 
    return 
    (
        is_unique_ ? new db_field
        (
            data.TINYINT, db_field.DEFAULT_SIZE, db_field.WRONG_DECIMALS, 
            db_field.WRONG_DEFAULT, new String[] { db_field.KEY_UNIQUE }
        ) 
        : new db_field(data.TINYINT)
    ); 
}

public static db_field get_field_error() { return get_field_string(MAX_SIZE_ERROR); }

public static db_field get_field_int() { return get_field_int(false); }

public static db_field get_field_int(boolean is_unique_) 
{ 
    return new db_field
    (
        data.INT, db_field.DEFAULT_SIZE, db_field.WRONG_DECIMALS, db_field.DEFAULT_DEFAULT, 
        (is_unique_ ? new String[] { db_field.KEY_UNIQUE } : null)
    ); 
}

public static db_field get_field_string() { return get_field_string(DEFAULT_SIZE_STRING); }

public static db_field get_field_string(int size_) { return get_field_string(size_, false); }

public static db_field get_field_string(boolean is_unique_) 
{ 
    return get_field_string(DEFAULT_SIZE_STRING, is_unique_); 
}

public static db_field get_field_string(int size_, boolean is_unique_) 
{ 
    return get_field_string(size_, is_unique_, null); 
}

public static db_field get_field_string(int size_, boolean is_unique_, String default_) 
{ 
    return new db_field
    (
        data.STRING, size_, db_field.WRONG_DECIMALS, 
        (strings.is_ok(default_) ? default_ : db_field.WRONG_DEFAULT), 
        (is_unique_ ? new String[] { db_field.KEY_UNIQUE } : null)
    ); 
}

public static db_field get_field_string_big() 
{ 
    return new db_field(data.STRING_BIG, 0, db_field.WRONG_DECIMALS, null, null); 
}

public static db_field get_field_is_enc() { return get_field_boolean(false); }

public static db_field get_field_boolean(boolean default_) 
{ 
    return new db_field
    (
        data.BOOLEAN, db_field.DEFAULT_SIZE, db_field.WRONG_DECIMALS, default_, null
    ); 
}

public static db_field get_field_time(String format_) 
{ 
    return new db_field
    (
        data.STRING, dates.get_length(format_), 
        db_field.WRONG_DECIMALS, 
        dates.get_default(format_), null
    ); 
}

public static db_field get_field_date(String format_) 
{ 
    return new db_field
    (
        data.STRING, dates.get_length(format_), 
        db_field.WRONG_DECIMALS, dates.get_default(format_), null
    ); 
}

permalink

public static db_field get_field_symbol(boolean is_unique_) 
{ 
    return db_common.get_field_string(contracts.MAX_LENGTH_SYMBOL_US_ANY, is_unique_); 
}

public static db_field get_field_order_id(boolean is_unique_) 
{ 
    return 
    (
        is_unique_ ? new db_field
        (
            data.INT, db_field.DEFAULT_SIZE, db_field.WRONG_DECIMALS, 
            ib.common.WRONG_ORDER_ID, new String[] { db_field.KEY_UNIQUE }
        ) 
        : new db_field(data.INT)
    ); 
}

public static db_field get_field_money() 
{ 
    return db_common.get_field_decimal(common.MAX_SIZE_MONEY); 
}

public static db_field get_field_quantity() { return db_common.get_field_decimal(); }

public static db_field get_field_size_volume() 
{ 
    return db_common.get_field_decimal(common.MAX_SIZE_VOLUME); 
}

public static db_field get_field_price() 
{ 
    return new db_field(data.DECIMAL, common.MAX_SIZE_PRICE, 2); 
}

public static db_field get_field_halted() { return db_common.get_field_tiny(); }

public static db_field get_field_halted_tot() { return db_common.get_field_tiny(); }

public static db_field get_field_time() 
{ 
    return db_common.get_field_time(ib.common.FORMAT_TIME); 
}

public static db_field get_field_time2() 
{ 
    return db_common.get_field_time(ib.common.FORMAT_TIME2); 
}

public static db_field get_field_time_elapsed() 
{ 
    return db_common.get_field_time(ib.common.FORMAT_TIME_ELAPSED); 
}

public static db_field get_field_elapsed_ini() { return new db_field(data.LONG); }

public static db_field get_field_user() { return get_field_user(false); }

public static db_field get_field_user(boolean is_unique_) 
{ 
    return db_common.get_field_string(common.MAX_SIZE_USER, is_unique_); 
}

public static db_field get_field_status_type() { return get_field_status_type(null); }

public static db_field get_field_status_type(String default_) 
{ 
    return db_common.get_field_string(db_common.DEFAULT_SIZE_STRING, false, default_); 
}

public static db_field get_field_app(boolean is_unique_) 
{ 
    return get_field_name(db_ib.common.MAX_SIZE_APP_NAME, is_unique_); 
}

public static db_field get_field_name(int size_, boolean is_unique_) 
{ 
    return db_common.get_field_string(size_, is_unique_); 
}

public static db_field get_field_request() { return db_common.get_field_int(true); }	

public static db_field get_field_date() 
{ 
    return db_common.get_field_date(ib.common.FORMAT_DATE); 
}

permalink

Classes and collections

The classes arrays and generic in the accessory package provide an additional layer of friendliness by virtually removing the differences between the native small/big types like int/Integer. Without trying to critise Java's implementation or even wanting to go deeper into these issues and possible alternatives, the way in which such a duplicity is being handled by default is simply incompatible with my expectations. That's why the small and big types are used interchangeably virtually anywhere across these libraries, what includes not just single variables (e.g., long/Long) but also arrays (e.g., byte[]/Byte[]). If in doubt, just call accessory.generic.are_equal.

public static boolean are_equal
(
    Class<?> input1_, Class<?> input2_
) 
{ return classes_are_equal(input1_, input2_); }

permalink

private static boolean classes_are_equal(Class<?> class1_, Class<?> class2_)
{
    boolean is_ok1 = is_ok(class1_);
    boolean is_ok2 = is_ok(class2_);
    if (!is_ok1 || !is_ok2) return (is_ok1 == is_ok2);

    if (class1_.equals(class2_)) return true;

    for (Entry<Class<?>, Class<?>[]> item: get_all_class_equivalents().entrySet())
    {
        Class<?> key = item.getKey();

        for (Class<?> val: item.getValue())
        {
            if 
            (
                (class1_.equals(key) && class2_.equals(val)) || 
                (class1_.equals(val) && class2_.equals(key))
            ) 
            { return true; }	
        }
    }

    return false;
}

permalink

static HashMap<Class<?>, Class<?>[]> populate_all_class_equivalents()
{
    HashMap<Class<?>, Class<?>[]> output = new HashMap<Class<?>, Class<?>[]>();

    output.put(Double.class, new Class<?>[] { double.class });
    output.put(Long.class, new Class<?>[] { long.class });
    output.put(Integer.class, new Class<?>[] { int.class });
    output.put(Boolean.class, new Class<?>[] { boolean.class });
    output.put(Byte.class, new Class<?>[] { byte.class });
    output.put(Character.class, new Class<?>[] { char.class });
    output.put(Double[].class, new Class<?>[] { double[].class });
    output.put(Long[].class, new Class<?>[] { long[].class });
    output.put(Integer[].class, new Class<?>[] { int[].class });
    output.put(Boolean[].class, new Class<?>[] { boolean[].class });
    output.put(Byte[].class, new Class<?>[] { byte[].class });
    output.put(Character[].class, new Class<?>[] { char[].class });
    output.put(Array.class, new Class<?>[] { Object[].class });

    return output;
}

permalink

The default Java support for compatibility between small/big single variables, although not perfect, doesn't really need to be complemented in a relevant way. Conversely, an important part of the accessory.arrays code is required to properly account for the more complex reality of small/big arrays. The to_big/to_small methods can be used to convert to/from these classes, but this is just a small part of the whole story.

public static Character[][] to_big(char[][] input_)
{
    int tot = get_size(input_);
    if (tot < 1) return null;

    Character[][] output = new Character[tot][];

    for (int i = 0; i < tot; i++) 
    { 
        if (input_[i] == null) continue;

        int tot2 = input_[i].length;

        output[i] = new Character[tot2];

        for (int i2 = 0; i2 < tot2; i2++) output[i][i2] = (Character)input_[i][i2];
    }

    return output;
}

public static double[] to_small(Double[] input_)
{
    int tot = get_size(input_);
    if (tot < 1) return null;

    double[] output = new double[tot];

    for (int i = 0; i < tot; i++) 
    { 
        output[i] = (input_[i] != null ? input_[i] : DEFAULT_DECIMAL); 
    }

    return output;
}

permalink

Additionally to the small/big support, most of the public methods in the accessory.arrays class can be used with any of the supported collections. That is, you can use the same method with compatible ArrayList, Hashmap or array instances. Depending on its specific functionality, the given method can support all the possible types or just some of them, a difference which should be intuitively clear. For example, there are two types of accessory.arrays.to_string methods: the overloads accepting variables of all the supported collections and the ones focused on maximising the HashMap peculiarities.

public static <x, y> String to_string
(
    HashMap<x, y> input_, String sep_items_, String sep_keyval_, String[] keys_ignore_
) 
{ return to_string(input_, sep_items_, sep_keyval_, keys_ignore_, true, true); }

public static String to_string
(
    Object input_, String sep_items_, String sep_keyval_, String[] keys_ignore_
) 
{ 
    return 
    (
        generic.are_equal(generic.get_class(input_), HashMap.class) ? 
        to_string(input_, sep_items_, sep_keyval_, keys_ignore_, false, true) : strings.DEFAULT
    ); 
}

public static String to_string(double[] input_, String separator_) 
{ 
    return to_string(to_big(input_), separator_, true); 
}

public static String to_string(long[] input_, String separator_) 
{ 
    return to_string(to_big(input_), separator_, true); 
}

public static String to_string(int[] input_, String separator_) 
{ 
    return to_string(to_big(input_), separator_, true); 
}

public static String to_string(boolean[] input_, String separator_) 
{ 
    return to_string(to_big(input_), separator_, true); 
}

public static String to_string(byte[] input_, String separator_) 
{ 
    return strings.from_bytes_base64(input_); 
}

public static String to_string(char[] input_, String separator_) 
{ 
    return to_string(to_big(input_), separator_, true); 
}

public static <x> String to_string(x[] input_, String separator_) 
{ 
    return to_string(input_, separator_, true); 
}

public static <x> String to_string(ArrayList<x> input_, String separator_) 
{ 
    return to_string(input_, separator_, true); 
}

@SuppressWarnings("unchecked")
public static <x> String to_string(Object input_, String separator_, boolean include_brackets_)
{
    String output = strings.DEFAULT;

    Class<?> type = generic.get_class(input_);
    if (!generic.is_array(type)) return output;

    if (type.equals(double[].class)) return to_string((double[])input_, separator_);
    else if (type.equals(long[].class)) return to_string((long[])input_, separator_);
    else if (type.equals(int[].class)) return to_string((int[])input_, separator_);
    else if (type.equals(boolean[].class)) return to_string((boolean[])input_, separator_);
    else if (type.equals(byte[].class)) return to_string((byte[])input_, separator_);
    else if (type.equals(char[].class)) return to_string((char[])input_, separator_);

    boolean is_array = generic.is_array_class(type);
    boolean is_arraylist = generic.are_equal(type, ArrayList.class);
    if (!is_array && !is_arraylist) return output;

    output = "";
    String separator = (strings.is_ok(separator_) ? separator_ : SEPARATOR_ARRAY);
    boolean first_time = true;

    if (generic.are_equal(type, Byte[].class)) 
    {
        output = strings.from_bytes_base64((Byte[])input_);
    }
    else if (is_array)
    {
        for (x item: (x[])input_)
        {
            if (first_time) first_time = false;
            else output += separator;

            output += to_string_val(item);
        }	
    }
    else if (is_arraylist)
    {
        for (x item: (ArrayList<x>)input_)
        {
            if (first_time) first_time = false;
            else output += separator;

            output += to_string_val(item);
        }
    }

    if (include_brackets_ && strings.is_ok(output)) 
    {
        output = misc.BRACKET_MAIN_OPEN + output + misc.BRACKET_MAIN_CLOSE;
    }

    return output;
}

@SuppressWarnings("unchecked")
public static <x, y> String to_string
(
    Object input_, String sep_items_, String sep_keyval_, 
    String[] keys_ignore_, boolean is_xy_, boolean include_brackets_
)
{
    if 
    (
        !is_ok(input_) || !generic.are_equal
        (
            generic.get_class(input_), HashMap.class
        )
    ) 
    { return strings.DEFAULT; }

    String output = "";
    String separator1 = (strings.is_ok(sep_items_) ? sep_items_ : SEPARATOR_HASHMAP_1);
    String separator2 = (strings.is_ok(sep_keyval_) ? sep_keyval_ : SEPARATOR_HASHMAP_2);

    boolean first_time = true;

    Object input = get_new_hashmap(input_, is_xy_);

    for (Object item: (is_xy_ ? (HashMap<x, y>)input : (HashMap<x, x>)input).entrySet())
    {
        Object[] temp = get_entry_key_val(item, is_xy_);
        Object key = temp[0];
        Object val = temp[1];					
        if (!generic.is_ok(key) || value_exists(keys_ignore_, key)) continue; 

        if (first_time) first_time = false;
        else output += separator1;

        String key2 = (generic.is_array(key) ? key.toString() : strings.to_string(key));
        String val2 = to_string_val(val);

        output += (key2 + separator2 + val2);
    }

    if (include_brackets_ && strings.is_ok(output)) 
    {
        output = misc.BRACKET_MAIN_OPEN + output + misc.BRACKET_MAIN_CLOSE;
    }

    return output;
}

permalink

The friendliness associated with such a comprehensive support for "arrays" is evident, although its implementation isn't precisely straightforward. More detailed explanations about the way in which accessory_java deals with collections will be part of future works.

Safety

I am including here both writing code safely and writing safe code. Typos happen and, although there is no replacement for the good old proper work (e.g., full attention, constant checks, lots of tests, etc.), any helpful bits are surely welcome, mainly when you write a lot of code at a relatively fast pace, as in my case. Furthermore, the created code is expected to run as smoothly as possible regardless of anything else. This also includes reliably logging all the important issues and avoiding uncaught exceptions or other undesired outcomes. In summary, making sure that each scenario is gracefully managed, that no erratic, uncontrolled behaviours happen. Generic safety measures like encrypting sensitive information are included in this second priority too.

Examples:

  • All the conventions involving names starting/ending with underscores are mostly meant for safety purposes. Warning about locked methods/classes all the way up or about variables being global or local (method parameters or not) is supposed to minimise the chances of making mistakes when writing code. Even the fact of knowing that the given class is loaded at startup is very useful to avoid errors. Bear in mind that most of the resources of these libraries, which are systematically being used everywhere, don’t work well at the very beginning, and having to implement workarounds isn’t unusual when developing these parts.
public static boolean start(int id_, String type_, boolean force_running_, boolean is_test_)
{
    boolean output = false;

    if (is_test_) wrapper_errors.log_errors(false);
    else update_is_connected(false);

    String error = null;

    if (!id_is_ok(id_)) error = ERROR_ID;
    else if (!type_is_ok(type_)) error = ERROR_TYPE;

    if (error != null)
    {
        errors.manage(error);

        return output;
    }

    _id = id_;
    _type = type_;

    update_port(type_);

    _wrapper = new wrapper();
    _client = _wrapper.get_client();

    output = connect(force_running_, is_test_);

    if (is_test_) wrapper_errors.log_errors(wrapper_errors.DEFAULT_LOG_ERRORS);

    return output;
}

permalink

  • The default accessibility rules are also eminently focused on safety. Making all the methods private, for example, is a good way to force yourself to think twice before making them public, with all that it entails.
public static double to_number(Object input_)
{
    double output = 0.0;

    Class<?> type = generic.get_class(input_);

    if (generic.are_equal(type, Double.class)) output = (double)input_;
    else if (generic.are_equal(type, Integer.class)) output = (double)((int)input_);
    else if (generic.are_equal(type, Long.class)) output = (double)((long)input_);
    else if (generic.are_equal(type, Boolean.class)) output = (double)to_int((boolean)input_);
    else if (generic.are_equal(type, String.class)) output = to_decimal((String)input_);

    return output;
}

permalink

  • All the checks and adaptations in the first version of most of the accessory.db methods, additionally to making their usage tremendously friendly, were mainly meant for safety purposes. The safety of the original versions of the querying methods relies on those checks for the most part, but all the queries are executed inside a try-catch block anyway (currently only in accessory.db_sql.execute_query). All the accessory.db_quicker querying methods have their own try-catch blocks.
private static ArrayList<HashMap<String, String>> execute_query
(
    String source_, String query_, boolean return_data_, 
    String[] cols_, Connection conn_, boolean is_static_
)
{
    db.update_is_ok(source_, false, is_static_);

    db._last_query = query_;

    if (db._print_all_queries) generic.to_screen(query_);

    ArrayList<HashMap<String, String>> output = new ArrayList<HashMap<String, String>>();
    if (conn_ == null) return output;

    try 
    {
        PreparedStatement statement = conn_.prepareStatement(query_);

        if (!return_data_) 
        {
            statement.executeUpdate();

            db.update_is_ok(source_, true, is_static_);

            return output;
        }

        try
        {
            ResultSet data = statement.executeQuery();

            String[] cols = execute_query_get_cols(source_, data, cols_, is_static_);
            if (!arrays.is_ok(cols)) return output;

            while (data.next()) 
            {
                HashMap<String, String> row = new HashMap<String, String>();

                for (String col : cols)
                {
                    String val = 
                    (
                        (strings.is_ok(col) && data.findColumn(col) > -1) ? 
                        data.getString(col) : strings.DEFAULT
                    );
                    if (!strings.is_ok(val)) val = strings.DEFAULT;

                    row.put(col, val);
                }

                output.add(row);
            }	

            db.update_is_ok(source_, true, is_static_);
        }
        catch (Exception e) 
        { 
            db.manage_error(source_, db.ERROR_QUERY, query_, e, null, is_static_); 
        }
    } 
    catch (Exception e) 
    { 
        db.manage_error(source_, db.ERROR_QUERY, query_, e, null, is_static_); 
    } 
    finally { disconnect(source_, conn_, is_static_); }

    return output;
}

permalink

public static boolean exists
(
    String type_, String source_, String count_col_, String where_cols_
) 
{
    initialise();

    boolean output = false;

    try { output = (select_count(type_, source_, count_col_, where_cols_) > 0); }
    catch (Exception e) 
    { 
        output = false;

        manage_error
        (
            type_, strings.to_string(source_), e, null, null, null, 
            strings.to_string(where_cols_), null, db.WRONG_INT, null
        );
    }

    return output; 
}

public static String select_one_string
(
    String type_, String source_, String col_, String where_cols_, String order_cols_
) 
{ 
    initialise();

    String output = db.WRONG_STRING;

    try
    {	
        Object temp = get_val
        (
            col_, select_one
            (
                type_, source_, new String[] { col_ }, 
                where_cols_, order_cols_
            ), 
            data.STRING
        );

        if (temp != null) output = (String)temp;
    }
    catch (Exception e) 
    {
        output = db.WRONG_STRING;

        manage_error
        (
            type_, strings.to_string(source_), e, strings.to_string(col_), 
            null, null, strings.to_string(where_cols_), 
            strings.to_string(order_cols_), db.WRONG_INT, null
        ); 
    }

    return output; 
}

permalink

  • Error management is handled by the accessory.errors class, mainly via its manage methods which internally rely on accessory.logs as needed. These methods can be used anywhere when required, but they are always called inside try-catch blocks. Note that the default error logging will work right away with no additional configuration. Or, in other words, all the resources in accessory_java/ib can be used with the certainty that any relevant problem will be reported as defined by the current configuration which, by default, includes on-screen messages and file logs stored in accessory.paths.get_dir(accessory.paths.DIR_LOGS_ERRORS).
public static void manage(String type_, Exception e_) { manage(get_message(type_, e_)); }

public static void manage(String type_, Exception e_, String further_) 
{ 
    manage(get_message(type_, e_, further_)); 
}

public static void manage(HashMap<String, Object> info_) 
{ 
    HashMap<String, Object> info = new HashMap<String, Object>(info_);

    String type = null;

    if (info != null && info.containsKey(TYPE))
    {
        type = (String)info.get(TYPE);

        info.remove(TYPE);
    }

    manage(type, null, info); 
}

public static void manage(String type_, Exception e_, HashMap<String, Object> info_) 
{ 
    manage(get_message(type_, e_, get_message(info_))); 
}

public static void manage(String[] info_) { manage(null, null, info_); }

public static void manage(String type_, Exception e_, String[] info_) 
{ 
    manage(get_message(type_, e_, get_message(info_))); 
}

permalink

  • When working with encryption, the main class is accessory.crypto. There are also other resources facilitating the management of encrypted information, which is the default option in case of dealing with sensitive data like credentials or the IB's account.
private static void populate_account_ib(String account_ib_, boolean ignore_ib_info_) 
{ 
    String account_ib = strings.DEFAULT;

    if (!ignore_ib_info_)
    {
        account_ib = (strings.is_ok(account_ib_) ? account_ib_ : get_account_ib_from_file());

        if (strings.is_ok(account_ib) && !account_ib.equals(get_account_ib(true)))
        {
            account_ib = encrypt_account_ib(account_ib);

            db_ib.basic.update_account_ib(account_ib); 
        }

        account_ib = get_account_ib(false);
    }

    _account_ib = account_ib;
}

permalink

  • For evident reasons, safety is a major concern in ib. The classes ib.orders and ib.execs, for example, include multiple methods which retrieve the most recent updates and which are being systematically called when performing virtually any relevant action to ensure the most up-to-date information.
public static boolean is_submitted(int order_id_main_) 
{ 
    return order_is_common(order_id_main_, STATUS_SUBMITTED); 
}

public static boolean is_submitted(String symbol_) 
{ 
    return order_is_common(symbol_, STATUS_SUBMITTED); 
}

public static boolean __is_submitted_ib(int order_id_main_) 
{ 
    return sync_orders.is_submitted(order_id_main_, sync.__get_orders()); 
}	

public static boolean __is_submitted_both(int order_id_main_) 
{ 
    return (is_submitted(order_id_main_) || __is_submitted_ib(order_id_main_)); 
}

public static boolean is_filled(int order_id_main_) 
{ 
    return order_is_common(order_id_main_, STATUS_FILLED); 
}	

public static boolean is_filled(String symbol_) 
{ 
    return order_is_common(symbol_, STATUS_FILLED); 
}	

public static boolean __is_filled_ib(int order_id_main_) 
{ 
    return is_filled_ib(order_id_main_, sync.__get_orders()); 
}	

public static boolean __is_filled_both(int order_id_main_) 
{ 
    return (is_filled(order_id_main_) || __is_filled_ib(order_id_main_)); 
}

public static boolean is_filled_ib
(
    int order_id_main_, HashMap<Integer, String> orders_
) 
{ return sync_orders.is_filled(order_id_main_, orders_); }	

public static boolean is_inactive(int order_id_main_) 
{ 
    return 
    (
        !db_ib.orders.exists(order_id_main_, true) || 
        order_is_common(order_id_main_, STATUS_INACTIVE)
    ); 
}	

public static boolean is_inactive(String symbol_) 
{ 
    return (!db_ib.orders.exists(symbol_) || order_is_common(symbol_, STATUS_INACTIVE)); 
}	

public static boolean __is_inactive_ib(int order_id_main_) 
{ 
    return sync_orders.is_inactive(order_id_main_, sync.__get_orders()); 
}	

public static boolean is_active(int order_id_main_) { return !is_inactive(order_id_main_); }	

public static boolean is_active(String symbol_) { return !is_inactive(symbol_); }	

permalink

public static boolean is_filled(int order_id_main_) 
{ 
    return db_ib.execs.is_filled(order_id_main_); 
}

public static boolean is_filled(String symbol_) 
{ 
    return arrays.is_ok(db_ib.execs.get_order_ids_filled(symbol_)); 
}

public static boolean is_completed(int order_id_main_) 
{ 
    return db_ib.execs.is_completed(order_id_main_); 
}

public static boolean is_completed(String symbol_) 
{ 
    return arrays.is_ok(db_ib.execs.get_order_ids_completed(symbol_)); 
}

public static ArrayList<Integer> get_order_ids_filled(String symbol_) 
{ 
    return db_ib.execs.get_order_ids_filled(symbol_); 
}

public static ArrayList<Integer> get_order_ids_completed(String symbol_) 
{ 
    return db_ib.execs.get_order_ids_completed(symbol_); 
}

public static ArrayList<HashMap<String, String>> get_all_filled() 
{ 
    return db_ib.execs.get_all_filled(); 
}

permalink

Efficiency

Writing quick code and relying on the less resource-consuming alternatives, despite being kind of a natural tendency for me, can't compete with friendliness or safety, mainly in these libraries.

accessory.db_field

I consider accessory.db_field a good example of non-static class, a valid exception for the preferred static option. Although the primary use of this class, namely defining the sources at startup, represents an important efficiency bottleneck for these libraries, probably the most relevant one. All the sources are loaded at startup and kept in memory for as long as the libraries are in use. In principle, there is no problem with it because this information is very important and, even under extreme conditions, not too "heavy" (at least not in a truly efficient implementation). Although using a non-static class like accessory.db_field so much (i.e., one instance per field/column in each source/table) is anything but efficient.

public static final String KEY_UNIQUE = _types.DB_FIELD_FURTHER_KEY_UNIQUE;
public static final String KEY_PRIMARY = _types.DB_FIELD_FURTHER_KEY_PRIMARY;
public static final String TIMESTAMP = _types.DB_FIELD_FURTHER_TIMESTAMP;
public static final String AUTO_INCREMENT = _types.DB_FIELD_FURTHER_AUTO_INCREMENT;

public static final String SERIALISATION_LABEL_TYPE = "type";
public static final String SERIALISATION_LABEL_SIZE = "size";
public static final String SERIALISATION_LABEL_DECIMALS = "decimals";
public static final String SERIALISATION_LABEL_DEFAULT = "default";
public static final String SERIALISATION_LABEL_FURTHER = "further";

public static final int MAX_DECIMALS = size.MAX_DECIMALS;
public static final int MIN_DECIMALS = size.MIN_DECIMALS;

public static final String WRONG_TYPE = strings.DEFAULT;
public static final long WRONG_SIZE = 0l;
public static final int WRONG_DECIMALS = size.WRONG_DECIMALS;
public static final Object WRONG_DEFAULT = null;

public static final String DEFAULT_TYPE = data.STRING;
public static final long DEFAULT_SIZE = WRONG_SIZE;
public static final int DEFAULT_DECIMALS = numbers.DEFAULT_DECIMALS;
public static final Object DEFAULT_DEFAULT = WRONG_DEFAULT;

private String _type = WRONG_TYPE;
private long _size = WRONG_SIZE;
private int _decimals = WRONG_DECIMALS;
private Object _default = WRONG_DEFAULT;
private String[] _further = null;

private String _temp_type = strings.DEFAULT;
private long _temp_size = 0;
private int _temp_decimals = 0;

public static boolean are_equal(db_field field1_, db_field field2_) 
{ 
    return are_equal_common(field1_, field2_); 
}

public static boolean further_is_ok(String[] further_)
{
    if (!arrays.is_ok(further_)) return true;

    for (String further: further_)
    {
        if (!further_is_ok(further)) return false;
    }

    return true;
}

permalink

In its current implementation, ib uses 14 sources with an average of 13 fields each, what, as per my current and most common requirements, is a pretty big setup. For my use case, the overall memory consumption is acceptable and, although the loading times are slightly slow, I still don’t see any problem. On the other hand, I am sure that much more demanding setups aren't exactly exceptional. Converting accessory.db_field into an efficiency-focused alternative would certainly have a very positive impact on both loading times and memory usage. An extreme example would be something on the lines of Object[] variables storing the different types of information in certain positions (e.g., 0 for data type, 1 for size and so on). Performing an implementation like this right away wouldn't be an option (very unfriendly, even "dirty"), but moving to that from the clean, working-fine accessory.db_field class would be a different story. It would be neither difficult nor unfriendly. Only the parts dealing with information storage/retrieval would have to be changed and most of the current structure might even be left commented for clarity purposes.

public static final String SOURCE_MARKET = _types.CONFIG_DB_IB_MARKET_SOURCE;
public static final String SOURCE_EXECS = _types.CONFIG_DB_IB_EXECS_SOURCE;
public static final String SOURCE_BASIC = _types.CONFIG_DB_IB_BASIC_SOURCE;
public static final String SOURCE_REMOTE = _types.CONFIG_DB_IB_REMOTE_SOURCE;
public static final String SOURCE_ORDERS = _types.CONFIG_DB_IB_ORDERS_SOURCE;
public static final String SOURCE_TRADES = _types.CONFIG_DB_IB_TRADES_SOURCE;
public static final String SOURCE_WATCHLIST = _types.CONFIG_DB_IB_WATCHLIST_SOURCE;
public static final String SOURCE_APPS = _types.CONFIG_DB_IB_APPS_SOURCE;

public static final String SOURCE_EXECS_OLD = _types.CONFIG_DB_IB_EXECS_OLD_SOURCE;
public static final String SOURCE_BASIC_OLD = _types.CONFIG_DB_IB_BASIC_OLD_SOURCE;
public static final String SOURCE_REMOTE_OLD = _types.CONFIG_DB_IB_REMOTE_OLD_SOURCE;
public static final String SOURCE_ORDERS_OLD = _types.CONFIG_DB_IB_ORDERS_OLD_SOURCE;
public static final String SOURCE_TRADES_OLD = _types.CONFIG_DB_IB_TRADES_OLD_SOURCE;
public static final String SOURCE_APPS_OLD = _types.CONFIG_DB_IB_APPS_OLD_SOURCE;

public static final String FIELD_SYMBOL = _types.CONFIG_DB_IB_FIELD_SYMBOL;
public static final String FIELD_PRICE = _types.CONFIG_DB_IB_FIELD_PRICE;
public static final String FIELD_SIZE = _types.CONFIG_DB_IB_FIELD_SIZE;
public static final String FIELD_TIME = _types.CONFIG_DB_IB_FIELD_TIME;
public static final String FIELD_OPEN = _types.CONFIG_DB_IB_FIELD_OPEN;
public static final String FIELD_CLOSE = _types.CONFIG_DB_IB_FIELD_CLOSE;
public static final String FIELD_LOW = _types.CONFIG_DB_IB_FIELD_LOW;
public static final String FIELD_HIGH = _types.CONFIG_DB_IB_FIELD_HIGH;
public static final String FIELD_VOLUME = _types.CONFIG_DB_IB_FIELD_VOLUME;		
public static final String FIELD_ASK = _types.CONFIG_DB_IB_FIELD_ASK;
public static final String FIELD_ASK_SIZE = _types.CONFIG_DB_IB_FIELD_ASK_SIZE;
public static final String FIELD_BID = _types.CONFIG_DB_IB_FIELD_BID;
public static final String FIELD_BID_SIZE = _types.CONFIG_DB_IB_FIELD_BID_SIZE;
public static final String FIELD_HALTED = _types.CONFIG_DB_IB_FIELD_HALTED;
public static final String FIELD_HALTED_TOT = _types.CONFIG_DB_IB_FIELD_HALTED_TOT;
public static final String FIELD_ENABLED = _types.CONFIG_DB_IB_FIELD_ENABLED;

public static final String FIELD_USER = _types.CONFIG_DB_IB_FIELD_USER;
public static final String FIELD_ORDER_ID = _types.CONFIG_DB_IB_FIELD_ORDER_ID;
public static final String FIELD_QUANTITY = _types.CONFIG_DB_IB_FIELD_QUANTITY;
public static final String FIELD_SIDE = _types.CONFIG_DB_IB_FIELD_SIDE;
public static final String FIELD_FEES = _types.CONFIG_DB_IB_FIELD_FEES;
public static final String FIELD_EXEC_ID = _types.CONFIG_DB_IB_FIELD_EXEC_ID;

public static final String FIELD_MONEY = _types.CONFIG_DB_IB_FIELD_MONEY;
public static final String FIELD_MONEY_INI = _types.CONFIG_DB_IB_FIELD_MONEY_INI;
public static final String FIELD_ACCOUNT_IB = _types.CONFIG_DB_IB_FIELD_ACCOUNT_IB;
public static final String FIELD_CURRENCY = _types.CONFIG_DB_IB_FIELD_CURRENCY;
public static final String FIELD_MONEY_FREE = _types.CONFIG_DB_IB_FIELD_MONEY_FREE;

public static final String FIELD_START = _types.CONFIG_DB_IB_FIELD_START;
public static final String FIELD_START2 = _types.CONFIG_DB_IB_FIELD_START2;
public static final String FIELD_STOP = _types.CONFIG_DB_IB_FIELD_STOP;
public static final String FIELD_ORDER_ID_MAIN = _types.CONFIG_DB_IB_FIELD_ORDER_ID_MAIN;
public static final String FIELD_ORDER_ID_SEC = _types.CONFIG_DB_IB_FIELD_ORDER_ID_SEC;
public static final String FIELD_STATUS = _types.CONFIG_DB_IB_FIELD_STATUS;
public static final String FIELD_STATUS2 = _types.CONFIG_DB_IB_FIELD_STATUS2;
public static final String FIELD_IS_MARKET = _types.CONFIG_DB_IB_FIELD_IS_MARKET;
public static final String FIELD_PERC_MONEY = _types.CONFIG_DB_IB_FIELD_PERC_MONEY;
public static final String FIELD_REQUEST = _types.CONFIG_DB_IB_FIELD_REQUEST;
public static final String FIELD_TYPE_ORDER = _types.CONFIG_DB_IB_FIELD_TYPE_ORDER;

public static final String FIELD_TYPE_PLACE = _types.CONFIG_DB_IB_FIELD_TYPE_PLACE;
public static final String FIELD_TYPE_MAIN = _types.CONFIG_DB_IB_FIELD_TYPE_MAIN;
public static final String FIELD_TYPE_SEC = _types.CONFIG_DB_IB_FIELD_TYPE_SEC;

public static final String FIELD_TIME_ELAPSED = _types.CONFIG_DB_IB_FIELD_TIME_ELAPSED;
public static final String FIELD_ELAPSED_INI = _types.CONFIG_DB_IB_FIELD_ELAPSED_INI;
public static final String FIELD_UNREALISED = _types.CONFIG_DB_IB_FIELD_UNREALISED;
public static final String FIELD_IS_ACTIVE = _types.CONFIG_DB_IB_FIELD_IS_ACTIVE;
public static final String FIELD_INVESTMENT = _types.CONFIG_DB_IB_FIELD_INVESTMENT;
public static final String FIELD_END = _types.CONFIG_DB_IB_FIELD_END;
public static final String FIELD_REALISED = _types.CONFIG_DB_IB_FIELD_REALISED;

public static final String FIELD_PRICE_INI = _types.CONFIG_DB_IB_FIELD_PRICE_INI;
public static final String FIELD_PRICE_MIN = _types.CONFIG_DB_IB_FIELD_PRICE_MIN;
public static final String FIELD_PRICE_MAX = _types.CONFIG_DB_IB_FIELD_PRICE_MAX;
public static final String FIELD_VOLUME_INI = _types.CONFIG_DB_IB_FIELD_VOLUME_INI;
public static final String FIELD_VOLUME_MIN = _types.CONFIG_DB_IB_FIELD_VOLUME_MIN;
public static final String FIELD_VOLUME_MAX = _types.CONFIG_DB_IB_FIELD_VOLUME_MAX;

public static final String FIELD_FLU = _types.CONFIG_DB_IB_FIELD_FLU;
public static final String FIELD_FLU2 = _types.CONFIG_DB_IB_FIELD_FLU2;
public static final String FIELD_FLU2_MIN = _types.CONFIG_DB_IB_FIELD_FLU2_MIN;
public static final String FIELD_FLU2_MAX = _types.CONFIG_DB_IB_FIELD_FLU2_MAX;
public static final String FIELD_FLUS_PRICE = _types.CONFIG_DB_IB_FIELD_FLUS_PRICE;
public static final String FIELD_VAR_TOT = _types.CONFIG_DB_IB_FIELD_VAR_TOT;

public static final String FIELD_APP = _types.CONFIG_DB_IB_FIELD_APP;
public static final String FIELD_CONN_ID = _types.CONFIG_DB_IB_FIELD_CONN_ID;
public static final String FIELD_CONN_TYPE = _types.CONFIG_DB_IB_FIELD_CONN_TYPE;
public static final String FIELD_CONN_IS_ON = _types.CONFIG_DB_IB_FIELD_CONN_IS_ON;
public static final String FIELD_ERROR = _types.CONFIG_DB_IB_FIELD_ERROR;
public static final String FIELD_ADDITIONAL = _types.CONFIG_DB_IB_FIELD_ADDITIONAL;
public static final String FIELD_TIME2 = _types.CONFIG_DB_IB_FIELD_TIME2;

public static final String FIELD_ID = _types.CONFIG_DB_IB_FIELD_ID;
public static final String FIELD_TYPE = _types.CONFIG_DB_IB_FIELD_TYPE;
public static final String FIELD_DATA_TYPE = _types.CONFIG_DB_IB_FIELD_DATA_TYPE;

public static final String FIELD_DATE = _types.CONFIG_DB_IB_FIELD_DATE;

public static final String FIELD_KEY = _types.CONFIG_DB_IB_FIELD_KEY;
public static final String FIELD_VALUE = _types.CONFIG_DB_IB_FIELD_VALUE;

permalink

DB

The main reason for the two quick modes is, evidently, efficiency. The whole evolution of this part provides a good example of how the main priorities are being assessed, of how inefficiency only became a concern after the higher priorities were fully addressed.

public static boolean select_one_boolean
(
    String type_, String source_, String col_, String where_cols_, String order_cols_
) 
{ 
    initialise();

    boolean output = db.WRONG_BOOLEAN;

    try
    {	
        Object temp = get_val
        (
            col_, select_one
            (
                type_, source_, new String[] { col_ }, where_cols_, order_cols_
            ), 
            data.BOOLEAN
        );

        if (temp != null) output = (boolean)temp;
    }
    catch (Exception e) 
    {
        output = db.WRONG_BOOLEAN;

        manage_error
        (
            type_, strings.to_string(source_), e, strings.to_string(col_), 
            null, null, strings.to_string(where_cols_), 
            strings.to_string(order_cols_), db.WRONG_INT, null
        ); 
    }

    return output; 
}

public static HashMap<String, String> select_one
(
    String type_, String source_, String[] cols_, String where_cols_, String order_cols_
) 
{ 
    initialise();

    HashMap<String, String> output = new HashMap<String, String>();

    try
    {	
        ArrayList<HashMap<String, String>> temp = select
        (
            type_, source_, cols_, where_cols_, 1, order_cols_
        );	

        if (arrays.is_ok(temp)) output = temp.get(0);
    }
    catch (Exception e) 
    {
        output = new HashMap<String, String>();

        manage_error
        (
            type_, strings.to_string(source_), e, null, null, 
            cols_, strings.to_string(where_cols_), 
            strings.to_string(order_cols_), db.WRONG_INT, null
        );
    }

    return output; 
}

permalink

ib.async_data[*]_quicker

The code in these classes is an excellent example of development focused on efficiency (for asynchronous scenarios). Almost anything in there has been done to achieve the best performance, at least up to the current point, assumed to be good enough. No need to go further after that, not at the moment and as a top priority.

static int _get_id(String symbol_, boolean lock_) 
{ 
    if (lock_) __lock();

    int id = WRONG_ID;

    id = get_id_i
    (
        symbol_, async_data_apps_quicker.get_symbols(), 
        async_data_apps_quicker.get_last_id(), 
        async_data_apps_quicker.get_max_id(), true
    ); 

    if (lock_) __unlock();

    return id;
}

static int get_id_i(String symbol_, String[] symbols_, int last_, int max_, boolean is_id_) 
{
    int min = 0;
    int wrong = WRONG_I;	

    if (is_id_)
    {
        min = MIN_ID;	
        wrong = WRONG_ID;	
    }

    if (!strings.is_ok(symbol_) || last_ <= wrong) return wrong;

    int id = last_;

    while (true)
    {	
        id--;
        if (id < min) id = max_; 

        boolean is_last = (id == last_);

        int i = get_i(id, is_id_);
        if (is_last || (symbols_[i] != null && symbols_[i].equals(symbol_))) 
        {
            if (is_last) id = wrong;

            break;
        }
    }

    return id;
}

permalink

Adaptability

After having adequately addressed the three previous priorities, it is time to worry about code adaptability and scalability, understood as ease to deal with conditions and scenarios which are unknown and uncertain at the moment.

  • Most of the conventions defining the words to be used are eminently meant to ease adaptability, to minimise the problems provoked by an uncertain future growth. All the rules about package/class structure, where descriptive and common words should be used, are, for example, being considered here. Also, including the name of the main methods in the intermediate ones is mostly expected to reduce the chances of mixing up potential new resources and existing ones.
private static String remove_escape_replace_many
(
    String[] needles_, String haystack_, String replacement_, String action_
)
{
    if (!is_ok(haystack_, true)) return DEFAULT;

    String output = haystack_;
    if (!arrays.is_ok(needles_)) return output;

    for (String needle: needles_) 
    { 
        if (!is_ok(needle, true)) continue;

        output = remove_escape_replace(needle, output, replacement_, action_);
    }

    return output;
}

permalink

  • The structure of names and values of the accessory._types constants is mainly motivated by a potential, uncertain future. It is an efficient way to store as many layers of information as required. The management of these constants might be not too friendly, although this is where the surprisingly simple but effective trick of local copies comes into picture. That is, using, for example, the friendlier db_ib.apps.SOURCE which points to the more adaptable accessory_ib._types.CONFIG_DB_IB_APPS_SOURCE.
private static _types _instance = new _types(); 

public _types() { }
public static void populate() { _instance.populate_internal(); }

public static final String SEPARATOR = misc.SEPARATOR_NAME;
public static final String CONFIG = "config";
public static final String CONFIG_BASIC = "config_basic";
public static final String CONFIG_BASIC_NAME = "config_basic_name";
public static final String CONFIG_BASIC_DIR = "config_basic_dir";
public static final String CONFIG_BASIC_DIR_APP = "config_basic_dir_app";
public static final String CONFIG_BASIC_DIR_INI = "config_basic_dir_ini";
public static final String CONFIG_BASIC_DIR_CREDENTIALS = "config_basic_dir_credentials";
public static final String CONFIG_BASIC_DIR_LOGS = "config_basic_dir_logs";
public static final String CONFIG_BASIC_DIR_LOGS_ERRORS = "config_basic_dir_logs_errors";
public static final String CONFIG_BASIC_DIR_LOGS_ACTIVITY = "config_basic_dir_logs_activity";
public static final String CONFIG_BASIC_DIR_CRYPTO = "config_basic_dir_crypto";
public static final String CONFIG_BASIC_DIR_SOUNDS = "config_basic_dir_sounds";
public static final String CONFIG_BASIC_DIR_BACKUPS = "config_basic_dir_backups";
public static final String CONFIG_BASIC_DIR_BACKUPS_DBS = "config_basic_dir_backups_dbs";
public static final String CONFIG_BASIC_DIR_BACKUPS_FILES = "config_basic_dir_backups_files";
public static final String CONFIG_BASIC_DIR_INFO = "config_basic_dir_info";

permalink

  • The main DB structure, in its three modes, is clearly DB-type independent, even though the whole development was performed for MySQL and there are no plans to include other types. In other words, the code of any part which isn't handled homogeneously by all the DB types is stored in the corresponding specific class (e.g., accessory.db_mysql) and called when that type is being used (e.g., a call to accessory.db.get_valid_type returns accessory._types.CONFIG_DB_SETUP_TYPE_MYSQL).
public static ArrayList<HashMap<String, String>> execute_query
(
    String type_, String source_, String query_, boolean return_data_, String[] cols_
) 
{ 
    ArrayList<HashMap<String, String>> output = null;

    if (type_.equals(db_quicker_mysql.TYPE)) 
    {
        output = db_quicker_mysql.execute_query(source_, query_, return_data_, cols_);
    }

    return output; 
}

permalink

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment