G2Engine Reference

Prepare Environment

In [1]:
import com.senzing.g2.engine.G2Engine;
import com.senzing.g2.engine.G2JNI;
import com.senzing.g2.engine.Result;

Helper class for Json Rendering

In [2]:
%%loadFromPOM
<dependency>
    <groupId>org.glassfish</groupId>
    <artifactId>javax.json</artifactId>
    <version>1.1.4</version>
</dependency>
In [3]:
import javax.json.*;
import static java.util.Collections.*;
import static javax.json.stream.JsonGenerator.PRETTY_PRINTING;
In [4]:
public class JsonUtil {
    private static final JsonWriterFactory PRETTY_FACTORY
        = Json.createWriterFactory(singletonMap(PRETTY_PRINTING, true));
        
    private static final JsonWriterFactory UGLY_FACTORY
        = Json.createWriterFactory(emptyMap());
    
    public static String toJsonText(JsonValue val) {
        return toJsonText(val, true);
    }

    public static String toJsonText(JsonValue val, boolean prettyPrint) {
        JsonWriterFactory factory = (prettyPrint) ? PRETTY_FACTORY : UGLY_FACTORY;
        StringWriter sw = new StringWriter();
        JsonWriter writer = factory.createWriter(sw);
        writer.write(val);
        sw.flush();
        return sw.toString();
    }
    
    public static JsonObject parseJsonObject(String jsonText) {
        if (jsonText == null) return null;
        StringReader sr = new StringReader(jsonText);
        JsonReader jsonReader = Json.createReader(sr);
        return jsonReader.readObject();
  }

    public static JsonArray parseJsonArray(String jsonText) {
        if (jsonText == null) return null;
        StringReader sr = new StringReader(jsonText);
        JsonReader jsonReader = Json.createReader(sr);
        return jsonReader.readArray();
  }
    
    
}
In [5]:
import java.util.UUID;
public static void RenderJSON(Object obj){
    String str = obj.toString();
    JsonObject json = JsonUtil.parseJsonObject(str);
    String Config = JsonUtil.toJsonText(json, false);
    UUID id = UUID.randomUUID();
    String uuid = id.toString();
    String div = "<div id=\""+ uuid +"\" style=\"height:100%; width:100%; background-color: LightCyan\"></div>";
    display(div, "text/html");
    String jav = "require([\"https://rawgit.com/caldwell/renderjson/master/renderjson.js\"], function() {document.getElementById(\'"+ uuid +"\').appendChild(renderjson("+json+"))});";
    display(jav, "application/javascript");
}

Initialize Senzing configuration

Using environment variables and default values, create senzingConfigJson. This value is used when instantiating Senzing objects.

In [6]:
// Get variables used in constructing Senzing Engine configuration.

String configPath = System.getenv("SENZING_ETC_DIR");
if (configPath == null) {
    configPath = "/etc/opt/senzing";
}

String supportPath = System.getenv("SENZING_DATA_VERSION_DIR");
if (supportPath == null) {
    supportPath = "/opt/senzing/data";
}

String g2Path = System.getenv("SENZING_G2_DIR");
if (g2Path == null) {
    g2Path = "/opt/senzing/g2";
}

String resourcePath = g2Path + "/resources";

String sqlConnection = System.getenv("SENZING_SQL_CONNECTION");
if (sqlConnection == null) {
    sqlConnection = "sqlite3://na:na@/var/opt/senzing/sqlite/G2C.db";
}

// Construct the JSON string used for Senzing Engine configuration.

String senzingConfigJson = "{"
   + "\"PIPELINE\": {"
   +     "\"CONFIGPATH\": \"" + configPath + "\","
   +     "\"SUPPORTPATH\": \"" + supportPath + "\","
   +     "\"RESOURCEPATH\": \"" + resourcePath + "\""
   + "},"
   + "\"SQL\": {"
   +     "\"CONNECTION\": \"" + sqlConnection + "\""
   + "}}";

RenderJSON(senzingConfigJson);

G2Engine

To start using Senzing G2Engine, create and initialize an instance. This should be done once per process. The initV2() method accepts the following parameters:

  • module_name: A short name given to this instance of the G2Engine object.
  • senzing_config_json: A JSON string containing configuration parameters.
  • verbose_logging: A boolean which enables diagnostic logging.
  • config_id: (optional) The identifier value for the engine configuration can be returned here.

Calling this function will return "0" upon success.

In [7]:
G2Engine g2engine = new G2JNI();

String moduleName = "ExampleG2Engine";
boolean verboseLogging = true;
int return_code = g2engine.initV2(moduleName, senzingConfigJson, verboseLogging);
System.out.print(return_code);
0

initWithConfigIDV2

Alternatively initWithConfigIDV2() can be used to specify a configuration.

reinitV2

The reinitV2() function may be used to reinitialize the engine with a configuration

primeEngine

The primeEngine() method may optionally be called to pre-initialize some of the heavier weight internal resources of the G2 engine.

In [8]:
int return_code= g2engine.primeEngine();
if(return_code!=0)
    System.out.print(g2engine.getLastException());
System.out.print(return_code)
0

getActiveConfigID

Call getActiveConfigID() to return an identifier for the loaded Senzing engine configuration. The call will assign a long integer to a user-designated variable -- the function itself will return "0" upon success. The getActiveConfigID() method accepts one parameter as input:

  • configuration_id: The identifier value for the engine configuration. The result of function call is returned here
In [9]:
Result<Long> configID = new Result<Long>();
g2engine.getActiveConfigID(configID);

System.out.print(configID.getValue());
1777871070

exportConfig

Call exportConfig() to retrieve your Senzing engine's configuration. The call will assign a JSON document to a user-designated buffer, containing all relevant configuration information -- the function itself will return "0" upon success. The exportConfig function accepts the following parameters as input:

  • response_bytearray: The memory buffer to retrieve the JSON configuration document
  • config_id_bytearray: The identifier value for the engine configuration can be returned here.
In [10]:
StringBuffer response = new StringBuffer();

g2engine.exportConfig(response, configID);
System.out.print("Config ID: "+configID.getValue());
Config ID: 1777871070

stats

Call stats() to retrieve workload statistics for the current process. These statistics will automatically reset after retrieval.

  • response: A memory buffer for returning the response document. If an error occurred, an error response is stored here.
In [11]:
String response = g2engine.stats();
RenderJSON(response);

getRepositoryLastModifiedTime

Call getRepositoryLastModifiedTime() to obtain the last modified time of the Senzing repository, measured in the number of seconds between the last modified time and January 1, 1970 12:00am GMT (epoch time). The call will assign a long integer to a user-designated buffer -- the function itself will return "0" upon success. The getRepositoryLastModifiedTime() method accepts one parameter as input:

  • lastModifiedTime: The last modified time. The result of function call is returned here
In [12]:
Result<Long> lastModifiedTime = new Result<Long>();

g2engine.getRepositoryLastModifiedTime(lastModifiedTime);

System.out.print(lastModifiedTime.getValue());
1594078674465

Insert

addRecord

Once the Senzing engine is initialized, use addRecord() to load a record into the Senzing repository -- addRecord() can be called as many times as desired and from multiple threads at the same time. The addRecord() function returns "0" upon success, and accepts four parameters as input:

  • datasource_code: The name of the data source the record is associated with. This value is configurable to the system
  • record_id: The record ID, used to identify distinct records
  • data_string: A JSON document with the attribute data for the record
  • load_id: The observation load ID for the record; value can be null and will default to data_source
In [13]:
String dataSourceCode = "TEST";
String recordID = "1";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JANE\", \"NAME_LAST\": \"SMITH\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"653 STATE ROUTE 7\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\"}";
String loadID = null;

/** addRecord */
int return_code= g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);

/** addRecordWithReturnedRecordID */
StringBuffer recordID2 = new StringBuffer();
int ret2 = g2engine.addRecordWithReturnedRecordID(dataSourceCode, recordID2, jsonData, loadID);

System.out.print("New Record ID: " + recordID2.toString());

/** addRecordWithInfo */
StringBuffer responseBuffer = new StringBuffer();
int flags = 0;
int ret3 = g2engine.addRecordWithInfo(dataSourceCode, recordID, jsonData, loadID, flags, responseBuffer);

RenderJSON(responseBuffer.toString());
New Record ID: E8DCD9E0D0494ADCA1A83C035F85ECFA62CE4DB4

getRecordV2

Use getRecord() to retrieve a single record from the data repository; the record is assigned in JSON form to a user-designated buffer, and the function itself returns "0" upon success. Once the Senzing engine is initialized, getRecord() can be called as many times as desired and from multiple threads at the same time. The getRecord() function accepts the following parameters as input:

  • data_source: The name of the data source the record is associated with. This value is configurable to the system
  • record_id: The record ID, used to identify the record for retrieval
  • response: A memory buffer for returning the response document; if an error occurred, an error response is stored here
In [14]:
int flags = 0;
String dataSourceCode = "TEST";
String recordID = "1";
StringBuffer response = new StringBuffer();
int return_code= g2engine.getRecordV2(dataSourceCode, recordID, flags, response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response)

getEntityByRecordIDV2

Use getEntityByRecordIDV2() to retrieve entity data based on the record ID of a particular data record. This function accepts the following parameters as input:

  • datasource_code: The name of the data source the record is associated with. This value is configurable to the system
  • record_id: The record ID for a particular data record
  • g2_engine_flags: Control flags for specifying what data about the entity to retrieve.
  • response: A memory buffer for returning the response document; if an error occurred, an error response is stored here.
In [15]:
String dataSourceCode = "TEST";
String recordID = "1";

StringBuffer response = new StringBuffer();

int return_code = g2engine.getEntityByRecordIDV2(dataSourceCode, recordID, flags, response);
int start = response.indexOf("\"ENTITY_ID\":")+12;
int end = response.indexOf(",");
long entityID=Integer.parseInt(response.substring(start, end));
if(return_code!=0)
    RenderJSON(g2engine.getLastException());
else
    RenderJSON(response);

getEntityByEntityIDV2

Entity searching is a key component for interactive use of Entity Resolution intelligence. The core Senzing engine provides real-time search capabilities that are easily accessed via the Senzing API. Senzing offers methods for entity searching, all of which can be called as many times as desired and from multiple threads at the same time (and all of which return "0" upon success).

Use getEntityByEntityIDV2() to retrieve entity data based on the ID of a resolved identity. This function accepts the following parameters as input:

  • entity_id: The numeric ID of a resolved entity
  • g2_engine_flags: Control flags for specifying what data about the entity to retrieve.
  • response_bytearray: A memory buffer for returning the response document; if an error occurred, an error response is stored here.
In [16]:
response = new StringBuffer();

int return_code = g2engine.getEntityByEntityIDV2(entityID, flags, response);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);

searchByAttributes

Use searchByAttributesV2() to retrieve entity data based on a user-specified set of entity attributes. This function accepts the following parameters as input:

  • data_string: A JSON document with the attribute data to search for
  • response: A memory buffer for returning the response document; if an error occurred, an error response is stored here.
In [17]:
String dataSourceCode = "TEST";
String recordID = "1";
int flags = 0;
response = new StringBuffer();

int return_code = g2engine.searchByAttributesV2(jsonData, flags, response);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);

Finding Paths

The FindPathByEntityID() and FindPathByRecordID() functions can be used to find single relationship paths between two entities. Paths are found using known relationships with other entities.

Entities can be searched for by either Entity ID or by Record ID, depending on which function is chosen.

These functions have the following parameters:

  • entityID1: The entity ID for the starting entity of the search path
  • entityID2: The entity ID for the ending entity of the search path
  • dataSourceCode1: The data source for the starting entity of the search path
  • recordID1: The record ID for the starting entity of the search path
  • dataSourceCode2: The data source for the ending entity of the search path
  • recordID2: The record ID for the ending entity of the search path
  • maxDegree: The number of relationship degrees to search

First you will need to create some records so that you have some that you can compare. Can you see what is the same between this record and the previous one?

In [18]:
String dataSourceCode = "TEST";
String recordID = "2";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JOHN\", \"NAME_LAST\": \"SMITH\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"753 STATE ROUTE 8\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\"}";
String loadID = null;

int return_code = g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);

if(return_code!=0)
    System.out.print(g2engine.getLastException());

StringBuffer recordID2 = new StringBuffer();
int return_code = g2engine.addRecordWithReturnedRecordID(dataSourceCode, recordID2, jsonData, loadID);

System.out.print("New Record ID: " + recordID2.toString());
if(return_code!=0)
    System.out.print(g2engine.getLastException());
New Record ID: C9C736537C22CB7B0EBCC834CB424308EC4A0B7F
In [19]:
String dataSourceCode = "TEST";
String recordID = "2";
StringBuffer response = new StringBuffer();
g2engine.getEntityByRecordIDV2(dataSourceCode, recordID, flags, response);
JsonObject jsonObject = JsonUtil.parseJsonObject(response.toString());
long entityID = jsonObject.getJsonObject("RESOLVED_ENTITY").getJsonNumber("ENTITY_ID").longValue();

FindPathByEntityIDV2

In [20]:
long entityID1 = entityID;
int maxDegree = 3;

StringBuffer response = new StringBuffer();

int ret_code = g2engine.findPathByEntityID(entityID1,entityID2,maxDegree,response);

if(ret_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
|   int ret_code = g2engine.findPathByEntityID(entityID1,entityID2,maxDegree,response);
cannot find symbol
  symbol:   variable entityID2

FindPathByRecordID()

In [21]:
String dataSourceCode1 = new String("TEST");
String recordID1 = new String("1");
String dataSourceCode2 = new String("TEST");
String recordID2 = new String("2");

ret_code = g2engine.findPathByRecordID(dataSourceCode1,recordID1,dataSourceCode2,recordID2,maxDegree,response);

if(ret_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
|   ret_code = g2engine.findPathByRecordID(dataSourceCode1,recordID1,dataSourceCode2,recordID2,maxDegree,response);
cannot find symbol
  symbol:   variable ret_code

Finding Paths with Exclusions

The FindPathExcludingByEntityID() and FindPathExcludingByRecordID() functions can be used to find single relationship paths between two entities. Paths are found using known relationships with other entities. In addition, it will find paths that exclude certain entities from being on the path.

Entities can be searched for by either Entity ID or by Record ID, depending on which function is chosen. Additionally, entities to be excluded can also be specified by either Entity ID or by Record ID.

When excluding entities, the user may choose to either (a) strictly exclude the entities, or (b) prefer to exclude the entities, but still include them if no other path is found. By default, entities will be strictly excluded. A "preferred exclude" may be done by specifying the G2_FIND_PATH_PREFER_EXCLUDE control flag.

These functions have the following parameters:

  • entityID1: The entity ID for the starting entity of the search path
  • entityID2: The entity ID for the ending entity of the search path
  • dataSourceCode1: The data source for the starting entity of the search path
  • recordID1: The record ID for the starting entity of the search path
  • dataSourceCode2: The data source for the ending entity of the search path
  • recordID2: The record ID for the ending entity of the search path
  • maxDegree: The number of relationship degrees to search
  • excludedEntities: Entities that should be avoided on the path (JSON document)
  • flags: Operational flags
In [22]:
String dataSourceCode = "TEST";
String recordID = "3";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"SAM\", \"NAME_LAST\": \"MILLER\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"753 STATE ROUTE 8\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;

int return_code= g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);

if(return_code!=0)
    System.out.print(g2engine.getLastException());

StringBuffer recordID2 = new StringBuffer();
int return_code2 = g2engine.addRecordWithReturnedRecordID(dataSourceCode, recordID2, jsonData, loadID);

System.out.print("New Record ID: " + recordID2.toString());

if(return_code2!=0)
    System.out.print(g2engine.getLastException());
New Record ID: 253356D57BAE6A15EB5EE0D7976D50920F12C732

FindPathExcludingByEntityID()

In [23]:
int maxDegree = 4;
String excludedEntities = new String("{\"ENTITIES\":[{\"ENTITY_ID\":\"1\"}]}");
int flags = G2Engine.G2_EXPORT_DEFAULT_FLAGS;

StringBuffer response = new StringBuffer();

int ret_code = g2engine.findPathExcludingByEntityID(entityID1,entityID2,maxDegree,excludedEntities,flags,response);

if(ret_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
|   int ret_code = g2engine.findPathExcludingByEntityID(entityID1,entityID2,maxDegree,excludedEntities,flags,response);
cannot find symbol
  symbol:   variable entityID2

FindPathExcludingByRecordID()

In [24]:
String dataSourceCode1 = new String("TEST");
String recordID1 = new String("2");
String dataSourceCode2 = new String("TEST");
String recordID2 = new String("3");
String excludedRecords = new String("{\"RECORDS\":[{\"RECORD_ID\":\"1\",\"DATA_SOURCE\":\"TEST\"}]}");

return_code = g2engine.findPathExcludingByRecordID(dataSourceCode1,recordID1,dataSourceCode2,recordID2,maxDegree,excludedRecords,flags,response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);

Finding Paths with Required Sources

The FindPathIncludingSourceByEntityID() and FindPathIncludingSourceByRecordID() functions can be used to find single relationship paths between two entities. In addition, one of the enties along the path must include a specified data source.

Entities can be searched for by either Entity ID or by Record ID, depending on which function is chosen. The required data source or sources are specified by a json document list.

Specific entities may also be excluded, using the same methodology as the FindPathExcludingByEntityID() and FindPathExcludingByRecordID() functions use.

These functions have the following parameters:

  • entityID1: The entity ID for the starting entity of the search path
  • entityID2: The entity ID for the ending entity of the search path
  • dataSourceCode1: The data source for the starting entity of the search path
  • recordID1: The record ID for the starting entity of the search path
  • dataSourceCode2: The data source for the ending entity of the search path
  • recordID2: The record ID for the ending entity of the search path
  • maxDegree: The number of relationship degrees to search
  • excludedEntities: Entities that should be avoided on the path (JSON document)
  • requiredDsrcs: Entities that should be avoided on the path (JSON document)
  • flags: Operational flags
In [25]:
int maxDegree = 4;
String excludedEntities = new String("{\"ENTITIES\":[{\"ENTITY_ID\":\"1\"}]}");
String requiredDsrcs = new String("{\"DATA_SOURCES\":[\"TEST\"]}");
int flags = 0;

StringBuffer response = new StringBuffer();

int ret_code = g2engine.findPathIncludingSourceByEntityID(entityID1,entityID2,maxDegree,excludedEntities,requiredDsrcs,flags,response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
String dataSourceCode1 = new String("TEST");
String recordID1 = new String("2");
String dataSourceCode2 = new String("TEST");
String recordID2 = new String("3");
String excludedRecords = new String("{\"RECORDS\":[{\"RECORD_ID\":\"1\",\"DATA_SOURCE\":\"TEST\"}]}");

return_code = g2engine.findPathIncludingSourceByRecordID(dataSourceCode1,recordID1,dataSourceCode2,recordID2,maxDegree,excludedRecords,requiredDsrcs,flags,response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
|   int ret_code = g2engine.findPathIncludingSourceByEntityID(entityID1,entityID2,maxDegree,excludedEntities,requiredDsrcs,flags,response);
cannot find symbol
  symbol:   variable entityID2

Finding Networks

The findNetworkByEntityID() and findNetworkByRecordID() functions can be used to find all entities surrounding a requested set of entities. This includes the requested entities, paths between them, and relations to other nearby entities.

Entities can be searched for by either Entity ID or by Record ID, depending on which function is chosen.

These functions have the following parameters:

  • entity_list: A list of entities, specified by Entity ID (JSON document)
  • record_list: A list of entities, specified by Record ID (JSON document)
  • max_degree: The maximum number of degrees in paths between search entities
  • buildout_degree: The number of degrees of relationships to show around each search entity
  • max_entities: The maximum number of entities to return in the discovered network

They also have various arguments used to return response documents

The functions return a JSON document that identifies the path between the each set of search entities (if the path exists), and the information on the entities in question (search entities, path entities, and build-out entities.

findNetworkByEntityID

In [26]:
StringBuffer response = new StringBuffer();
int max_degree = 2;
int buildout_degree = 1;
int max_entities = 12;
String entity_list = "{\"ENTITIES\": [{\"ENTITY_ID\":"+entityID+"}, {\"ENTITY_ID\":"+entityID2+"}]}";

int return_code = g2engine.findNetworkByEntityIDV2(entity_list, max_degree, buildout_degree, max_entities, flags, response);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
|   String entity_list = "{\"ENTITIES\": [{\"ENTITY_ID\":"+entityID+"}, {\"ENTITY_ID\":"+entityID2+"}]}";
cannot find symbol
  symbol:   variable entityID2

findNetworkByRecordIDV2

In [27]:
StringBuffer response = new StringBuffer();
int max_degree = 2;
int buildout_degree = 1;
int max_entities = 12;
String record_list = "{\"RECORDS\": [{\"RECORD_ID\": \""+recordID1+"\", \"DATA_SOURCE\": \""+dataSourceCode1+"\" }, {\"RECORD_ID\": \""+recordID2+"\", \"DATA_SOURCE\": \""+dataSourceCode2+"\" }]}";

int return_code = g2engine.findNetworkByRecordIDV2(record_list, max_degree, buildout_degree, max_entities, flags, response);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);

Connection details

Why records belong to an entity

The WhyEntityByEntityID() and WhyEntityByRecordID() functions can be used to determine why records belong to their resolved entities. These functions will compare the record data within an entity against the rest of the entity data, and show why they are connected. This is calculated based on the features that record data represents.

Records can be chosen by either Record ID or by Entity ID, depending on which function is chosen. If a single record ID is used, then comparison results for that single record will be generated, as part of its entity. If an Entity ID is used, then comparison results will be generated for every record within that entity.

These functions have the following parameters:

  • entityID: The entity ID for the entity to be analyzed
  • dataSourceCode: The data source for the record to be analyzed
  • recordID: The record ID for the record to be analyzed
  • flags: Control flags for outputting entities They also have various arguments used to return response documents.

The functions return a JSON document that gives the results of the record analysis. The document contains a section called "WHY_RESULTS", which shows how specific records relate to the rest of the entity. It has a "WHY_KEY", which is similar to a match key, in defining the relevant connected data. It shows candidate keys for features that initially cause the records to be analyzed for a relationship, plus a series of feature scores that show how similar the feature data was.

The response document also contains a separate ENTITIES section, with the full information about the resolved entity. (Note: When working with this entity data, Senzing recommends using the flags G2_ENTITY_OPTION_INCLUDE_INTERNAL_FEATURES and G2_ENTITY_OPTION_INCLUDE_FEATURE_STATS. This will provide detailed feature data that is not included by default, but is useful for understanding the WHY_RESULTS data.)

In [28]:
/** define input variables */
long entityID = 1;

/** buffer response variables */
StringBuffer response = new StringBuffer();

/** find the why-information */
int return_code = g2engine.whyEntityByEntityIDV2(entityID,flags,response);

/** print the results */
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
In [29]:
/** define input variables */
String dataSourceCode = new String("PEOPLE");
String recordID = new String("1001");

/** find the why-information */
ret_code = g2engine.whyEntityByRecordIDV2(dataSourceCode,recordID,flags,response);

/** print the results */
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);
|   ret_code = g2engine.whyEntityByRecordIDV2(dataSourceCode,recordID,flags,response);
cannot find symbol
  symbol:   variable ret_code

Replace

Replace the record

Use the replaceRecord() function to update or replace a record in the data repository (if record doesn't exist, a new record is added to the data repository. Like the above functions, replaceRecord() returns "0" upon success, and it can be called as many times as desired and from multiple threads at the same time. The replaceRecord() function accepts four parameters as input:

  • dataSourceCode: The name of the data source the record is associated with. This value is configurable to the system
  • recordID: The record ID, used to identify distinct records
  • jsonData: A JSON document with the attribute data for the record
  • loadID: The observation load ID for the record; value can be null and will default to dataSourceCode
In [30]:
String dataSourceCode = "TEST";
String recordID = "1";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JANE\", \"NAME_LAST\": \"ADAMS\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"653 STATE ROUTE 7\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\"}";
String loadID = null;

int return_code= g2engine.replaceRecord(dataSourceCode, recordID, jsonData, loadID);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    System.out.print(return_code)
0

Do getRecord() again to see the changes

In [31]:
String dataSourceCode = "TEST";
String recordID = "1";
StringBuffer response = new StringBuffer();

g2engine.getRecord(dataSourceCode, recordID, response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);

replaceRecordWithInfo

In [32]:
String dataSourceCode = "TEST";
String recordID = "1";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JANE\", \"NAME_LAST\": \"ADAMS\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"653 STATE ROUTE 7\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\"}";
String loadID = null;
StringBuffer response = new StringBuffer();

int return_code= g2engine.replaceRecordWithInfo(dataSourceCode, recordID, jsonData, loadID, flags, response);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response)

Re-evaluate

reevaluateRecord

In [33]:
int return_code = g2engine.reevaluateRecord(dataSourceCode1, recordID1, flags);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    System.out.print(return_code);
0

ReevaluateRecordWithInfo

In [34]:
StringBuffer response = new StringBuffer();

int return_code = g2engine.reevaluateRecordWithInfo(dataSourceCode1, recordID1, flags, response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);

ReevaluateEntity

Find an entity

In [35]:
String dataSourceCode = "TEST";
String recordID = "2";
StringBuffer response = new StringBuffer();
g2engine.getEntityByRecordIDV2(dataSourceCode, recordID, flags, response);
JsonObject jsonObject = JsonUtil.parseJsonObject(response.toString());
long entityID = jsonObject.getJsonObject("RESOLVED_ENTITY").getJsonNumber("ENTITY_ID").longValue();

Re-evaluate the entity.

In [36]:
int return_code = g2engine.reevaluateEntity(entityID, flags);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    System.out.print(return_code);
0

reevaluateEntityWithInfo

In [37]:
StringBuffer response = new StringBuffer();

return_code = g2engine.reevaluateEntityWithInfo(entityID, flags, response);

if(return_code!=0)
    System.out.print(return_code);
else
    RenderJSON(response);

Reporting

Export JSON Entity Report

There are three steps to exporting resolved entity data from the G2Engine object in JSON format. First, use the exportJSONEntityReport() method to generate a long integer, referred to here as an 'exportHandle'. The exportJSONEntityReport() method accepts one parameter as input:

  • flags: An integer specifying which entity details should be included in the export. See the "Entity Export Flags" section for further details.

Second, use the fetchNext() method to read the exportHandle and export a row of JSON output containing the entity data for a single entity. Note that successive calls of fetchNext() will export successive rows of entity data. The fetchNext() method accepts the following parameters as input:

  • exportHandle: A long integer from which resolved entity data may be read and exported
  • response: A memory buffer for returning the response document; if an error occurred, an error response is stored here.
In [38]:
int flags = g2engine.G2_EXPORT_INCLUDE_ALL_ENTITIES;

long exportHandle = g2engine.exportJSONEntityReport(flags);

String response = g2engine.fetchNext(exportHandle);
RenderJSON(response);

g2engine.closeExport(exportHandle);

Export CSV Entity Report

There are three steps to exporting resolved entity data from the G2Engine object in CSV format. First, use the exportCSVEntityReportV2() method to generate a long integer, referred to here as an 'exportHandle'. The exportCSVEntityReportV2() method accepts one parameter as input:

  • csvColumnList: A String containing a comma-separated list of column names for the CSV export.
  • flags: An integer specifying which entity details should be included in the export. See the "Entity Export Flags" section in the link below for further details.

Second, use the fetchNext() method to read the exportHandle and export a row of CSV output containing the entity data for a single entity. Note that the first call of fetchNext() may yield a header row, and that successive calls of fetchNext() will export successive rows of entity data. The fetchNext() method accepts the following parameters as input:

  • exportHandle: A long integer from which resolved entity data may be read and exported
  • response: A memory buffer for returning the response document; if an error occurred, an error response is stored here
In [39]:
int flags = g2engine.G2_EXPORT_INCLUDE_ALL_ENTITIES;
String headers = "RESOLVED_ENTITY_ID,RESOLVED_ENTITY_NAME,RELATED_ENTITY_ID,MATCH_LEVEL,MATCH_KEY,IS_DISCLOSED,IS_AMBIGUOUS,DATA_SOURCE,RECORD_ID,JSON_DATA,LAST_SEEN_DT,NAME_DATA,ATTRIBUTE_DATA,IDENTIFIER_DATA,ADDRESS_DATA,PHONE_DATA,RELATIONSHIP_DATA,ENTITY_DATA,OTHER_DATA";

long exportHandle = g2engine.exportCSVEntityReportV2(headers, flags);

String response = g2engine.fetchNext(exportHandle);
while (response != null)
{
    System.out.print(response);
    response = g2engine.fetchNext(exportHandle);
}

g2engine.closeExport(exportHandle);
RESOLVED_ENTITY_ID,RESOLVED_ENTITY_NAME,RELATED_ENTITY_ID,MATCH_LEVEL,MATCH_KEY,IS_DISCLOSED,IS_AMBIGUOUS,DATA_SOURCE,RECORD_ID,JSON_DATA,LAST_SEEN_DT,NAME_DATA,ATTRIBUTE_DATA,IDENTIFIER_DATA,ADDRESS_DATA,PHONE_DATA,RELATIONSHIP_DATA,ENTITY_DATA,OTHER_DATA
1,"JANE SMITH",0,0,"",0,0,"TEST","E8DCD9E0D0494ADCA1A83C035F85ECFA62CE4DB4","{""NAME_TYPE"":""PRIMARY"",""NAME_FIRST"":""JANE"",""NAME_LAST"":""SMITH"",""ADDR_TYPE"":""HOME"",""ADDR_LINE1"":""653 STATE ROUTE 7"",""ADDR_CITY"":""FRESNO"",""ADDR_STATE"":""CA"",""ADDR_POSTAL_CODE"":""55073-1234"",""DATA_SOURCE"":""TEST"",""ENTITY_TYPE"":""TEST"",""DSRC_ACTION"":""A"",""LENS_CODE"":""DEFAULT""}","2020-07-08 23:34:12.964","PRIMARY: SMITH JANE","","","HOME: 653 STATE ROUTE 7 FRESNO CA 55073-1234","","","",""
2,"JOHN SMITH",0,0,"",0,0,"TEST","2","{""NAME_TYPE"":""PRIMARY"",""NAME_FIRST"":""JOHN"",""NAME_LAST"":""SMITH"",""ADDR_TYPE"":""HOME"",""ADDR_LINE1"":""753 STATE ROUTE 8"",""ADDR_CITY"":""FRESNO"",""ADDR_STATE"":""CA"",""ADDR_POSTAL_CODE"":""55073-1234"",""DATA_SOURCE"":""TEST"",""ENTITY_TYPE"":""TEST"",""DSRC_ACTION"":""A"",""LENS_CODE"":""DEFAULT"",""RECORD_ID"":""2""}","2020-07-08 23:34:13.523","PRIMARY: SMITH JOHN","","","HOME: 753 STATE ROUTE 8 FRESNO CA 55073-1234","","","",""
2,"JOHN SMITH",0,0,"+EXACTLY_SAME",0,0,"TEST","C9C736537C22CB7B0EBCC834CB424308EC4A0B7F","{""NAME_TYPE"":""PRIMARY"",""NAME_FIRST"":""JOHN"",""NAME_LAST"":""SMITH"",""ADDR_TYPE"":""HOME"",""ADDR_LINE1"":""753 STATE ROUTE 8"",""ADDR_CITY"":""FRESNO"",""ADDR_STATE"":""CA"",""ADDR_POSTAL_CODE"":""55073-1234"",""DATA_SOURCE"":""TEST"",""ENTITY_TYPE"":""TEST"",""DSRC_ACTION"":""A"",""LENS_CODE"":""DEFAULT""}","2020-07-08 23:34:13.556","PRIMARY: SMITH JOHN","","","HOME: 753 STATE ROUTE 8 FRESNO CA 55073-1234","","","",""
3,"SAM MILLER",0,0,"",0,0,"TEST","253356D57BAE6A15EB5EE0D7976D50920F12C732","{""NAME_TYPE"":""PRIMARY"",""NAME_FIRST"":""SAM"",""NAME_LAST"":""MILLER"",""ADDR_TYPE"":""HOME"",""ADDR_LINE1"":""753 STATE ROUTE 8"",""ADDR_CITY"":""FRESNO"",""ADDR_STATE"":""CA"",""ADDR_POSTAL_CODE"":""55073-1234"",""SSN_NUMBER"":""111-11-1111"",""DATA_SOURCE"":""TEST"",""ENTITY_TYPE"":""TEST"",""DSRC_ACTION"":""A"",""LENS_CODE"":""DEFAULT""}","2020-07-08 23:34:13.971","PRIMARY: MILLER SAM","","SSN: 111-11-1111","HOME: 753 STATE ROUTE 8 FRESNO CA 55073-1234","","","",""
3,"SAM MILLER",0,0,"+EXACTLY_SAME",0,0,"TEST","3","{""NAME_TYPE"":""PRIMARY"",""NAME_FIRST"":""SAM"",""NAME_LAST"":""MILLER"",""ADDR_TYPE"":""HOME"",""ADDR_LINE1"":""753 STATE ROUTE 8"",""ADDR_CITY"":""FRESNO"",""ADDR_STATE"":""CA"",""ADDR_POSTAL_CODE"":""55073-1234"",""SSN_NUMBER"":""111-11-1111"",""DATA_SOURCE"":""TEST"",""ENTITY_TYPE"":""TEST"",""DSRC_ACTION"":""A"",""LENS_CODE"":""DEFAULT"",""RECORD_ID"":""3""}","2020-07-08 23:34:13.926","PRIMARY: MILLER SAM","","SSN: 111-11-1111","HOME: 753 STATE ROUTE 8 FRESNO CA 55073-1234","","","",""
4,"JANE ADAMS",0,0,"",0,0,"TEST","1","{""NAME_TYPE"":""PRIMARY"",""NAME_FIRST"":""JANE"",""NAME_LAST"":""ADAMS"",""ADDR_TYPE"":""HOME"",""ADDR_LINE1"":""653 STATE ROUTE 7"",""ADDR_CITY"":""FRESNO"",""ADDR_STATE"":""CA"",""ADDR_POSTAL_CODE"":""55073-1234"",""DATA_SOURCE"":""TEST"",""ENTITY_TYPE"":""TEST"",""DSRC_ACTION"":""A"",""LENS_CODE"":""DEFAULT"",""RECORD_ID"":""1""}","2020-07-08 23:34:14.911","PRIMARY: ADAMS JANE","","","HOME: 653 STATE ROUTE 7 FRESNO CA 55073-1234","","","",""

Redo Processing

Redo records are automatically created by Senzing when certain conditions occur where it believes more processing may be needed. Some examples:

  • A value becomes generic and previous decisions may need to be revisited
  • Clean up after some record deletes
  • Detected related entities were being changed at the same time
  • A table inconsistency exists, potentially after a non-graceful shutdown First we will need to have a total of 6 data sources so let's add 4 more
In [40]:
String dataSourceCode = "TEST";
String recordID = "4";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JANE\", \"NAME_LAST\": \"ADAMS\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int return_code= g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
System.out.print(return_code);

String dataSourceCode = "TEST";
String recordID = "5";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"LILY\", \"NAME_LAST\": \"OWENS\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int return_code= g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
System.out.print(return_code);

String dataSourceCode = "TEST";
String recordID = "6";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"AUGUST\", \"NAME_LAST\": \"Bauler\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int return_code= g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
System.out.print(return_code);

String dataSourceCode = "TEST";
String recordID = "7";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JACK\", \"NAME_LAST\": \"MILLER\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int return_code= g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
System.out.print(return_code);

String dataSourceCode = "TEST";
String recordID = "8";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"LOGAN\", \"NAME_LAST\": \"WILLIAMS\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int return_code= g2engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(return_code!=0)
    System.out.print(g2engine.getLastException());
System.out.print(return_code);
00000

Counting the number of redos

This returns the number of redos within the processed records that are awaiting processing.

In [41]:
long response = g2engine.countRedoRecords();
if(response<0)
    System.out.print(g2engine.getLastException());

Geting a redo record

Gets a redo record so that it can be processed

In [42]:
StringBuffer response = new StringBuffer();
g2engine.getRedoRecord(response);
System.out.print(response);
<UMF_DOC><OBS><OBS_ENT><ENT_SRC_KEY>B65FE9D17056FD897658BA6FCCCE0255CE072AFE</ENT_SRC_KEY><ENT_SRC_DESC>AUGUST Bauler</ENT_SRC_DESC><RECORD_ID>6</RECORD_ID><ETYPE_CODE>TEST</ETYPE_CODE></OBS_ENT><LENS_LIST><LENS_CODE>DEFAULT</LENS_CODE></LENS_LIST><DSRC_CODE>TEST</DSRC_CODE><DSRC_ACTION>X</DSRC_ACTION><OBS_SRC_KEY>REDO_QUEUE</OBS_SRC_KEY></OBS><REDO_EVALUATION_FOR_GENERIC><FEATURE_ID>12</FEATURE_ID><LENS_ID>1</LENS_ID><ECLASS_ID>1</ECLASS_ID></REDO_EVALUATION_FOR_GENERIC></UMF_DOC>
In [43]:
StringBuffer response_string = new StringBuffer();
int response = g2engine.getRedoRecord(response_string);
if(response==0 && response_string.length()>0)
    response = g2engine.process(response_string.toString());
if(response!=0)
    System.out.print(g2engine.getLastException());
System.out.print(response);
0

Delete

Deleting Records

use deleteRecord() to remove a record from the data repository (returns "0" upon success) ; deleteRecord() can be called as many times as desired and from multiple threads at the same time. The deleteRecord() function accepts three parameters as input:

  • dataSourceCode: The name of the data source the record is associated with. This value is configurable to the system
  • recordID: The record ID, used to identify distinct records
  • loadID: The observation load ID for the record; value can be null and will default to dataSourceCode
In [44]:
String dataSourceCode = "TEST";
String recordID = "1";
String loadID = null;

/** deleteRecord */
int return_code= g2engine.deleteRecord(dataSourceCode, recordID, loadID);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    System.out.print(return_code);
0

deleteRecordWithInfo

In [45]:
String recordID = "2";
String dataSourceCode = "TEST";
/** deleteRecordWithInfo */
StringBuffer response = new StringBuffer();
int flags = 0;
int return_code = g2engine.deleteRecordWithInfo(dataSourceCode, recordID, loadID, flags, response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    RenderJSON(response);

Attempt to get the record again. It should error and give an output similar to "Unknown record" and a return value of -2.

In [46]:
StringBuffer response = new StringBuffer();
int return_code = g2engine.getRecord(dataSourceCode, recordID, response);

if(return_code!=0)
    System.out.print(g2engine.getLastException());
0033E|Unknown record: dsrc[TEST], record[2]

Cleanup

Purge Repository

To purge the G2 repository, use the aptly named purgeRepository() method. This will remove every record in your current repository.

In [47]:
int return_code= g2engine.purgeRepository();
if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    System.out.print(return_code)
0

destroy

Once all searching is done in a process call destroy() to uninitialize Senzing and clean up resources. You should always do this once at the end of each process.

In [48]:
int return_code = g2engine.destroy();

if(return_code!=0)
    System.out.print(g2engine.getLastException());
else
    System.out.print(return_code);
0