G2Engine Redo Processing
Version 3.x
Version 2.x
Redo Processing
Redo records are automatically created when certain conditions occur during entity resolution where additional processing is required.
The most common reasons are:
- A feature value becomes generic and previous decisions need to be revisited
- Clean up after some record deletes
- Detected related entities were being changed at the same time
- A table inconsistency exists, potentially after a non-graceful shutdown
countRedoRecords
countRedoRecords() returns the integer count of the remaining internally queued redo records in the Senzing repository.
g2_engine.countRedoRecords()
Click to expand `countRedoRecords()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
result_count = g2_engine.countRedoRecords()
g2_engine.destroy()
print(result_count)
except G2Exception as err:
print(g2_engine.getLastException())
Response message:
getRedoRecord
getRedoRecord() retrieves the next internally queued redo record into the Senzing repository.
Unlike processRedoRecord(), getRedoRecord() does not actually process the record.
To process the record, use process()
g2_engine.getRedoRecord(response_bytearray)
Parameters
- response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `getRedoRecord()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.getRedoRecord(response_bytearray)
print(response_bytearray.decode())
except G2Exception as err:
print(err)
Output
{
"REASON": "LIB_FEAT_ID[13] of FTYPE_ID[7] went generic for CANDIDATES ECLASS_ID[1] in LENS_ID[1]",
"DATA_SOURCE": "TEST",
"RECORD_ID": "39F84F26E1F85ABCCE20D5FDBCA7D08729DD829A",
"ENTITY_TYPE": "GENERIC",
"DSRC_ACTION": "X"
}
process
process() processes the redo record retrieved using getRedoRecord().
g2_engine.process(response_bytearray)
Parameters
- response_bytearray: (bytearray) the redo record obtained by
getRedoRecord() to process.
Click to expand `process()` example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.getRedoRecord(response_bytearray)
g2_engine.process(response_bytearray)
print(response_bytearray.decode())
except G2Exception as err:
print(err)
processWithInfo
processWithInfo() processes the redo record retrieved using getRedoRecord(), and returns a JSON document containing the ENTITY_ID values of the affected entities.
g2_engine.processWithInfo(response_bytearray, process_response_bytearray)
Parameters
- response_bytearray: (bytearray) the redo record obtained by
getRedoRecord() to process.
- process_response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `processWithInfo()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
with_info_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.getRedoRecord(response_bytearray)
g2_engine.processWithInfo(
response_bytearray,
with_info_bytearray)
print(with_info_bytearray.decode())
except G2Exception as err:
print(err)
Output
{
"REASON": "LIB_FEAT_ID[1307] of FTYPE_ID[14] went generic for CANDIDATES AND SCORING ECLASS_ID[1] in LENS_ID[1]",
"DATA_SOURCE": "CUSTOMERS",
"RECORD_ID": "2181",
"ENTITY_TYPE": "GENERIC",
"DSRC_ACTION": "X"
}
processRedoRecord
processRedoRecordWithInfo() processes the next redo record in the redo queue.
If processRedoRecord() returns a blank response_bytearray, then the redo queue is empty at the point of execution.
Additional redo records can be created after reevaluation.
g2_engine.processRedoRecord(response_bytearray)
Parameters
- response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `processRedoRecord()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.processRedoRecord()
except G2Exception as err:
print(err)
Output
{
"DATA_SOURCE": "CUSTOMERS",
"RECORD_ID": "2192",
"AFFECTED_ENTITIES": [
{
"ENTITY_ID": 105
}
],
"INTERESTING_ENTITIES": {
"ENTITIES": []
}
}
processRedoRecordWithInfo
processRedoRecordWithInfo() processes the next redo record in the redo queue, and returns a JSON document containing the ENTITY_ID values of the affected entities.
g2_engine.processRedoRecordWithInfo(response_bytearray, with_info_bytearray)
Parameters
- response_bytearray: (bytearray) Object to store the output of the method.
- info_bytearray: (bytearray) Returns the “withInfo” section
Click to expand `processRedoRecordWithInfo()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
with_info_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.processRedoRecordWithInfo(
response_bytearray,
with_info_bytearray)
print(with_info_bytearray.decode())
except G2Exception as err:
print(err)
Output
{
"DATA_SOURCE": "CUSTOMERS",
"RECORD_ID": "2192",
"AFFECTED_ENTITIES": [
{
"ENTITY_ID": 105
}
],
"INTERESTING_ENTITIES": {
"ENTITIES": []
}
}