Create Tables
The connector described in A Complete Synchronize Action always ingests every file, even those that have not changed since the last synchronize. You could improve this connector by implementing incremental synchronize.
To do this, you can create a Datastore table to store the modified date of each file. When the connector is about to ingest a file it can check this table and only ingest the file if it is new or has been modified:
Reference | LastModified |
---|---|
C:\data\file1.txt | 2013-08-05 18:05 |
C:\data\file2.txt | 2013-07-30 14:36 |
C:\data\file3.txt | 2012-12-24 09:00 |
You can create the table like this:
datastore.createTable( "Documents", new String[]{"Reference", "LastModified"} );
This creates a new table called "Documents" with two columns, "Reference" and "LastModified", provided the table does not exist already. If there is already a table called "Documents", this line verifies the table has the expected structure.
This example creates two text columns, but a table can also store numeric columns. Datastore.createTable
method.
Primary Keys and Automatic IDs
In ConnectorLib Java, you can specify the columns to use as the primary key in an overload of the Datastore.createTable
method.
In addition to the columns you specify when calling createTable
, each table also includes a column that holds a unique number to identify each row. This number is assigned automatically when a new record is inserted. The assigned values are expected to increase but are not guaranteed to be consecutive. The name of this column is specified in Datastore.DATASTORE_KEY
and is automatically included in the columns retrieved by the recordSelect
and recordSelectOne
methods. The unique number contained in this column is a 64-bit integer so you can retrieve it from a record like this:
long recordId = record.getLong(Datastore.DATASTORE_KEY);