Gazetteer upload

Owned by Robert Sikorski
Last updated: 09 May 2016
2 min read

Upload script

Gazetteer script file is a very specific script, since it does not contain a list of files to be imported. Instead it holds a path to the gazetteer source folder as well as destination table name.

  • data source – this is the path to the folder containing gazetteer CSV (comma separated values) files in DTF standard.
  • destination – this is the name of the gazetteer table in Earthlight database.

An example of gazetteer.script is provided below.

 

1
2
3
4
5
6
7
8
9
<?xml version="1.0"?>
<Script xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <Actions>
    <Action xsi:type="Load">
      <Source>data source="s:\Data To Upload\Gazetteer\*";table name="";cartridge=dtf</Source>
      <Destination>llpg</Destination>
    </Action>
  </Actions>
</Script>

 

The script file is usually accompanied by gazetteer.destination file which provides the connection string to Earthlight’s database.

database=statmap;user id=USER;password=PASSWORD;timeout=15;pooling=True;enlist=False;integrated security=False;initial catalog=EarthlightDB;data source=dbserver\sqlexpress;cartridge=SqlServer;schema=dbo

 

Please note that there is no table name in the .destination file.

 

Regular updates

  1. Create a folder where you wish to place DTF files for upload into your Earthlight address gazetteer.
  2. Copy a full DTF extract of your gazetteer to this newly created location.
  3. Execute the initial upload of the full gazetteer extract – using the .script and .destination files (as described above).
  4. Set up your .script file to point at the folder and find all files with a particular suffix (i.e. .csv or .xml).
  5. Set up your gazetteer management software to export a new DTF CoU file into this folder on a daily (or whichever interval you feel adequate) basis.
  6. Schedule DataPump to run a batch file which points at the gazetteer upload .script file, and set it to run at an interval similar to that which you have set the CoU DTF to be produced. Obviously this should be scheduled for a later time than the CoU DTF export from your gazetteer management software.

You must leave the original full DTF extract in that folder location – and all subsequent CoU DTF extracts (be they daily, weekly, monthly, etc.) in that same folder. Each time DataPump is activated, it will rebuild the gazetteer from the initial full extract, followed by all subsequent CoU files – to be processed in reverse chronological order.