Download this Jupyter notebook from github

---

Downloading and registering “catalogue” datasets

The geoslurp tools come with a set of “standard” datasets which are ready to be downloaded and used in the database. When running tasks using the geoslurper.py command line tool, it will consult a cached catalogue with dataset classes. Whenever a dataset class is needed it will be instantiated, so the pull and register routines can be called on it.

Although this is covered by the functionality of the geoslurper.py script, the catalogue can also be consulted directly in other python scripts.

The following example demonstrates how the catalogue is used to download and register a dataset from the natural earth collection.

from geoslurp.config.catalogue import geoslurpCatalogue
from geoslurp.db import Settings
from geoslurp.config import setInfoLevel

from geoslurp.db import geoslurpConnect

setInfoLevel()

gpcon=geoslurpConnect(readonlyuser=False) # this will be a connection based on the readonly userfrom geoslurp.db.geo

#Some datasets need info from the server side settings so we need to load these
conf=Settings(gpcon)

#refresh catalogue (note this only needs to be done when a catalogue exists already but new classes have been added to the paths)
# geoslurpCatalogue.refresh(conf)

Once the catalogue is loaded, we can query for dataset classes using regular expressions. The catalogue will return classes, which still need to be instantiated in order to be useful.

As a remark: Note that the following operation could also have been achieved with the following shell command:

geoslurper.py --pull --register -d "globalgis.ne_110m_admin_1_states_provinces.\*"

# find all datasetclasses which obey a certain regex (needs to match the entire string)
for dsclass in geoslurpCatalogue.getDatasets(conf,"globalgis\.ne_110m_admin_1_states_provinces.*"):
    # create an instance of the class
    dsobject=dsclass(gpcon)
    dsobject.pull()
    dsobject.register()

<class 'abc.ne_110m_admin_1_states_provinces'>
<class 'abc.ne_110m_admin_1_states_provinces_lakes'>
<class 'abc.ne_110m_admin_1_states_provinces_lines'>
<class 'abc.ne_110m_admin_1_states_provinces_scale_rank'>